Commit 5847f5b
authored
Reduce complexity of CriticalPathQueuingDurationDataProvider (#194)
This provider currently performs several full loops over the Bazel
profiles. This results in it not scaling well performance and memory
consumption-wise.
Also at the profile level, some expensive copying and re-sorting of
bazel profile data is performed each time it is requested by the
providers.
Allocations:
<img width="3079" height="828" alt="image"
src="https://github.com/user-attachments/assets/164a37a3-8193-42fb-a860-6bc961d277ac"
/>
CPU:
<img width="3098" height="988" alt="image"
src="https://github.com/user-attachments/assets/13ed0e2c-f0b4-42d4-8562-04c000f76325"
/>
This PR aims to reduce the amount of loops over the data in
`CriticalPathQueuingDurationDataProvider` to just one, enabling it to
scale better.
Also, cleans the expensive copying and re-sorting of bazel profile data
each time it is requested, by switching thread building to a Builder
pattern, which sorts the entries internally as needed before returning
the immutable sorted result.
Thanks to @iamricard for doing a lot of the initial work and
investigation on this (the branch was opened in another repository so I
didn't know how to cleanly credit him in the commits I moved over to
here)
---------
Signed-off-by: Felipe <afrueda97@outlook.com>1 parent 3fa9602 commit 5847f5b
6 files changed
Lines changed: 500 additions & 328 deletions
File tree
- analyzer
- javatests/com/engflow/bazel/invocation/analyzer/bazelprofile
- java/com/engflow/bazel/invocation/analyzer
- bazelprofile
- dataproviders/remoteexecution
Lines changed: 15 additions & 5 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
35 | 35 | | |
36 | 36 | | |
37 | 37 | | |
| 38 | + | |
38 | 39 | | |
39 | 40 | | |
40 | 41 | | |
41 | 42 | | |
42 | 43 | | |
43 | 44 | | |
| 45 | + | |
44 | 46 | | |
45 | 47 | | |
46 | 48 | | |
| |||
82 | 84 | | |
83 | 85 | | |
84 | 86 | | |
| 87 | + | |
| 88 | + | |
| 89 | + | |
| 90 | + | |
85 | 91 | | |
86 | 92 | | |
87 | | - | |
| 93 | + | |
88 | 94 | | |
89 | 95 | | |
90 | 96 | | |
| 97 | + | |
91 | 98 | | |
92 | 99 | | |
93 | 100 | | |
| |||
116 | 123 | | |
117 | 124 | | |
118 | 125 | | |
119 | | - | |
120 | | - | |
| 126 | + | |
| 127 | + | |
121 | 128 | | |
122 | 129 | | |
123 | 130 | | |
124 | | - | |
| 131 | + | |
125 | 132 | | |
126 | 133 | | |
127 | 134 | | |
128 | 135 | | |
129 | | - | |
| 136 | + | |
130 | 137 | | |
131 | 138 | | |
132 | 139 | | |
| |||
143 | 150 | | |
144 | 151 | | |
145 | 152 | | |
| 153 | + | |
| 154 | + | |
| 155 | + | |
146 | 156 | | |
147 | 157 | | |
148 | 158 | | |
| |||
0 commit comments