Skip to content

Avoid creation of empty CopyOnWriteArrayList for span links#10822

Open
dougqh wants to merge 1 commit intomasterfrom
dougqh/span-links-allocation
Open

Avoid creation of empty CopyOnWriteArrayList for span links#10822
dougqh wants to merge 1 commit intomasterfrom
dougqh/span-links-allocation

Conversation

@dougqh
Copy link
Contributor

@dougqh dougqh commented Mar 12, 2026 •
edited
Loading

What Does This Do

This change shares an "EMPTY" list between spans when there are no span links.

This avoids the construction of a CopyOnWriteArrayList, empty Object[], and internal lock Object for the common case where there are no links.

Motivation

This elimination of the CopyOnWriteArrayList in the empty case, reduced allocation by 5% and GC time by 7% in a span creation stress test.

Additional Notes

Contributor Checklist

Jira ticket: [PROJ-IDENT]

Note: Once your PR is ready to merge, add it to the merge queue by commenting /merge. /merge -c cancels the queue request. /merge -f --reason "reason" skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.

@dougqh
Avoid creation of empty CopyOnWriteArrayList for span links
40fc417 
This change shares an "EMPTY" list between span when there are no span links.

This avoids the construction of a CopyOnWriteArrayList, empty Object[], and internal lock Object for the common case where there are no links.

This elimination of the CopyOnWriteArrayList in the empty case, reduced allocation by 5% and GC time by 7% in a span creation stress test.
@dougqh dougqh requested a review from a team as a code owner March 12, 2026 15:12
@dougqh dougqh requested a review from amarziali March 12, 2026 15:12
@dougqh dougqh added type: enhancement Enhancements and improvements comp: core Tracer core tag: performance Performance related changes labels Mar 12, 2026
amarziali
amarziali approved these changes Mar 12, 2026
View reviewed changes
Copy link
Contributor

@amarziali amarziali left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for the improvement

@dougqh dougqh self-assigned this Mar 12, 2026
@dougqh dougqh requested a review from PerfectSlayer March 12, 2026 15:18
amarziali
amarziali reviewed Mar 12, 2026
View reviewed changes
dd-trace-core/src/main/java/datadog/trace/core/DDSpan.java
private volatile int longRunningVersion = 0;

protected final List<AgentSpanLink> links;
private static final List<AgentSpanLink> EMPTY = Collections.emptyList();
Copy link
Contributor

@amarziali amarziali Mar 12, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There is an issue with SpanPointersProcessor. In fact TagsPostProcessors manipulate links directly and adding something to an emptyList will cause an UnsupportedOperationException. What can be done, is to use an empty non threadsafe list when calling tag processors if the original is empty and eventually batch adding the list at the end if the processors added something

Copy link
Contributor

@PerfectSlayer PerfectSlayer Mar 12, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Few ideas here:

  1. There is a TODO from Doug about the processor that mutates the span links:

dd-trace-java/dd-trace-core/src/main/java/datadog/trace/core/tagprocessor/SpanPointersProcessor.java

Line 42 in d9df78f

// DQH - TODO - There's a lot room to optimize this using TagMap's capabilities

Does it mean it can skip span links and use tags directly?

  1. As processors only add span links, we can replace the span links collection by a method reference to the addSpanLink method as Consumer<SpanLink>. This will avoid exposing the span link implementation and we can keep it null until there is effectively span link stored.

Copy link
Contributor Author

@dougqh dougqh Mar 12, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's a good catch.
I do think we need to improve the encapsulation; otherwise, these changes are going to be hard to make.

As for my prior TODO, TagMap has a bunch of methods for removeAndGetEntry, getString, stringValue, etc that could be used in the various *spanPointer methods. In theory, they could simplify the code significantly.

Copy link
Contributor Author

@dougqh dougqh Mar 12, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

On a related note, I'm curious why spanLinks was put on DDSpan rather than DDSpanContext.
It seems like it would be more consistent to put on DDSpanContext and then we wouldn't have this issue.

I also think we might be pool DDSpanContext in the future, so I'd like to avoid adding more to DDSpan.

Copy link
Contributor

@PerfectSlayer PerfectSlayer Mar 13, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm curious why spanLinks was put on DDSpan rather than DDSpanContext.

SpanContext was supposed to describe "a span context", not to hold the span data model.
We should be able to use SpanContext as reference descriptor for spans, but they end up holding way too much data.
So when I add to implement span links, I made sure to leave them in Span, not SpanContext.

I had a look at its Javadoc and it basically should only be IDs (trace + span), sampling, a bit of (W3C) trace state and baggage, nothing else :)

@pr-commenter
Copy link

pr-commenter bot commented Mar 12, 2026

Benchmarks

Startup

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master dougqh/span-links-allocation
git_commit_date 1773312120 1773328264
git_commit_sha 93c3816 40fc417
release_version 1.61.0-SNAPSHOT~93c3816062 1.61.0-SNAPSHOT~40fc417db7
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1773330010 1773330010
ci_job_id 1500802965 1500802965
ci_pipeline_id 102158397 102158397
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-2-dyta6wri 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-2-dyta6wri 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module Agent Agent
parent None None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 64 metrics, 7 unstable metrics.

Startup time reports for insecure-bank 
gantt
    title insecure-bank - global startup overhead: candidate=1.61.0-SNAPSHOT~40fc417db7, baseline=1.61.0-SNAPSHOT~93c3816062

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1059300
Total [baseline] (8.821 s) : 0, 8821382
Agent [candidate] (1.063 s) : 0, 1063068
Total [candidate] (8.838 s) : 0, 8838074
section iast
Agent [baseline] (1.224 s) : 0, 1223967
Total [baseline] (9.529 s) : 0, 9529275
Agent [candidate] (1.237 s) : 0, 1237236
Total [candidate] (9.626 s) : 0, 9626169
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent iast 1.224 s 164.667 ms (15.5%)
Total tracing 8.821 s -
Total iast 9.529 s 707.893 ms (8.0%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.063 s -
Agent iast 1.237 s 174.168 ms (16.4%)
Total tracing 8.838 s -
Total iast 9.626 s 788.095 ms (8.9%) 
gantt
    title insecure-bank - break down per module: candidate=1.61.0-SNAPSHOT~40fc417db7, baseline=1.61.0-SNAPSHOT~93c3816062

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.2 ms) : 0, 1200
crashtracking [candidate] (1.209 ms) : 0, 1209
BytebuddyAgent [baseline] (629.253 ms) : 0, 629253
BytebuddyAgent [candidate] (631.084 ms) : 0, 631084
AgentMeter [baseline] (29.137 ms) : 0, 29137
AgentMeter [candidate] (29.33 ms) : 0, 29330
GlobalTracer [baseline] (257.389 ms) : 0, 257389
GlobalTracer [candidate] (258.262 ms) : 0, 258262
AppSec [baseline] (31.691 ms) : 0, 31691
AppSec [candidate] (31.741 ms) : 0, 31741
Debugger [baseline] (58.888 ms) : 0, 58888
Debugger [candidate] (59.039 ms) : 0, 59039
Remote Config [baseline] (595.757 µs) : 0, 596
Remote Config [candidate] (595.549 µs) : 0, 596
Telemetry [baseline] (8.715 ms) : 0, 8715
Telemetry [candidate] (8.606 ms) : 0, 8606
Flare Poller [baseline] (6.405 ms) : 0, 6405
Flare Poller [candidate] (7.119 ms) : 0, 7119
section iast
crashtracking [baseline] (1.189 ms) : 0, 1189
crashtracking [candidate] (1.191 ms) : 0, 1191
BytebuddyAgent [baseline] (793.967 ms) : 0, 793967
BytebuddyAgent [candidate] (803.35 ms) : 0, 803350
AgentMeter [baseline] (11.288 ms) : 0, 11288
AgentMeter [candidate] (11.643 ms) : 0, 11643
GlobalTracer [baseline] (246.966 ms) : 0, 246966
GlobalTracer [candidate] (249.807 ms) : 0, 249807
IAST [baseline] (25.094 ms) : 0, 25094
IAST [candidate] (25.417 ms) : 0, 25417
AppSec [baseline] (26.346 ms) : 0, 26346
AppSec [candidate] (26.664 ms) : 0, 26664
Debugger [baseline] (62.715 ms) : 0, 62715
Debugger [candidate] (62.757 ms) : 0, 62757
Remote Config [baseline] (544.341 µs) : 0, 544
Remote Config [candidate] (524.48 µs) : 0, 524
Telemetry [baseline] (15.056 ms) : 0, 15056
Telemetry [candidate] (15.208 ms) : 0, 15208
Flare Poller [baseline] (4.389 ms) : 0, 4389
Flare Poller [candidate] (4.408 ms) : 0, 4408
Loading
Startup time reports for petclinic 
gantt
    title petclinic - global startup overhead: candidate=1.61.0-SNAPSHOT~40fc417db7, baseline=1.61.0-SNAPSHOT~93c3816062

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.059 s) : 0, 1059055
Total [baseline] (11.047 s) : 0, 11046504
Agent [candidate] (1.062 s) : 0, 1061759
Total [candidate] (11.048 s) : 0, 11047624
section appsec
Agent [baseline] (1.242 s) : 0, 1242156
Total [baseline] (11.106 s) : 0, 11105845
Agent [candidate] (1.246 s) : 0, 1245594
Total [candidate] (11.137 s) : 0, 11137046
section iast
Agent [baseline] (1.231 s) : 0, 1231135
Total [baseline] (11.331 s) : 0, 11330714
Agent [candidate] (1.225 s) : 0, 1225207
Total [candidate] (11.328 s) : 0, 11327532
section profiling
Agent [baseline] (1.186 s) : 0, 1186246
Total [baseline] (11.016 s) : 0, 11015656
Agent [candidate] (1.177 s) : 0, 1177500
Total [candidate] (10.937 s) : 0, 10936657
Loading
  • baseline results
Module Variant Duration Δ tracing
Agent tracing 1.059 s -
Agent appsec 1.242 s 183.101 ms (17.3%)
Agent iast 1.231 s 172.079 ms (16.2%)
Agent profiling 1.186 s 127.19 ms (12.0%)
Total tracing 11.047 s -
Total appsec 11.106 s 59.341 ms (0.5%)
Total iast 11.331 s 284.21 ms (2.6%)
Total profiling 11.016 s -30.849 ms (-0.3%)
  • candidate results
Module Variant Duration Δ tracing
Agent tracing 1.062 s -
Agent appsec 1.246 s 183.834 ms (17.3%)
Agent iast 1.225 s 163.447 ms (15.4%)
Agent profiling 1.177 s 115.74 ms (10.9%)
Total tracing 11.048 s -
Total appsec 11.137 s 89.422 ms (0.8%)
Total iast 11.328 s 279.908 ms (2.5%)
Total profiling 10.937 s -110.968 ms (-1.0%) 
gantt
    title petclinic - break down per module: candidate=1.61.0-SNAPSHOT~40fc417db7, baseline=1.61.0-SNAPSHOT~93c3816062

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.191 ms) : 0, 1191
crashtracking [candidate] (1.193 ms) : 0, 1193
BytebuddyAgent [baseline] (626.743 ms) : 0, 626743
BytebuddyAgent [candidate] (629.019 ms) : 0, 629019
AgentMeter [baseline] (29.033 ms) : 0, 29033
AgentMeter [candidate] (29.31 ms) : 0, 29310
GlobalTracer [baseline] (257.074 ms) : 0, 257074
GlobalTracer [candidate] (258.276 ms) : 0, 258276
AppSec [baseline] (31.81 ms) : 0, 31810
AppSec [candidate] (31.726 ms) : 0, 31726
Debugger [baseline] (59.859 ms) : 0, 59859
Debugger [candidate] (59.763 ms) : 0, 59763
Remote Config [baseline] (601.437 µs) : 0, 601
Remote Config [candidate] (597.14 µs) : 0, 597
Telemetry [baseline] (8.705 ms) : 0, 8705
Telemetry [candidate] (8.699 ms) : 0, 8699
Flare Poller [baseline] (8.048 ms) : 0, 8048
Flare Poller [candidate] (7.228 ms) : 0, 7228
section appsec
crashtracking [baseline] (1.186 ms) : 0, 1186
crashtracking [candidate] (1.196 ms) : 0, 1196
BytebuddyAgent [baseline] (655.577 ms) : 0, 655577
BytebuddyAgent [candidate] (657.095 ms) : 0, 657095
AgentMeter [baseline] (11.987 ms) : 0, 11987
AgentMeter [candidate] (12.039 ms) : 0, 12039
GlobalTracer [baseline] (257.567 ms) : 0, 257567
GlobalTracer [candidate] (258.529 ms) : 0, 258529
IAST [baseline] (23.865 ms) : 0, 23865
IAST [candidate] (23.91 ms) : 0, 23910
AppSec [baseline] (177.319 ms) : 0, 177319
AppSec [candidate] (177.575 ms) : 0, 177575
Debugger [baseline] (65.348 ms) : 0, 65348
Debugger [candidate] (65.788 ms) : 0, 65788
Remote Config [baseline] (568.582 µs) : 0, 569
Remote Config [candidate] (573.403 µs) : 0, 573
Telemetry [baseline] (8.932 ms) : 0, 8932
Telemetry [candidate] (8.962 ms) : 0, 8962
Flare Poller [baseline] (3.561 ms) : 0, 3561
Flare Poller [candidate] (3.655 ms) : 0, 3655
section iast
crashtracking [baseline] (1.193 ms) : 0, 1193
crashtracking [candidate] (1.179 ms) : 0, 1179
BytebuddyAgent [baseline] (798.964 ms) : 0, 798964
BytebuddyAgent [candidate] (794.847 ms) : 0, 794847
AgentMeter [baseline] (11.538 ms) : 0, 11538
AgentMeter [candidate] (11.298 ms) : 0, 11298
GlobalTracer [baseline] (247.758 ms) : 0, 247758
GlobalTracer [candidate] (246.743 ms) : 0, 246743
IAST [baseline] (25.21 ms) : 0, 25210
IAST [candidate] (25.042 ms) : 0, 25042
AppSec [baseline] (26.47 ms) : 0, 26470
AppSec [candidate] (26.311 ms) : 0, 26311
Debugger [baseline] (64.443 ms) : 0, 64443
Debugger [candidate] (64.253 ms) : 0, 64253
Remote Config [baseline] (530.288 µs) : 0, 530
Remote Config [candidate] (529.686 µs) : 0, 530
Telemetry [baseline] (14.276 ms) : 0, 14276
Telemetry [candidate] (14.84 ms) : 0, 14840
Flare Poller [baseline] (4.383 ms) : 0, 4383
Flare Poller [candidate] (4.268 ms) : 0, 4268
section profiling
crashtracking [baseline] (1.175 ms) : 0, 1175
crashtracking [candidate] (1.17 ms) : 0, 1170
BytebuddyAgent [baseline] (685.8 ms) : 0, 685800
BytebuddyAgent [candidate] (680.448 ms) : 0, 680448
AgentMeter [baseline] (8.674 ms) : 0, 8674
AgentMeter [candidate] (8.579 ms) : 0, 8579
GlobalTracer [baseline] (216.603 ms) : 0, 216603
GlobalTracer [candidate] (214.84 ms) : 0, 214840
AppSec [baseline] (31.976 ms) : 0, 31976
AppSec [candidate] (31.716 ms) : 0, 31716
Debugger [baseline] (63.935 ms) : 0, 63935
Debugger [candidate] (65.024 ms) : 0, 65024
Remote Config [baseline] (585.487 µs) : 0, 585
Remote Config [candidate] (582.679 µs) : 0, 583
Telemetry [baseline] (9.72 ms) : 0, 9720
Telemetry [candidate] (8.12 ms) : 0, 8120
Flare Poller [baseline] (3.451 ms) : 0, 3451
Flare Poller [candidate] (3.453 ms) : 0, 3453
ProfilingAgent [baseline] (93.286 ms) : 0, 93286
ProfilingAgent [candidate] (93.098 ms) : 0, 93098
Profiling [baseline] (93.849 ms) : 0, 93849
Profiling [candidate] (93.658 ms) : 0, 93658
Loading

Load

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master dougqh/span-links-allocation
git_commit_date 1773312120 1773328264
git_commit_sha 93c3816 40fc417
release_version 1.61.0-SNAPSHOT~93c3816062 1.61.0-SNAPSHOT~40fc417db7
See matching parameters
Baseline Candidate
application insecure-bank insecure-bank
ci_job_date 1773330584 1773330584
ci_job_id 1500802966 1500802966
ci_pipeline_id 102158397 102158397
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-1-88mdjkcj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-1-88mdjkcj 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 2 performance regressions! Performance is the same for 18 metrics, 15 unstable metrics.

scenario Δ mean agg_http_req_duration_p50 Δ mean agg_http_req_duration_p95 Δ mean throughput candidate mean agg_http_req_duration_p50 candidate mean agg_http_req_duration_p95 candidate mean throughput baseline mean agg_http_req_duration_p50 baseline mean agg_http_req_duration_p95 baseline mean throughput
scenario:load:insecure-bank:profiling:high_load better
[-189.337µs; -57.217µs] or [-10.508%; -3.175%]		 unstable
[-1241.836µs; -359.831µs] or [-21.627%; -6.267%]		 unstable
[-37.568op/s; +478.380op/s] or [-1.948%; +24.809%]		 1.679ms 4.941ms 2148.656op/s 1.802ms 5.742ms 1928.250op/s
scenario:load:insecure-bank:iast_GLOBAL:high_load worse
[+102.794µs; +268.546µs] or [+3.776%; +9.865%]		 worse
[+172.172µs; +675.871µs] or [+2.199%; +8.634%]		 unstable
[-217.055op/s; +69.117op/s] or [-16.434%; +5.233%]		 2.908ms 8.252ms 1246.781op/s 2.722ms 7.828ms 1320.750op/s
Request duration reports for insecure-bank 
gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~40fc417db7, baseline=1.61.0-SNAPSHOT~93c3816062
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.178 ms) : 1167, 1190
.   : milestone, 1178,
iast (3.166 ms) : 3125, 3207
.   : milestone, 3166,
iast_FULL (5.998 ms) : 5936, 6059
.   : milestone, 5998,
iast_GLOBAL (3.471 ms) : 3421, 3522
.   : milestone, 3471,
profiling (2.354 ms) : 2331, 2377
.   : milestone, 2354,
tracing (1.746 ms) : 1732, 1759
.   : milestone, 1746,
section candidate
no_agent (1.196 ms) : 1184, 1208
.   : milestone, 1196,
iast (3.163 ms) : 3120, 3206
.   : milestone, 3163,
iast_FULL (6.001 ms) : 5940, 6062
.   : milestone, 6001,
iast_GLOBAL (3.679 ms) : 3623, 3736
.   : milestone, 3679,
profiling (2.104 ms) : 2084, 2125
.   : milestone, 2104,
tracing (1.775 ms) : 1761, 1790
.   : milestone, 1775,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.178 ms [1.167 ms, 1.19 ms] -
iast 3.166 ms [3.125 ms, 3.207 ms] 1.988 ms (168.7%)
iast_FULL 5.998 ms [5.936 ms, 6.059 ms] 4.819 ms (409.0%)
iast_GLOBAL 3.471 ms [3.421 ms, 3.522 ms] 2.293 ms (194.6%)
profiling 2.354 ms [2.331 ms, 2.377 ms] 1.176 ms (99.8%)
tracing 1.746 ms [1.732 ms, 1.759 ms] 567.162 µs (48.1%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 1.196 ms [1.184 ms, 1.208 ms] -
iast 3.163 ms [3.12 ms, 3.206 ms] 1.967 ms (164.5%)
iast_FULL 6.001 ms [5.94 ms, 6.062 ms] 4.805 ms (401.8%)
iast_GLOBAL 3.679 ms [3.623 ms, 3.736 ms] 2.484 ms (207.7%)
profiling 2.104 ms [2.084 ms, 2.125 ms] 908.139 µs (75.9%)
tracing 1.775 ms [1.761 ms, 1.79 ms] 579.125 µs (48.4%)
Request duration reports for petclinic 
gantt
    title petclinic - request duration [CI 0.99] : candidate=1.61.0-SNAPSHOT~40fc417db7, baseline=1.61.0-SNAPSHOT~93c3816062
    dateFormat X
    axisFormat %s
section baseline
no_agent (18.203 ms) : 18018, 18388
.   : milestone, 18203,
appsec (18.855 ms) : 18663, 19046
.   : milestone, 18855,
code_origins (17.96 ms) : 17780, 18140
.   : milestone, 17960,
iast (17.617 ms) : 17443, 17792
.   : milestone, 17617,
profiling (18.476 ms) : 18292, 18660
.   : milestone, 18476,
tracing (18.04 ms) : 17860, 18219
.   : milestone, 18040,
section candidate
no_agent (18.076 ms) : 17894, 18259
.   : milestone, 18076,
appsec (18.941 ms) : 18749, 19132
.   : milestone, 18941,
code_origins (18.327 ms) : 18144, 18511
.   : milestone, 18327,
iast (17.727 ms) : 17553, 17901
.   : milestone, 17727,
profiling (18.622 ms) : 18434, 18810
.   : milestone, 18622,
tracing (18.27 ms) : 18089, 18451
.   : milestone, 18270,
Loading
  • baseline results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.203 ms [18.018 ms, 18.388 ms] -
appsec 18.855 ms [18.663 ms, 19.046 ms] 651.422 µs (3.6%)
code_origins 17.96 ms [17.78 ms, 18.14 ms] -243.289 µs (-1.3%)
iast 17.617 ms [17.443 ms, 17.792 ms] -585.75 µs (-3.2%)
profiling 18.476 ms [18.292 ms, 18.66 ms] 272.588 µs (1.5%)
tracing 18.04 ms [17.86 ms, 18.219 ms] -163.316 µs (-0.9%)
  • candidate results
Variant Request duration [CI 0.99] Δ no_agent
no_agent 18.076 ms [17.894 ms, 18.259 ms] -
appsec 18.941 ms [18.749 ms, 19.132 ms] 864.548 µs (4.8%)
code_origins 18.327 ms [18.144 ms, 18.511 ms] 251.203 µs (1.4%)
iast 17.727 ms [17.553 ms, 17.901 ms] -348.906 µs (-1.9%)
profiling 18.622 ms [18.434 ms, 18.81 ms] 545.603 µs (3.0%)
tracing 18.27 ms [18.089 ms, 18.451 ms] 193.626 µs (1.1%)

Dacapo

Parameters

Baseline Candidate
baseline_or_candidate baseline candidate
git_branch master dougqh/span-links-allocation
git_commit_date 1773312120 1773328264
git_commit_sha 93c3816 40fc417
release_version 1.61.0-SNAPSHOT~93c3816062 1.61.0-SNAPSHOT~40fc417db7
See matching parameters
Baseline Candidate
application biojava biojava
ci_job_date 1773330241 1773330241
ci_job_id 1500802967 1500802967
ci_pipeline_id 102158397 102158397
cpu_model Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version Linux runner-zfyrx7zua-project-304-concurrent-0-i4qg0zp6 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux Linux runner-zfyrx7zua-project-304-concurrent-0-i4qg0zp6 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat 
gantt
    title tomcat - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~40fc417db7, baseline=1.61.0-SNAPSHOT~93c3816062
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.474 ms) : 1463, 1486
.   : milestone, 1474,
appsec (3.844 ms) : 3621, 4067
.   : milestone, 3844,
iast (2.254 ms) : 2185, 2323
.   : milestone, 2254,
iast_GLOBAL (2.311 ms) : 2241, 2381
.   : milestone, 2311,
profiling (2.099 ms) : 2044, 2155
.   : milestone, 2099,
tracing (2.065 ms) : 2012, 2119
.   : milestone, 2065,
section candidate
no_agent (1.475 ms) : 1464, 1487
.   : milestone, 1475,
appsec (3.815 ms) : 3591, 4038
.   : milestone, 3815,
iast (2.258 ms) : 2189, 2327
.   : milestone, 2258,
iast_GLOBAL (2.3 ms) : 2230, 2369
.   : milestone, 2300,
profiling (2.527 ms) : 2360, 2693
.   : milestone, 2527,
tracing (2.077 ms) : 2023, 2131
.   : milestone, 2077,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.474 ms [1.463 ms, 1.486 ms] -
appsec 3.844 ms [3.621 ms, 4.067 ms] 2.37 ms (160.8%)
iast 2.254 ms [2.185 ms, 2.323 ms] 779.925 µs (52.9%)
iast_GLOBAL 2.311 ms [2.241 ms, 2.381 ms] 836.663 µs (56.8%)
profiling 2.099 ms [2.044 ms, 2.155 ms] 625.234 µs (42.4%)
tracing 2.065 ms [2.012 ms, 2.119 ms] 591.001 µs (40.1%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 1.475 ms [1.464 ms, 1.487 ms] -
appsec 3.815 ms [3.591 ms, 4.038 ms] 2.339 ms (158.6%)
iast 2.258 ms [2.189 ms, 2.327 ms] 782.986 µs (53.1%)
iast_GLOBAL 2.3 ms [2.23 ms, 2.369 ms] 824.51 µs (55.9%)
profiling 2.527 ms [2.36 ms, 2.693 ms] 1.051 ms (71.3%)
tracing 2.077 ms [2.023 ms, 2.131 ms] 601.481 µs (40.8%)
Execution time for biojava 
gantt
    title biojava - execution time [CI 0.99] : candidate=1.61.0-SNAPSHOT~40fc417db7, baseline=1.61.0-SNAPSHOT~93c3816062
    dateFormat X
    axisFormat %s
section baseline
no_agent (14.891 s) : 14891000, 14891000
.   : milestone, 14891000,
appsec (14.848 s) : 14848000, 14848000
.   : milestone, 14848000,
iast (18.037 s) : 18037000, 18037000
.   : milestone, 18037000,
iast_GLOBAL (17.576 s) : 17576000, 17576000
.   : milestone, 17576000,
profiling (15.316 s) : 15316000, 15316000
.   : milestone, 15316000,
tracing (15.285 s) : 15285000, 15285000
.   : milestone, 15285000,
section candidate
no_agent (15.002 s) : 15002000, 15002000
.   : milestone, 15002000,
appsec (15.107 s) : 15107000, 15107000
.   : milestone, 15107000,
iast (18.244 s) : 18244000, 18244000
.   : milestone, 18244000,
iast_GLOBAL (17.942 s) : 17942000, 17942000
.   : milestone, 17942000,
profiling (15.008 s) : 15008000, 15008000
.   : milestone, 15008000,
tracing (14.983 s) : 14983000, 14983000
.   : milestone, 14983000,
Loading
  • baseline results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 14.891 s [14.891 s, 14.891 s] -
appsec 14.848 s [14.848 s, 14.848 s] -43.0 ms (-0.3%)
iast 18.037 s [18.037 s, 18.037 s] 3.146 s (21.1%)
iast_GLOBAL 17.576 s [17.576 s, 17.576 s] 2.685 s (18.0%)
profiling 15.316 s [15.316 s, 15.316 s] 425.0 ms (2.9%)
tracing 15.285 s [15.285 s, 15.285 s] 394.0 ms (2.6%)
  • candidate results
Variant Execution Time [CI 0.99] Δ no_agent
no_agent 15.002 s [15.002 s, 15.002 s] -
appsec 15.107 s [15.107 s, 15.107 s] 105.0 ms (0.7%)
iast 18.244 s [18.244 s, 18.244 s] 3.242 s (21.6%)
iast_GLOBAL 17.942 s [17.942 s, 17.942 s] 2.94 s (19.6%)
profiling 15.008 s [15.008 s, 15.008 s] 6.0 ms (0.0%)
tracing 14.983 s [14.983 s, 14.983 s] -19.0 ms (-0.1%)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@amarziali amarziali amarziali approved these changes

@PerfectSlayer PerfectSlayer Awaiting requested review from PerfectSlayer

Assignees

@dougqh dougqh

@PerfectSlayer PerfectSlayer

Labels

comp: core Tracer core tag: performance Performance related changes type: enhancement Enhancements and improvements

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@dougqh @PerfectSlayer @amarziali