feat: pass span_kinds_stats_computed and peer_tags to span concentrator by duncanpharvey · Pull Request #124 · DataDog/serverless-components

duncanpharvey · 2026-04-30T14:30:05Z

What does this PR do?

Pass span_kinds_stats_computed and peer_tags to span concentrator.
Include span_kinds_stats_computed and peer_tags in /info endpoint response.

Motivation

Compute stats for certain span kinds even if they are not top level or measured spans.
- server
- client
- producer
- consumer
Add peer tags as additional tags on computed stats

https://datadoghq.atlassian.net/browse/SVLS-8977

Additional Notes

Span kinds from Go agent
Peer tags from Go agent
Updates libdatadog revision for feat: use ip quantization when aggregating peer tags for trace stats libdatadog#1944
Updated shutdown logic due to flaky test test_mini_agent_named_pipe_with_real_flushers. Now, when a shutdown signal arrives, each connection gets a chance to finish its current request before closing, rather than potentially being killed mid-response.

    thread 'test_mini_agent_named_pipe_with_real_flushers' (2036) panicked at crates\datadog-trace-agent\tests\integration_test.rs:49:5:
    Timed out after 60s waiting for request at /api/v0.2/stats

Describe how to test/QA your changes

Unit tests
- Expected peer tags are parsed from json
- Parsed peer tags are sorted and do not contain duplicates
Integration tests
- /info endpoint contains expected values for span_kinds_stats_computed and peer_tags
- Stats computed for a non-top-level, non-measured, eligible span kind (server)
- Stats not computed for a non-top-level, non-measured, ineligible span kind ('internal`)
- Peer tags included on flushed stats
End to end test (manual)
- Deployed Azure Function with Service Bus trigger and validated that stats are computed for service bus spans with relevant peer tags

Copilot

Pull request overview

This PR extends the serverless trace agent’s stats computation configuration by (1) ensuring specific span.kind values can trigger stats computation even when spans aren’t top-level/measured, and (2) including “peer tag” keys so computed stats can be enriched with additional peer-related dimensions.

Changes:

Add a peer_tags configuration field derived from an embedded JSON mapping, and expose it via /info.
Pass SPAN_KINDS_STATS_COMPUTED and configured peer_tags into the span concentrator creation path.
Add integration + unit test coverage to validate /info output and peer-tag/span-kind driven stats behavior.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
crates/datadog-trace-agent/tests/integration_test.rs	Adds integration assertions for `/info` (`span_kinds_stats_computed`, `peer_tags`) and verifies peer tags + eligible span kinds affect computed stats.
crates/datadog-trace-agent/tests/common/helpers.rs	Adds helpers to generate span.kind/peer-tag traces and decode gzipped msgpack stats payloads in integration tests.
crates/datadog-trace-agent/src/trace_processor.rs	Updates test config construction to include the new `peer_tags` config field.
crates/datadog-trace-agent/src/stats_concentrator_service.rs	Introduces `SPAN_KINDS_STATS_COMPUTED` and wires span kinds + peer tags into `SpanConcentrator::new` during concentrator initialization.
crates/datadog-trace-agent/src/peer_tags.rs	New module to parse embedded mappings and produce a sorted/deduped peer tag key list, with unit tests.
crates/datadog-trace-agent/src/peer_tags_mappings.json	Embedded peer tag mappings source used to derive the peer tag key set.
crates/datadog-trace-agent/src/mini_agent.rs	Exposes `span_kinds_stats_computed` and `peer_tags` on `/info`, and passes peer tags into the handler.
crates/datadog-trace-agent/src/lib.rs	Exports the new `peer_tags` module.
crates/datadog-trace-agent/src/config.rs	Adds `peer_tags` to config and initializes it from the embedded mappings.
crates/datadog-trace-agent/Cargo.toml	Adds `flate2` as a dev-dependency for gzip decoding in tests.
Cargo.lock	Updates lockfile to include `flate2`.

duncanpharvey · 2026-05-05T14:32:43Z

                    let concentrator = self
                        .concentrators
                        .entry(Arc::clone(&metadata))
-                        .or_insert_with(new_concentrator);
+                        .or_insert_with(|| {
+                            Self::new_span_concentrator(self.config.peer_tags.clone())
+                        });


Fixed in 09bd187

duncanpharvey · 2026-05-04T15:29:37Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 09bd1877fe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 09bd1877fe

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-04T15:34:26Z

+        { "name": "http.server_name", "provider": "datadog", "type": "string" },
+        { "name": "out.host", "provider": "datadog", "type": "string" },
+        { "name": "dns.hostname", "provider": "datadog", "type": "string" },
+        { "name": "network.destination.ip", "provider": "otel", "type": "string" }


Avoid aggregating raw destination IP peer tags

When client/producer/consumer spans carry network.destination.ip, this new default key is passed into both the agent-side SpanConcentrator and /info for tracer-side stats. I checked the pinned libdd-trace-stats implementation at d7eef80: it copies configured peer tag values directly into ClientGroupedStats.peer_tags via format!("{k}:{v}") without IP quantization, so OTel spans that set destination IPs will create one stats aggregation group per raw remote IP and can explode stats cardinality/cost. Please omit this key until the concentrator quantizes IP peer tags, or add quantization before advertising/using it.

Useful? React with 👍 / 👎.

quantize_peer_ip_addresses is available in libdatadog but not yet applied to trace stats aggregation in libdd-trace-stats. Checking with libdatadog to see when this change can be applied.

Fixed in DataDog/libdatadog#1944

with tracer.trace("test_span") as span: span.set_tag("span.kind", "client") span.set_tag("peer.hostname", "127.0.0.1") span.set_tag("_dd.flattened.peer.tags", "peer.hostname:127.0.0.1")

with tracer.trace("test_span") as span: span.set_tag("span.kind", "client") span.set_tag("peer.hostname", "192.0.2.1") span.set_tag("_dd.flattened.peer.tags", "peer.hostname:192.0.2.1")

…1944) # What does this PR do? Quantize IP addresses for peer tags when aggregating trace stats. # Motivation Needed for agent side stats computation in the Serverless Compatibility Layer (used as the agent in Azure Functions) in DataDog/serverless-components#124. # Additional Notes Anything else we should know when reviewing? # How to test the change? Describe here in detail how the change can be validated. Co-authored-by: Eldolfin <oscar.ledauphin@datadoghq.com>

duncanpharvey · 2026-05-06T16:53:43Z

@codex review

chatgpt-codex-connector · 2026-05-06T16:57:40Z

Codex Review: Didn't find any major issues. Swish!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-05-06T17:00:00Z

Codex Review: Didn't find any major issues. Chef's kiss.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

litianningdatadog force-pushed the main branch from 1e0c4af to 30b1e00 Compare May 1, 2026 14:23

duncanpharvey force-pushed the duncan-harvey/trace-stats-kind-peer-tags branch 3 times, most recently from 674ae04 to 97a2f9e Compare May 4, 2026 15:14

duncanpharvey marked this pull request as ready for review May 4, 2026 15:14

duncanpharvey requested a review from a team as a code owner May 4, 2026 15:14

Copilot AI review requested due to automatic review settings May 4, 2026 15:14

duncanpharvey requested a review from a team as a code owner May 4, 2026 15:14

duncanpharvey requested review from apiarian-datadog, jchrostek-dd and lucaspimentel and removed request for a team May 4, 2026 15:14

Copilot started reviewing on behalf of duncanpharvey May 4, 2026 15:15 View session

Copilot AI reviewed May 4, 2026

View reviewed changes

chatgpt-codex-connector Bot reviewed May 4, 2026

View reviewed changes

Comment thread crates/datadog-trace-agent/src/peer_tags_mappings.json

chatgpt-codex-connector Bot reviewed May 4, 2026

View reviewed changes

duncanpharvey mentioned this pull request May 4, 2026

feat: use ip quantization when aggregating peer tags for trace stats DataDog/libdatadog#1944

Merged

duncanpharvey added 5 commits May 6, 2026 10:28

add span_kinds_stats_computed to stats concentrator service

54afa97

add peer_tags to stats concentrator service

2685b63

add integration tests

102abbc

clone peer_tags before entry(...) call to avoid borrow conflict

03497a2

update libdatadog revision to 4ae8ebe252451374c292efd159ce254c3f5a72e0

0162d7f

duncanpharvey force-pushed the duncan-harvey/trace-stats-kind-peer-tags branch from 7818b4b to 0162d7f Compare May 6, 2026 14:30

duncanpharvey added 3 commits May 6, 2026 10:35

fix integration tests

f95eea1

use different port for integration test

b4f8fdc

fix flaky graceful shutdown

905b2e2

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: pass span_kinds_stats_computed and peer_tags to span concentrator#124

feat: pass span_kinds_stats_computed and peer_tags to span concentrator#124
duncanpharvey wants to merge 8 commits intomainfrom
duncan-harvey/trace-stats-kind-peer-tags

duncanpharvey commented Apr 30, 2026 •

edited

Loading

Uh oh!

Copilot AI left a comment

Uh oh!

duncanpharvey May 5, 2026

Uh oh!

duncanpharvey commented May 4, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot May 4, 2026

Uh oh!

duncanpharvey May 4, 2026

Uh oh!

duncanpharvey May 6, 2026

Uh oh!

duncanpharvey May 6, 2026

Uh oh!

duncanpharvey commented May 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

duncanpharvey commented Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Additional Notes

Describe how to test/QA your changes

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

duncanpharvey May 5, 2026

Choose a reason for hiding this comment

Uh oh!

duncanpharvey commented May 4, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot May 4, 2026

Choose a reason for hiding this comment

Uh oh!

duncanpharvey May 4, 2026

Choose a reason for hiding this comment

Uh oh!

duncanpharvey May 6, 2026

Choose a reason for hiding this comment

Uh oh!

duncanpharvey May 6, 2026

Choose a reason for hiding this comment

Uh oh!

duncanpharvey commented May 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 6, 2026

Uh oh!

chatgpt-codex-connector Bot commented May 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

duncanpharvey commented Apr 30, 2026 •

edited

Loading