Skip to content

[APPSEC-60097] Add more data collection for downstream requests#5206

Merged
Strech merged 21 commits intomasterfrom
appsec-60097-add-new-data-collection-to-downstream-requests
Jan 16, 2026
Merged

[APPSEC-60097] Add more data collection for downstream requests#5206
Strech merged 21 commits intomasterfrom
appsec-60097-add-new-data-collection-to-downstream-requests

Conversation

@Strech
Copy link
Copy Markdown
Member

@Strech Strech commented Jan 7, 2026

What does this PR do?

Add the first part of the API10 RFC. Where we supply more data to the WAF on downstream requests, more stats and metrics. Ideally we should process request/response body and track redirects.

  1. Add new WAF addresses <------------- you are here
  2. Add request/response body processing
  3. Add best-effort redirects tracking

Motivation:

I want to start with small adjustments for RASP methods and metrics and see where it will lead without doing a refactoring, we are missing some other metrics and telemetry and before changing it would be good to see the full picture.

Change log entry

Yes. AppSec: Add analysis of the downstream requests.

Additional Notes:

First of all this is not a perfect implementation, we already notice an accumulation of the look a like methods and modules. And that will be addresses in rework of the event engine we using.

Second, this PR fixes few overlooks and enable (partially) ignore typespecs. With typespecs I decide to go as low as I can to avoid PR bloating and prepare a ticket to clean the rest.

How to test the change?

CI + ST

@github-actions github-actions Bot added integrations Involves tracing integrations appsec Application Security monitoring product labels Jan 7, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 7, 2026

Thank you for updating Change log entry section 👏

Visited at: 2026-01-15 13:54:00 UTC

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Jan 12, 2026

Typing analysis

Ignored files

This PR clears 2 ignored files. It increases the percentage of typed files from 39.22% to 39.44% (+0.22%).

Ignored files (+0-2)Cleared:
lib/datadog/appsec.rb
lib/datadog/appsec/contrib/excon/ssrf_detection_middleware.rb

Note: Ignored files are excluded from the next sections.

Untyped methods

This PR introduces 1 untyped method, and clears 1 untyped method. It increases the percentage of typed methods from 57.9% to 58.21% (+0.31%).

Untyped methods (+1-1)Introduced:
sig/datadog/appsec/metrics/collector.rbs:44
└── def initialize: () -> void
Cleared:
sig/datadog/appsec/metrics/collector.rbs:33
└── def initialize: () -> void

If you believe a method or an attribute is rightfully untyped or partially typed, you can add # untyped:accept to the end of the line to remove it from the stats.

@pr-commenter
Copy link
Copy Markdown

pr-commenter Bot commented Jan 12, 2026

Benchmarks

Benchmark execution time: 2026-01-16 10:40:12

Comparing candidate commit 292314b in PR branch appsec-60097-add-new-data-collection-to-downstream-requests with baseline commit eea0f39 in branch master.

Found 1 performance improvements and 3 performance regressions! Performance is the same for 40 metrics, 2 unstable metrics.

scenario:line instrumentation - targeted

  • 🟥 throughput [-14251.162op/s; -13762.506op/s] or [-9.048%; -8.738%]

scenario:line instrumentation - untargeted

  • 🟥 throughput [-5810.617op/s; -5703.530op/s] or [-9.761%; -9.581%]

scenario:method instrumentation

  • 🟥 throughput [-12837.310op/s; -12302.970op/s] or [-7.242%; -6.941%]

scenario:profiling - Allocations ()

  • 🟩 throughput [+416460.918op/s; +424845.961op/s] or [+14.312%; +14.600%]

@datadog-datadog-prod-us1
Copy link
Copy Markdown
Contributor

datadog-datadog-prod-us1 Bot commented Jan 14, 2026

✅ Tests

🎉 All green!

❄️ No new flaky tests detected
🧪 All tests passed

🎯 Code Coverage
Patch Coverage: 87.38%
Overall Coverage: 95.19% (-0.03%)

View detailed report

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 292314b | Docs | Datadog PR Page | Was this helpful? Give us feedback!

@Strech Strech force-pushed the appsec-60097-add-new-data-collection-to-downstream-requests branch from 200ebf9 to 10e1a50 Compare January 15, 2026 09:55
@Strech
Copy link
Copy Markdown
Member Author

Strech commented Jan 15, 2026

@codex find critical bugs and corner cases in this PR's code

Copy link
Copy Markdown

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 10e1a50b70

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread lib/datadog/appsec/contrib/faraday/ssrf_detection_middleware.rb Outdated
@Strech Strech changed the title [APPSEC-60097] Add new data collection to downstream requests [APPSEC-60097] Add more data collection for downstream requests Jan 15, 2026
@Strech Strech force-pushed the appsec-60097-add-new-data-collection-to-downstream-requests branch from d4ebd84 to 3a663fd Compare January 15, 2026 13:59
@Strech Strech marked this pull request as ready for review January 15, 2026 14:06
@Strech Strech requested review from a team as code owners January 15, 2026 14:06
Comment thread lib/datadog/appsec/contrib/excon/ssrf_detection_middleware.rb
Comment thread lib/datadog/appsec/contrib/excon/ssrf_detection_middleware.rb
Comment thread lib/datadog/appsec/contrib/rest_client/request_ssrf_detection_patch.rb Outdated
Comment thread lib/datadog/appsec/contrib/rest_client/request_ssrf_detection_patch.rb Outdated
@y9v
Copy link
Copy Markdown
Member

y9v commented Jan 15, 2026

nice work! Thanks for fixing the url for excon instrumentation!

* Add additional notes to clarify internals of the library used.
@Strech Strech merged commit acfeeff into master Jan 16, 2026
637 checks passed
@Strech Strech deleted the appsec-60097-add-new-data-collection-to-downstream-requests branch January 16, 2026 13:36
@github-actions github-actions Bot added this to the 2.27.0 milestone Jan 16, 2026
@lloeki lloeki mentioned this pull request Jan 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

appsec Application Security monitoring product integrations Involves tracing integrations

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants