Skip to content

chore(deps): bump libdatadog to db05e1f and adapt to HttpClientTrait API#1218

Merged
duncanista merged 7 commits intomainfrom
jordan.gonzalez/dependencies/libdatadog-32.0.0
May 5, 2026
Merged

chore(deps): bump libdatadog to db05e1f and adapt to HttpClientTrait API#1218
duncanista merged 7 commits intomainfrom
jordan.gonzalez/dependencies/libdatadog-32.0.0

Conversation

@duncanista
Copy link
Copy Markdown
Contributor

@duncanista duncanista commented May 4, 2026

Summary

Bumps the libdd-* git revs in bottlecap from c8121f42 (~v31.x) to db05e1f8408a76075efb37ecec544d2e74217e57 (current libdatadog main), bumps the dogstatsd / datadog-fips revs to 5b68f50f49c9defbfed4d25bd621e2a86405a972 (current serverless-components main, which already sits on the same libdatadog db05e1f), and adapts bottlecap to the breaking changes that ship between those revs.

What changed and why

Upstream libdatadog changes that motivated this PR

  • feat(capabilities)!: trait architecture http libdatadog#1555 (feat: capability traits architecture for HTTP) — replaced the raw hyper client API with an HttpClientTrait abstraction. SendData::send and send_with_retry now require H: HttpClientTrait, and stats_utils::send_stats_payload_with_client was removed in favor of a generic send_stats_payload<H: HttpClientTrait>(…) that constructs its own client.
  • ObfuscationConfig restructured — flat fields (http_remove_path_digits, obfuscate_memcached, …) replaced with nested per-engine structs (http: HttpConfig, memcached: MemcachedConfig, redis: RedisConfig, plus valkey, credit_cards, sql, elasticsearch, opensearch, mongodb).
  • fix(crypto): use ring for non-fips builds libdatadog#1816 + #1872 + #1943 (crypto provider gating) — moved ring behind libdd-common/https and aws-lc-rs behind libdd-common/fips, then progressively gated the internal crates (libdd-trace-utils, libdd-trace-obfuscation, libdd-capabilities-impl, libdd-trace-stats, libdd-data-pipeline, libdd-dogstatsd-client, libdd-telemetry) so downstream consumers can pick exactly one provider. #1943 also added a workspace-wide CI guard in libdatadog that rejects any PR which puts both ring and aws-lc-rs in the dep graph at the same time.

Other commits in the range (c8121f42..db05e1f) are sidecar / FFE / tracer-flare / telemetry changes that don't touch surfaces bottlecap consumes.

Code changes in this PR

bottlecap/src/traces/http_client.rs — wrap the client to implement HttpClientTrait

The bottlecap HTTP client must keep proxy + custom CA + skip-SSL support (FIPS, DD_PROXY_HTTPS, DD_TLS_CERT_FILE, DD_SKIP_SSL_VALIDATION). The upstream DefaultHttpClient from libdd-capabilities-impl hardcodes Connector::default() and supports none of those — using it would be a regression.

So HttpClient is now a newtype around GenericHttpClient<ProxyConnector<Connector>> that implements libdd_capabilities::HttpClientTrait. The trait's request() maps http::Request<Bytes>Body::from_bytes(…) → hyper request, then collects the response body back to Bytes. ~30 lines, reuses libdatadog's encoding/retry/header logic.

HttpClientTrait::new_client() is required by the trait but doesn't fit our model (we need a configured client, not a default). It now routes through create_client(None, None, false) so the failure surface is consistent with the rest of the module — and it's never invoked on production paths (we always go through create_client(proxy, tls_cert, skip_ssl)).

bottlecap/src/traces/stats_flusher.rs — inline the stats POST

The new send_stats_payload<H: HttpClientTrait>(data, target, api_key) calls H::new_client() internally — meaning callers can't supply a pre-configured client anymore. That would lose Lambda's pool_max_idle_per_host(0) tuning, which exists specifically to avoid stale connections after Lambda freeze/resume cycles.

Replaced the removed call with a tiny in-module send_stats_payload helper that builds the same POST request (msgpack + gzip + DD-API-KEY) and invokes our HttpClient's request() method directly. Per Copilot review: each attempt is wrapped in tokio::time::timeout(target.timeout_ms, …) so the retry loop stays bounded by config, and the error-body capture is bounded to 512 bytes (lossy UTF-8) with the HTTP status surfaced in the message instead of silently emptying on non-UTF8 responses.

bottlecap/src/bin/bottlecap/main.rs — flatten → nested ObfuscationConfig

Maps the two HTTP fields we configure (apm_config_obfuscation_http_remove_paths_with_digits, apm_config_obfuscation_http_remove_query_string) into the new HttpConfig. Everything else flows through ..Default::default(), which preserves the previous behavior (memcached/redis disabled).

bottlecap/Cargo.toml + bottlecap/Cargo.lock

  • Bumps all libdd-* revs to db05e1f and dogstatsd / datadog-fips revs to 5b68f50 (serverless-components main).
  • Adds libdd-capabilities (source of HttpClientTrait) and http (now used directly in stats_flusher) as direct dependencies.
  • Sets default-features = false on all libdd-* deps and forwards libdd-*/https from bottlecap's default feature and libdd-*/fips from bottlecap's fips feature — the consumer-side pattern that #1872's description prescribed.

bottlecap/LICENSE-3rdparty.csv

Regenerated via dd-rust-license-tool write to include the new libdd-capabilities / libdd-capabilities-impl / libdd-shared-runtime / http entries.

FIPS

Previously a known issue — the FIPS clippy job failed because libdd-trace-stats (and libdd-data-pipeline) pulled libdd-capabilities-impl with default features = https = ring, with no downstream-side workaround possible. DataDog/libdatadog#1943 fixed this upstream, and this PR now forwards libdd-trace-stats/fips from the fips feature.

Verified locally:

$ cargo clippy --workspace --all-targets --no-default-features --features fips
warning: bottlecap@0.1.0: FIPS feature is enabled, checking for forbidden dependencies...
warning: bottlecap@0.1.0: No ring dependency found. FIPS compliance check passed
warning: bottlecap@0.1.0: No openssl dependency found. FIPS compliance check passed
warning: bottlecap@0.1.0: No boringssl dependency found. FIPS compliance check passed
warning: bottlecap@0.1.0: All dependency checks passed.

Companion PRs (already merged)

Test plan

  • cargo check --all-targets (default features)
  • cargo clippy --workspace --all-targets --features default
  • cargo clippy --workspace --all-targets --no-default-features --features fips — FIPS dependency check now passes locally (was the long-standing CI blocker)
  • cargo test --no-run
  • cargo fmt --all -- --check
  • Production layer build via ARCHITECTURE=arm64 FIPS=false ./scripts/build_bottlecap_layer.sh — built successfully, binary 11.17 MiB stripped / layer zip 5.08 MiB (slightly smaller than origin/main's 11.23 MiB / 5.11 MiB)
  • End-to-end smoke test in a real Lambda environment

Copilot AI review requested due to automatic review settings May 4, 2026 18:49
@duncanista duncanista requested review from a team as code owners May 4, 2026 18:49
@duncanista duncanista requested a review from lym953 May 4, 2026 18:49
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR bumps libdatadog crates to v32.0.0 and updates bottlecap to accommodate upstream breaking changes, primarily the new capability-based HTTP client trait (HttpClientTrait) and the restructured trace obfuscation configuration.

Changes:

  • Update libdd-* git rev pins to v32.0.0 and add new direct deps (libdd-capabilities, http).
  • Introduce a wrapped, pre-configured Hyper client that implements HttpClientTrait to preserve proxy/TLS/skip-SSL behavior.
  • Replace removed upstream stats-sending helper with an in-module POST helper and migrate obfuscation config to nested structs.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
bottlecap/Cargo.toml Bumps libdd-* revs and adds new direct dependencies needed by the v32 API.
bottlecap/Cargo.lock Updates the resolved dependency graph to match the v32 bump and newly introduced crates.
bottlecap/src/traces/http_client.rs Wraps the existing configured Hyper client and implements HttpClientTrait for libdd v32 compatibility.
bottlecap/src/traces/stats_flusher.rs Replaces removed upstream stats sender with a local helper that uses the shared configured client.
bottlecap/src/bin/bottlecap/main.rs Updates construction of ObfuscationConfig to the new nested configuration shape.
bottlecap/LICENSE-3rdparty.csv Regenerates third-party license inventory to reflect updated dependencies.

Comment thread bottlecap/src/traces/stats_flusher.rs Outdated
Comment thread bottlecap/src/traces/stats_flusher.rs Outdated
Comment thread bottlecap/src/traces/http_client.rs Outdated
duncanista added 2 commits May 4, 2026 14:59
libdatadog #1555 replaced raw hyper-client APIs with a HttpClientTrait
abstraction. SendData::send and send_with_retry now require
H: HttpClientTrait, and stats_utils::send_stats_payload_with_client
was removed in favor of a generic that constructs its own client
internally.

- Wrap the proxy/TLS-aware hyper client in a struct that implements
  libdd_capabilities::HttpClientTrait so it can be passed to libdd
  trace senders without losing custom proxy / CA / skip-SSL config.
- Inline the stats POST in stats_flusher to keep using the configured
  HttpClient (preserves Lambda's pool_max_idle_per_host(0) tuning that
  avoids stale connections after freeze/resume).
- Update obfuscation_config::ObfuscationConfig construction for the
  new nested HttpConfig/MemcachedConfig/RedisConfig layout.
@duncanista duncanista force-pushed the jordan.gonzalez/dependencies/libdatadog-32.0.0 branch from 0b25a69 to 8a94026 Compare May 4, 2026 19:00
@duncanista duncanista changed the title chore(deps): bump libdatadog to v32.0.0 chore(deps): bump libdatadog to 0a70516 May 4, 2026
@duncanista duncanista changed the title chore(deps): bump libdatadog to 0a70516 chore(deps): bump libdatadog to 0a70516 and adapt to HttpClientTrait API May 4, 2026
duncanista added 2 commits May 4, 2026 15:07
- Enforce target.timeout_ms on each stats POST attempt so the retry
  loop stays bounded by configuration, instead of blocking on a
  hung connection past the configured flush timeout.
- Bound the error-body preview captured from non-202 responses to 512
  bytes, fall back to lossy UTF-8 instead of silently emptying on
  invalid encoding, and include the HTTP status in the error message.
- Route HttpClientTrait::new_client through create_client(None,
  None, false) so the never-invoked fallback shares the same
  construction path and failure surface as the rest of the module.
Apply the consumer-side pattern from DataDog/libdatadog#1872:
  libdd-* deps with default-features = false, with explicit
  https/fips forwarding from bottlecap's own default/fips features.

This is the configuration the PR description recommends for downstream
consumers. It does not on its own fix the FIPS clippy job, because
libdd-trace-stats still pulls libdd-capabilities-impl with default
features inside its own Cargo.toml — that path needs an upstream
follow-up to #1872 to fully gate.
@duncanista
Copy link
Copy Markdown
Contributor Author

Blocked by DataDog/libdatadog#1943

…/fips

db05e1f is the merge of DataDog/libdatadog#1943 which gates the
remaining internal crates that #1872 missed. With those gates in
place, bottlecap's `fips` feature can now forward to
`libdd-trace-stats/fips` which closes the last path that pulled
ring through libdd-capabilities-impl.

Verified locally: `cargo clippy --no-default-features --features fips`
now reports `No ring dependency found. FIPS compliance check passed`.
@duncanista duncanista changed the title chore(deps): bump libdatadog to 0a70516 and adapt to HttpClientTrait API chore(deps): bump libdatadog to db05e1f and adapt to HttpClientTrait API May 5, 2026
@duncanista duncanista merged commit 3d4c862 into main May 5, 2026
55 of 59 checks passed
@duncanista duncanista deleted the jordan.gonzalez/dependencies/libdatadog-32.0.0 branch May 5, 2026 21:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants