Propagate docker_run env vars to discovery-stability check's compose command by vitkyrka · Pull Request #24533 · DataDog/integrations-core

vitkyrka · 2026-07-13T14:32:18Z

What does this PR do?

Makes the env vars used to interpolate a docker_run compose file (whether passed via env_vars=, set by shared_logs(), or set by any other wrapper) available to the docker compose ps/logs commands that assert_all_discovery_candidates_stable shells out to, even when that call happens in a separate ddev env test process from the one that started the environment. Also removes the per-integration docker-compose.yaml fallback defaults (temporal, pulsar, ray) that were added last week as workarounds for this same issue.

Motivation

https://datadoghq.atlassian.net/browse/DSCVR-614

During ddev env test, the dd_environment fixture that calls docker_run is replaced with a no-op, since the environment was already started by a separate ddev env start process. Any env vars used to interpolate the compose file were therefore missing from os.environ in that later process. When assert_all_discovery_candidates_stable shelled out to docker compose ps/logs to inspect the running container, compose would reject the file with errors like invalid spec: empty section between colons for any referenced variable with no value and no fallback default.

Rather than have every integration that adopts assert_all_discovery_candidates_stable add :-<default> fallbacks to its compose file, docker_run now snapshots the env vars introduced by any of its wrappers and persists them alongside the compose metadata it already saves across process boundaries (compose file, service name, project name). _get_compose_container_id then passes those env vars through explicitly to the subprocess it invokes.

Review checklist (to be filled by reviewers)

Feature or bugfix MUST have appropriate tests (unit, integration, e2e)
Add qa/required if this PR needs QA validation, or qa/skip-qa if it does not. Exactly one of the two is required.
If you need to backport this PR to another branch, you can add the backport/<branch-name> label to the PR and it will automatically open a backport PR once this one is merged

datadog-datadog-prod-us1-2 · 2026-07-13T14:40:17Z

Tests

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 92.31%
• Overall Coverage: 88.37% (+0.10%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: f4762a7 | Docs | Datadog PR Page | Give us feedback!}

vitkyrka · 2026-07-13T15:08:14Z

Propagate docker_run env vars to discovery-stability check's compose command #24533 : 3 dependent PRs (#24479 , #24487 , #24494 ) 👈 (View in Graphite)
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 0fd521c3de

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

During `ddev env test`, the `dd_environment` fixture that calls `docker_run` never runs (it's replaced with a no-op since the environment was already started by a separate `ddev env start` process). Any env vars docker_run's caller passed for compose-file interpolation were therefore missing when assert_all_discovery_candidates_stable shelled out to `docker compose ps`, causing compose to reject the file if a referenced variable had no fallback. docker_run now persists its env_vars alongside the other compose metadata it already saves across process boundaries, and _get_compose_container_id passes them through to the `docker compose ps` subprocess.

…y_candidates_stable temporal, pulsar, and ray each added a `:-<default>` fallback to a compose-file variable so `docker compose ps`/`logs` wouldn't fail during e2e discovery-stability tests. That was a workaround for docker_run not propagating its env vars across the separate `ddev env start`/`ddev env test` processes, now fixed centrally in docker.py.

…ars param CI failed on temporal, pulsar, and ray with the same compose-parsing error the previous commit fixed for the caller-supplied `env_vars` param: `docker compose ps` still rejected the file over TEMPORAL_LOG_FOLDER, DD_LOG_1, and RAY_LOG_FOLDER. Those integrations set those vars via `shared_logs()`/a custom wrapper passed through `wrappers=`, not through docker_run's own `env_vars=` param, so the previous fix's plain `env_vars or {}` never saw them. Instead of tracking one specific parameter, snapshot os.environ before any wrapper runs and diff against it once every wrapper (docker_run's own env_vars, mount_logs's shared_logs, and any caller-supplied wrappers) has been entered. That captures whatever env vars any of them introduced, regardless of which mechanism they used to set them.

dd-octo-sts · 2026-07-14T14:30:41Z

Validation Report

All 21 validations passed.

Show details

Validation	Description	Status
`agent-reqs`	Verify check versions match the Agent requirements file	✅
`ci`	Validate CI configuration and code coverage settings	✅
`codeowners`	Validate every integration has a CODEOWNERS entry	✅
`config`	Validate default configuration files against spec.yaml	✅
`dep`	Verify dependency pins are consistent and Agent-compatible	✅
`http`	Validate integrations use the HTTP wrapper correctly	✅
`imports`	Validate check imports do not use deprecated modules	✅
`integration-style`	Validate check code style conventions	✅
`jmx-metrics`	Validate JMX metrics definition files and config	✅
`labeler`	Validate PR labeler config matches integration directories	✅
`legacy-signature`	Validate no integration uses the legacy Agent check signature	✅
`license-headers`	Validate Python files have proper license headers	✅
`licenses`	Validate third-party license attribution list	✅
`metadata`	Validate metadata.csv metric definitions	✅
`models`	Validate configuration data models match spec.yaml	✅
`openmetrics`	Validate OpenMetrics integrations disable the metric limit	✅
`package`	Validate Python package metadata and naming	✅
`qa-label`	Validate the pull request declares whether it needs QA for the next Agent release	✅
`readmes`	Validate README files have required sections	✅
`saved-views`	Validate saved view JSON file structure and fields	✅
`version`	Validate version consistency between package and changelog	✅

View full run

temporal-github-worker-1 Bot added agent/review-requested ecosystems/review-requested product/review-requested labels Jul 13, 2026

vitkyrka added the qa/skip-qa Automatically skip this PR for the next QA label Jul 13, 2026 — with ddtool CLI

dd-octo-sts Bot added integration/pulsar dev_package integration/temporal integration/ray labels Jul 13, 2026

vitkyrka added a commit that referenced this pull request Jul 13, 2026

Add changelog entry for #24533

0622785

vitkyrka mentioned this pull request Jul 13, 2026

fluentd: add container-based config discovery support #24479

Draft

3 tasks

vitkyrka changed the title ~~Propagate docker_run env vars across e2e process boundaries~~ Propagate docker_run env vars to the compose commands used by discovery-stability checks Jul 13, 2026

vitkyrka changed the title ~~Propagate docker_run env vars to the compose commands used by discovery-stability checks~~ Propagate docker_run env vars to discovery-stability check's compose command Jul 13, 2026

vitkyrka mentioned this pull request Jul 13, 2026

boundary: add container-based config discovery support #24494

Draft

3 tasks

vitkyrka marked this pull request as ready for review July 14, 2026 07:02

vitkyrka requested a review from a team as a code owner July 14, 2026 07:02

dd-octo-sts Bot added the team/agent-integrations label Jul 14, 2026

chatgpt-codex-connector Bot reviewed Jul 14, 2026

View reviewed changes

Comment thread datadog_checks_dev/datadog_checks/dev/docker.py Outdated

vitkyrka added a commit that referenced this pull request Jul 14, 2026

Add changelog entry for #24533

af7a573

vitkyrka force-pushed the vwhitchurch/docker-run-env-vars-e2e branch from f3908fc to 3251f62 Compare July 14, 2026 09:55

vitkyrka mentioned this pull request Jul 14, 2026

gearmand: add container-based config discovery support #24487

Draft

3 tasks

vitkyrka added 5 commits July 14, 2026 14:28

Add changelog entry for #24533

debb542

fixup

f4762a7

vitkyrka force-pushed the vwhitchurch/docker-run-env-vars-e2e branch from 3251f62 to f4762a7 Compare July 14, 2026 14:28

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Propagate docker_run env vars to discovery-stability check's compose command#24533

Propagate docker_run env vars to discovery-stability check's compose command#24533
vitkyrka wants to merge 5 commits into
masterfrom
vwhitchurch/docker-run-env-vars-e2e

vitkyrka commented Jul 13, 2026 •

edited

Loading

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Jul 13, 2026 •

edited

Loading

Uh oh!

vitkyrka commented Jul 13, 2026 •

edited

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

Uh oh!

dd-octo-sts Bot commented Jul 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

vitkyrka commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What does this PR do?

Motivation

Review checklist (to be filled by reviewers)

Uh oh!

datadog-datadog-prod-us1-2 Bot commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

vitkyrka commented Jul 13, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

dd-octo-sts Bot commented Jul 14, 2026

Validation Report

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

vitkyrka commented Jul 13, 2026 •

edited

Loading

datadog-datadog-prod-us1-2 Bot commented Jul 13, 2026 •

edited

Loading

vitkyrka commented Jul 13, 2026 •

edited

Loading