Migrate dynamic config smoke tests to JUnit by PerfectSlayer · Pull Request #12148 · DataDog/dd-trace-java

PerfectSlayer · 2026-08-05T15:57:02Z

What Does This Do

This PR migrates the Spring Boot RabbitMQ smoke tests to the new JUnit smoke test framework.

Motivation

The new tests checks the RC messages (payload and behavior) against the CI test agent, and no longer our own implementation, to avoid validating our implementation by ourselves.

Additional Notes

The CI test agent runs as side car in CI. It will be run using testcontainers on local environment.

Contributor Checklist

Format the title according to the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any other useful labels
Avoid using close, fix, or any linking keywords when referencing an issue
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, migration, or deletion
Update public documentation with any new configuration flags or behaviors
Once approved, use merge queue to merge the PR

Jira ticket: APMLP-1247

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eaee938397

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector · 2026-08-05T16:11:56Z

+
+  @Test
+  void activatesAppSecViaRemoteConfig() {
+    assumeFalse(isOracleJDK8(), "Telemetry product-change event flakes on Oracle JDK 8");


Preserve the flaky tag for the Oracle JDK 8 skip

When this suite runs on Oracle JDK 8, replacing the old @Flaky marker with an in-method assumption leaves the migrated flaky case invisible to Gradle's JUnit tag filtering (-PskipFlakyTests/-PrunFlakyTests); those filters select by the flaky tag at discovery, while this assumption is reached only after the static SmokeCliApp extension has already started the backend and app. Keep this skip in the flaky-test tagging path rather than only aborting inside the test body.

AGENTS.md reference: AGENTS.md:L65-L65

Useful? React with 👍 / 👎.

This will be part of some other changes. Outside the scope of this PR. Conditional Flaky annotations are not supported in JUnit.

chatgpt-codex-connector · 2026-08-05T16:11:56Z

+        "datadog/2/ASM_FEATURES/asm_features_activation/config", "{\"asm\":{\"enabled\":true}}");
+
+    // The tracer reports the applied change via a telemetry configuration event.
+    agent.telemetry().waitForFlat(AppSecActivationSmokeTest::appsecEnabledFromRemoteConfig);


Restore the longer AppSec telemetry wait

On slower smoke-test jobs where the AppSec config-change telemetry arrives more than 10 seconds after the RC payload is pushed but still within the app's 15-second lifetime, this assertion can now time out early because Telemetry.waitForFlat(...) uses a 10-second default; the migrated Spock helper used the suite's 30-second polling timeout. Use the overload with the previous timeout, or otherwise align the wait with the app lifetime, to avoid introducing CI-only false negatives.

Useful? React with 👍 / 👎.

I'll use the app timeout instead and set the default timeout to 30s as groovy in a PR down the stack

dd-octo-sts · 2026-08-05T16:24:44Z

🟢 Java Benchmark SLOs — All performance SLOs passed

Suite	Status
Startup	🟢 pass

SLO thresholds are defined here based on automatically generated metrics. A warning is raised when results are within 5% of the threshold.

PR vs. master results

Scenario	Candidate	master	Δ (95% CI of mean)
startup:insecure-bank:iast:Agent	14.01 s	13.98 s	[-0.5%; +0.9%] (no difference)
startup:insecure-bank:tracing:Agent	12.95 s	12.95 s	[-0.7%; +0.8%] (no difference)
startup:petclinic:appsec:Agent	17.01 s	16.57 s	[+1.8%; +3.6%] (significantly worse)
startup:petclinic:iast:Agent	16.98 s	17.07 s	[-1.3%; +0.3%] (no difference)
startup:petclinic:profiling:Agent	16.99 s	16.92 s	[-0.7%; +1.5%] (no difference)
startup:petclinic:sca:Agent	16.76 s	16.80 s	[-1.2%; +0.7%] (no difference)
startup:petclinic:tracing:Agent	16.13 s	15.69 s	[-1.6%; +7.3%] (no difference)

Commit: f71af087 · CI Pipeline · Benchmarking Platform UI

Load and DaCapo benchmarks can be triggered manually in the GitLab pipeline. Results will appear in the Benchmarking Platform UI after completion.

datadog-official

More details

The JUnit migration preserves the prior RC assertions and compiles cleanly; shared smoke/backend tests also pass. The two changed smoke tests could not execute their child JVMs here because Docker is unavailable for TestAgentBackend, so end-to-end app behavior remains to be confirmed in CI.

Was this helpful? React 👍 or 👎

_{Open Bits AI session}

_{🤖 Datadog Autotest · Commit eaee938 · What is Autotest? · @DataDog review to ask questions · Any feedback? Reach out in #autotest}

datadog-official · 2026-08-06T11:29:02Z

Bits has a CI fix ready

🟢 Investigated · 🟢 Fix prepared · ⚪ Validation skipped · 🟠 Ready

Reformatted the newly added AppSecActivationSmokeTest.java invocation of Telemetry.waitForFlat to match the repository’s Spotless formatting rules. The separate create-multiarch-lib-injection-image jq null-iteration error is unrelated to the smoke-test migration.

Commit fix to this PR

View in Datadog | Reviewed commit 76437d7 · Any feedback? Reach out in #deveng-pr-agent

PerfectSlayer requested a review from a team as a code owner August 5, 2026 15:57

PerfectSlayer requested a review from ygree August 5, 2026 15:57

PerfectSlayer added type: feature Enhancements and improvements tag: no release notes Changes to exclude from release notes comp: remote config Configuration at Runtime labels Aug 5, 2026

PerfectSlayer mentioned this pull request Aug 5, 2026

Improving smoke test capabilities #12051

Draft

chatgpt-codex-connector Bot reviewed Aug 5, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

datadog-official Bot reviewed Aug 5, 2026

View reviewed changes

PerfectSlayer force-pushed the bbujon/smoke-tests-dynamic-config branch from eaee938 to b3e4942 Compare August 6, 2026 11:24

PerfectSlayer force-pushed the bbujon/smoke-tests-spring-boot-rabbit branch from 353ea51 to 3b1d4ed Compare August 6, 2026 11:24

PerfectSlayer force-pushed the bbujon/smoke-tests-dynamic-config branch from b3e4942 to 76437d7 Compare August 6, 2026 11:47

PerfectSlayer force-pushed the bbujon/smoke-tests-spring-boot-rabbit branch from 3b1d4ed to bb468da Compare August 6, 2026 11:47

feat(dynamic-config): Migrate smoke test to JUnit

f71af08

PerfectSlayer force-pushed the bbujon/smoke-tests-dynamic-config branch from 76437d7 to f71af08 Compare August 6, 2026 12:36

Navigation Menu

Uh oh!

Migrate dynamic config smoke tests to JUnit - #12148

Migrate dynamic config smoke tests to JUnit#12148
PerfectSlayer wants to merge 1 commit into
bbujon/smoke-tests-spring-boot-rabbitfrom
bbujon/smoke-tests-dynamic-config

PerfectSlayer commented Aug 5, 2026 •

edited by atlassian Bot

Loading

Uh oh!

chatgpt-codex-connector Bot left a comment

Uh oh!

chatgpt-codex-connector Bot Aug 5, 2026

Uh oh!

PerfectSlayer Aug 6, 2026

Uh oh!

Uh oh!

chatgpt-codex-connector Bot Aug 5, 2026

Uh oh!

PerfectSlayer Aug 6, 2026

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Aug 5, 2026 •

edited

Loading

Uh oh!

datadog-official Bot left a comment

Uh oh!

datadog-official Bot commented Aug 6, 2026 •

edited by datadog-prod-us1-4 Bot

Loading

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Uh oh!

Conversation

PerfectSlayer commented Aug 5, 2026 • edited by atlassian Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Additional Notes

Contributor Checklist

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

chatgpt-codex-connector Bot Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

chatgpt-codex-connector Bot Aug 5, 2026

Choose a reason for hiding this comment

Uh oh!

PerfectSlayer Aug 6, 2026

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

dd-octo-sts Bot commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🟢 Java Benchmark SLOs — All performance SLOs passed

Uh oh!

datadog-official Bot left a comment

Choose a reason for hiding this comment

Uh oh!

datadog-official Bot commented Aug 6, 2026 • edited by datadog-prod-us1-4 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Bits has a CI fix ready

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

PerfectSlayer commented Aug 5, 2026 •

edited by atlassian Bot

Loading

dd-octo-sts Bot commented Aug 5, 2026 •

edited

Loading

datadog-official Bot commented Aug 6, 2026 •

edited by datadog-prod-us1-4 Bot

Loading