docs(changelog): Flaky Tests AI investigations delivered via webhooks

[samgutentag]

What shipped

AI-powered flaky test investigations now run automatically when a test first becomes flaky. Trunk kicks off a background AI investigation that gathers context from CI runs, Git history, and related code, then emits a structured payload to the configured webhook endpoint.

Source

• Eng PR: trunk-io/trunk2#3475 — "Part 8/9: Trigger AI FT investigations and emit results to Svix"
• Linear: TRUNK-17938 (no absorbed duplicates)
• Date basis: source eng PR mergedAt = 2026-04-06

Wired into all four sites

1. changelog/2026-04-06-flaky-tests-ai-investigations-webhooks.mdx (new entry)
2. docs.json nav (Changelog 2026 group)
3. changelog/index.mdx (new April 6, 2026 Update block)
4. flaky-tests/changelog.mdx (new April 2026 section + Update block)

Docs link: https://docs.trunk.io/flaky-tests/webhooks

🤖 Generated with Claude Code

[mintlify]

Preview deployment for your docs. Learn more about Mintlify Previews.

Project	Status	Preview	Updated (UTC)
trunk	🟢 Ready	View Preview	May 29, 2026, 5:34 AM

💡 Tip: Enable Workflows to automatically generate PRs for you.

[samgutentag]

Verification status (2026-05-28): live

Verified: customers can use this. Ready to publish.

• Eng PR: trunk-io/trunk2#3475 (stacked, combining child PRs #3371 and #3372), merged 2026-04-06. Merge commit is intact on main.
• Flags: emitFlakyTestsInvestigationEvents ("Flaky Tests Emit Investigate Event") and emitInvestigationResultWebhooks ("Flaky Tests Emit Investigation Result Event"), plus flakyTestInvestigationMonthlyTestCap. Backend-services LaunchDarkly project.
• Signals:
  • LaunchDarkly bot: both event flags "updated in Production" on 2026-04-23 and again 2026-04-27, after the eng PR merge date.
  • #team-flaky-tests thread (2026-04-27): feature actively running against real customer orgs in prod ("30 successful investigations", rate-limit tuning, re-enable and redrive). This is production behavior, not staging-only.

Next action: ready to publish. Mark the PR ready for review and merge.

[samgutentag]

Code verification (2026-05-28): 4 confirmed / 0 contradicted / 0 ambiguous / 0 unverifiable

The entry makes only behavioral claims (no field names, limits, or paths to check). All four resolve against trunk-io/trunk2.

Claim	Verdict	Source
Investigation triggers when a test first becomes flaky	confirmed	detection-engine-webhook-event-handler/src/investigation-event.ts:16-17
Runs in the background	confirmed	investigation-event.ts:43
Gathers context from CI runs, Git history, and related code	confirmed	ai/generation/flaky-tests-investigation/
Emits a structured payload to your webhook	confirmed	svix-publish-schemas/src/event-types.ts

No contradictions. Entry is safe to publish as written.

---

Source #1 — Triggers when a test first becomes flaky (confirmed)

File: trunk-io/trunk2/ts/apps/detection-engine-webhook-event-handler/src/investigation-event.ts#L16-L17

  event.previous_status === "HEALTHY" &&
  (event.new_status === "FLAKY" || event.new_status === "BROKEN");

Reasoning: The enqueue guard only fires on a HEALTHY to FLAKY (or BROKEN) transition, i.e. the first time a healthy test becomes flaky. This matches "when a test first becomes flaky." Note the trigger also covers the HEALTHY to BROKEN case, which the entry does not call out, but the flaky case is accurate.

Source #2 — Runs in the background (confirmed)

File: investigation-event.ts#L43

  const queueUrl = process.env["FLAKY_TESTS_INVESTIGATION_QUEUE_URL"];

Reasoning: The detection handler does not run the investigation inline. It enqueues an SQS FIFO message onto the investigation queue, which the separate flaky-tests-investigation-poller app consumes asynchronously. That is the "kicks off in the background" behavior. PR #3371 also notes the handler isolates errors so a failed enqueue does not break detection.

Source #3 — Gathers context from CI runs, Git history, and related code (confirmed)

File: ts/packages/ai/generation/flaky-tests-investigation/

ci-logs-analyzer/        # CI runs
git-blame/               # Git history
file-co-change/          # Git history (co-changed files)
co-failure/              # related code / tests
failure-mode-insight-extractor/
test-purpose-summarizer/

Reasoning: The investigation package's subdirectories map directly to the three named context sources: ci-logs-analyzer (CI runs), git-blame + file-co-change (Git history), and co-failure + test-purpose-summarizer (related code). Confirmed at the package-structure level rather than a single line, which is appropriate for a multi-tool agent.

Source #4 — Emits a structured payload to your webhook (confirmed)

File: ts/packages/tools/svix-publish-schemas/src/event-types.ts

  "test_case.investigation_completed":

Reasoning: PR #3372 emits a v2.test_case.investigation_completed Svix webhook on successful investigation completion, with a schema-defined payload (full repository and test_case metadata, facts with citations). The event type is registered in the Svix publish schemas. Emission is skipped if the org has no webhookAppId, so delivery is conditional on the customer having webhooks configured, consistent with "delivered to your configured webhook endpoint."