Skip to content

docs(flaky-tests): configurable minimum POR count for pass-on-retry monitor#176

Draft
samgutentag wants to merge 1 commit into
mainfrom
sam-gutentag/por-count-monitor
Draft

docs(flaky-tests): configurable minimum POR count for pass-on-retry monitor#176
samgutentag wants to merge 1 commit into
mainfrom
sam-gutentag/por-count-monitor

Conversation

@samgutentag
Copy link
Copy Markdown
Member

Summary

Documents the new minPorCount setting for the pass-on-retry monitor, which lets teams require multiple commits to show pass-on-retry behavior before a test is flagged as flaky.

Changes:

  • Added Minimum POR count row to the configuration table in flaky-tests/detection/pass-on-retry-monitor.mdx
  • Added ### Minimum POR Count subsection with guidance on when to raise the setting
  • Added draft changelog entry in flaky-tests/changelog.mdx for June 2, 2026

Source

Test plan

  • Check Mintlify preview link (posted on the PR by the Mintlify GitHub app)

Generated by Claude Code

…n-retry monitor

Adds the Minimum POR count setting to the configuration table and adds
a dedicated subsection explaining when to raise it. Also adds a draft
changelog entry.

Source: trunk-io/trunk2#4045

Co-Authored-By: Claude <noreply@anthropic.com>
@mintlify
Copy link
Copy Markdown
Contributor

mintlify Bot commented Jun 2, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
trunk 🟢 Ready View Preview Jun 2, 2026, 12:10 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

Copy link
Copy Markdown
Member Author

Verification status (June 2, 2026): pending

Eng merged but flag off in prod. Hold off.

  • Eng PR: trunk-io/trunk2#4045
  • Flag: showPorCountMonitorConfig (project: frontend-web-2)
  • Flag state: ON in production but fallthrough serves false (variation 1). Only users where impersonatorEmail ends with tyler@trunk.io receive true. Customers see the feature as off.
  • Signals:
    • trunk-io/trunk2#4045 merged 2026-06-02T05:28:59Z: detection endpoint accepts minPorCount, schema and MV path are in place.
    • usePorCountDetect defaults to false in the request schema; callers must explicitly opt in.
    • Dagster pipeline update (step 1 in PR remaining work) has not landed: showPorCountMonitorConfig not referenced in any dagster/pipeline code in trunk-io/trunk2.
    • UI config and preview (step 2 in PR remaining work) are gated behind showPorCountMonitorConfig -- flag is on in prod env but fallthrough is false, so the UI panel is hidden for all customers.
    • Test env: flag fallthrough is true (feature visible to all in test).
    • Rollout (step 4) is listed as future work.

Next step: wait for dagster wiring and UI rollout. Re-run verification after showPorCountMonitorConfig fallthrough is flipped to true in production.


Generated by Claude Code

@samgutentag samgutentag added pending Verify docs PR: eng merged but flag off in prod. Hold off. changelog PR touches the changelog (auto-generated drafts, hosting, formatting, indexing). labels Jun 2, 2026 — with Claude
Copy link
Copy Markdown
Member Author

Verification status: pending - June 2, 2026

Eng merged but flag off in prod. Hold off.

  • Flag state: LaunchDarkly not consulted; flag key unknown. Eng PR trunk-io/trunk2#4045 merged today (2026-06-02) but PR body explicitly lists remaining work: UI config screen, dagster pipeline update with LD flag, and full rollout.
  • Eng PR links: trunk-io/trunk2#4045 (merged 2026-06-02)
  • Flag: Unknown; rollout listed as remaining work item in trunk2#4045
  • Signals checked: trunk2 PR merge status; PR body remaining-work list
  • Suggested next action: Wait for the UI and dagster LD-flag updates, then re-run sweep after eng confirms rollout complete.

Generated by Claude Code

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog PR touches the changelog (auto-generated drafts, hosting, formatting, indexing). pending Verify docs PR: eng merged but flag off in prod. Hold off.

Development

Successfully merging this pull request may close these issues.

2 participants