fix(ci): dogfood gate passes cancelled runs through — a cancelled matrix is not a failed one - #3671
Merged
Merged
Conversation
…rix is not a failed one Closes #3668. With cancel-in-progress on, every consecutive push cancelled the in-flight dogfood matrix (the longest job in the workflow) and the gate's catch-all branch turned that into a red X on the superseded SHA — two observed on #3660 alone. False reds train everyone to ignore the one check that must never be ignored. Safety premise verified experimentally before landing (per the issue's own ask): run 30271824408 executed a fail-fast matrix where shard 1 really failed and fail-fast cancelled shard 2 mid-run — the aggregate needs.<job>.result reads 'failure', not 'cancelled'. Failure dominates, so an aggregate of 'cancelled' can only come from the whole run being stopped externally (supersession or manual cancel) and passing it masks no real regression. The manual-cancel case going green is the issue's accepted trade-off; the rejected alternative (skipping the gate via !cancelled()) would republish the #3622 required-context deadlock. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ECTCrcCdZpCHw5zFSgcmGt
|
The latest updates on your projects. Learn more about Vercel for GitHub. 1 Skipped Deployment
|
os-zhuang
marked this pull request as ready for review
July 27, 2026 14:11
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #3668.
Why
With
cancel-in-progresson, every consecutive push cancels the in-flight run — and the dogfood matrix, as the longest job in the workflow, is almost always the one still running. The gate's catch-all*)branch turnedneeds.dogfood.result == "cancelled"intoexit 1, painting a false red on every superseded SHA (two observed on #3660 alone, logs identical:dogfood matrix aggregate result: cancelled). False reds on a regression gate train people and agents to ignore the one check that must never be ignored.What
The issue's prescribed one-line fix:
cancelledjoins the pass branch —— plus an inline comment recording the evidence and the rejected alternative.
Safety premise — verified experimentally before landing, per the issue's own ask
The fix is only safe if a real shard failure can never surface as an aggregate
cancelled. Ran the experiment (run 30271824408, temporary workflow since removed from the branch): afail-fast: truematrix where shard 1 exits 1 immediately and shard 2 sleeps until fail-fast cancels it mid-run — the harshest version of "failure and cancellation coexist".Result: shard 1 concluded
failure, shard 2 concludedcancelled, and the aggregate observer printedneeds.matrix.result = failure. Failure dominates. An aggregate ofcancelledtherefore only occurs when the whole run is stopped from outside (concurrency supersession or manual cancel) with no leg having failed — passing it masks no regression.Notes recorded in the inline comment:
if: !cancelled()on the job was deliberately rejected (as the issue prescribes): a skipped gate publishes no required-check context on that SHA — the chore(ci): cut PR wall-clock ~9.5min → ~5-6min — dedupe dogfood, shard it, cache lint's build #3622 merge-deadlock again.grep 'needs\..*\.result' .github/workflows/*.ymlconfirms this is the only aggregate gate of this shape, so no sibling occurrences to fix.🤖 Generated with Claude Code
https://claude.ai/code/session_01ECTCrcCdZpCHw5zFSgcmGt