Skip to content

design: verdict-stability backoff guards the whole pass instead of the verdict derivation, so every bounded retry chain needs an exemption #10227

Description

@JSONbored

The bug class, stated once

#10204 placed two halves of the verdict-stability backoff (#10184) in two different kinds of place, and only one of them was right.

half placement consequence
record inside runAgentMaintenancePlanAndExecute, immediately after persistDecisionRecord — the single ledger write every verdict passes through correct, and its own comment says why: "so no caller can bypass it"
skip at the entry of the whole publish-and-maintain pass wrong: suppresses everything the pass owed, not just the redundant re-derivation

Guarding the pass entry means backoff cancels work that has nothing to do with re-deriving a verdict:

#10222 fixes the first, second and fourth by relocating the guard and adding an explicitlyRequested escape hatch. It does not fix the class.

Why the current shape cannot be made safe by adding cases

The escape hatch is an allowlist of exemptions:

explicitlyRequested: options.force === true || previewPollAttempt !== undefined

Every bounded retry chain in the system must remember to add itself. The default for anything not listed is "suppress me." So:

That is the same shape as the audit finding in #10170's comment: a rule whose correctness depends on every future author remembering an unwritten obligation.

Proposed redesign: guard the derivation, not the pass

Move the skip to the same kind of place the record half already occupies — the single choke point where the verdict is derived and published — and let everything else in the pass run unconditionally.

Then, by construction:

This restores the symmetry #10204 intended: one place every verdict passes through, for both writing it and deciding whether to re-derive it.

Scope

In:

  1. Identify the single choke point where a pass derives-and-publishes a verdict (start from maybePublishPrPublicSurface, and confirm maybeRunAgentMaintenance cannot reach a verdict independently of it).
  2. Move the skip there. Delete the pass-entry guard added by perf(queue): back off re-evaluating a PR whose verdict never changes #10204/fix(queue): the verdict-stability backoff misses the webhook path, eats a manual retrigger, and ignores force #10222.
  3. Reduce explicitlyRequested to options.force only; drop the previewPollAttempt exemption if the poll's bookkeeping now sits outside the guard (verify, don't assume).
  4. Wire the webhook path (handlePullRequestWebhookEvent) — the whole reason this matters, and currently unguarded. Confirm visual(capture): bound the render-failure retry chain with a durable per-head budget #10061's recapture chain still spends its full MAX_CAPTURE_RETRY_ATTEMPTS budget with the guard live.
  5. A mechanical drift check in the scripts/check-*.ts family: fail the build if a publish-and-maintain entry point exists that does not route its verdict derivation through the guarded choke point. Same idiom as checkers-wired:check / dead-source-files:check.
  6. A property test per bounded retry chain: with backoff engaged on a settled verdict, each chain still completes its full attempt budget. This is the regression proof that generalises to chains not yet written.

Out:

  • Changing the backoff policy — the fingerprint, VERDICT_BACKOFF_* constants, head-SHA keying and fail-open posture are all correct and stay as they are.
  • The record half. It is already in the right place.

Acceptance criteria

Note on how this was found

#10204 shipped the wiring with no test at all (grep -rn "review_skipped_stable_verdict" test/ returned nothing before #10222). The three defects were found by reading the merged diff and reproducing against the Orb, not by CI. Whatever lands here should leave behind the tests that would have caught them.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions