Catalogue: gates that could not fail — the pattern, not a fixed count (9 filed, 8 added 2026-08-02, 3 added 2026-08-20)

[TortoiseWolfe]

➕ SIX ADDED 2026-08-21 — and two of them were guards written for THIS issue

Every one was caught by running the mutation, never by reading the code. In two cases the
reasoning had already concluded the guard was fine.

1 — a wiring guard that matched its own docblock. #896 added
scripts/__tests__/payment-queue-drain-is-wired.test.js to assert startConnectionListener
has a caller. Deleting the mount left it GREEN: referencingFiles() grepped raw source,
and the component's own docblock names the symbol while explaining why it exists. The guard
shipped with the exact defect it was written to catch. Fixed by stripping comments first.

2 — a REVOKE that reports success and changes nothing. #897. A migration
REVOKE ... FROM authenticated issued by postgres against an existing database prints
no privileges could be revoked and leaves the grant in place, because the grantor is
supabase_admin. #565 wrote exactly such a revoke specifically to correct already-provisioned
databases — and production still held the privilege three weeks later. A fix that cannot
fail is the same defect as a gate that cannot fail.

3 — twelve source citations pointing at nothing. #892/#893. Every clause citation in
docs/messaging/AUTHORIZATION-CONTRACT.md — the doc its own header calls the canonical
contract — resolved to the wrong line, two of them to blank lines. Nothing had ever compared a
citation to its target, and an append-only 3,535-line migration guarantees they rot.

4 — an exit criterion that could never pass. #575's was phrased as a grep that must return
empty, while a scheduled maintenance workflow legitimately matches it forever. A gate that
cannot return the passing answer is not a gate; it is a note people learn to ignore. #894
restated it on trigger, not presence.

5 — a module with zero callers AND zero tests, reading as a working feature. #895.
connection-listener.ts, 99 lines, existed to drain the offline payment queue on reconnect.
Nothing called it. The generalisation is worth keeping: a module can be fully unit-tested and
called by nobody, because the test exercises the function, not the application's use of it.
A guard therefore has to assert the wiring.

6 — a wait-loop that reported success against a PR where nothing had started. Mine, in
session tooling: pending == 0 is indistinguishable from "no checks have reported yet". It
printed RESOLVED, 0 pending on an untested PR. Harmless only because auto-merge was doing the
real gating.

The anti-vacuity floor earned its place this session. #893, #894, #903 and the #897 grants
test each needed an explicit assertion that the scan found something, because in every case
emptying the input — a renamed symbol, an emptied column, a wrong project ref — would otherwise
report success. #903's version is the sharpest statement of the rule: a drift check silently
inspecting an empty set is worse than no check, because it reports reassurance.

A guard is not finished when it passes. It is finished when you have watched it fail.

✅ 2026-08-21 — the two highest-value items below are DONE, and one of them is now a merge-blocking gate.

Both were called out as "the highest-value things in this issue, because they are generic rather than per-instance":

• Per-shard assertion counts — tests/e2e/reporters/assertion-count-reporter.ts counts them, and as of Make the zero-assertion reporter a gate, not a printout #861 a passing test that ran zero assertions fails the required lane. First armed run: 24 shards, three browsers, ~2,000 test executions, every shard reporting a real non-zero count. It fails on a missing verdict file and on observed: 0 as well as on offenders, so it cannot pass by not running.
• Assertions inside conditionals — the nine tests that pattern was hiding are all settled: fix(#861): make the four axe accessibility tests assert visibly #875 (four accessibility tests invisible to the reporter because checkA11y throws instead of calling expect), fix(#861): settle the last four zero-assertion tests #876 (two tests for features deleted or never built, one with no assertion in either branch, one asserting only inside if (failures.length > 0)), fix(#862): assert FR-005 where the failure is actually induced #878 (one guarding on an error banner no Retry control accompanies).

The habit this catalogue describes is unchanged, and new instances should still be added. What changed is that this particular shape can no longer reach main silently — it now has a gate that was itself mutation-tested in both directions, and whose three separable pieces are each pinned by scripts/__tests__/zero-assertion-gate-armed.test.js.

📈 RE-BASELINED 2026-08-02 — "nine" is a frozen count on a catalogue that keeps growing, and at least one checklist item below is done.

The title's number is the problem. This is a living catalogue of a recurring pattern, not a fixed set of nine defects. 2026-08-02 alone added eight more instances, recorded in #115's audit trail — and three of those eight were mine:

1. A ticket describing a symptom nobody had verified (payment-realtime: a row landing before the channel joins is lost forever — neither hook resyncs on first SUBSCRIBED #497 — "zero events delivered"; two tests in the same shard received those events seconds later).
2. A probe reading HTTP 500s as findings (.pay-verify/landmark.mjs reported mains: 0 for /contact, a page with <main> on line 18).
3. A watchdog probing one URL (docker/healthcheck.sh — / at 200 kept [self-heal] silent while /contact served the exact 5xx it exists to catch; fixed in A bake corrupts the dev server's .next, and #230's self-heal is blind to it (200 + dead chunk ≠ 5xx) #298).
4. A watchdog that would have re-dispatched every red commit forever (coverage defined as conclusion == "success"; caught before shipping, Post-merge E2E on main is silently cancelled by the next queued run, so main can be unverified while everything reads green #444).
5. An annotation invisible on passing runs (Playwright's list reporter prints none, so a latency trend could never be seen on the runs that would show it).
6. A coverage floor set above its own achievable maximum (a gate asserting ≥40 paths when 36 is all that exists).
7. A ticket of my own filed with an over-claimed severity (Realtime consumers never resync on channel join — /payment-result can strand permanently (messaging half RETRACTED: bounded at 10s by a poll) #499 — "lost until reload"; ConversationView.tsx:249 polls every 10s), retracted the same day.
8. A fix asserted without checking it would work — I wrote into A bake corrupts the dev server's .next, and #230's self-heal is blind to it (200 + dead chunk ≠ 5xx) #298 that "a second URL would have caught this in ~60s"; /docs/ was 200 throughout that incident. Struck in place.

Checklist correction — instance 4 is DONE. "Make validate:breakpoints run — a gate that exists and is never invoked is the cheapest possible fix." It runs now: .github/workflows/ci.yml:58 and scripts/validate-ci.sh:89. The box below is still unticked.

Two of the remaining items are the highest-value things in this issue, because they are generic rather than per-instance:

• Assertions inside conditionals (if (x) { expect(...) }) — a skipped assertion and a passing one are indistinguishable in the report.
• Per-shard assertion counts — a spec running zero assertions should not look like one running twelve.

Do not treat the nine below as a to-do list to drain. The pattern is the finding; the instances are evidence.

Nine times now, this repo has shipped a gate that could not fail. Each one was green, named after the thing it was supposed to protect, and reported nothing when that thing was removed.

This is the catalogue. It exists because the pattern keeps recurring across unrelated subsystems — .NET conformance, auth config, mobile layout, session persistence, accessibility, typography, CSS tooling — which means it is not a subsystem problem. It is a habit.

The standing rule: when adding or trusting a guard, ask what it would still report if the thing it guards were removed — then actually run that. Not reason about it. Run it. Instances 7 and 8 below were both caught only because someone executed the mutation instead of reasoning about it, and in both cases the reasoning had said the test was fine.

---

The nine

1 — .NET conformance passed with the rule deleted.
Row-state-only assertions ("no row was created") cannot prove the server enforces anything. Deleting a C# authorization rule left all 25 cases green: RLS caught it, the request 500'd, and the provider threw anyway — so the row was absent for the wrong reason. Fix: ConformanceConfig.assertRefusal pins a 403/400 from the server's own check, never a 5xx.

2 — the auth-config drift gate passed while CAPTCHA did not exist.
tests/unit/auth-config-validity.test.ts validated the tracked config file against itself rather than against the live project, so it stayed green through a period when security_captcha_enabled was false in production.

3 — mobile-horizontal-scroll.spec.ts: 12 of 16 assertions neutralised by an ancestor.
layout.tsx's overflow-hidden made the frame a non-scrolling scroll container. Descendants could not overflow the viewport, so the assertions were true by construction — the same ancestor also silently killed every position: sticky on the site. A gate cannot observe what an ancestor has already made impossible.

4 — validate:breakpoints was already failing and was never run.
Not wired into CI or a hook. A gate nobody executes has the same value as no gate, and worse reputation.

5 — all three Remember Me tests (#375).
session-persistence.spec.ts:61 puts its assertion inside if (authCookie) — the implicit OAuth flow sets no cookie, so the block never runs. :96 is a tautology. :218 logs "storage sync not available" and skips. Three green tests over a feature that is completely inert.

6 — all form inputs have labels (#391).
filter({ hasNot: page.locator('[type="hidden"]') }) keeps elements that do not contain a descendant matching the inner locator. An <input> is void, so the condition is trivially true and hidden inputs are never excluded. The filter has no effect at all.

This entry originally also claimed a real unlabelled control on /sign-in. That half is retracted — main is green. The "reproduction" ran page.goto('/sign-in') against BASE_URL=…/ScriptHammer; an absolute path replaces the basePath rather than appending, so the test measured a 404 page. Proven both ways on one commit: passes against a real root build, fails against the basePath URL. The broken-filter half above was never environment-dependent and stands.

7 — the #388 ladder test passed with the defect reintroduced.
It asserted a heading was larger at 500px than at 390px. But --text-* is clamp(1.75rem, 1.5rem + 1.5vw, 2.25rem) — fluid, so a heading frozen at text-4xl still grows with viewport width. Measured with the bug present: 44.775px → 47.25px. Larger, and still broken. Fix: compare against a probe element carrying the target class, never the same element at two widths.

8 — the @source inline test manufactured its own evidence (#377).
The test asserted that @source inline("sh-plate sh-well sh-groove") in globals.css was what made those utilities compile. It passed with that line deleted. Tailwind's source scanner reads tests/, so the spec's own literal string sh-plate was enough to make Tailwind emit .sh-plate. The test caused the condition it then went looking for. Fix: assemble the class names at runtime ('sh' + '-' + 'plate') so the literal never appears in the file.

This one generalises well past Tailwind: any test that names the artefact it is checking for can bring that artefact into existence — snapshot files, generated fixtures, scanned identifiers, cache keys.

9 — the icon set was green on two visibly broken drawings (#377).
settings rendered as a starburst indistinguishable from sun; theme rendered as a spiral instead of a contrast circle. Every test passed: path data differed so the duplicate check found nothing, and both cleared the coordinate-bounds and half-unit-grid checks.

This is a different failure mode from 1–8 and worth separating. The gates were not broken and were not bypassed — they simply could not observe the property that mattered. No assertion over path strings can see "these two icons look the same." Caught only by rendering them and looking. Where correctness is perceptual, a test suite bounds the search; it does not close it.

---

A related discipline point, not a tenth instance

A documented mutation nobody re-runs is a claim, not evidence. While writing #377's type spec I recorded MUTATION CHECK: remove the @theme mapping → verified red. On execution it stayed green — FONT_FAMILIES independently leads with var(--font-archivo), so body and headings survive without that mapping. The claim was plausible, written in good faith, and wrong. It was corrected in place, and different tests now pin that mapping.

Mutation notes in comments age exactly as badly as the code around them. They are a record of one execution, not a standing guarantee.

---

What would actually help

Not a process document. Candidates, roughly in order of value per effort:

• Make validate:breakpoints run (instance 4) — done, and it had rotted into a
  different member of this family. It runs at ci.yml:65 inside the required Test (20.x)
  lane. But the exit code is errorCount > 0 ? 1 : 0 and its coverage check pushed
  severity: 'warning', so it ran and could not fail: pointing CRITICAL_MOBILE_WIDTHS
  at an uncovered 999px printed ✅ Critical widths coverage validated above the warning
  and exited 0. Promoted to error and both success lines gated on the error list not
  growing (PR fix(#396): the breakpoint gate ran in CI and could not fail #841). Exit 1 with the mutant, 0 without.
• Fix the filter({ hasNot }) locator (instance 6) — done.
  tests/e2e/tests/accessibility.spec.ts:129 is now
  input:not([type="hidden"]), select, textarea, with the hasNot trap and the Turnstile
  cause recorded in the comment above it.
• Audit the remaining "no row was created" conformance cases — done.
  assertRefusal exists on the contract (messaging-provider.contract.ts:163) with 5 call
  sites, and the Supabase implementation pins the error TYPE and MESSAGE, noting explicitly
  that both backends must refuse with an authorization error "rather than a 5xx". The
  tests/rls/* row-absence assertions are a different category and are correct: there, zero
  rows IS the property under test.
• Grep the E2E suite for assertions inside conditionals — done 2026-08-20, and it
  found a live defect. Two specs guarded every assertion behind if (elements.length > 0)
  while visiting /, which has zero of either: mobile-form-inputs (0 inputs, 0 <form>)
  and mobile-card-layout (0 cards — the 2a refresh replaced them with grids). Pointing them
  at /contact/ and /blog/ and asserting the precondition immediately exposed the contact
  form's inputs rendering 40px on production, under the 44px minimum those very specs
  claim to enforce (Two mobile E2E specs assert nothing: they guard on elements the page they visit does not have #842, PR fix(#842): two mobile specs asserted nothing — and fixing them found a real 40px touch target #843). Proven side by side: guard shape → 3 passed, assertion
  shape → 2 failed, same page.
• Assertion counts in the per-shard reporter — still open, and still the highest-value
  remaining item: it would make every instance of this family visible without anyone
  suspecting a specific test.
• [~] Fail loudly when an E2E test lands on the 404 route — half done (PR fix(#396): an accessibility spec proved a setting survives navigating to the 404 page #844). The
  static half asserts every literal goto('/...') in the suite resolves to a real app-router
  route, with two reasoned allowlist entries. It found colorblind-toggle.spec.ts navigating
  to /about — a route this app has never had — to prove a setting "survives navigation";
  it was proving survival across the 404 template, and passing 7/7 either way.
  The runtime half is deliberately not done: a fixture wrapping every navigation would
  also catch computed paths, but it changes behaviour for every spec on the required lane and
  the sweep found only that one instance. Worth doing on its own terms, not smuggled in.

The rule needs a second half

The standing rule above — ask what a gate would report if the thing it guards were removed — only covers false negatives. Instance 6 was a false positive, and that turned out to be the more expensive kind.

A test that wrongly passes wastes a run. A test that wrongly fails gets believed, written up, filed as a ticket, and propagated: #115 carried an instruction to disregard that test as "known red" for about two hours, which is training the next person to ignore a real failure. Nobody checked which page had actually loaded before trusting the failure's story about why.

So: ask what a red gate is actually measuring, before you believe its explanation of why.

New instances, 2026-08-20 — three from one night, and two are the same gate failing differently

A. A CSS rule nothing could ever render, so no gate and no reviewer could see it (#838).
globals.css carried rgb(0 0 0 / .3) on .btn:hover in the three house themes through the
whole #376/#426 epic. It was never wrong-looking to a reviewer because it was never
rendered: the resting rule scores (0,7,0) from a five-:not() chain and the hover rule
(0,3,0), so the state rule lost the cascade. :active lost too, at (0,6,0).

Measured on a restarted dev server: resting, hover and active on all three themes resolve to
the identical --sh-plate. depth-tokens.spec.ts reads only resting state, so the gate that
exists for exactly this could not see it either. A gate that never enters a state cannot
observe a rule that only applies in that state — which is a new shape for this catalogue:
not a selector that stopped matching, but a gate whose coverage of states was never
enumerated at all.

B. A detector whose boundary excluded the form the defect took (#835).
no-build-in-dev-container.test.ts matched the banned build command with the boundary
[\s'"&|;]|$ — no backtick. So it matched fenced code blocks and silently missed markdown
inline code, which is the form the occurrence serving on production was in:

...after running `docker compose exec scripthammer pnpm run build` - they contain...

With the defect deliberately reintroduced, the scan reported zero. Caught only by
mutation-testing; reading the regex would not have found it, because the boundary looks
exhaustive until you ask which characters actually terminate the token in the corpus being
scanned.

C. A guard that passed against zero call sites.
The same night, playwright-install-resilience.test.js still asserted "lets apt give up on its
own, at every call site" — green, against zero call sites, because the installs it
guarded had been removed. Its own header had warned that a guard named for a property that
turned out to be harmful is worse than no guard. It reached that state anyway.

What these three add to the rule

The existing entries are mostly "the selector stopped matching". These are three different
mechanisms:

	how it failed
A	the gate never entered the state the rule lives in
B	the detector's boundary excluded the form the defect actually took
C	the subject disappeared and the assertion stayed green

C is the one worth generalising: every catalogue entry should be paired with a non-vacuity
assertion on its own subject — "I found N things to check, and N > 0". A and B both survived
review; only mutation-testing found B, and only a hand measurement found A.

---

Cross-references

#373 §B1/§B2 · #375 · #388 / #390 · #391 · #377 / #394 · #115 (where the running tally lives)

---

New instance, 2026-07-28 — a gate that stops measuring when a class name changes

The touch-target gate in tests/e2e/tests/mobile-touch-targets.spec.ts selects by a hard-coded allowlist of class names:

page.locator('nav button, nav label, a.btn')

#379 introduced a second button vocabulary — the 2a design's .sh-btn, alongside DaisyUI's .btn. The moment the home page's CTAs moved from one to the other, this selector silently stopped matching them. The buttons had not shrunk and had not moved; they were simply no longer being looked at.

This is the good news story for the pattern. The coverage floor added in a previous pass caught it immediately:

Error: Only 5 nav touch targets were measured, down from 6.

Without that floor the run would have gone green while measuring one fewer target than the day before — the failure this catalogue is about. The floor is the reason this is a two-line fix rather than a defect discovered months later.

Fixed by naming both vocabularies in the selector, with a note that a third class inherits the same invisibility for free.

The generalisable point: a gate that selects by class name has a silent dependency on that class name. Renaming a class, or introducing a parallel design system, removes elements from measurement without removing them from the page — and nothing about that reads as a regression. The defences that work are the ones already used here:

1. A coverage floor, so "measured zero" cannot be mistaken for "found zero failures". This is what saved it.
2. Selecting by role or computed property rather than class where practical.
3. Naming the assumption in the test, so the next person adding a button class sees the dependency.

Refs #379, #411.