Skip to content

ci: 22 of 42 test:ci drift checks run in no workflow — including the meta-guard that detects unwired checkers #10269

Description

@JSONbored

Problem

npm run test:ci is documented as the local gate, and ci.yml is what actually blocks a merge. They have
drifted: 22 of the 42 drift/wiring checks reachable from test:ci run in no workflow at all.

They are therefore enforced only when a human happens to run the full local aggregate. On a repo whose gate
auto-merges on green CI, a check nothing in CI runs is not a gate — it is a convention.

Measured against main by walking npm run references transitively from test:ci and matching each
resulting *:check / *drift-check / *:audit script against every file in .github/workflows/:

check script
db:migrations:immutable:check scripts/check-released-migrations-immutable.ts
turbo-inputs:check scripts/check-turbo-typecheck-inputs.ts
workspace-dep-ranges:check scripts/check-workspace-dep-ranges.ts
control-plane:contract:check control-plane contract drift
control-plane:openapi:check control-plane OpenAPI drift
coverage-boltons:check scripts/check-coverage-bolt-on-filenames.ts
ui-derived-types:check scripts/check-ui-derived-types.ts
server-manifest:check scripts/check-server-manifest.ts
dead-exports:check scripts/check-dead-exports.ts
publishable-deps:check scripts/check-publishable-deps.ts
fixture-clock-races:check scripts/check-fixture-clock-races.ts
typecheck-coverage:check scripts/check-typecheck-coverage.ts
test-wiring:check scripts/check-test-coverage-wiring.ts
checkers-wired:check scripts/check-checkers-wired.ts
focus-manifest-fields:check scripts/check-focus-manifest-fields.ts
release-commit-parsing:check scripts/check-release-commit-parsing.ts
releasable-commit-types:check scripts/check-releasable-commit-types.ts
maintainer-associations:check scripts/check-maintainer-association-copies.ts
release-linked-versions:check scripts/check-release-linked-versions.ts
mcp:tool-reference:check MCP tool-reference drift
mcp:client-config:check MCP client-config drift
contract:api-schemas:check being wired by #10268

Why this is worth acting on rather than noting

Three of these are load-bearing in a way that makes the gap self-concealing:

  • checkers-wired:check is the meta-guard from Generate or guard the remaining hand-maintained lists — three shipped red-main incidents traced to one class #9860 that fails when a scripts/check-*.ts runs
    nowhere. It itself runs nowhere in CI. So the mechanism that detects unwired checkers is unwired.
  • db:migrations:immutable:check guards the invariant that a shipped migrations/*.sql is never
    edited. Violating it breaks boot for every upgraded Orb — the highest-blast-radius rule in the repo,
    and CI does not check it.
  • test-wiring:check and typecheck-coverage:check exist to stop coverage and typecheck silently
    narrowing. Narrowing them is invisible precisely because these do not run.

This is not hypothetical: #10237 is the same class having already fired twice on one artifact, both times
discovered by a human running the local gate rather than by CI.

The shape of the fix is a decision, not an obvious default

Wiring all 22 is not simply "add 22 steps". Each needs a needs.changes.outputs.* gate chosen from its real
inputs — the mistake #10268 had to fix for its own check was that no filter watched
packages/loopover-contract/** at all, so the naive gating would have skipped the exact PRs that matter.
CI runtime is also an explicit constraint here (it is what moved selfhost.yml post-merge).

Roughly:

A. Wire all 22 into ci.yml, individually gated. Highest fidelity, most review effort, some CI cost.
Most are sub-second node scripts, so the cost is likely dominated by job setup rather than the checks.

B. One drift-checks job that runs the whole set ungated on every PR. Cheapest to write and to keep
correct — no per-check gating to get wrong, and no filter can be missing. Pays one job's setup cost per PR.
Loses per-check path gating.

C. Wire only the load-bearing subset (db:migrations:immutable, checkers-wired, test-wiring,
typecheck-coverage, dead-exports, publishable-deps), leave the rest local-only, and document that
test:ci is deliberately a superset.

D. Add a mechanical guard that fails when a check reachable from test:ci is absent from every
workflow, with a reasoned allowlist — the ALLOWED_UNWIRED idiom check-checkers-wired.ts already uses.
Composes with any of A–C and is what stops the gap reopening.

Recommendation: B + D. B removes the per-check gating surface entirely, which is where the fidelity bugs
actually live, and one extra job on a PR is cheap next to the existing fan-out. D is what makes it stay
true. A is the "right" answer on paper but its 22 hand-chosen gates are precisely the hand-maintained list
#9860 exists to reject, and C leaves the meta-guard unguarded.

Deliverables

  • Decide between A–D above.
  • Every check reachable from test:ci either runs in a workflow or is explicitly recorded as
    local-only with a reason.
  • A mechanical guard so a newly added test:ci check cannot silently be CI-invisible.

How this was found

Walking npm run references transitively out of test:ci and diffing against every workflow file, while
fixing the single instance in #10237. The single-instance fix is #10268; this is the class.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions