feat(catalog): make tier requirements data, and hold every caller to them - #178
Conversation
…them `--tier scheduled` needs a GitHub token, two external hosts and the tag refs. That was recorded once, as comments beside maintenance.yml's egress allow-list, and enforced nowhere. Three jobs invoked the tier; one granted what it needs. release.yml ran every tier with no token, no tags and an allow-list omitting both SDK hosts, so a release stopped in preflight on four capability failures -- before its own promotion gate, and long before the missing evidence manifest that is the known blocker. It had never been noticed because the graph had never run: at 0.13.3 `resolve` did not invoke validate_all at all, and the step that does landed ten days after the last release. The advisory tier now splits by what a check reaches for. CALENDAR reaches no further than the checkout; EXTERNAL needs a token or a named host. `--tier release` is core plus CALENDAR, so a release still refuses to ship an expired external fact without a publishing graph reaching the network to find out. It passes with no token and no reachable host. catalog/validation-tiers.yml records what each check needs and which job may run which tier. check_validation_tier_contract.py reads membership from validate_all, the requirement from the catalog, and the grant from the job. Discovery is fail-closed both ways: an undeclared caller is a finding, and so is a declared caller that no longer exists. Six mutations were each caught, including release.yml reverting to every tier.
|
Note on a flaky check, so the record is accurate. Between writing this PR's verification section and pushing, one local full-tier run It is not a regression from this branch — The cause is the defect tracked in #177, which That is scheduled-tier only, so it does not gate this PR, and it is the next piece of |
Closes the last of the three callers, and lands the contract that makes all three
permanent. Ordered deliberately: #170 and #172/#176 fixed
ci.ymlandmaintenance.ymlfirst, so this contract arrives green rather than red.The single root cause behind three findings
validate_all.py --tier scheduledneeds a GitHub token, two external hosts, andthe tag refs. That requirement was written down once — as comments beside
maintenance.yml's egress allow-list — and enforced nowhere. Three jobs invoked thetier; one granted what it needs.
maintenance.ymlrelease.yml::resolveci.ymlno-base fallbackThe P1
release.yml::resolveranvalidate_all.py --— every tier — in a job with noneof the four. Every release would have stopped in preflight on four capability
failures: before its own promotion gate, and long before the missing promotion
evidence manifest (#157) that is the known blocker.
Nothing noticed because the graph had never run. At tag
0.13.3the release wasresolve → publishandresolvedid not invokevalidate_allat all; the "Fullvalidation sweep" step arrived in
25b2514on 2026-08-12, ten days after the lastrelease. Every green historical release run describes a different graph.
The fix keeps the intent
The old comment said a release "gets everything" because an immutable artifact must
not ship carrying an expired external fact. That intent is preserved, not dropped.
The advisory tier now splits by what a check reaches for:
CALENDAR— product-fact expiry, runtime-coverage waivers, release ledger vstags, documentation links — reaches no further than the checkout.
EXTERNAL— the two SDK pins, and the two GitHub-API checks — does.--tier releaseis core +CALENDAR. It passes with no token and no reachablehost, which is the executable proof that the preflight is satisfiable by the job
that has to run it:
resolvealso now checks out withfetch-tags: true.The contract
catalog/validation-tiers.ymldeclares what each check needs and which job may runwhich tier.
check_validation_tier_contract.pyreads:validate_allitself, so the catalog cannot describe a tierthat does not exist;
env, checkout inputs.Discovery is fail-closed both ways: a job running
validate_all.pythat thecatalog does not declare is a finding, and a declared caller that no longer exists is
a finding too. A closed allowlist that only checks what it was told is how this
problem stayed invisible in the first place.
release.ymlreverts to every tier (the original P1)maintenance.ymlloses its tokenmaintenance.ymlloses the Qt hostmaintenance.ymlstops fetching tagsci.ymlvalidate-touchedinvokes the advisory tierNot done here
The promotion gate is untouched — no role, signature, freshness or exact-SHA binding
is relaxed. #157 remains the blocker after this one, and this PR does not close it.
Verification
validate_all— all tiers OK withGH_TOKEN;--tier releaseOK without oneactionlint— clean ·zizmor@1.26.1 --persona pedantic --min-severity low— no findings