Target-parity contract for populace-us: gate + wire missing SSA SSI families#404
Merged
Conversation
…rget registry The retired us-data/eCPS pipeline calibrated SSI recipients (nation/ssa/ssi_recipients) but the populace registry silently dropped them. The v8 consumer feed carries ssa_supplement.ssi_recipients (7,404,820 national plus 51 states) and ssa_supplement.ssi_payments by area/category, none of which compiled — only the six oasdi_ssi_payments aggregates did. Add a source_name == "ssa" dispatch (_ssa_ssi_reference_from_fact) mirroring the census_pep / census_stc idiom: - oasdi_ssi_payments: delegated unchanged to _direct_reference_from_fact, so the six certified ssa targets (social_security_total ... ssi_total) never regress. - ssi_recipients.by_area_category "total": indicator sum of engine ssi receipt (role ssi_recipients), national and state (state_fips) grain. - ssi_payments.by_area_category "total": dollar sum of engine ssi (role ssi_state_payments), state rows only — the national all_areas_total duplicates the OASDI-table ssi_total already compiled. - aged/blind/disabled eligibility categories: dropped here, documented as reviewed exclusions (PolicyEngine-US has no SSI eligibility-category split). Registry 5,586 -> 5,689 specs, deterministic. Engine-computed counterparts (ssi), not reported-survey proxies. Six targeted tests. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The target-side analog of the input-column coverage contract, as a launch gate. Every administrative target family the retired us-data/eCPS pipeline calibrated to must be compiled into the populace registry or carry a reviewed exclusion naming its evidence — silent omission is the failure this kills (the SSI recipient family was dropped with no record of it). - release_target_parity.py: family-id derivation (namespace.concept of the ledger source_record_id), the TargetFamily/TargetParityManifest model, the us_release_target_parity_gate (fails the build when a compiled family is absent from the registry, or a reviewed exclusion has started compiling — cannot-rot per #286/#337), and assert_target_parity_manifest_current (manifest vs the checked-in sha-pinned feed inventory, plus the red line that the core SSA SSI recipient family can never be downgraded off compiled). - target_parity_manifest.json + target_parity_feed_families.json: generated, sha-pinned to consumer_facts_buildh_v8 (94b7155f). 25 compiled families, 55 reviewed exclusions (52 feed-only, each with a classification + evidence, and 3 source-absent us-data families: BLS CE, WIC, HUD housing). - build_us_target_parity_manifest.py: regenerates both artifacts; halts if a feed family neither compiles nor has a declared, evidence-naming exclusion. - Wired into build_us_fiscal_refresh_release.py on the compiled + substituted registry, before the diagnostic JCT skip, mirroring the input-coverage gate. - 28 tests mirroring the input-coverage suite (gate green/red/cannot-rot, anti-rot drift + red-line, feed reconciliation, guarded regeneration). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
test_main_writes_diagnostics_before_post_calibration_gate_failure drives main() with a stub target registry, so it must stub the new target-parity gate and anti-rot exactly as it already stubs target_profile_coverage_gate and assert_release_input_coverage_manifest_current (a stub registry cannot carry the full 25-family surface the real gate requires). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…bent-repo token CI (wheels + spec-only tests) surfaced two shipping-contract violations: - the us country package must declare every file it ships: add target_parity_manifest.json and target_parity_feed_families.json to country_package.json resources. - the live tree may not name the retired incumbent data package: reword the source-absent provenance / us_data_source strings in the manifest generator (and regenerate the manifest) to "retired us-data pipeline (archived)" instead of the hyphenated incumbent package name, which test_no_incumbent_data_package_references_in_live_tree forbids outside the three allowlisted eCPS-parity-reference files. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This was referenced Jul 13, 2026
Chesterton's-fence audit: wire NIPA wages/proprietors, net worth, LIHEAP; fence every exclusion
#405
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Target-parity contract for populace-us
The target-side analog of the input-column coverage contract (#368), now a launch gate. Every target the retired us-data/eCPS pipeline calibrated to — if it is a valid ledger-shaped administrative target — must be compiled into the populace registry, or carry a reviewed exclusion naming its evidence. Silent omission is the failure mode this kills. This gates Build M.
The gap this closes
The retired pipeline calibrated
nation/ssa/ssi_recipients(SSA Annual Statistical Supplement — 7,404,820 SSI recipients in 2024). The pinned feed (consumer_facts_buildh_v8, sha94b7155f) carries thessa_supplement.ssi_recipientsfacts (208 rows: national + 51 states × aged/blind/disabled/total) and a parallelssa_supplement.ssi_paymentsby-area table (208 rows). None of them compiled — the registry landed only the sixoasdi_ssi_paymentsaggregates and dropped the recipient family with no record of the omission. Export-mass parity cannot catch this: a target family that never compiles contributes zero rows, not wrong ones.Audit correction to the starting premise
The brief flagged the
$63.079B/$51.075BSSI/OASDI payment measures and "core Social Security (OASDI) aggregates" as also uncompiled. The live compile onorigin/mainshows otherwise, and the contract records the true state:ssi_payments= $63,079,493,000 compiles today as rolessi_total;social_security_dependents_benefits= $51,075,000,000 compiles asssa_dependents_total. Both, plus all five OASDI aggregates, are the certifiedssa_supplement.oasdi_ssi_payments:6(added in Use Ledger-backed fiscal target references #65, present on main).ssi_total) and the aged/blind/disabled eligibility sub-rows.So this PR wires the recipients and state payments; it does not re-wire OASDI/national-SSI payments (already present), and documents that finding rather than restating the premise.
What this PR does
1. Wires the missing SSA SSI families (
fiscal_targets.py) — asource_name == "ssa"dispatch (_ssa_ssi_reference_from_fact) mirroring thecensus_pep/census_stcidiom:oasdi_ssi_payments→ delegated unchanged to_direct_reference_from_fact; the six certifiedssatargets never regress.ssi_recipients.by_area_category"total" → indicator sum of enginessireceipt (rolessi_recipients), national + state (state_fips). Engine-computed counterpart, not a survey proxy — the same materialization path as thesnap_householdscaseload (Compile USDA SNAP state and national caseload targets from FNS average-monthly facts #371), flippingmeasure_modetoindicator_sumon the samessivariable the existingssi_totaldollar target already materializes.ssi_payments.by_area_category"total" → dollar sum of enginessi(rolessi_state_payments), state rows only.Registry 5,586 → 5,689 specs; national SSI recipients materialize to exactly 7,404,820; deterministic.
2. The contract (
release_target_parity.py+target_parity_manifest.json) — a family-granularity register. A family isnamespace.conceptof the ledgersource_record_id(us_target_family_id) — finer than the source family (sossaalone can't hide the recipient gap behind compiled OASDI) and coarser than the row. Each family iscompiledorreviewed_exclusionwith aclassification+reason+evidence.3. The gate (
us_release_target_parity_gate) — wired intobuild_us_fiscal_refresh_release.pyon the compiled + substituted registry, before the diagnostic JCT skip. Fails the build when acompiledfamily is absent from the registry, or a reviewed exclusion has started compiling (cannot-rot, #286/#337).4. Anti-rot (
assert_target_parity_manifest_current) — the manifest declares exactly the feed's family surface (checked against the sha-pinnedtarget_parity_feed_families.json, so it runs in CI without the 131 MB feed) plus documented source-absent us-data families, and the red line: the coressa_supplement.ssi_recipientsfamily must staycompiledand can never be quietly downgraded.5. Tests — 6 SSA wiring cases + 28 contract cases mirroring the input-coverage suite (gate green/red/cannot-rot, anti-rot drift + red-line + feed reconciliation, guarded regeneration proving the committed artifacts reproduce from the pinned feed).
The classified diff — (us-data targets ∪ feed families) − registry
Compiled — 25 families (registry targets | feed facts):
cbo.revenue_projectioncensus_pep.national_resident_population_agecensus_pep.v2024census_stc.individual_income_tax_collectionscms_aca.oep2024cms_medicaid.state_enrollmentcms_medicare.part_b_premium_incomehhs_acf_tanf.cash_assistanceirs_soi.filing_season_week47irs_soi.historic_table_2irs_soi.state_2022irs_soi.table_1_1/1_2/1_4/2_1/2_5/4_3jct.tax_expendituresssa_supplement.oasdi_ssi_paymentsssa_supplement.ssi_recipients(wired)ssa_supplement.ssi_payments(wired)usda_snap.national_average_monthly_households/national_benefits/state_average_monthly_households/state_benefitsReviewed exclusions — 55 families, each with evidence:
macro_control_totalsurvey_derivedcensus_acs.acs1_2023— ACS is a sample survey; also CD-only, off by defaultoff_by_defaultirs_soi.congressional_district_2022— CD targets opt-in for the national buildsupersededcensus.popproj2023(superseded by observedcensus_pep),kff.marketplace_effectuated_enrollment(superseded by CMScms_aca)not_modeledtip_incomesupport)deferredinput_sidenon_linearcms_nhe.medicaid_title_xix_expenditures— alreadyvalidation_only(person-weight-dependent allocation)source_absentProvenance
us-data target surface enumerated from the archived
policyengine-us-data(db/etl_national_targets.py,db/etl_*.py,utils/national_target_parity.py), read via the GitHub archive, not the local fork. Feed inventory + registry compile are sha-pinned and reproduce deterministically (tools/build_us_target_parity_manifest.py).Ruff clean; 243 tests green across the three touched suites. Please do not merge — this is for conductor review. Gates Build M.
🤖 Generated with Claude Code