Problem
The Populace US pipeline sometimes stages an artifact in its own immutable release that is deliberately not the certified default — for example a local-area build (populace_us_2024_acs_local.h5: ~1.6M households, ACS 2024 multispine, PUMA-assigned CD-119/county/state, calibrated to state administrative totals and state/CD population). Today there is no way to load such an artifact by name:
data_releases.{country}.datasets is the only named registry that resolve_dataset_reference / managed_microsimulation read.certify_data_release rewrites data_releases.{country} wholesale from the certified release manifest, so anything hand-added there is wiped by the next certification.
We want a staged artifact to be loadable by an explicit name, pinned to its own immutable revision and sha256, without ever becoming the certified default and without being wiped on re-certification.
Proposal
Add a sibling dataset_overlays.{country} section to the bundle manifest that get_release_manifest merges into the resolvable dataset registry:
- strictly additive — an overlay may not shadow the certified default or any certified dataset;
- resolves by name to its own pinned, sha-verified revision;
- survives re-certification, because certification only rewrites
data_releases.
Register the local-area artifact as the first overlay (populace_us_2024_acs_local) and document loading it and filtering it for state/CD breakdowns.
Also
Capture a fill-in-the-id Build M US Populace certification runbook (replaying #470) so the next default-flip is a mechanical exercise, and so it documents that overlays are preserved across certification.
Problem
The Populace US pipeline sometimes stages an artifact in its own immutable release that is deliberately not the certified default — for example a local-area build (
populace_us_2024_acs_local.h5: ~1.6M households, ACS 2024 multispine, PUMA-assigned CD-119/county/state, calibrated to state administrative totals and state/CD population). Today there is no way to load such an artifact by name:data_releases.{country}.datasetsis the only named registry thatresolve_dataset_reference/managed_microsimulationread.certify_data_releaserewritesdata_releases.{country}wholesale from the certified release manifest, so anything hand-added there is wiped by the next certification.We want a staged artifact to be loadable by an explicit name, pinned to its own immutable revision and sha256, without ever becoming the certified default and without being wiped on re-certification.
Proposal
Add a sibling
dataset_overlays.{country}section to the bundle manifest thatget_release_manifestmerges into the resolvable dataset registry:data_releases.Register the local-area artifact as the first overlay (
populace_us_2024_acs_local) and document loading it and filtering it for state/CD breakdowns.Also
Capture a fill-in-the-id Build M US Populace certification runbook (replaying #470) so the next default-flip is a mechanical exercise, and so it documents that overlays are preserved across certification.