Add named non-default access to the US Populace local-area dataset

[MaxGhenis]

Fixes #471

What

Two related changes to the US Populace data path.

1. Named, non-default access to the local-area artifact. The Build L artifact populace_us_2024_acs_local.h5 (~1.6M households, ACS 2024 multispine, PUMA-assigned CD-119/county/state, calibrated to state administrative totals and state/CD population; four reviewed limitations in its release gate summary) is published in its own immutable release and is deliberately not the certified default. This makes it loadable by name:

import policyengine as pe
sim = pe.us.managed_microsimulation(dataset="populace_us_2024_acs_local")

2. Build M certification runbook. A fill-in-the-id runbook that replays #470, so tonight's default-flip is mechanical.

How — the dataset_overlays mechanism (design note)

The registry that resolution reads (data_releases.{country}.datasets) is regenerated wholesale by certify_data_release on every certification. So a dataset hand-added there would be wiped by the very next certification (i.e. tonight's Build M). Putting the local-area entry directly in data_releases.us.datasets would therefore be self-defeating.

Instead this adds a sibling top-level manifest section, dataset_overlays.{country}, that:

• get_release_manifest merges into the resolvable dataset registry, so resolve_dataset_reference / resolve_managed_dataset_reference / managed_microsimulation load it by name at its own pinned, sha-verified revision;
• is strictly additive — an overlay may not shadow the certified default or any certified dataset (guarded with a hard error), so default resolution is provably untouched;
• survives re-certification with zero manual steps, because certify_data_release only ever rewrites data_releases, never dataset_overlays.

Default resolution is unchanged: default_dataset stays populace_us_2024, default_dataset_uri stays the certified Build J URI, and region_datasets.national is untouched. The certified-bundle TRO composition is unchanged except for rebinding the bundle-manifest sha256 (the overlay is not a certified artifact, so it does not enter the TRO).

Changes

• src/policyengine/provenance/manifest.py — _apply_dataset_overlays + merge in get_release_manifest.
• src/policyengine/data/bundle/manifest.json — dataset_overlays.us.populace_us_2024_acs_local (Build L revision + sha256).
• src/policyengine/data/bundle/us.trace.tro.jsonld — rebind manifest sha256 (required by TestVendoredSidecarBinding; regenerated US-only, UK preserved).
• tests/test_release_manifests.py — regression test that the overlay never changes default resolution; overlay resolves to its Build L tag; additive/guard/survive-recertification unit tests (TestDatasetOverlays).
• docs/microsim.md — load + state/CD filtering guidance. docs/release-bundles.md — overlay mechanism. docs/engineering/runbooks/build-m-us-populace-certification.md — Build M runbook (+ pointer from the data-certification skill).

Verification

• python scripts/bundle.py check exits 0; ruff format --check + ruff check clean.
• 240 passing across the manifest/dataset/region/model/TRO/cert/bundle blast radius, including the new overlay tests.
• All four touched docs render under Quarto 1.9.36.
• The runbook's steps are checked against Certify US Populace default: Build J 57k (rmloss100) #470's actual diff (manifest.json, us.trace.tro.jsonld, three test constants, auto-written changelog fragment; pyproject.toml and snapshots untouched because policyengine-us stayed 1.764.6).

🤖 Generated with Claude Code