Add versioned computation trace export (policyengine.trace.v1)

[edithatogo]

What's changed

Closes #512.

Adds an opt-in, versioned JSON-compatible computation trace export:

• FullTracer.to_trace(format="policyengine.trace.v1") exports calculation roots, structured nodes, dependencies, parameter reads, values, and timings.
• Simulation.to_trace() requires trace=True and adds best-effort model metadata.
• Unsupported format identifiers fail with ValueError.
• Node identity comes from structured TraceNode fields rather than parsing display keys.

The export does not alter formula evaluation, add an external schema dependency, or impose tracing cost when tracing is disabled.

Verification

• Regression and format-validation tests added.
• pytest tests/core/test_tracers.py passes locally (45 tests at the recorded revision).
• Ruff check and format pass for touched files.
• Changelog fragment added.
• make format && make documentation awaits the upstream workflow run.
• Issue linked and project item updated.

Compatibility, performance, and limits

The API is opt-in and built from the existing tracer. V1 intentionally omits calculation entity/count and parameter source/version metadata. Full vector values may be large for population-scale traces; summarization is deferred. The identifier is distinct from PolicyEngine's TRACE Transparent Research Object namespace: this PR covers per-computation provenance, while TRO covers release/composition provenance.

Maintainer action required

GitHub is holding workflow execution for this first-time fork contribution. Please approve the non-secret workflow run and confirm the policyengine.trace.v1 identifier before merge.

[MaxGhenis]

Thanks for this — it's a clean, well-scoped, and genuinely well-tested contribution, and computation provenance is a direction we actively want to build out. I reviewed the diff in full and ran the suite locally against master.

What works well

• No hot-path cost. to_trace() builds on the existing get_serialized_flat_trace() and only runs when explicitly called; it's gated on FullTracer, and SimpleTracer stays the default, so tracing-disabled simulations pay nothing. The raise ValueError guard in Simulation.to_trace makes the requirement explicit. That's exactly the compatibility posture core needs, since every country model sits on it.
• Forward-compatible versioning. A top-level format string plus a hard ValueError on anything but policyengine.trace.v1 is the right way to keep the document evolvable.
• Grounded in existing surfaces. _engine_metadata() reads get_runtime_metadata() and the {name, version, git_sha} keys line up; node/parameter ids reuse FlatTrace.key(); the changelog fragment is present.
• Verified locally: tests/core/test_tracers.py → 45 passed; test_simulations.py + test_tracers.py regression → 51 passed.

Suggestions (non-blocking)

1. Recover structured fields from TraceNode, not the string key. to_trace() builds calculation.roots from structured node attributes (tree.name, tree.period, tree.branch_name) but builds nodes by regex-parsing the serialized key via parse_trace_key. The regex assumes branch names contain no parentheses and periods no angle brackets — true today, but it couples the export to the exact string format. Since browse_trace() already yields TraceNodes with these fields structured, consider building nodes from those directly and keeping parse_trace_key only for the flat parameter-key map (which has no node object to read from).
2. calculation entity/count + parameter source metadata. #512's "minimum useful fields" also listed entity/count on calculation and source/version on parameters; roots currently omit entity/count and parameters omit source/version. Fine to defer — worth a line in the format notes so consumers know they're intentionally v1-absent.
3. Value size at population scale. Nodes embed the full serialized vector, which is ideal for the household-audit case in #512; for population traces it can get large. #512 mentioned "values or summaries," so an optional summarization/threshold could follow later.
4. Coordinate with the existing policyengine.py trace surface. policyengine already ships policyengine.provenance, which emits TRACE Transparent Research Objects (JSON-LD, TROv vocab, namespace https://policyengine.org/trace/0.1#) that pin the model wheel, dataset, manifest, and reform by sha256 — i.e. release/composition provenance. This PR's policyengine.trace.v1 is per-computation provenance, so the two are complementary rather than overlapping, but both live under the PolicyEngine "trace" name. Two things would help: (a) confirm the policyengine.trace.v1 format id and the …/trace/0.1# TRO namespace are intentionally distinct, and (b) note that a per-simulation TRO could embed or reference a policyengine.trace.v1 document as its runtime payload — that composition is the natural end state for provenance visualization. A short written schema for policyengine.trace.v1 (pairs well with #517) would also make the format consumable downstream.

CI status

No checks have run yet — the status rollup is empty because this is a first-time fork contribution and GitHub is holding the workflow runs for maintainer approval (not a pass/fail). The touched code exercises tests/core unit tests plus a docs build, none of which need repository secrets, so Lint / check-changelog / Test should go green once a maintainer approves the run; only the multi-package smoke test is occasionally fork-flaky.

Overall this is close, and I'd be glad to see it land after the structured-field cleanup and a quick maintainer call on the format/namespace coordination in (4).

[codecov]

Codecov Report

❌ Patch coverage is 70.00000% with 33 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.12%. Comparing base (c94a745) to head (768c986).
⚠️ Report is 37 commits behind head on master.

Files with missing lines	Patch %	Lines
policyengine_core/simulations/simulation.py	9.09%	20 Missing ⚠️
policyengine_core/tracers/full_tracer.py	72.91%	8 Missing and 5 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master     #515      +/-   ##
==========================================
+ Coverage   84.69%   86.12%   +1.42%     
==========================================
  Files         223      230       +7     
  Lines       11830    12883    +1053     
  Branches     1169     1253      +84     
==========================================
+ Hits        10019    11095    +1076     
+ Misses       1496     1444      -52     
- Partials      315      344      +29     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[edithatogo]

Pushed additional unit tests covering Simulation.to_trace (FullTracer required, model metadata inference, core-package skip) and the unparseable parameter-key path in FullTracer.to_trace, then rebased onto current master. Aimed at the Codecov patch gaps on simulation.py / full_tracer.py.

[edithatogo]

Follow-up to @MaxGhenis review

Addressed the structured-field cleanup and format notes:

1. to_trace nodes now take variable / period / branch from TraceNode via browse_trace() (same for parameter entries). parse_trace_key remains for flat-key callers only.
2. Docstring records v1-absent entity/count + parameter source/version, TRO complementarity (policyengine.trace.v1 vs …/trace/0.1#), and deferred population-scale summarization.

Also: CI still shows no checks — first-time fork contribution. Could you approve the workflow runs when you have a moment so Lint/Test can go green?

Happy to iterate further if you want a separate short schema page (pairs with #517) before merge.