Proposal: versioned computation trace export for household calculations

[edithatogo]

I have been testing PolicyEngine's existing tracing machinery for downstream audit and conformance workflows. The current FullTracer/FlatTrace API already exposes the core information needed for a deterministic trace export: calculation nodes, dependencies, parameter reads, values, branches, periods, and timings.

Would maintainers consider a small, versioned export API over the existing trace data, for example:

sim = Simulation(situation=situation, trace=True)
sim.calculate("snap", "2024-01")
trace = sim.tracer.to_trace(format="policyengine.trace.v1")

Minimum useful fields

• format: a stable format/version string, such as policyengine.trace.v1
• engine: policyengine-core version, model package name/version, and optionally git revision
• calculation: requested variable, period, branch, and entity/count metadata
• nodes: dependency-ordered calculation nodes with id, variable, period, branch, dependencies, parameters, value, and optional timing fields
• parameters: accessed parameter names, instants/effective dates, scalar/vector values or summaries, and source/version metadata where available

Why this may be useful

• Audit trails for household-level calculations
• Debugging and reproducible explanation of why a calculation changed
• Cross-engine comparison where result equality is not enough and users need to compare calculation paths
• Downstream documentation or conformance formats without requiring PolicyEngine to adopt those formats directly

Local evidence (SNAP household)

• policyengine-core==3.28.0 @ f761573c2a13adecc3826be04af1980d13657e1d
• policyengine-us==1.755.5 @ fc64cef64ab55c3c48309c7fb304c35e5f3c9184
• Household SNAP result: snap [291.0]
• Trace shape: one root tree, 2,833 serialized flat-trace nodes
• Root dependencies: takes_up_snap_if_eligible, snap_normal_allotment, snap_emergency_allotment, and dc_snap_temporary_local_benefit

I also built a small external prototype that projects sim.tracer.get_serialized_flat_trace() into a dependency-ordered trace document and validates it against a downstream JSON Schema (2,833 steps / 164 inferred leaf inputs). That is evidence the existing tracer already has enough structure for a stable export; it is not a request that PolicyEngine support that downstream schema.

Questions

1. Would a versioned FullTracer.to_trace() or Simulation.to_trace() helper be welcome in policyengine-core?
2. Should this live as a JSON-compatible export only, or should PolicyEngine expose a typed Python dataclass layer first?
3. What model/package version metadata should be considered stable enough for a trace export?
4. Is parameter source/version metadata available today in a form maintainers would be comfortable exposing?

Out of scope

• No runtime AI or explanation generation
• No replacement for PolicyEngine's native formulas or tests
• No normative external schema dependency
• No claim that microsimulation traces are cheap enough for population-scale use

Evidence packet (external): https://github.com/edithatogo/rulesandprocesses/tree/main/external/policyengine

[edithatogo]

Tidied the issue body to remove internal draft framing and point evidence links at the public packet in edithatogo/rulesandprocesses. Substance of the proposal is unchanged.