PolicyBench error review surfaced SNAP cases where models counted vehicle value in the asset/resource test while PolicyEngine ignored vehicles through the generic snap_assets source list.
Current code:
policyengine_us/variables/gov/usda/snap/eligibility/snap_assets.py documents that vehicles are excluded, subject to state fair-market-value exemptions.
snap_assets currently adds only gov.usda.snap.asset_test.sources, which does not appear to include vehicle excess value for states that still apply a vehicle value/resource rule.
Why this matters:
Some states, including Texas, have vehicle-related SNAP resource rules. In benchmark cases with listed household vehicle value, an LLM may deny or reduce SNAP based on vehicle resources. If PolicyEngine does not model the relevant state vehicle rule, those model responses can be penalized against a reference output that omits a policy condition.
Suggested follow-up:
- Audit SNAP vehicle/resource rules by state and whether Broad-Based Categorical Eligibility bypasses the asset test.
- Add state-specific vehicle excess value/countable equity variables where SNAP still counts them.
- Add tests for at least Texas and one broad-based-categorical-eligibility state where vehicles should remain excluded.
- Update
snap_assets documentation to state exactly which vehicle rules are implemented.
This is a modeling coverage issue rather than a confirmed arithmetic bug in a specific scenario.
PolicyBench error review surfaced SNAP cases where models counted vehicle value in the asset/resource test while PolicyEngine ignored vehicles through the generic
snap_assetssource list.Current code:
policyengine_us/variables/gov/usda/snap/eligibility/snap_assets.pydocuments that vehicles are excluded, subject to state fair-market-value exemptions.snap_assetscurrently adds onlygov.usda.snap.asset_test.sources, which does not appear to include vehicle excess value for states that still apply a vehicle value/resource rule.Why this matters:
Some states, including Texas, have vehicle-related SNAP resource rules. In benchmark cases with listed household vehicle value, an LLM may deny or reduce SNAP based on vehicle resources. If PolicyEngine does not model the relevant state vehicle rule, those model responses can be penalized against a reference output that omits a policy condition.
Suggested follow-up:
snap_assetsdocumentation to state exactly which vehicle rules are implemented.This is a modeling coverage issue rather than a confirmed arithmetic bug in a specific scenario.