The Stage 1 enhanced CPS Medicaid validator currently compares 2025 medicaid_enrolled behavior against the 2024 Medicaid enrollment target file:
- Dataset:
EnhancedCPS_2024
- Target file:
policyengine_us_data/storage/calibration_targets/medicaid_enrollment_2024.csv
- Formula period:
sim.calculate("medicaid_enrolled", map_to="household", period=2025)
Recent pipeline runs failed this validator for New Hampshire, with NH exceeding the 45% tolerance. This may indicate a real Medicaid behavior issue, but it may also be a validator logic mismatch because the dataset and target file are 2024 while the formula period is 2025.
I traced the period=2025 calculation back to the original validator addition in July 2025, so the mismatch itself does not appear to be newly introduced. What changed recently is that this validator became part of the Stage 1 post-build validation lane.
The validator has been temporarily disabled with the verify_behavior_skip_temporarily pytest marker while we confirm the intended behavior.
@daphnehanse11, could you confirm whether the 2025 formula period was intentional here? If it was intentional, we should document why a 2024 dataset/target is expected to validate 2025 Medicaid enrollment. If not, we likely need to switch the validator to 2024 or update it to use a 2025 dataset/target pair.
The Stage 1 enhanced CPS Medicaid validator currently compares 2025
medicaid_enrolledbehavior against the 2024 Medicaid enrollment target file:EnhancedCPS_2024policyengine_us_data/storage/calibration_targets/medicaid_enrollment_2024.csvsim.calculate("medicaid_enrolled", map_to="household", period=2025)Recent pipeline runs failed this validator for New Hampshire, with NH exceeding the 45% tolerance. This may indicate a real Medicaid behavior issue, but it may also be a validator logic mismatch because the dataset and target file are 2024 while the formula period is 2025.
I traced the
period=2025calculation back to the original validator addition in July 2025, so the mismatch itself does not appear to be newly introduced. What changed recently is that this validator became part of the Stage 1 post-build validation lane.The validator has been temporarily disabled with the
verify_behavior_skip_temporarilypytest marker while we confirm the intended behavior.@daphnehanse11, could you confirm whether the 2025 formula period was intentional here? If it was intentional, we should document why a 2024 dataset/target is expected to validate 2025 Medicaid enrollment. If not, we likely need to switch the validator to 2024 or update it to use a 2025 dataset/target pair.