Skip to content

fix: preserve region stability across theme drift#153

Merged
abrichr merged 2 commits into
mainfrom
agent/fix-region-stable-theme-overhalt
Jul 19, 2026
Merged

fix: preserve region stability across theme drift#153
abrichr merged 2 commits into
mainfrom
agent/fix-region-stable-theme-overhalt

Conversation

@abrichr

@abrichr abrichr commented Jul 19, 2026

Copy link
Copy Markdown
Member

Summary

  • keep REGION_STABLE outcome evidence armed across light/dark palette changes
  • match the same sealed, localized postcondition crop on Canny edge geometry after the stricter grayscale matcher and before the existing structural pHash
  • retain fail-closed behavior for true region changes, low-information edge crops, and missing evidence
  • add a metadata-only fixture for the exact v1.16.1 3/3 over-halt and regenerate the synthetic MockMed evidence during E2E

Root cause

The v1.16.1 benchmark substituted a new parameterized note under ?drift=theme.
The save succeeded and the independent final-state oracle confirmed the exact
note, patient, and encounter type, but the final saved-region pHash crossed its
fixed-position threshold because the region contained both the new text and
the palette change.

Measured on the exact reproducer:

  • correct theme + parameter substitution: grayscale template 0.143,
    structural edge template 0.860, pHash distance 32
  • true theme + blocking modal: structural edge template 0.245

The structural acceptance threshold is 0.80; the original grayscale check,
sealed/local search boundary, and pHash remain in place. The postcondition is
not removed and its pHash tolerance is not relaxed.

Verification

  • 112 passed:
    tests/test_vision.py tests/test_replayer.py tests/test_drift_oracle.py tests/test_postcondition_fuzz.py tests/test_seal_template_crops.py
  • all three exact generated v1.16.1 regression notes: 3/3 pass with the
    independent effect oracle and zero model calls
  • adversarial theme + blocking modal: still refuses; both region_stable and
    text_present fail
  • existing theme-heal and modal-refusal E2E contracts: 2 passed
  • Ruff source/targeted tests and full format check: pass
  • strict safety-path mypy for runtime/replayer.py: pass
  • consistency, claims, PHI, and release-version gates: pass
  • final wheel/sdist license boundary: pass; no openIMIS/AGPL material in either
    artifact

Final local artifact hashes:

  • wheel: 2dec0ba2e4640dcd1b33e0f7a9ce3f4d0d2fc1ee2db30331adde786f4b437a82
  • sdist: bd4fff313a2134ce9b0c6acdd80d4ed2e6c2bd155a24ebf817e68e05e6494aa9

Coordination

Draft #152 also edits runtime/replayer.py, but its only nearby hunk inserts
_attended_transition_observation before _structural_changed; it does not
modify the postcondition evaluator. Semantic overlap is none. Merge this small
fix first, then rebase #152 to resolve the file-level ordering cleanly.

Related evaluation: OpenAdaptAI/openadapt-evals#270.

@abrichr
abrichr marked this pull request as ready for review July 19, 2026 02:55
@abrichr
abrichr merged commit 30512ec into main Jul 19, 2026
14 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant