fix(tests): the at-rest plaintext probe was both flaky and blind — decode before matching

[wshallwshall]

tests/test_store_aad_binding.py asserted "DOE" not in on_disk against an mfenc cell. The payload is base64 of a random nonce + AES-GCM ciphertext, so matching a 3-character needle against that text is a coin flip.

It flaked

Measured at this payload size: ~0.12% of runs, about 1 in 850. It red-X'd PR #7, whose diff touched release.yml and leak-gate docs — nothing within reach of the store.

The flake is the smaller half — it was also blind

Handed a cell that base64-encodes the plaintext verbatim, the old check evaluates True — "no plaintext here" — because base64 of DOE^JANE contains no literal DOE.

plaintext-bearing cell -> new probe says absent? False   (correct: it IS present)
same cell, old base64-text check:                 True   (MISSED IT)

So it fired at random on safe data and would have passed a real PHI-at-rest leak. For an ASVS 11.3.3 at-rest binding test, that's the wrong failure mode in both directions.

And the obvious fix is a trap

Reaching for a longer, more distinctive needle — "DOE^JANE" — makes it vacuous: ^ is not in the base64 alphabet, so the assertion could never match, would pass unconditionally, and would look stronger than what it replaced.

The fix

Decode first, then match the full plaintext against the at-rest bytes. A false positive now needs a collision across the whole message rather than three characters, and a cell carrying its plaintext fails loudly.

test_the_plaintext_probe_can_actually_fail pins the non-vacuity directly — it feeds the probe a plaintext-bearing cell and asserts the probe reports it. Without that, the round-trip test would pass no matter what the store wrote at rest.

Not changed — not every short needle is wrong

Site	Verdict
test_support_bundle.py:81 — "2575" not in blob	blob is json.dumps(summary), plaintext. A deterministic redaction check; a short needle is exactly right. Left alone.
test_uploads.py:52 — "MRN123" not in blob	Same ciphertext shape, but a 6-char needle is ~400,000× safer (~3×10⁻⁷%). Left alone rather than churn it.

The pattern to avoid is a short needle against ciphertext, not needle length itself.

🤖 Generated with Claude Code