test(mcp): replace exact context-bytes pins with tolerance bands - #6603
Conversation
The exact-equality pins on standalone.context_bytes and the dictionary definition counts forced every schema-touching PR to re-pin the constants, guaranteeing merge conflicts between any two in-flight schema PRs (#6571). Replace them with the invariants that matter: the standalone context stays within [200 KiB, 400 KiB], the shared dictionary stays substantial, and prompt cleanup strictly reduces definitions. The ratio comparisons and the payload-consistency equality are unchanged and carry the experiment's real signal. Closes the snapshot-pin half of #6571; the budget-headroom half stays open pending the shared-dictionary graduation. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
There was a problem hiding this comment.
Ladon verdict: Approve
Approve — clean test-hygiene change.
This PR touches only tests/mcp-schema-analysis.test.cjs (+20/-5), replacing three brittle exact-equality snapshot pins (context_bytes == 287_357, dictionary_definitions == 148 and == 127) with a tolerance band plus strict-reduction inequalities. Ratio comparisons and the payload-consistency self-check are untouched, so context_bytes is still validated against its computed source rather than a frozen literal. No published protocol surface (schemas, docs/reference, dist artifacts, task defs) is touched, so no changeset is required. Well-justified fix for real merge-conflict churn (#6279, #6571).
Decision-table walk: no critical/high findings (row 1 no); gated_paths false (row 2 no); high_risk false, no deletions/modifications-with-medium (rows 3–5 no); no prior decision (row 6 no); no no-auto-approve team match (row 7 no); zero medium findings (row 8 no). Falls through to row 9 → approve.
Nit (non-blocking): line 18 lower-bound literal 200_000 is mislabeled as "200 KiB" in the error message.
The exact-equality pins on
standalone.context_bytesand the dictionary-definition counts intests/mcp-schema-analysis.test.cjsforce every schema-touching PR to re-pin the constants — guaranteeing merge conflicts between any two in-flight schema PRs. PR #6279 was knocked from MERGEABLE back to CONFLICTING four times in two days by exactly this line.This replaces the pins with the invariants that matter:
The ratio comparisons (cleanup < 0.82×, shared dictionary < 0.37×) and the payload-consistency equality are unchanged — they carry the experiment's real signal.
Closes the snapshot-pin half of #6571; the budget-headroom half stays open pending the shared-dictionary graduation.
🤖 Generated with Claude Code