fix(#396): an accessibility spec proved a setting survives navigating to the 404 page - #844
Merged
Merged
Conversation
… to the 404 page `colorblind-toggle.spec.ts` navigated to `/about` to prove a colour-vision setting survives a page change. **This app has never had an `/about` route.** So "navigate to another page" navigated to `not-found.tsx`, and the persistence the test claims to prove was only ever proven across the error page. It passed 7/7 either way, because the chrome it asserts on renders there too. That is #396's "an E2E test that lands on the 404 route" class: nothing errors, the spec keeps running, and a green result means something weaker than it reads. Pointed at `/blog/`, which exists. Still 7/7 against production — the fix is a strict improvement, not a repair of a broken test. THE GUARD is static, and that is a deliberate scope choice. #396 asks to "fail loudly when an E2E test lands on the 404 route". A runtime fixture wrapping every navigation would catch more — including computed paths like `goto(\`/blog/${slug}\`)` — but it changes behaviour for every spec on the REQUIRED lane, and sweeping the suite found exactly ONE real instance. The zero-risk half is worth having now; the runtime half stays recorded in #396 rather than being smuggled in behind a one-line bug fix. So `e2e-gotos-resolve.test.js` enumerates the app router's real routes (handling `(groups)`, `[dynamic]` and `[...catchall]`) and asserts every literal `goto('/...')` in the suite resolves to one. Two paths are allowlisted, each with a reason: `/non-existent-page` (the test IS "404 page handles non-existent routes") and color-contrast's `TEMPLATE_PROBES` path, which must reach not-found.tsx deliberately because a template has no URL of its own (#425). The allowlist requires a reason string longer than a token, asserted by the test, so it cannot rot into a place where real mistakes get parked. Stated in the file so a green run is not over-read: this cannot see computed paths, redirects that end on a 404, or routes that exist as files but fail to build. Mutation-verified, mutant confirmed present first, two ways: restoring the `/about` navigation (caught, naming `colorblind-toggle.spec.ts:174 -> /about`), and emptying the allowlist (caught, proving the allowlist is load-bearing rather than decorative). test:scripts 424/424; type-check and lint clean. Refs #396
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The bug
colorblind-toggle.spec.tsnavigated to/aboutto prove a colour-vision setting survives a page change. This app has never had an/aboutroute.So "navigate to another page" navigated to
not-found.tsx, and the persistence the test claims to prove was only ever proven across the error page. It passed 7/7 either way, because the chrome it asserts on renders there too.That's #396's "an E2E test that lands on the 404 route" class: nothing errors, the spec keeps running, and a green result means something weaker than it reads.
Pointed at
/blog/, which exists. Still 7/7 against production — a strict improvement, not a repair of a broken test.The guard is static, deliberately
#396 asks to "fail loudly when an E2E test lands on the 404 route." A runtime fixture wrapping every navigation would catch more — including computed paths like
goto(`/blog/${slug}`)— but it changes behaviour for every spec on the required lane, and sweeping the whole suite found exactly one real instance.I took the zero-risk half now rather than smuggling a lane-wide behaviour change in behind a one-line bug fix. The runtime half stays recorded in #396.
e2e-gotos-resolve.test.jsenumerates the app router's real routes — handling(groups),[dynamic]and[...catchall]— and asserts every literalgoto('/...')in the suite resolves to one.Two allowlisted paths, each with a reason
/non-existent-page/__contrast-probe-unmatched-route__/color-contrast.spec.ts'sTEMPLATE_PROBES— a template has no URL of its own, so not-found.tsx must be reached deliberately (#425)The test asserts each reason is longer than a token, so the allowlist can't rot into a place where real mistakes get parked.
Verification
Mutation-verified with the mutant confirmed present in the file first:
/aboutnavigation → caught, namingcolorblind-toggle.spec.ts:174 -> /abouttest:scripts424/424;type-checkandlintclean.Stated in the file so a green run isn't over-read: this cannot see computed paths, redirects that end on a 404, or routes that exist as files but fail to build.
Refs #396