From db944e9a83683e38ed2a4b4908379709bc12a6db Mon Sep 17 00:00:00 2001 From: DonislawDev Date: Sat, 25 Jul 2026 09:09:07 +0200 Subject: [PATCH] test(repo): guard prose that has an expiry date The 2b/2c drift cleaned up in #43 was not a set of false claims. All four sentences were TRUE when written and were supposed to die when a stage landed, and nothing enforced the expiry: convention 5 covers claims that are wrong, and check_notes.py deliberately does not check prose at all. So they outlived the code they described by several PRs, telling every later session the opposite of what the code did. Discipline alone produced four stale sites in one transition, so this adds the mechanical step. Prose that is only true until a stage lands now carries a PENDING marker naming that stage, and the open ids live in exactly one place. Closing a stage is one deletion from that set, which turns the guard red on every marker still pointing at it - so the prose is corrected in the same commit as the code. - test_no_stale_pending_markers scans every .py and .md for markers and checks BOTH directions: an id that is not open fails, and an open id that nothing references fails (a leftover entry) - OPEN_PENDING is the single source of open ids and is deliberately NOT a roadmap - it is the set of ids that PROSE points at, which is what makes the second check meaningful - mutation-checked both ways, not asserted: a probe marker for an unlisted stage and an unreferenced id each turned the test red, with file:line in the message - first real marker goes on the SocketEvent comment, itself a leftover of 2c: proto/remote_ip/remote_port/outbound are still unconsumed, so the comment now says so and PENDING(socket-event-fields) forces the decision to be revisited - rejected, measured not guessed: scanning for the word "yet" gives 25 hits of which ~21 are permanently true, a false-positive rate that gets a guard switched off within a week - convention 44, the sub-rule under process rule 5 and a new definition-of-done step live in the private notes, which this test cannot see - that half stays manual on purpose Co-Authored-By: Claude Opus 5 --- CHANGELOG-INTERNAL.md | 35 +++++++++++++++++ beantester/socketwatch.py | 6 ++- tests/test_repo_conventions.py | 72 ++++++++++++++++++++++++++++++++++ 3 files changed, 111 insertions(+), 2 deletions(-) diff --git a/CHANGELOG-INTERNAL.md b/CHANGELOG-INTERNAL.md index 08a6c75..c51104a 100644 --- a/CHANGELOG-INTERNAL.md +++ b/CHANGELOG-INTERNAL.md @@ -42,6 +42,41 @@ a `### BREAKING` section placed FIRST in that version, and each such line is pre - Help text and the flag tables in both READMEs now state that the flag is valid on its own. - Version bump deliberately NOT taken (convention 34): the owner closes it in `VERSION.txt`. +### Tests: a mechanical guard for prose with an expiry date (convention 44) + +The 2b/2c drift fixed below was not a set of FALSE claims - it was four claims that were true when +written and were supposed to die when a stage landed, with nothing enforcing the expiry. +Convention 5 covers claims that are wrong; `check_notes.py` deliberately does not check prose at +all. So the sentences outlived the code they described by several PRs. This adds the missing +mechanical step, because discipline alone produced four stale sites in a single transition. + +- New test: `tests/test_repo_conventions.py::test_no_stale_pending_markers` - scans every `.py` + and `.md` in the repo for `PENDING()` markers and checks BOTH directions. A marker whose id + is not in `OPEN_PENDING` fails (the stage closed, so the prose beside the marker is now a lie), + and an id in `OPEN_PENDING` that nothing references fails (a leftover entry). Closing a stage is + therefore ONE deletion from `OPEN_PENDING`, which turns the guard red on every marker still + pointing at it - so the prose gets corrected in the same commit as the code that outdated it. +- `OPEN_PENDING` (same file) is the single source of open stage ids, and is deliberately NOT a + roadmap: it is the set of ids that PROSE points at, which is what makes the second check + meaningful rather than bureaucratic. +- MUTATION-CHECKED both ways instead of asserted (convention 5): a probe file carrying a marker + for an unlisted stage, and an unreferenced id added to the set, were each confirmed to turn the + test red with the offending `file:line` in the message. The probe was then removed. +- First real marker: the `SocketEvent` comment in `socketwatch.py`, which was itself a leftover + ("carried for the connection log later (2c)"). `proto` / `remote_ip` / `remote_port` / + `outbound` are still unconsumed - the comment now says so plainly and carries + `PENDING(socket-event-fields)`, so whichever way that decision goes, the guard forces the + sentence to be revisited rather than quietly kept. +- The guard skips its own file (that file holds the ids rather than pointing at them), and the + `` placeholder form used in prose does not match the pattern, so docs and changelogs can + name the token without registering it. +- Rejected, MEASURED not guessed: scanning for the word "yet". 25 hits across `beantester/` and + `tests/`, of which roughly 21 are permanently true ("width 1 == not laid out yet", "no honest + answer yet"). A guard with that false-positive rate is switched off within a week. +- Convention 44, a sub-rule under process rule 5, and a new definition-of-done step (grep for the + id when closing a stage) live in the private notes, which this test cannot see - the notes are + not in this repo, so that half stays manual on purpose. + ### Docs: de-stale the 2b/2c prose - targeting DOES resolve against the live socket map Prose drift of the kind convention 5 exists for, with a twist: every one of these sentences was diff --git a/beantester/socketwatch.py b/beantester/socketwatch.py index 4a468e9..91bebc2 100644 --- a/beantester/socketwatch.py +++ b/beantester/socketwatch.py @@ -55,8 +55,10 @@ _ADD = frozenset({BIND, CONNECT, LISTEN, ACCEPT}) # One socket-layer event, normalised away from the ctypes struct so the map logic -# (and its tests) never touch pydivert. remote_ip/remote_port are carried for the -# connection log later (2c); the map itself keys on local_port only. +# (and its tests) never touch pydivert. The map keys on local_port alone: proto, +# remote_ip, remote_port and outbound are carried but NOT consumed anywhere yet, +# and whether the connection log should use them or they should be cut is still +# open - PENDING(socket-event-fields). SocketEvent = namedtuple( "SocketEvent", "kind pid proto local_port remote_ip remote_port outbound") diff --git a/tests/test_repo_conventions.py b/tests/test_repo_conventions.py index 3f8b7c3..31bba01 100644 --- a/tests/test_repo_conventions.py +++ b/tests/test_repo_conventions.py @@ -135,3 +135,75 @@ def test_no_em_or_en_dashes_in_repo_text(): offenders.append(f"{os.path.relpath(path, ROOT)}: {label}") check("no em/en dashes outside licenses/ (use '-')", not offenders, f"({offenders[:8]}{'...' if len(offenders) > 8 else ''})") + + +# -- prose with an expiry date -------------------------------------------------- # +# Convention 5 guards claims that are FALSE. This guards the OTHER failure mode, +# which cost four stale sites in a single transition: a claim that was TRUE when it +# was written and was supposed to die when a stage landed. Chunks 2a->2d wired the +# SocketWatcher into the engine and targeting, but comments went on saying "NOT read +# by targeting yet - that is 2c" long after it was read, because nothing enforced +# the expiry: check_notes.py deliberately does not check prose, and a comment cannot +# fail a build by itself. +# +# So prose that is only true UNTIL a stage lands carries a marker naming that stage, +# and the ids still open live here, in exactly one place. Closing a stage means +# deleting its id from this set - which turns this test red on every marker still +# pointing at it, so the prose is corrected in the same commit as the code that +# outdated it. +# +# This set is NOT a roadmap. It is the set of stage ids that PROSE currently points +# at, which is why an id nothing references is a failure too: it means the marker is +# gone and the entry was left behind. +OPEN_PENDING = { + # SocketEvent carries remote_ip / remote_port / proto / outbound, but the map + # consumes only kind / pid / local_port. Use them or cut them - undecided. + "socket-event-fields", +} + +PENDING_EXTS = (".py", ".md") + + +def test_no_stale_pending_markers(): + """Every expiry-dated marker names an open stage, and every open stage is marked. + + Both directions were MUTATION-CHECKED (2026-07-25) rather than assumed, because + a guard nobody has deliberately broken is a guard whose shape nobody knows: a + marker naming an id that is not in ``OPEN_PENDING`` fails, and deleting the last + marker for a listed id fails. Confirmed red for each. + + The scan skips THIS file, which holds the ids rather than pointing at them, so a + future example in the text above cannot break the guard. Note that the + placeholder form used in prose and changelogs does not match the pattern (``<`` + is not a valid id character), so documentation can name the token freely. + """ + import re + + marker = re.compile(r"PENDING\(([a-z0-9][a-z0-9-]*)\)") + seen = {} + for dirpath, dirnames, filenames in os.walk(ROOT): + dirnames[:] = [d for d in dirnames + if d not in (".git", "__pycache__", ".pytest_cache", + "licenses", "build", "dist", ".hypothesis")] + for name in filenames: + if not name.endswith(PENDING_EXTS): + continue + if name == os.path.basename(__file__): + continue # the registry does not scan itself + path = os.path.join(dirpath, name) + text = open(path, encoding="utf-8", errors="replace").read() + for lineno, line in enumerate(text.splitlines(), 1): + for found in marker.findall(line): + seen.setdefault(found, []).append( + f"{os.path.relpath(path, ROOT)}:{lineno}") + + stale = sorted(f"{key} ({', '.join(where)})" + for key, where in seen.items() if key not in OPEN_PENDING) + check("every PENDING marker names a stage still open in OPEN_PENDING " + "(a closed stage means the prose beside the marker is now a lie)", + not stale, f"({stale})") + + unmarked = sorted(OPEN_PENDING - set(seen)) + check("every id in OPEN_PENDING is still referenced by a marker " + "(an id nothing points at is a leftover entry, not an open stage)", + not unmarked, f"({unmarked})")