test(chaos): put the SocketWatcher into the concurrency chaos suite#47
Merged
Merged
Conversation
This file's charter is "many threads hammering one engine", and it names the failure it exists to catch: threads that were only ever tested in isolation. Since the SOCKET-layer work the session runs a FOURTH thread, and since the connection-log fix the capture thread reads the watcher's live map WITHOUT a lock - yet no test here ever passed the engine a socket_source, so neither the watcher's lifecycle nor that lock-free read took part in any chaos at all. - test_the_socket_watcher_survives_start_stop_cycles: CYCLES rounds with a live event stream, asserting fail-open (running implies a live capture thread AND a watcher), the watcher cleared on stop, and no bean-socket-watcher thread outliving its session - checked after a 0.3 s grace, because SocketWatcher.stop() only joins for 0.25 s - test_the_capture_thread_reads_the_live_socket_map_under_churn: the capture thread resolving pids while the watcher mutates the map, the watchdog republishes it and settings and targeting churn underneath. It runs on the SAME ports the event source announces - otherwise pid_for always misses and a green run proves nothing - the crashlog watch IS the test: _pid_for/_process_for swallow into crashlog.once by design, so a read that started raising would leave every other assertion green. Patching once() also defeats its _once_seen dedupe, so repeated failures stay visible instead of collapsing into a single entry - conclusiveness is a CONDITION the test waits for (MIN_STAMPED rows carrying the event stream's pid), never a duration - same reasoning as MIN_BUILDS/MIN_ROWS - both claims mutation-confirmed: a pid_for that raises turns it red through the crashlog watch, and an engine reverted to poller-only stamps 0 rows out of 2.8 million packets. The first mutation also independently confirmed the failure-domain split from the previous commit - it reported engine.ports AND engine.ports.pid rather than one collapsed entry - written into the docstring, so nobody assumes otherwise: this does NOT catch putting the lock back into pid_for. A lock contends, it does not raise, and this test does not measure contention - that guard lives in test_socketwatch.py - _LiveSocketSource is paced deliberately; unpaced it saturates a core and starves the very threads the test is about Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
test_concurrency_chaos.pyexists for one reason, and its own docstring says it: the suite testedeach thread's job in isolation, which is how an off-main-thread Tk call survived. Since the
SOCKET-layer work the session runs a fourth thread, and since #46 the capture thread reads the
watcher's live
port -> pidmap without a lock - yet no test in that file ever passed the enginea
socket_source. So the watcher's lifecycle and that lock-free read were exactly the thing the filewas written to prevent: concurrency covered only in isolation.
Two tests, both following this file's existing conventions rather than inventing new ones.
test_the_socket_watcher_survives_start_stop_cycles-CYCLESrounds with a real event streamrunning: fail-open (running implies a live capture thread and a watcher), the watcher cleared and
its handle released on stop, and no
bean-socket-watcherthread outliving its session. The threadcheck waits 0.3 s first, because
SocketWatcher.stop()only joins for 0.25 s.test_the_capture_thread_reads_the_live_socket_map_under_churn- the real new surface: thecapture thread resolving pids while the watcher thread mutates the map, the watchdog republishes it
wholesale, and settings and targeting churn underneath. It runs on the same ports the event
source announces, because otherwise
pid_foralways misses and a green run proves nothing.What a reviewer should know
_pid_for/_process_forswallow theirexceptions into
crashlog.onceby design - a broken port table must not kill a session over adisplay name. So a read that started raising under concurrency would leave every other assertion
in the test green, and the only trace would be a crash entry nobody looks at. Patching
once()also defeats its
_once_seendedupe, so repeated failures stay visible instead of collapsing intoone entry.
SocketWatcher.pid_forthat raises turnsthe test red through the crashlog watch; an engine reverted to poller-only stamps 0 rows out of
2.8 million packets. The first mutation also independently confirmed the failure-domain split
from fix(engine): stamp the connection log from the live socket map #46 - it reported
engine.portsandengine.ports.pidrather than one collapsed entry.pid_for. Alock contends, it does not raise, and this test does not measure contention. That property has its
own guard (
test_socketwatch.py::test_pid_for_takes_no_lock_because_the_capture_thread_calls_it).This file documents what it cannot see, and I kept that habit.
MIN_STAMPEDrows carryingthe event stream's pid, exactly as the model-worker test waits for
MIN_BUILDS/MIN_ROWS- awall-clock budget would let machine speed decide whether the test proved anything, which this file
already learned the hard way (a threshold from a dev machine vs 8342 packets on a CI runner).
_LiveSocketSourceis paced deliberately. Unpaced it saturates a core and starves the verythreads the test is about, which would turn a green run into noise.
_start_lockedrace (the unsynchronisedself._targetingreadI flagged in the review). 25 start/stop cycles with concurrent
apply_settingspassed cleanly.That is not evidence the race is absent - it is evidence the window is too narrow to hit in a few
seconds, and it will be handled as its own change.
Checklist
python -m pytest testspasses locally. (666 tests, 0 failures, on an elevated shell, sothe two known non-admin failures are absent rather than excused. Verified through pytest's own
exit code.
smoke_gui.py: OK.)tests/for the style). - the change is tests, and both oftheir central claims were mutation-checked.
lang/en.jsonandlang/pl.jsonupdated. -n/a: no UI text.
CHANGELOG.md; technical ones and new tests inCHANGELOG-INTERNAL.md, under[Unreleased]. - internal only (convention 39): new tests, nobehaviour change.
type(scope): summary).VERSION.txt.🤖 Generated with Claude Code