fix(engine): stamp the connection log from the live socket map#46
Merged
Conversation
The connection log asked the POLLER who owns a local port - a snapshot refreshed a few times a second - so a flow that opened AND finished inside one refresh interval left a row with no owner at all. Short-lived connections are what this tool gets pointed at (a browser opens hundreds a minute), so that was the common case, not an edge one. The engine now asks the SOCKET-event map first and falls back to the poller, and the CONNECT event lands ~0.1 ms before the SYN reaches the NETWORK layer, so the row is stamped from its very first packet. The read had to become LOCK-FREE first, and that is the substance of this change. pid_for used to take _lock, which the watcher thread holds on every socket event and reconcile holds across a whole snapshot merge - calling that from _log_conn would have let the CAPTURE THREAD queue behind maintenance, which is the stall convention 20 exists to prevent. So pid_for reads the reference once and does a C-level get on int keys (the idiom PortTable.pid_for already uses), and reconcile builds the new state to the side and publishes it by REASSIGNMENT, so its O(n) pass is atomic to a reader instead of being observed half-applied. - names stay cheap=True (cache or nothing), so a pid can reach a row before its name does; that is written down rather than glossed over - the shared lookup is _live_pid, deliberately WITHOUT a handler: the first attempt had _process_for delegate to _pid_for, which swallows, so a broken port table reported one failure instead of two and the name domain lost its voice. test_processes.py caught it; the two callers are two failure domains and each wraps the raising helper itself - _Broken in that test now models what the engine actually calls, and the insight in its comment (a fake missing the keyword raises TypeError, so the test passes while exercising the wrong failure) moved to the keyword that now matters - four new tests, each mutation-checked: poller-only turns the wiring test red, restoring the lock turns the no-lock guard red, mutating reconcile in place turns the identity guard red - the lock-free safety claim is RUN, not asserted: a reader hammering pid_for while another thread inserts, deletes and republishes the map never raised and only ever saw the real pid - test_hot_path.py (packet threads must never reach the OS) stays green: both lookups are dict reads Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What and why
The Connections table asked the poller who owns a local port - a socket-table snapshot
refreshed a few times a second - so a flow that opened AND finished inside one refresh interval
left a row with no owner at all. Short-lived connections are what this tool gets pointed at (a
browser opens hundreds a minute), so that was the common case rather than an edge one: the row's
Process and PID columns simply stayed blank, and
_log_conn's retry could not help a flow that wasalready over.
The engine now asks the SOCKET-event map first and falls back to the poller. The CONNECT event lands
~0.1 ms before the SYN reaches the NETWORK layer (measured 2026-07-22), so the row is stamped from
its very first packet.
This is the follow-up to #45. That PR established that the four dead
SocketEventfields boughtnothing, because the packet is a better source for everything about the traffic. This one uses the
thing a packet genuinely cannot tell us: who owns the socket.
The lock-free read is the substance of this change
pid_forused to take_lock- which the watcher thread holds on every socket event, and whichreconcileholds across a whole snapshot merge. Calling that from_log_connwould have let thecapture thread queue behind maintenance, which is exactly the stall convention 20 exists to
prevent: WinDivert keeps diverting into a queue nobody drains while the UI still says "running". So:
pid_forreads the reference once into a local and does a C-levelgeton int keys - theidiom
PortTable.pid_foralready uses. No lock, no OS call.reconcilebuilds the new state to the side and publishes it by reassignment, so its O(n)pass is atomic to a reader instead of being observed half-applied.
The safety claim is run, not asserted.
test_a_lock_free_reader_survives_writes_in_flighthammers
pid_forfrom one thread while another inserts, deletes and republishes the whole mapunderneath it: no exception, and the only value ever seen was the real pid.
What a reviewer should know
first version had
_process_fordelegate to_pid_for, which swallows and records underengine.ports.pid- so a broken port table reported ONE failure instead of two and the namelookup lost its own voice.
test_processes.py::test_engine_records_a_broken_port_table_instead_of_going_quietfailed on exactly that. The shared lookup is now
_live_pid, deliberately without a handler,and each caller wraps it in its own failure domain - the same principle the watchdog already
applies to refresh-vs-trim.
process_for_port, so_Brokennow models what it does call (pid_for+name_of(cheap=)). The insight in that fake'scomment - a fake missing the keyword raises
TypeError, so the test passes while exercising thewrong failure - is still true, so it moved to the keyword that now matters instead of being
deleted.
names are warmed by the watchdog from the poller's snapshot. That is stated in the docstring and
in the user-facing changelog rather than being glossed over. Warming names from the watcher's map
as well would close the gap, and is deliberately not in this PR: it adds per-pid OS calls to
the watchdog and deserves its own measurement.
_live_pidrefactor because the code had changed shape since: poller-only turns the wiring test red,
restoring the lock turns the no-lock guard red, mutating
reconcilein place turns the identityguard red.
than that the capture thread happened to win a race with the watcher thread.
test_hot_path.py(packet threads must never reach the OS) was run explicitly and stays green.Checklist
python -m pytest testspasses locally. (664 tests, 0 failures, on an elevated shell, sothe two known non-admin failures are absent rather than excused. Verified via pytest's own
exit code this time - piping to
tailhad been masking it.smoke_gui.py: OK.)tests/for the style). - four, each mutation-checked, plus thelock-free claim exercised under concurrent writes.
lang/en.jsonandlang/pl.jsonupdated. -n/a: no new UI text; the affected columns and their labels already exist.
CHANGELOG.md; technical ones and new tests inCHANGELOG-INTERNAL.md, under[Unreleased]. - both, since a tester sees columns fill inthat used to be blank (convention 39).
type(scope): summary).VERSION.txt.🤖 Generated with Claude Code