Skip to content

fix(engine): stamp the connection log from the live socket map#46

Merged
donislawdev merged 1 commit into
masterfrom
fix/conn-log-pid-from-live-socket-map
Jul 25, 2026
Merged

fix(engine): stamp the connection log from the live socket map#46
donislawdev merged 1 commit into
masterfrom
fix/conn-log-pid-from-live-socket-map

Conversation

@donislawdev

Copy link
Copy Markdown
Owner

What and why

The Connections table asked the poller who owns a local port - a socket-table snapshot
refreshed a few times a second - so a flow that opened AND finished inside one refresh interval
left a row with no owner at all. Short-lived connections are what this tool gets pointed at (a
browser opens hundreds a minute), so that was the common case rather than an edge one: the row's
Process and PID columns simply stayed blank, and _log_conn's retry could not help a flow that was
already over.

The engine now asks the SOCKET-event map first and falls back to the poller. The CONNECT event lands
~0.1 ms before the SYN reaches the NETWORK layer (measured 2026-07-22), so the row is stamped from
its very first packet.

This is the follow-up to #45. That PR established that the four dead SocketEvent fields bought
nothing, because the packet is a better source for everything about the traffic. This one uses the
thing a packet genuinely cannot tell us: who owns the socket.

The lock-free read is the substance of this change

pid_for used to take _lock - which the watcher thread holds on every socket event, and which
reconcile holds across a whole snapshot merge. Calling that from _log_conn would have let the
capture thread queue behind maintenance, which is exactly the stall convention 20 exists to
prevent: WinDivert keeps diverting into a queue nobody drains while the UI still says "running". So:

  • pid_for reads the reference once into a local and does a C-level get on int keys - the
    idiom PortTable.pid_for already uses. No lock, no OS call.
  • reconcile builds the new state to the side and publishes it by reassignment, so its O(n)
    pass is atomic to a reader instead of being observed half-applied.

The safety claim is run, not asserted. test_a_lock_free_reader_survives_writes_in_flight
hammers pid_for from one thread while another inserts, deletes and republishes the whole map
underneath it: no exception, and the only value ever seen was the real pid.

What a reviewer should know

  • The suite caught a regression I introduced, and the fix was to the design, not the test. My
    first version had _process_for delegate to _pid_for, which swallows and records under
    engine.ports.pid - so a broken port table reported ONE failure instead of two and the name
    lookup lost its own voice. test_processes.py::test_engine_records_a_broken_port_table_instead_of_going_quiet
    failed on exactly that. The shared lookup is now _live_pid, deliberately without a handler,
    and each caller wraps it in its own failure domain - the same principle the watchdog already
    applies to refresh-vs-trim.
  • A test fake had to be de-staled with it. The engine no longer calls process_for_port, so
    _Broken now models what it does call (pid_for + name_of(cheap=)). The insight in that fake's
    comment - a fake missing the keyword raises TypeError, so the test passes while exercising the
    wrong failure - is still true, so it moved to the keyword that now matters instead of being
    deleted.
  • Honest limit of the win: the PID reaches the row immediately, but the NAME can lag, because
    names are warmed by the watchdog from the poller's snapshot. That is stated in the docstring and
    in the user-facing changelog rather than being glossed over. Warming names from the watcher's map
    as well would close the gap, and is deliberately not in this PR: it adds per-pid OS calls to
    the watchdog and deserves its own measurement.
  • Every new guard was mutation-checked, and the first one was re-checked after the _live_pid
    refactor because the code had changed shape since: poller-only turns the wiring test red,
    restoring the lock turns the no-lock guard red, mutating reconcile in place turns the identity
    guard red.
  • The engine test drives a gated divert, so it asserts that the live map is consulted rather
    than that the capture thread happened to win a race with the watcher thread.
  • test_hot_path.py (packet threads must never reach the OS) was run explicitly and stays green.

Checklist

  • python -m pytest tests passes locally. (664 tests, 0 failures, on an elevated shell, so
    the two known non-admin failures are absent rather than excused. Verified via pytest's own
    exit code this time - piping to tail had been masking it. smoke_gui.py: OK.)
  • New behaviour has tests (see tests/ for the style). - four, each mutation-checked, plus the
    lock-free claim exercised under concurrent writes.
  • UI text goes through i18n keys, with both lang/en.json and lang/pl.json updated. -
    n/a: no new UI text; the affected columns and their labels already exist.
  • User-facing changes noted in CHANGELOG.md; technical ones and new tests in
    CHANGELOG-INTERNAL.md, under [Unreleased]. - both, since a tester sees columns fill in
    that used to be blank (convention 39).
  • Commits follow Conventional Commits (type(scope): summary).
  • No version bump - the owner closes a version via VERSION.txt.

🤖 Generated with Claude Code

The connection log asked the POLLER who owns a local port - a snapshot refreshed
a few times a second - so a flow that opened AND finished inside one refresh
interval left a row with no owner at all. Short-lived connections are what this
tool gets pointed at (a browser opens hundreds a minute), so that was the common
case, not an edge one. The engine now asks the SOCKET-event map first and falls
back to the poller, and the CONNECT event lands ~0.1 ms before the SYN reaches
the NETWORK layer, so the row is stamped from its very first packet.

The read had to become LOCK-FREE first, and that is the substance of this change.
pid_for used to take _lock, which the watcher thread holds on every socket event
and reconcile holds across a whole snapshot merge - calling that from _log_conn
would have let the CAPTURE THREAD queue behind maintenance, which is the stall
convention 20 exists to prevent. So pid_for reads the reference once and does a
C-level get on int keys (the idiom PortTable.pid_for already uses), and reconcile
builds the new state to the side and publishes it by REASSIGNMENT, so its O(n)
pass is atomic to a reader instead of being observed half-applied.

- names stay cheap=True (cache or nothing), so a pid can reach a row before its
  name does; that is written down rather than glossed over
- the shared lookup is _live_pid, deliberately WITHOUT a handler: the first
  attempt had _process_for delegate to _pid_for, which swallows, so a broken port
  table reported one failure instead of two and the name domain lost its voice.
  test_processes.py caught it; the two callers are two failure domains and each
  wraps the raising helper itself
- _Broken in that test now models what the engine actually calls, and the insight
  in its comment (a fake missing the keyword raises TypeError, so the test passes
  while exercising the wrong failure) moved to the keyword that now matters
- four new tests, each mutation-checked: poller-only turns the wiring test red,
  restoring the lock turns the no-lock guard red, mutating reconcile in place
  turns the identity guard red
- the lock-free safety claim is RUN, not asserted: a reader hammering pid_for
  while another thread inserts, deletes and republishes the map never raised and
  only ever saw the real pid
- test_hot_path.py (packet threads must never reach the OS) stays green: both
  lookups are dict reads

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@donislawdev
donislawdev merged commit 8f82434 into master Jul 25, 2026
8 checks passed
@donislawdev
donislawdev deleted the fix/conn-log-pid-from-live-socket-map branch July 25, 2026 10:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant