feat(targeting): event-driven process targeting via the WinDivert SOCKET layer (chunk 2)#38
Merged
Merged
Conversation
…t SOCKET events Chunk 2a of the targeting overhaul: the event-driven replacement for polling the OS socket table. The poller takes a snapshot a few times a second, so a connection that opens and closes between two snapshots is never seen and its whole life slips through unimpaired - measured as a large share of Chrome's short-lived connections. WinDivert 2.2 exposes a SOCKET layer that delivers BIND/CONNECT/ACCEPT/LISTEN/ CLOSE with the owning ProcessId, via a sniff-only handle (SNIFF | RECV_ONLY, which cannot drop or modify anything). SocketWatcher folds those into a live local_port->pid map: add on the first four, remove on CLOSE (pid-checked, so a late CLOSE cannot evict a port the OS has recycled to a different process). reconcile() seeds from a portmap snapshot and prunes a port absent for TWO passes (grace against evicting a socket opened microseconds before the snapshot was taken). Names/ancestors delegate to portmap (no duplication); the event source is injected, so the map is unit-tested without WinDivert. Why SOCKET, not FLOW (measured spike, elevated, sniff-only): SOCKET_CONNECT arrives ~0.1 ms BEFORE the outbound SYN reaches the NETWORK layer (closes the race); FLOW_ESTABLISHED arrives ~28 ms AFTER (post-handshake, SYN already slipped). Two sniff handles (NETWORK+SOCKET) were confirmed to coexist, and the real source was smoke-verified end to end (a known connection's local port mapped to os.getpid(), removed on close). Scope: the module in ISOLATION. It is NOT wired into BeanEngine or ProcessTargeting yet (2b wires the lifecycle + bootstrap + fallback; 2c makes targeting read the live map). The polling path is untouched and stays the fallback for --simulate / tests / non-Windows. New tests: tests/test_socketwatch.py. import beantester stays pydivert-free (the real source constructs pydivert lazily, only in start()). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The engine now owns a SocketWatcher: created and started in _start_locked, stopped in _stop_locked, next to the TargetResolver (both hold OS handles, both live as long as the session). A new start(..., socket_source=None) parameter injects the event source for tests. Bootstrap: start reconciles the watcher from a forced portmap snapshot, so connections open before the session are known from the first packet (events only announce NEW sockets). The watchdog folds a fresh snapshot in each tick as the safety net - a missed CLOSE ages out, a dropped event is recovered. Started only on the real-WinDivert path (divert is None) or with an injected source; on the synthetic/simulate/test path self._socketwatch stays None and the poller stands, so the testable-without-WinDivert contract holds. A SOCKET handle that cannot open DEGRADES to the poller (recorded, not swallowed) rather than failing the session. No behaviour change yet: targeting still reads the polling table; the watcher is kept live but unused until 2c. Verified by a real-path smoke (elevated, narrow pass-through filter): the NETWORK impairing handle and the SOCKET watcher ran together (retiring the coexistence risk) and a known connection's local port mapped to os.getpid(). New tests: tests/test_socketwatch_wiring.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…p (chunk 2c) This is the payoff of chunk 2. ProcessTargeting gained set_table(), and the engine now points it at the SocketWatcher's live map when a session has one (_targeting_table() -> watcher, else portmap), rebinding in _start_locked (targeting is usually built before start, against the poller). _start_locked was reordered so the watcher is created before the initial synchronous resolve, so the very first resolve already reads the live map. Effect: a connection of the targeted process is in impairment scope the instant its SOCKET_CONNECT event arrives - and since that event precedes the SYN, before its first packet. Verified end to end on real WinDivert (elevated, narrow pass-through filter): targeting bound to the watcher, and a fresh outbound connection read in_scope in ~0 ms. The polling path stays as the fallback, unchanged, so short-lived connections only still escape when there is no real WinDivert (--simulate, tests, non-Windows, or a SOCKET handle that could not open). Only the LIVE path changed: engine.target_for builds against _targeting_table(). The one-shot reporting helpers (find_process_ports -> resolve_ports, make_targeting) keep resolving against portmap - they are display snapshots, not session targeting. Prose corrected (rules 5/6): the targeting.py docstring claimed the race "cannot be closed, only made small" - true for the poller, false for the watcher. It now names which table closes it. test_gui_state.py: its FakeTable is now injected into engine._ports (the one shared table targeting resolves against) rather than only monkeypatching portmap.default_table after the engine was built - the same injection the resolver and socket-watcher tests use. New tests: tests/test_targeting_socketwatch.py. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Both READMEs' targeting notes said process targeting is 'a race with the system (and stays one)' via a ~2-3x/second socket-table poll. That is now the FALLBACK, not the primary path: on Windows targeting follows the system's SOCKET events, so a connection is in scope from the moment it opens (before its first packet, for outbound). Updated the two targeting bullets and the field note in EN and PL, and softened the exclusion note (unidentified is no longer 'every new connection'). Also a user-facing CHANGELOG entry: targeting now catches connections as they open, including the short-lived ones a busy app like Chrome makes, which the poll used to miss. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…r stop Two fixes from a field crash log + a "first start takes seconds" report. 1) SocketWatcher._loop recorded the WinError 995 that stop() induces (closing the SOCKET handle unblocks the parked recv() with "I/O aborted") as a crash, so every STOP left a spurious socketwatch.loop entry in crashes/. Now guarded by `if not self._stopping.is_set()`, like the capture loop's `if self._running`; a real socket-stream failure while running is still recorded. 2) The first target-start blocked ~2 s. Resolving a target walks every socket-owning PID and its process tree; the first protected ancestor that could not be opened individually made portmap.info fall back to a whole-system psutil.process_iter() (~2.9 s), synchronously, on the start/apply path. A PID the OS will not let us open is never one of the user's apps, so targeting now passes allow_bulk=False (new flag on info/name_of/ancestors); the bulk stays for warm_names() (the connection log) on the watchdog thread. Cold resolve 2148 ms -> 371 ms. This also relieves the yes/no flicker at start: the resolver hit the same bulk, so targeting was stale for ~2 s and early connections slipped. Behaviour change: a protected process can no longer be targeted by name (it will not match); the connection log still shows its name. test_processes.py's fake psutil gained a Process class (individual resolution, matching real psutil), and the targeting/watcher test fakes gained the allow_bulk kwarg. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…s target-by-name)
The previous commit's allow_bulk=False made the first target-start fast by SKIPPING
the process-name bulk scan - and that broke targeting Chrome BY NAME: the field
reported "BRAK pasującego procesu" while targeting Chrome's PID worked. Skipping the
bulk skipped the only path that can name a HARDENED process (Chrome's network service
and renderers refuse the OpenProcess that psutil.Process(pid) needs), and name
matching needs the name where PID matching does not.
The real problem was never "walk the tree", it was that the bulk fallback was
psutil.process_iter at ~2.6-2.9 s. Replace it with a native CreateToolhelp32Snapshot
(portmap._toolhelp_process_table): it reads every process's name+ppid WITHOUT opening
any of them, measured ~6 ms for 350 processes, and it names hardened processes. So
the bulk is affordable on the synchronous start/apply resolve AND resolves Chrome by
name. Cold ProcessTargeting.refresh('chrome') 2148 ms -> ~365 ms, matched=True.
- Reverts the allow_bulk flag entirely (portmap/socketwatch/targeting + the test
fakes). Making the bulk fast is the fix; skipping it was the bug.
- portmap._process_table() prefers toolhelp; psutil.process_iter is the fallback off
Windows / in tests (which flip portmap._ALLOW_NATIVE_PROCESSES off).
- Keeps the SocketWatcher-stop crash guard from the previous commit.
- Corrects PortTable.info's docstring (convention 5): create_time() is ~5 ms/PID
(not 0.005 ms) - so the per-refresh recycle check is ~180 ms with a browser open,
a measured perf follow-up, not the sub-millisecond it claimed.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…e signal Field report from a browsing session: rows were coloured orange while the column read "impaired: no" (and "yes" rows had no colour) - "something is wrong with the connections table". Cause: the column reads the STORED per-flow scoped record (a session-long "was ever in impairment scope"), but ConnsPage._tag_of still asked the engine LIVE (in_scope_now -> local_port in target_ports). For a closed or idle flow the live check flips to False the instant the socket closes, so the colour and the column disagreed. The session in the screenshot was also Stopped, where the live scope is frozen and meaningless. _tag_of now reads the same stored c["scoped"]. Column, highlight, sort (views.py) and CSV (gui/app.py) are all one signal now and can never diverge - a row is orange exactly when its column says "yes". ConnsPage._in_scope removed; engine.in_scope_now / core.in_scope are now unused (left for a follow-up removal). tests/test_conns_columns.py updated: a scoped=True flow whose port is out of the current target is now "yes" AND highlighted (it used to be "yes" with an empty tag). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What & why
Targeting a process (e.g.
chrome.exe) worked out which connections belonged to the target bypolling the OS socket table a few times a second. A snapshot is always a little behind reality,
so a connection that opened and closed between two snapshots was never seen and its whole life
slipped through unimpaired. On a busy app like Chrome (many short-lived connections, QUIC, prefetch)
that meant a large share of the traffic escaped - the reported "part caught, part not".
This replaces the poll with WinDivert's SOCKET layer, which delivers a real-time event
(bind / connect / accept / close) carrying the owning
ProcessId. A sniff-only handle(
SNIFF | RECV_ONLY- it cannot drop or modify anything) keeps a livelocal_port -> pidmap, so aconnection is targeted the instant it opens. Measured:
SOCKET_CONNECTarrives ~0.1 ms beforethe outbound SYN reaches the NETWORK layer, so the mapping is ready before the first packet. (FLOW
layer was measured ~28 ms later, i.e. after the handshake, so SOCKET, not FLOW, is the source.)
How it was built (4 sub-chunks, one commit each)
beantester/socketwatch.py(SocketWatcher) in isolation: the live map (add onbind/connect/accept/listen, remove on close, pid-checked so a late close cannot evict a
recycled port), a snapshot
reconcile()safety net with a two-pass grace, names delegated toportmap, an injected event source so it is unit-tested without WinDivert.BeanEngineruns the watcher for the session lifecycle (start/stop next to the resolver),bootstraps it from a
portmapsnapshot, reconciles each watchdog tick. Only on the real-WinDivertpath; degrades to the poller if the SOCKET handle cannot open (does not fail the session).
ProcessTargeting.set_table()+engine._targeting_table(): targeting resolves againstthe live map when a session has one, the poller otherwise. This is where behaviour changes.
Fallback / testability
The polling path (
portmap/target_resolver) is untouched and stays the fallback for--simulate, the tests, and non-Windows - so the "engine testable without WinDivert" contractholds. Every unit test runs without WinDivert (
import beantesterstill does not import pydivert).Verification
tests/test_socketwatch.py,tests/test_socketwatch_wiring.py,tests/test_targeting_socketwatch.py. One pre-existing GUItest was updated to inject its fake table into
engine._ports(the shared table targeting nowresolves against), the same injection the resolver/watcher tests use.
impairment): SOCKET/FLOW layers deliver PID + 5-tuple; NETWORK + SOCKET handles coexist;
SOCKET_CONNECTprecedes the SYN; the engine opens both handles together; and a freshconnection of the targeted process read
in_scopein ~0 ms through the live map.Note on prose (rules 5/6)
The
targeting.pydocstring and both READMEs previously called the race "a limit of the method"that "cannot be closed". That was true of the poller and false of the event path; corrected to name
which table closes it. This also motivated a new PROJECT_NOTES rule: a recurring problem is a
wrong-primitive signal - read the dependency's capability surface before building a workaround.
(The SOCKET/FLOW layers were in pydivert the whole time.)
Before merge
Please run the real-Chrome check on your machine (GUI, target
chrome.exe, watch the Connectionstab): the sea of untargeted connections should be gone, and the intermittent
"BRAK pasującego procesu / no matching process" at apply time should not recur.
🤖 Generated with Claude Code