Per-host filtering for the osctrl-tls HTTP debug dump#882
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
osctrl-tlscan dump every incoming HTTP request to a debug log (--enable-http-debug/HTTP_DEBUG), but the toggle is all-or-nothing: either every request from every node isdumped, or none are. On a busy server this is a firehose — the operator cannot isolate the
traffic of a single host, which is the only case where request-level debugging is useful.
Change
Add an optional host filter to the existing HTTP debug feature. The filter is the osquery
node UUID (case-insensitive), configured via a new
hostIdentifierfield on thedebugconfig block, or
--http-debug-host/HTTP_DEBUG_HOST.debug.hostIdentifierempty (default) → behavior is identical to today: every request isdumped when
enableHttpis true. No semantic change for existing deployments.debug.hostIdentifierset → only requests whose node UUID matches are dumped. The expensivedump work (
httputil.DumpRequest+ lumberjack write) runs only for the matching host.The matching is keyed on the canonical node UUID:
enroll→ the request'shost_identifier(which becomes the node UUID, uppercased).config,log,queryRead,queryWrite,carveInit→node.UUIDfrom theGetByKeylookup the handler already performs.
flags,cert,verify,script, quick enroll/remove,enroll package, osquery-config-endpoint) and
carveBlockare skipped in filtered mode,since they carry no node UUID to match against. They still dump in legacy (no-filter) mode.
Performance
The filter is designed so that the non-matching path (the overwhelming majority of traffic on a
busy server) adds only a single string comparison on top of the JSON parse and node lookup
the handler already does — no body re-read, no extra DB/Redis lookup, no dump formatting, no
file write. This removes the dominant cost of the current debug mode (
httputil.DumpRequestre-reading and formatting every request body) for all non-matching traffic. The matching path
pays the dump cost, but only for one host, so the debug file stays readable.
The filtered dump uses a new
DebugHTTPDumpWithBodyhelper that re-wraps the already-bufferedrequest body bytes, so the existing
httputil-based formatter serializes the request withoutre-reading the stream.
Behavior changes
everything" path runs at the top of each handler, including malformed/unparseable bodies).
json.Unmarshal) and invalidnode_keyrequestsare no longer dumped, since the node UUID cannot be determined. This is intentional: the goal
is to see traffic from one specific host, not anonymous/malformed traffic.
Files
pkg/config/types.go,pkg/config/flags.go— newTargetHostIdentifierfield + flag/env.pkg/utils/http-utils.go—DebugHTTPDumpWithBodyhelper.cmd/tls/handlers/handlers.go—shouldDebugHTTP/debugHTTPAllgate helpers.cmd/tls/handlers/post.go— gate top-of-handler dumps and add filtered dumps on the sixnode-bearing endpoints.
deploy/config/tls.yml— document the newhostIdentifierfield.cmd/tls/handlers/debug_http_filter_test.go,pkg/utils/http-utils_test.go.Validation
go build ./...— passes.go vet ./cmd/tls/handlers/...— clean.go test ./cmd/tls/handlers/— passes, including newTestShouldDebugHTTP*(disabled,no-filter/legacy, filtered match/mismatch/unknown-node, nil-config) and
TestDebugHTTPDumpWithBody(show-body / omit-body).TestSendRequestinpkg/utilsfails in the sandbox because it binds anetwork port (disallowed here); unrelated to this change.
Usage
or
Follow-ups (not in this PR)
carveBlockis excluded from the filtered path (it carriessession_id, notnode_key).Wiring it in requires a
NodeManager.GetByIDto resolvecarve.NodeID→ UUID.settings-refresh goroutine in
cmd/tls/main.govia asettings.DebugHostFilterentry.