Skip to content

Per-host filtering for the osctrl-tls HTTP debug dump#882

Merged
javuto merged 1 commit into
developfrom
debug-uuid-specific
Jul 6, 2026
Merged

Per-host filtering for the osctrl-tls HTTP debug dump#882
javuto merged 1 commit into
developfrom
debug-uuid-specific

Conversation

@javuto

@javuto javuto commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Problem

osctrl-tls can dump every incoming HTTP request to a debug log (--enable-http-debug /
HTTP_DEBUG), but the toggle is all-or-nothing: either every request from every node is
dumped, or none are. On a busy server this is a firehose — the operator cannot isolate the
traffic of a single host, which is the only case where request-level debugging is useful.

Change

Add an optional host filter to the existing HTTP debug feature. The filter is the osquery
node UUID (case-insensitive), configured via a new hostIdentifier field on the debug
config block, or --http-debug-host / HTTP_DEBUG_HOST.

  • debug.hostIdentifier empty (default) → behavior is identical to today: every request is
    dumped when enableHttp is true. No semantic change for existing deployments.
  • debug.hostIdentifier set → only requests whose node UUID matches are dumped. The expensive
    dump work (httputil.DumpRequest + lumberjack write) runs only for the matching host.

The matching is keyed on the canonical node UUID:

  • enroll → the request's host_identifier (which becomes the node UUID, uppercased).
  • config, log, queryRead, queryWrite, carveInitnode.UUID from the GetByKey
    lookup the handler already performs.
  • Pre-enroll / no-node endpoints (flags, cert, verify, script, quick enroll/remove,
    enroll package, osquery-config-endpoint) and carveBlock are skipped in filtered mode,
    since they carry no node UUID to match against. They still dump in legacy (no-filter) mode.

Performance

The filter is designed so that the non-matching path (the overwhelming majority of traffic on a
busy server) adds only a single string comparison on top of the JSON parse and node lookup
the handler already does — no body re-read, no extra DB/Redis lookup, no dump formatting, no
file write. This removes the dominant cost of the current debug mode (httputil.DumpRequest
re-reading and formatting every request body) for all non-matching traffic. The matching path
pays the dump cost, but only for one host, so the debug file stays readable.

The filtered dump uses a new DebugHTTPDumpWithBody helper that re-wraps the already-buffered
request body bytes, so the existing httputil-based formatter serializes the request without
re-reading the stream.

Behavior changes

  • No filter → byte-for-byte identical to the previous implementation (the legacy "dump
    everything" path runs at the top of each handler, including malformed/unparseable bodies).
  • Filter set → malformed bodies (failed json.Unmarshal) and invalid node_key requests
    are no longer dumped, since the node UUID cannot be determined. This is intentional: the goal
    is to see traffic from one specific host, not anonymous/malformed traffic.

Files

  • pkg/config/types.go, pkg/config/flags.go — new TargetHostIdentifier field + flag/env.
  • pkg/utils/http-utils.goDebugHTTPDumpWithBody helper.
  • cmd/tls/handlers/handlers.goshouldDebugHTTP / debugHTTPAll gate helpers.
  • cmd/tls/handlers/post.go — gate top-of-handler dumps and add filtered dumps on the six
    node-bearing endpoints.
  • deploy/config/tls.yml — document the new hostIdentifier field.
  • Tests: cmd/tls/handlers/debug_http_filter_test.go, pkg/utils/http-utils_test.go.

Validation

  • go build ./... — passes.
  • go vet ./cmd/tls/handlers/... — clean.
  • go test ./cmd/tls/handlers/ — passes, including new TestShouldDebugHTTP* (disabled,
    no-filter/legacy, filtered match/mismatch/unknown-node, nil-config) and
    TestDebugHTTPDumpWithBody (show-body / omit-body).
  • The pre-existing TestSendRequest in pkg/utils fails in the sandbox because it binds a
    network port (disallowed here); unrelated to this change.

Usage

debug:
  enableHttp: true
  httpFile: ./debug-http-tls.log
  showBody: false
  hostIdentifier: "DEADBEEF-1234-ABCD"   # only dump this node's traffic

or

HTTP_DEBUG=true HTTP_DEBUG_HOST=deadbeef-1234-abcd osctrl-tls ...

Follow-ups (not in this PR)

  • carveBlock is excluded from the filtered path (it carries session_id, not node_key).
    Wiring it in requires a NodeManager.GetByID to resolve carve.NodeID → UUID.
  • The filter is currently startup-only (flag/YAML). A runtime-toggleable version could reuse the
    settings-refresh goroutine in cmd/tls/main.go via a settings.DebugHostFilter entry.

@javuto javuto added ✨ enhancement New feature or request osctrl-tls osctrl-tls related changes ⚙️ configuration Configuration related issues labels Jul 6, 2026
@javuto javuto merged commit a291fd2 into develop Jul 6, 2026
8 checks passed
@javuto javuto deleted the debug-uuid-specific branch July 6, 2026 20:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⚙️ configuration Configuration related issues ✨ enhancement New feature or request osctrl-tls osctrl-tls related changes

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant