feat(audit): consume signed events, classify integrity_status (PR2) - #5
Merged
Merged
Conversation
PR2 of the audit:events integrity remediation (PR1 #4, merged: the sign/verify helper, previously unwired anywhere). Wires classification into the worker without requiring any producer to sign yet -- every producer today writes only {"data": ...}, no `sig` field, and this PR keeps that fully compatible: unsigned traffic persists and ACKs exactly as before, now explicitly tagged integrity_status="unsigned" rather than untagged. AuditConfig.EVENT_SIGNING_SECRET reuses the existing JWT_SECRET env var and "change-me" fallback -- same convention audit/jwt_verify.py already uses, no new secret mechanism. audit/jwt_verify.py itself is untouched. consumers/processor.py::classify_event_integrity(service, signature, data, secret) is a new, separate function -- parse_audit_event()'s signature and its 3 existing callers are unchanged. The missing-signature check happens before verify_audit_event() is ever called (verified by a dedicated ordering test), and verification is always performed against the exact raw fields["data"] string, never a re-serialization of the parsed event -- audit/signing.py's own MAC covers the exact transmitted bytes, and reusing it unmodified was a hard constraint of this PR. worker/main.py::handle_message now classifies after a successful parse and persists the result via consumers/sink.py's new, purely additive integrity_status=event.get("integrity_status", "unsigned") lookup. Invalid/malformed signatures are persisted and ACKed, not left pending: retrying a signature verdict can never change it, so treating it like a transient parse/DB failure would only accumulate a permanent poison-pill pending entry, whereas persisting it preserves the forged attempt as durable evidence. A distinct "SIGNATURE INVALID" print fires only for that case; a dedicated test proves it never includes the secret, the signature, or the raw event data. Parse failures, DB failures, and duplicate-event_id dedup are all unchanged -- confirmed by re-running the pre-existing tests for those paths unmodified, byte-for-byte, alongside the new ones. db/models.py adds exactly one column, integrity_status (String(16), NOT NULL, server_default="unsigned"), via new migration 0002_integrity_status (down_revision=0001_audit_events, revision id 21 chars -- under the 32-char limit the installed alembic's alembic_version.version_num column enforces). No index: nothing queries this column yet, and the table has none on any column today, including ones already actively filtered on. Migration tests prove a row inserted under 0001, before this column existed, backfills to "unsigned" on upgrade to head -- not just that the column exists -- and that downgrading exactly one revision (head -> 0001) removes only this column, distinct from the existing full-downgrade-to-base test. 32 new tests: 12 classification (valid/invalid/malformed/missing/empty, service+data+signature mismatches, exact-raw-string preservation via a reordered-JSON-keys case, ordering of the missing-check before verify_audit_event()), 3 migration, 4 sink, 10 worker (including the secret/signature/data non-leakage security test), 3 real Redis+MySQL integration tests extending the existing PR-B0 fixture -- signing with whatever AuditConfig.EVENT_SIGNING_SECRET actually resolves to in the running test process rather than assuming which value it holds. Full suite 423 passed (391 baseline + 32 new), 0 regressions. ruff: 16 findings, all pre-existing or matching 0001's own established Union/ Sequence migration-file style; 0 new avoidable findings. git diff --check clean. Follow-up, deliberately not fixed here: the dev-compose security-audit-worker container currently has JWT_SECRET unset (falls back to "change-me") while other platform services have the real shared secret configured. Harmless today since no producer signs yet, but must be corrected before any producer-signing rollout, or valid producer signatures will be classified as invalid. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PR2 of the
audit:eventsintegrity remediation (PR1 #4, merged — the sign/verify helper, previously unwired anywhere). Wires classification into the worker without requiring any producer to sign yet.What changes
AuditConfig.EVENT_SIGNING_SECRET— reuses the existingJWT_SECRETenv var and"change-me"fallback, same conventionaudit/jwt_verify.pyalready uses. No new secret mechanism.audit/jwt_verify.pyitself is untouched.consumers/processor.py::classify_event_integrity(service, signature, data, secret) -> str— new, separate function.parse_audit_event()'s signature and its 3 existing callers are unchanged.worker/main.py::handle_message— classifies after a successful parse, using the exact rawfields["data"]string (never a re-serialization), persists via a purely additiveconsumers/sink.pylookup, then ACKs.db/models.py+alembic/versions/0002_integrity_status.py— exactly one new column,integrity_status(String(16),NOT NULL,server_default="unsigned"). No index — nothing queries it yet, and the table has none on any column today.Classification behavior
The missing-signature check happens before
verify_audit_event()— verified by a dedicated ordering test, not just asserted.ACK/retry behavior
"valid""unsigned""invalid"SIGNATURE INVALIDprintInvalid signatures are persisted and ACKed rather than left pending: a signature verdict can't change on retry, so leaving it pending would only accumulate a permanent poison-pill entry, whereas persisting it preserves the forged attempt as durable evidence. A dedicated test proves the distinct log line never includes the secret, the signature, or the raw event data.
Migration
0002_integrity_status,down_revision="0001_audit_events"(21-char revision id, under the installed alembic's 32-charalembic_version.version_numlimit — verified against source). Tests prove: a row inserted before this migration existed backfills to"unsigned"on upgrade (real backfill viaserver_default, not just schema presence), and downgrading exactly one revision removes only this column, distinct from the existing full-downgrade-to-basetest.Tests — 32 new
12 classification (valid/invalid/malformed/missing/empty, service+data+signature mismatches, exact-raw-string preservation via a reordered-JSON-keys case, missing-check ordering), 3 migration, 4 sink, 10 worker (including the secret/signature/data non-leakage security test), 3 real Redis+MySQL integration tests extending the existing PR-B0 fixture — signing with whatever
AuditConfig.EVENT_SIGNING_SECRETactually resolves to in the running test process, never assuming which value it holds.Compatibility
Every existing producer keeps working unmodified — none write
sigtoday; the real integration test proves today's exact traffic shape ({"data": ...}, nosig) classifies as"unsigned"and persists/ACKs against real Redis+MySQL. No API Gateway, Control Center, or other-repo changes.audit/signing.py— zero-line diff.Follow-up (deliberately not fixed here)
The dev-compose
security-audit-workercontainer currently hasJWT_SECRETunset (falls back to"change-me") while other platform services have the real shared secret configured. Harmless today since no producer signs yet, but must be corrected before any producer-signing rollout, or valid producer signatures will be classified as invalid.🤖 Generated with Claude Code