Fixes #28770: write server audit entries to audit.log#28771
Conversation
Audit-to-file logging silently broke in #23733: gutting AuditEventHandler removed the only emitter of the AUDIT log marker, so the audit.log appender (audit-only-filter-factory) creates the file but never writes to it. Re-emit each audit entry through the AUDIT-marked logger from AuditLogRepository.write(), gated on a successful (non-duplicate) DB insert so the publisher+consumer dual-write path does not double-log. Emit a compact one-line summary at INFO rather than the full ChangeEvent JSON. Harden conf/openmetadata.yaml: pin the audit logger to INFO so capture is independent of LOG_LEVEL, and exclude audit lines from the console appender so they land only in audit.log. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
✅ PR checks passedThe linked issue has a description and all required Shipping project fields set. Thanks! |
🔴 Playwright Results — 1 failure(s), 14 flaky✅ 4274 passed · ❌ 1 failed · 🟡 14 flaky · ⏭️ 88 skipped
Genuine Failures (failed on all attempts)❌
|
Resolved the add/add conflict in AuditLogRepositoryTest.java by combining this branch's audit-marker emission + dedup-gate tests with the lineage-persistence test added in #28426. AuditLogRepository.java and CollectionDAO.java auto-merged cleanly: main's lineage event types and IntakeForm DAO coexist with the insert-gated audit-to-file logging. Verified: 16/16 audit/logging tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Code Review ✅ ApprovedRestores audit logging by re-emitting audit entries through an AUDIT-marked logger within AuditLogRepository.write(), using row-affected gating to ensure exactly-once file writes without duplicate logging. OptionsDisplay: compact → Showing less information. Comment with these commands to change:
Was this helpful? React with 👍 / 👎 | Gitar |
|
Failed to cherry-pick changes to the 1.13 branch. |
|
Failed to cherry-pick changes to the 1.12.11 branch. |
Resolve 2 conflicts from the CollectionDAO/EntityRepository split meeting two upstream fixes: - #28771 (server audit entries to audit.log) changed the audit-log DAO insert to return int; that DAO moved to ActivityAuditDAOs in this branch, so ported the void->int signature there (AuditLogRepository now reads the affected-row count). Took the restructured CollectionDAO (--ours). - #28720 (lost-update race fix in PolicyConditionUpdater) rewrote the inline rewrite loop into rewriteMatchingPolicies/rewriteSinglePolicy/casWritePolicy with a FOR UPDATE + CAS retry. Took the upstream rewrite and re-applied this branch's cache-invalidation redirect (EntityRepository.invalidateCacheForEntity -> EntityCacheInvalidator.invalidateCacheForEntity) inside casWritePolicy; kept both the EntityDAO (upstream) and EntityCacheInvalidator (ours) imports. Service (main+test) and integration-tests compile; CollectionDAOCompositionTest, AuditLogRepositoryTest (3), and PolicyConditionUpdaterTest (41) pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>



Describe your changes:
Fixes #28770
Server audit logging to
audit.logsilently broke in #23733: guttingAuditEventHandlerremoved the only emitter of theAUDITSLF4J marker, so theaudit.logappender (guarded byaudit-only-filter-factory) still creates the file at startup but never writes a line to it. I re-emit each audit entry through theAUDIT-marked logger fromAuditLogRepository.write()— the single chokepoint all audit records flow through — gated on a successful, non-duplicate DB insert so the publisher + consumer dual-write path (and multi-server replays) can't double-log. The emitted line is a compact one-line summary atINFO, not the fullChangeEventJSON (which can be 1–2 MB/event).Type of change:
High-level design:
AuditLogDAO.insertnow returns rows-affected (int); since the insert is idempotent (ON CONFLICT DO NOTHING/INSERT IGNORE), the file line is emitted only when> 0, giving exactly-once file logging that matches the DB's exactly-once semantics. Config hardening inconf/openmetadata.yaml: pin the audit logger toINFOso capture is independent of the globalLOG_LEVEL, and addaudit-exclude-filter-factoryto the console appender so audit lines land only inaudit.log. The DB-backed audit subsystem (REST API + UI) is unchanged.Tests:
Use cases covered
audit.log.Unit tests
openmetadata-service/src/test/java/org/openmetadata/service/audit/AuditLogRepositoryTest.java— asserts theAUDITmarker is emitted on insert, and suppressed when the insert is de-duplicated (gate). Pure unit tests (mocked DAO + in-memory logbackListAppender), no DB/container.Backend integration tests
AuditLogResourceITcovers the DB-backed path).Ingestion integration tests
Playwright (UI) tests
Manual testing performed
mvn -pl openmetadata-service test -Dtest=AuditLogRepositoryTest— 2/2 pass; observed theINFO+AUDITsummary line emitted on insert and suppressed on dedup.mvn -pl openmetadata-service test -Dtest=LoggingConfigurationYamlTest— 2/2 pass; confirms the editedconf/openmetadata.yamlparses in both text and JSON logging modes.mvn spotless:apply -pl openmetadata-service— clean.UI screen recording / screenshots:
Not applicable.
Checklist:
Fixes <issue-number>: <short explanation>Fixes #<issue-number>above.