Skip to content

fix: address CodeQL concurrency warnings - #19827

Open
FrankChen021 wants to merge 2 commits into
apache:masterfrom
FrankChen021:codex/codeql-warning-concurrency
Open

fix: address CodeQL concurrency warnings#19827
FrankChen021 wants to merge 2 commits into
apache:masterfrom
FrankChen021:codex/codeql-warning-concurrency

Conversation

@FrankChen021

Copy link
Copy Markdown
Member

What

  • wake every blocked reader when input is added, completed, or failed
  • prove terminal stream transitions release multiple waiting readers
  • synchronize MapDB check/use sequences on MapDB's own monitor

Why

This addresses all five CodeQL warnings in the concurrency group:

  • 4 java/notify-instead-of-notify-all
  • 1 java/toctou-race-condition

Impact

Terminal input-stream transitions can no longer leave additional readers blocked. MapDB close and cache disposal cannot interleave between isClosed() and the corresponding state-changing operation.

Root cause

The stream implementations relied on an implicit single-waiter assumption. The MapDB lifecycle method was synchronized on the anonymous handler instance rather than on the database object used by MapDB's own synchronized APIs.

Checks

  • affected-module compilation and Checkstyle passed
  • processing concurrency tests: 17 passed
  • lookup cache lifecycle/concurrency tests: 57 passed
  • git diff --check

Created by GPT-5.6-Sol.

@FrankChen021
FrankChen021 marked this pull request as ready for review July 30, 2026 19:25
Copilot AI review requested due to automatic review settings July 30, 2026 19:25

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Not ready to approve

notifyAll() in AppendableByteArrayInputStream.add() can enable concurrent readers to proceed and race on shared read state, and the MapDB manager changes need small follow-ups for lock scope and final field consistency.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Pull request overview

This PR updates several synchronization points to address CodeQL concurrency warnings (notify vs notifyAll, and a TOCTOU check/use pattern around MapDB lifecycle) in Druid’s processing and lookup-cache components.

Changes:

  • Switch monitor signaling from notify() to notifyAll() in stream/channel paths to ensure terminal transitions (and chunk/backpressure transitions) don’t leave additional waiters blocked.
  • Add tests proving AppendableByteArrayInputStream terminal transitions (done, exceptionCaught) unblock multiple waiting readers.
  • Synchronize MapDB “check then use” sequences (isClosed() + delete/close) on the DB instance monitor.
File summaries
File Description
processing/src/test/java/org/apache/druid/java/util/http/client/io/AppendableByteArrayInputStreamTest.java Adds multi-reader blocking/unblock tests for done() and exceptionCaught().
processing/src/main/java/org/apache/druid/java/util/http/client/io/AppendableByteArrayInputStream.java Changes monitor notification to notifyAll() in producer/terminal methods.
processing/src/main/java/org/apache/druid/frame/channel/ReadableInputStreamFrameChannel.java Uses notifyAll() when releasing backpressure waiters on readMonitor.
extensions-core/lookups-cached-global/src/main/java/org/apache/druid/server/lookup/namespace/cache/OffHeapNamespaceExtractionCacheManager.java Synchronizes MapDB check/use sequences on the DB instance and documents the rationale.
Review details

Comments suppressed due to low confidence (1)

extensions-core/lookups-cached-global/src/main/java/org/apache/druid/server/lookup/namespace/cache/OffHeapNamespaceExtractionCacheManager.java:162

  • Per the project code style (“Always use final for fields and variables that are not reassigned”, see AGENTS.md:40), these atomic field references should be final since they are never reassigned.
  private final DB mmapDB;
  private final File tmpFile;
  private AtomicLong mapDbKeyCounter = new AtomicLong(0);
  private AtomicInteger cacheCount = new AtomicInteger(0);
  • Files reviewed: 4/4 changed files
  • Comments generated: 2
  • Review effort level: Low

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟢 Ready to approve

The concurrency fixes are narrowly scoped, align with the PR’s stated CodeQL goals, and are supported by targeted multi-waiter tests for the terminal stream transitions.

This review doesn't count toward merge requirements. Sign up for the private preview to control whether Copilot approvals count.

Review details
  • Files reviewed: 4/4 changed files
  • Comments generated: 0 new
  • Review effort level: Lite

We're testing this review assessment. Please use 👍 or 👎 to tell us if it's correct.

@FrankChen021
FrankChen021 marked this pull request as draft July 30, 2026 23:57
@FrankChen021
FrankChen021 marked this pull request as ready for review July 31, 2026 01:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants