Skip to content

fix: secure temporary files and resource lookups - #19822

Open
FrankChen021 wants to merge 2 commits into
apache:masterfrom
FrankChen021:codex/codeql-warning-temp-lookup
Open

fix: secure temporary files and resource lookups#19822
FrankChen021 wants to merge 2 commits into
apache:masterfrom
FrankChen021:codex/codeql-warning-temp-lookup

Conversation

@FrankChen021

Copy link
Copy Markdown
Member

What

  • replace insecure default temporary-file creation with owner-private NIO or test-managed temporary locations
  • make test resource lookup use declaring-class literals and close resource streams
  • preserve portable getconf PATH lookup with a narrow CodeQL suppression and rationale
  • strengthen temporary archive cleanup in the OSS, S3, and Cassandra segment pushers

Why

This addresses the 55 CodeQL warnings in the temporary-file and resource-lookup group:

  • 46 java/local-temp-file-or-directory-information-disclosure
  • 8 java/unsafe-get-resource
  • 1 java/relative-path-command

Impact

Temporary files are created with restrictive permissions or inside managed private directories. Resource lookup no longer depends on runtime subclass dispatch, and affected tests can load resources when packaged in JARs.

Root cause

The affected code used legacy File.createTempFile, runtime-class resource lookup, and one intentionally portable executable lookup that CodeQL could not distinguish from an attacker-controlled command.

Checks

  • affected 21-module reactor test compilation and Checkstyle passed
  • production-focused tests: 24 passed
  • processing-focused tests: 174 passed
  • cross-module focused tests: 100 passed
  • git diff --check

Created by GPT-5.6-Sol.

Process p = new ProcessBuilder("getconf", "CLK_TCK").start();
// getconf has no portable absolute location; the fixed command and argument intentionally use the service PATH.
// codeql[java/relative-path-command]
final Process p = new ProcessBuilder("getconf", "CLK_TCK").start();
@FrankChen021
FrankChen021 marked this pull request as ready for review July 30, 2026 17:56
Copilot AI review requested due to automatic review settings July 30, 2026 17:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR hardens temporary file creation and classpath resource loading to address CodeQL findings across Druid’s test suite and several deep-storage segment pushers, while preserving a portable getconf lookup with a narrowly scoped suppression.

Changes:

  • Replaced File.createTempFile usages with test-managed temp locations (TemporaryFolder, JUnit5 @TempDir) or Druid FileUtils.createTempDir, and improved cleanup paths.
  • Updated resource loading to use declaring-class literals and (in most places) close streams via try-with-resources.
  • Switched several segment pushers and cache components to NIO temp file creation and ensured temp artifacts are deleted.

Reviewed changes

Copilot reviewed 34 out of 34 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
sql/src/test/java/org/apache/druid/sql/calcite/BaseCalciteQueryTest.java Uses declaring-class resource lookup and try-with-resources when copying resources to temp files.
server/src/test/java/org/apache/druid/segment/loading/LocalDataSegmentPullerTest.java Uses JUnit-managed temp files instead of File.createTempFile.
server/src/test/java/org/apache/druid/metadata/input/SqlInputSourceTest.java Replaces custom temp-dir tracking/cleanup with TemporaryFolder.
server/src/test/java/org/apache/druid/metadata/input/SqlEntityTest.java Uses TemporaryFolder for temp files and closes file streams explicitly.
processing/src/test/java/org/apache/druid/segment/TestIndex.java Uses FileUtils.createTempDir for merge/persist temp directories.
processing/src/test/java/org/apache/druid/segment/SchemalessIndexTest.java Uses FileUtils.createTempDir for multiple temp directories used in index persistence/merge paths.
processing/src/test/java/org/apache/druid/segment/IndexMergerV9WithSpatialIndexTest.java Switches temp dirs to FileUtils.createTempDir and consolidates cleanup to delete the root temp dir.
processing/src/test/java/org/apache/druid/segment/filter/SpatialFilterTest.java Uses FileUtils.createTempDir and deleteOnExit for temp dirs used during persist/merge.
processing/src/test/java/org/apache/druid/segment/filter/SpatialFilterBonusTest.java Same temp-dir hardening approach as SpatialFilterTest.
processing/src/test/java/org/apache/druid/segment/EmptyIndexTest.java Uses FileUtils.createTempDir and deletes it in a finally block.
processing/src/test/java/org/apache/druid/query/DoubleStorageTest.java Uses TemporaryFolder and passes an explicit index directory into helper method.
processing/src/test/java/org/apache/druid/java/util/common/io/smoosh/SmooshedFileMapperTest.java Uses JUnit @TempDir folder-scoped files instead of global temp files.
processing/src/test/java/org/apache/druid/frame/FrameTest.java Uses a temp directory per test and cleans it via Closer with FileUtils.deleteDirectory.
processing/src/test/java/org/apache/druid/data/input/impl/prefetch/JsonIteratorTest.java Uses JUnit5 @TempDir for deterministic, test-managed temp files.
processing/src/main/java/org/apache/druid/java/util/metrics/CgroupCpuMonitor.java Adds rationale and narrow CodeQL suppression for portable getconf PATH lookup.
extensions-core/s3-extensions/src/main/java/org/apache/druid/storage/s3/S3DataSegmentPusher.java Uses NIO temp zip creation and ensures deletion in a finally block.
extensions-core/lookups-cached-global/src/main/java/org/apache/druid/server/lookup/namespace/cache/OffHeapNamespaceExtractionCacheManager.java Uses NIO temp file creation for MapDB backing file.
extensions-core/kinesis-indexing-service/src/test/java/org/apache/druid/indexing/kinesis/KinesisIndexTaskTest.java Uses test-managed temp file for reports output.
extensions-core/kafka-indexing-service/src/test/java/org/apache/druid/indexing/kafka/KafkaIndexTaskTest.java Uses test-managed temp file for reports output.
extensions-core/hdfs-storage/src/test/java/org/apache/druid/storage/hdfs/HdfsDataSegmentPullerTest.java Uses FileUtils.createTempDir and folder-scoped input file creation.
extensions-core/hdfs-storage/src/test/java/org/apache/druid/segment/loading/HdfsFileTimestampVersionFinderTest.java Same temp-dir and input-file hardening as HDFS puller test.
extensions-core/google-extensions/src/main/java/org/apache/druid/storage/google/GoogleDataSegmentPusher.java Uses NIO temp zip creation and deletes it in finally.
extensions-core/azure-extensions/src/main/java/org/apache/druid/storage/azure/AzureDataSegmentPusher.java Uses NIO temp zip creation and deletes it in finally.
extensions-contrib/moving-average-query/src/test/java/org/apache/druid/query/movingaverage/MovingAverageQueryTest.java Uses declaring-class resource lookup, non-null enforcement, and try-with-resources.
extensions-contrib/graphite-emitter/src/test/java/org/apache/druid/emitter/graphite/WhiteListBasedConverterTest.java Switches to declaring-class resource lookup and test-managed temp file.
extensions-contrib/compressed-bigdecimal/src/test/java/org/apache/druid/compressedbigdecimal/aggregator/CompressedBigDecimalAggregatorTimeseriesTestBase.java Closes resource streams and copies resources into temp files safely.
extensions-contrib/compressed-bigdecimal/src/test/java/org/apache/druid/compressedbigdecimal/aggregator/CompressedBigDecimalAggregatorGroupByTestBase.java Closes resource streams and enforces non-null resource lookup.
extensions-contrib/cloudfiles-extensions/src/main/java/org/apache/druid/storage/cloudfiles/CloudFilesDataSegmentPusher.java Uses NIO temp files for descriptor and zipped index creation.
extensions-contrib/cassandra-storage/src/main/java/org/apache/druid/storage/cassandra/CassandraDataSegmentPusher.java Uses NIO temp zip creation, wraps upload in try, and always deletes the temp zip.
extensions-contrib/ambari-metrics-emitter/src/test/java/org/apache/druid/emitter/ambari/metrics/WhiteListBasedDruidToTimelineEventConverterTest.java Switches to declaring-class resource lookup and test-managed temp file.
extensions-contrib/aliyun-oss-extensions/src/main/java/org/apache/druid/storage/aliyun/OssDataSegmentPusher.java Uses NIO temp zip creation and ensures deletion in finally.
benchmarks/src/test/java/org/apache/druid/benchmark/indexing/IndexMergeBenchmark.java Uses FileUtils.createTempDir and deletes the directory in finally.
benchmarks/src/test/java/org/apache/druid/benchmark/GenericIndexedBenchmark.java Uses a folder-scoped metadata file instead of a global temp file.
benchmarks/src/test/java/org/apache/druid/benchmark/FrontCodedIndexedBenchmark.java Uses folder-scoped metadata files instead of global temp files.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread sql/src/test/java/org/apache/druid/sql/calcite/BaseCalciteQueryTest.java Outdated

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 34 out of 34 changed files in this pull request and generated no new comments.

@FrankChen021
FrankChen021 marked this pull request as draft July 30, 2026 23:58
@FrankChen021
FrankChen021 marked this pull request as ready for review July 31, 2026 00:18
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants