feat: Add CONTAINERSIZE segment metadata analysis type - #19847
Open
cecemei wants to merge 2 commits into
Open
Conversation
Adds SegmentMetadataQuery.AnalysisType.CONTAINERSIZE, which reports each V10 segment file container's owning bundle name and on-disk byte size via SegmentAnalysis.getContainers(). Only populated for segments written in the V10 file format; pre-V10 segments report null, consistent with how other analysis types handle segments that predate them. - QueryableIndex.getFileContainers() (default null) sources this from SegmentFileMapper.getSegmentFileMetadata(), implemented by SimpleQueryableIndex/PartialQueryableIndex; SmooshedFileMapper (legacy) explicitly returns null since it has no container/bundle structure. - Merging sums sizes by bundle name rather than concatenating raw per-segment container lists, since a container has no identity across segments; single-sided merges pass the non-null side through unchanged. - SegmentAnalysis.ContainerAnalysis is nested (not top-level) since it has no identity outside that one field, and SegmentAnalysis.Builder gained bulk setters so production call sites don't need the raw constructor. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Removes a redundant parenthetical and condenses the explanation of why one-sided merges return unchanged rather than passing through the per-bundle collapse. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a new
SegmentMetadataQueryanalysis type,CONTAINERSIZE, that reports the on-disk byte size of each segment's internal storage containers, broken down by the base table and each aggregate projection. This lets a user (or an internal tool) inspect how much storage an individual aggregate projection is adding to a segment, on top of the base table it's derived from.When
CONTAINERSIZEis requested,SegmentAnalysis.getContainers()returns a list of(bundle, size)pairs, one per physical storage container in the segment's V10 file, wherebundleis either the base table or a projection's name. This is only available for segments written in the V10 segment file format; segments written in the (currently default) V9/legacy format reportnullhere.A few implementation notes:
QueryableIndex.getFileContainers()is the new extension point sourcing this data, implemented bySimpleQueryableIndex/PartialQueryableIndexviaSegmentFileMapper.getSegmentFileMetadata();SmooshedFileMapper(legacy) explicitly returnsnullsince the legacy format has no container/bundle structure to report.Release note
Added a new
CONTAINERSIZEsegment metadata analysis type that reports the on-disk byte size of each segment's storage containers, broken down by the base table and each aggregate projection. This makes it possible to see how much storage an individual projection is adding to a segment. Only available for segments written in the V10 segment file format.Key changed/added classes in this PR
SegmentMetadataQuerySegmentAnalysisSegmentAnalysis.ContainerAnalysisSegmentMetadataQueryRunnerFactorySegmentMetadataQueryQueryToolChestQueryableIndexThis PR has: