[flink] Expose scan.bucket for single-bucket manifest pruning#8117
[flink] Expose scan.bucket for single-bucket manifest pruning#8117wwj6591812 wants to merge 1 commit into
Conversation
17c2722 to
7e8c5d8
Compare
|
The failed test is not related to my modifications. |
|
The validation here still allows
Could we enforce that here, e.g. require the fixed bucket mode and configured bucket count > 0, and also check primary-key-ness if the option is intended only for primary-key tables? |
Background
ReadBuilder.withBucket(int) and manifest scanning already support reading a single bucket, but Flink SQL had no connector option to expose it. Operators often need to debug or scan one bucket of a fixed-bucket primary-key table without reading all buckets.
Why this PR
Expose scan.bucket in Flink so users can run:
SELECT * FROM t /*+ OPTIONS('scan.bucket' = '0') */
and plan splits only for that bucket.
What changes
Stage optimized: scan / manifest planning — fewer manifest entries and splits before read. No change to merge or per-record logic.
Tests
Test plan