Skip to content

[common] Collect min and max stats for binary columns - #8847

Open
ArnavBalyan wants to merge 3 commits into
apache:masterfrom
ArnavBalyan:arnavb/fix-bin-stat
Open

[common] Collect min and max stats for binary columns#8847
ArnavBalyan wants to merge 3 commits into
apache:masterfrom
ArnavBalyan:arnavb/fix-bin-stat

Conversation

@ArnavBalyan

Copy link
Copy Markdown
Member

Purpose

  • BINARY and VARBINARY columns don't have min/max stats when collected in memory (e.g. avro).
  • However the reader already has a comparator for byte[] which can prune the binary and varbin if the stats are provided.
  • Ensure we can pass valid statistics for binary/varbinary so that the reader can benefit from pruning/file skip.

Tests

  • UT

@JingsongLi
JingsongLi marked this pull request as draft July 26, 2026 07:10
@JingsongLi

Copy link
Copy Markdown
Contributor

Please keep this draft until CI passed.

@ArnavBalyan
ArnavBalyan force-pushed the arnavb/fix-bin-stat branch from 05490f0 to 28c1d53 Compare July 26, 2026 12:09
@ArnavBalyan
ArnavBalyan marked this pull request as ready for review July 27, 2026 06:33
@ArnavBalyan

Copy link
Copy Markdown
Member Author

cc @JingsongLi could you please take a look when possible thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants