Skip to content

[VL] test(iceberg): Fix disabled iceberg row group bytes test#12509

Open
infvg wants to merge 1 commit into
apache:mainfrom
infvg:testFIx
Open

[VL] test(iceberg): Fix disabled iceberg row group bytes test#12509
infvg wants to merge 1 commit into
apache:mainfrom
infvg:testFIx

Conversation

@infvg

@infvg infvg commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Previously, the row group bytes value was a hard boundary of the uncompressed value. Now the implementation causes the buffer to reach or slightly exceed the threshold.

@github-actions github-actions Bot added the VELOX label Jul 14, 2026
@infvg infvg changed the title test(iceberg): Fix disabled iceberg row group bytes test [VL] test(iceberg): Fix disabled iceberg row group bytes test Jul 14, 2026
@zhouyuan

Copy link
Copy Markdown
Member

related #12494

@JkSelf

JkSelf commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

@infvg @zhouyuan

This issue was introduced by a recent PR where the code writes the batch before checking if the buffer size exceeds bytesInRowGroup, causing the row group to exceed the limit.

Unlike Velox, the parquet-java reference implementation strictly ensures the row group size never exceeds bytesInRowGroup. Since Velox's current behavior deviates from this, I've opened a community issue to discuss how to define this in Velox. We should wait for that decision before making any changes here, as a fix on the Velox side would be preferred over patching it in Gluten.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants