Skip to content

[common] Add V2 batch format with statistics collection for filter pushdown #2885

@platinumhamburg

Description

@platinumhamburg

Search before asking

  • I searched in the issues and found nothing similar.

Motivation

Add V2 log record batch format that collects per-column min/max statistics to enable efficient filter│pushdown.

Solution

V2 batch layout: [Header (56 bytes)] [Statistics (variable)] [Records (variable)]. See : https://cwiki.apache.org/confluence/display/FLUSS/FIP-10%3A+Support+Log+RecordBatch+Filter+Pushdown

Anything else?

No response

Willingness to contribute

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions