Skip to content

Implement ArrowWriter pooling #444

@fresh-borzoni

Description

@fresh-borzoni

Search before asking

  • I searched in the issues and found nothing similar.

Description

Java pools ArrowWriter instances per table/schema via ArrowWriterPool. Reused writers keep their allocated vectors and seed the fullness threshold from the previous batch (recordsCount / 2), avoiding reallocation and the first-record size check on every new batch. We currently create a fresh MemoryLogRecordsArrowBuilder per batch.

Willingness to contribute

  • I'm willing to submit a PR!

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions