Skip to content

Commit fa8091e

Browse files
authored
refactor(read): split ArrowReader into DataFileReader, DataEvolutionReader and TableRead (#246)
Move the monolithic arrow/reader.rs into three focused modules under table/: - data_file_reader.rs: regular Parquet file reading with schema evolution - data_evolution_reader.rs: column-merge reading for data evolution mode - table_read.rs: TableRead extracted from read_builder.rs Also replaces assert! with proper error returns and adds bounds checking in attach_row_id.
1 parent bda2c7e commit fa8091e

8 files changed

Lines changed: 1036 additions & 1047 deletions

File tree

crates/paimon/src/arrow/mod.rs

Lines changed: 0 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -17,11 +17,8 @@
1717

1818
pub(crate) mod filtering;
1919
pub(crate) mod format;
20-
mod reader;
2120
pub(crate) mod schema_evolution;
2221

23-
pub use crate::arrow::reader::ArrowReaderBuilder;
24-
2522
use crate::spec::{
2623
ArrayType, BigIntType, BooleanType, DataField, DataType as PaimonDataType, DateType,
2724
DecimalType, DoubleType, FloatType, IntType, LocalZonedTimestampType, MapType, RowType,

crates/paimon/src/arrow/reader.rs

Lines changed: 0 additions & 945 deletions
This file was deleted.

crates/paimon/src/table/data_evolution_reader.rs

Lines changed: 455 additions & 0 deletions
Large diffs are not rendered by default.

0 commit comments

Comments
 (0)