Skip to content

Commit 39a2b71

Browse files
authored
Add additional Arrow type support (#9291)
# Which issue does this PR close? - Closes #9290 # Rationale for this change **NOTE TO REVIEWERS:** Over 1500 lines of this diff are tests. `arrow-avro` currently cannot encode/decode a number of Arrow `DataType`s, and some types have schema/encoding mismatches that can lead to incorrect data (even when encoding succeeds). The goal is: * **No more `ArrowError::NotYetImplemented` (or similar) when writing/reading an Arrow `RecordBatch` containing supported Arrow types**, excluding **Sparse Unions** (will be handled separately). * **When compiled with `feature = "avro_custom_types"`:** Arrow to Avro to Arrow should **round-trip the Arrow `DataType`** (including width/signedness/time units and relevant metadata using **Arrow-specific custom logical types** following the established `arrow.*` pattern. * **When compiled without `avro_custom_types`:** Arrow types should be encoded to the **closest standard Avro primitive / logical type**, with any necessary lossy conversions documented and consistently applied. # What changes are included in this PR? Implementation of all existing missing `arrow-avro` types except for Sparse Unions # Are these changes tested? Yes # Are there any user-facing changes? Yes, additional type support is being added which is user-facing.
1 parent d8946ca commit 39a2b71

6 files changed

Lines changed: 3240 additions & 142 deletions

File tree

arrow-avro/Cargo.toml

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,7 @@ apache-avro = "0.21.0"
9595
num-bigint = "0.4"
9696
object_store = { version = "0.13", default-features = false, features = ["fs"] }
9797
once_cell = "1.21.3"
98+
half = { version = "2.1", default-features = false }
9899
tokio = { version = "1.0", default-features = false, features = ["macros", "rt-multi-thread", "io-util", "fs"] }
99100

100101
[[bench]]

0 commit comments

Comments
 (0)