Commit 39a2b71
authored
Add additional Arrow type support (#9291)
# Which issue does this PR close?
- Closes #9290
# Rationale for this change
**NOTE TO REVIEWERS:** Over 1500 lines of this diff are tests.
`arrow-avro` currently cannot encode/decode a number of Arrow
`DataType`s, and some types have schema/encoding mismatches that can
lead to incorrect data (even when encoding succeeds).
The goal is:
* **No more `ArrowError::NotYetImplemented` (or similar) when
writing/reading an Arrow `RecordBatch` containing supported Arrow
types**, excluding **Sparse Unions** (will be handled separately).
* **When compiled with `feature = "avro_custom_types"`:** Arrow to Avro
to Arrow should **round-trip the Arrow `DataType`** (including
width/signedness/time units and relevant metadata using **Arrow-specific
custom logical types** following the established `arrow.*` pattern.
* **When compiled without `avro_custom_types`:** Arrow types should be
encoded to the **closest standard Avro primitive / logical type**, with
any necessary lossy conversions documented and consistently applied.
# What changes are included in this PR?
Implementation of all existing missing `arrow-avro` types except for
Sparse Unions
# Are these changes tested?
Yes
# Are there any user-facing changes?
Yes, additional type support is being added which is user-facing.1 parent d8946ca commit 39a2b71
6 files changed
Lines changed: 3240 additions & 142 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
95 | 95 | | |
96 | 96 | | |
97 | 97 | | |
| 98 | + | |
98 | 99 | | |
99 | 100 | | |
100 | 101 | | |
| |||
0 commit comments