Commit dc9c3cd
authored
[Variant] Implement row builders for cast_to_variant (#8299)
Note to reviewers: This PR includes 1600+ LoC of new unit tests. The
actual changes are half that big.
# Which issue does this PR close?
We generally require a GitHub issue to be filed for all bug fixes and
enhancements and this helps us generate change logs for our releases.
You can link an issue to this PR using the GitHub syntax.
- Closes #8310
# Rationale for this change
The original `cast_to_variant` code did columnar conversions of types to
variant. For primitive types this worked ok, but for deeply nested types
it means repeatedly creating new variants (and variant metadata), only
to re-code them by copying the variant values to new arrays (with new
metadata and field ids). Very expensive.
# What changes are included in this PR?
Follow the example of #8280, and
introduce a row builder concept that takes individual array values and
writes them to an `impl VariantBuilderExt`. Row builders for complex
types instantiate the appropriate list or object builder to pass to
their children.
# Are these changes tested?
Existing unit tests continue to pass. Extensive new unit tests added as
well.
# Are there any user-facing changes?
* `VariantBuilderExt` has a new `append_null` method.
* `ObjectFieldBuilder` moved to `builder.rs` and made public1 parent f751a45 commit dc9c3cd
7 files changed
Lines changed: 2151 additions & 737 deletions
File tree
- parquet-variant-compute/src
- parquet-variant-json/src
- parquet-variant/src
Large diffs are not rendered by default.
0 commit comments