Skip to content

WIP: Variant Type#6853

Draft
AdamGS wants to merge 8 commits intodevelopfrom
adamg/variant
Draft

WIP: Variant Type#6853
AdamGS wants to merge 8 commits intodevelopfrom
adamg/variant

Conversation

@AdamGS
Copy link
Contributor

@AdamGS AdamGS commented Mar 9, 2026

Just stashing results here to make the work more visible:

  1. Variant DType and Array
  2. ParquetVariant encoding
  3. VariantGet expression

@AdamGS AdamGS added the changelog/feature A new feature label Mar 9, 2026
AdamGS added 4 commits March 10, 2026 14:56
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
@codspeed-hq
Copy link

codspeed-hq bot commented Mar 10, 2026

Merging this PR will degrade performance by 20.89%

❌ 5 regressed benchmarks
✅ 995 untouched benchmarks
⏩ 1466 skipped benchmarks1

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation bench_many_nulls[0.9] 468.5 µs 533.8 µs -12.24%
Simulation patched_take_200k_first_chunk_only 4.8 ms 5.4 ms -10.69%
Simulation patched_take_200k_dispersed 4.7 ms 5.6 ms -16.59%
Simulation take_200k_first_chunk_only 3.3 ms 4.2 ms -20.89%
Simulation take_200k_dispersed 3.6 ms 4.5 ms -19.61%

Comparing adamg/variant (83828b2) with develop (1bb7e68)

Open in CodSpeed

Footnotes

  1. 1466 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

Signed-off-by: Adam Gutglick <adam@spiraldb.com>
/// [`I32`]: PType::I32
/// [`NonNullable`]: Nullability::NonNullable
#[derive(Debug, Clone, PartialEq, Eq, Hash)]
pub enum DType {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do you think we should make this #[non_exhaustive] ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think adding a new dtype should be very rare, but it is pretty painful to do right now.

Signed-off-by: Adam Gutglick <adam@spiraldb.com>
AdamGS added 2 commits March 10, 2026 17:00
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
Signed-off-by: Adam Gutglick <adam@spiraldb.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/feature A new feature

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants