Replace ToCanonical trait usages with execute#8609
Conversation
Merging this PR will not alter performance
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | Simulation | extend_from_array_zctl[(1000, 8)] |
238.9 µs | 404.6 µs | -40.96% |
| ❌ | Simulation | extend_from_array_non_zctl_overlapping[(1000, 8)] |
271.3 µs | 436.3 µs | -37.81% |
| ❌ | Simulation | extend_from_array_non_zctl_overlapping[(1000, 32)] |
735.7 µs | 900.5 µs | -18.31% |
| ❌ | Simulation | chunked_varbinview_into_canonical[(1000, 10)] |
169.1 µs | 205.6 µs | -17.73% |
| ❌ | Simulation | slice_empty_vortex |
339.4 ns | 397.8 ns | -14.66% |
| ❌ | Simulation | extend_from_array_zctl[(1000, 64)] |
1 ms | 1.2 ms | -13.91% |
| ❌ | Simulation | i32_small_overlapping |
39.9 µs | 44.8 µs | -11.08% |
| ❌ | Simulation | extend_from_array_zctl[(10000, 8)] |
1.9 ms | 2.2 ms | -10.77% |
| ❌ | Simulation | extend_from_array_non_zctl_overlapping[(10000, 8)] |
2.2 ms | 2.5 ms | -10.62% |
| ❌ | Simulation | compact_sliced[(4096, 90)] |
750 ns | 838.1 ns | -10.51% |
| ⚡ | Simulation | chunked_bool_canonical_into[(1000, 10)] |
26.3 µs | 16 µs | +64.78% |
| ⚡ | Simulation | bench_many_codes_few_values[1024] |
529.3 µs | 364.1 µs | +45.4% |
| ⚡ | Simulation | chunked_varbinview_canonical_into[(100, 100)] |
259.7 µs | 223.6 µs | +16.14% |
| ⚡ | Simulation | bitwise_not_vortex_buffer_mut[128] |
244.4 ns | 215.3 ns | +13.55% |
| ⚡ | Simulation | chunked_varbinview_into_canonical[(100, 100)] |
306.7 µs | 270.8 µs | +13.27% |
| ⚡ | Simulation | rebuild_naive |
109 µs | 98.5 µs | +10.69% |
| ⚡ | Simulation | bitwise_not_vortex_buffer_mut[1024] |
304.7 ns | 275.6 ns | +10.58% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing claude/nice-hypatia-g68zgt (947a966) with develop (bf2be52)
Footnotes
-
4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
This also removes ArrayBuilder::extend_from_array and moves that logic in to ArrayRef::append_to_builder Plumb ExecutionCtx through more methods instead of creating fresh one. Signed-off-by: Robert Kruszewski <github@robertk.io>
5e975ca to
f8f6a90
Compare
| builder: &mut dyn ArrayBuilder, | ||
| ctx: &mut ExecutionCtx, | ||
| ) -> VortexResult<()> { | ||
| let Some(builder) = builder.as_any_mut().downcast_mut::<ListBuilder<u64>>() else { |
There was a problem hiding this comment.
builder_with_capacity(DType::List, ...) returns a ListViewBuilder<u64, u64>, so generic callers that allocate the canonical builder and then append a ListArray now hit this branch and error out instead of appending. For example, zip/case_when build the default list builder and append sliced branches through append_to_builder. This should either append into the canonical ListViewBuilder or fall back through execute::<ListViewArray> before appending.
There was a problem hiding this comment.
This part needs reworking. Needs to do a different downcast here
| )? | ||
| .0; | ||
|
|
||
| let len = u.int_in_range(0..=2048)?; |
There was a problem hiding this comment.
len can now be as high as 2048, but random_strictly_sorted_ends still randomly chooses PType::U8 and later does u8::try_from(e).vortex_expect(...) after forcing the last end toward this target. Ordinary fuzzer input can therefore panic during Arbitrary instead of returning arbitrary::Error. Please choose an end type that can represent the generated max end, cap the target by the chosen type, or return an error instead of vortex_expect.
375f17b to
048b19b
Compare
`builder_with_capacity` produces a `ListViewBuilder` for `DType::List`, but `List::append_to_builder` downcast the builder to `ListBuilder<u64>` and bailed otherwise — so executing a `List`-encoded array into the canonical builder (the one `builder_with_capacity` returns) failed. `ListView::append_to_builder` had the mirror-image assumption, only accepting a `ListViewBuilder<u64, u64>`. Add two macros that enumerate the possible list builder offset/size integer types and try downcasting to each, rather than assuming the types produced by `builder_with_capacity`: - `match_each_list_builder!` enumerates `ListBuilder<O>`. - `match_each_listview_builder!` enumerates `ListViewBuilder<O, S>`. `List::append_to_builder` tries `ListBuilder` (its non-canonical, directly-supplied builder) and otherwise canonicalizes and dispatches, routing the canonical `ListViewBuilder` case through `ListView::append_to_builder`, which tries `ListViewBuilder`. Either encoding works with any unsigned offset/size types. Other encodings that yield list dtypes (constant, dict, chunked, ...) already route here through the canonical-dispatch fallback. Signed-off-by: Robert <robert@spiraldb.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Mkrgj5SuJpaKBXK9jydtSc
048b19b to
947a966
Compare
Rebased onto develop and made standalone (no longer stacked on #8609). Instead of deprecating the hidden global session, expose it as `vortex_array::legacy_session()` and add it to the central `clippy.toml` `disallowed-methods` list (denied via `clippy::all`). New call sites are now rejected by clippy; every current invocation is explicitly allowlisted with `#[allow(clippy::disallowed_methods)]` on its enclosing function. - `LEGACY_SESSION` static -> `legacy_session() -> &'static VortexSession` (the `LazyLock` is now a private static inside the accessor). - All call sites renamed and allowlisted across vortex-array, the encodings, vortex-ffi, vortex-cuda, vortex-cxx, vortex-layout, vortex-python and vortex-web. develop has more call sites than #8609 (incl. test/builder code and crates #8609 had refactored away), all covered. Verified with: - cargo clippy -p vortex-array -p vortex-alp -p vortex-runend -p vortex-fsst -p vortex-fastlanes -p vortex-zstd -p vortex-ffi -p vortex-layout -p vortex-web-wasm -p vortex-cxx -p vortex-python --all-targets --all-features (0 disallowed-method hits, 0 warnings) - cargo +nightly fmt --check (touched crates) vortex-cuda could not be compiled here (CUDA toolchain unavailable); its two call sites are allowlisted following the same pattern. Signed-off-by: Robert Kruszewski <robert@spiraldb.com> Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018E5DywGCpq26ndoMHd1vyg
This also removes ArrayBuilder::extend_from_array and moves that logic
in to ArrayRef::append_to_builder
Plumb ExecutionCtx through more methods instead of creating fresh one.
Fixes: #3235
Signed-off-by: Robert Kruszewski github@robertk.io