Skip to content

Replace ToCanonical trait usages with execute#8609

Open
robert3005 wants to merge 3 commits into
developfrom
claude/nice-hypatia-g68zgt
Open

Replace ToCanonical trait usages with execute#8609
robert3005 wants to merge 3 commits into
developfrom
claude/nice-hypatia-g68zgt

Conversation

@robert3005

@robert3005 robert3005 commented Jun 27, 2026

Copy link
Copy Markdown
Contributor

This also removes ArrayBuilder::extend_from_array and moves that logic
in to ArrayRef::append_to_builder

Plumb ExecutionCtx through more methods instead of creating fresh one.

Fixes: #3235

Signed-off-by: Robert Kruszewski github@robertk.io

@robert3005 robert3005 requested review from a team, 0ax1, AdamGS and gatesn June 27, 2026 00:25
@robert3005 robert3005 added the changelog/chore A trivial change label Jun 27, 2026
@codspeed-hq

codspeed-hq Bot commented Jun 27, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 7 improved benchmarks
❌ 10 regressed benchmarks
✅ 1578 untouched benchmarks
⏩ 4 skipped benchmarks1

Warning

Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation extend_from_array_zctl[(1000, 8)] 238.9 µs 404.6 µs -40.96%
Simulation extend_from_array_non_zctl_overlapping[(1000, 8)] 271.3 µs 436.3 µs -37.81%
Simulation extend_from_array_non_zctl_overlapping[(1000, 32)] 735.7 µs 900.5 µs -18.31%
Simulation chunked_varbinview_into_canonical[(1000, 10)] 169.1 µs 205.6 µs -17.73%
Simulation slice_empty_vortex 339.4 ns 397.8 ns -14.66%
Simulation extend_from_array_zctl[(1000, 64)] 1 ms 1.2 ms -13.91%
Simulation i32_small_overlapping 39.9 µs 44.8 µs -11.08%
Simulation extend_from_array_zctl[(10000, 8)] 1.9 ms 2.2 ms -10.77%
Simulation extend_from_array_non_zctl_overlapping[(10000, 8)] 2.2 ms 2.5 ms -10.62%
Simulation compact_sliced[(4096, 90)] 750 ns 838.1 ns -10.51%
Simulation chunked_bool_canonical_into[(1000, 10)] 26.3 µs 16 µs +64.78%
Simulation bench_many_codes_few_values[1024] 529.3 µs 364.1 µs +45.4%
Simulation chunked_varbinview_canonical_into[(100, 100)] 259.7 µs 223.6 µs +16.14%
Simulation bitwise_not_vortex_buffer_mut[128] 244.4 ns 215.3 ns +13.55%
Simulation chunked_varbinview_into_canonical[(100, 100)] 306.7 µs 270.8 µs +13.27%
Simulation rebuild_naive 109 µs 98.5 µs +10.69%
Simulation bitwise_not_vortex_buffer_mut[1024] 304.7 ns 275.6 ns +10.58%

Tip

Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.


Comparing claude/nice-hypatia-g68zgt (947a966) with develop (bf2be52)

Open in CodSpeed

Footnotes

  1. 4 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

This also removes ArrayBuilder::extend_from_array and moves that logic
in to ArrayRef::append_to_builder

Plumb ExecutionCtx through more methods instead of creating fresh one.

Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005 robert3005 force-pushed the claude/nice-hypatia-g68zgt branch from 5e975ca to f8f6a90 Compare June 27, 2026 15:12
Signed-off-by: Robert Kruszewski <github@robertk.io>
builder: &mut dyn ArrayBuilder,
ctx: &mut ExecutionCtx,
) -> VortexResult<()> {
let Some(builder) = builder.as_any_mut().downcast_mut::<ListBuilder<u64>>() else {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

builder_with_capacity(DType::List, ...) returns a ListViewBuilder<u64, u64>, so generic callers that allocate the canonical builder and then append a ListArray now hit this branch and error out instead of appending. For example, zip/case_when build the default list builder and append sliced branches through append_to_builder. This should either append into the canonical ListViewBuilder or fall back through execute::<ListViewArray> before appending.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This part needs reworking. Needs to do a different downcast here

)?
.0;

let len = u.int_in_range(0..=2048)?;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

len can now be as high as 2048, but random_strictly_sorted_ends still randomly chooses PType::U8 and later does u8::try_from(e).vortex_expect(...) after forcing the last end toward this target. Ordinary fuzzer input can therefore panic during Arbitrary instead of returning arbitrary::Error. Please choose an end type that can represent the generated max end, cap the target by the chosen type, or return an error instead of vortex_expect.

@robert3005 robert3005 force-pushed the claude/nice-hypatia-g68zgt branch from 375f17b to 048b19b Compare June 27, 2026 20:45
`builder_with_capacity` produces a `ListViewBuilder` for `DType::List`, but
`List::append_to_builder` downcast the builder to `ListBuilder<u64>` and bailed
otherwise — so executing a `List`-encoded array into the canonical builder (the
one `builder_with_capacity` returns) failed. `ListView::append_to_builder` had
the mirror-image assumption, only accepting a `ListViewBuilder<u64, u64>`.

Add two macros that enumerate the possible list builder offset/size integer types
and try downcasting to each, rather than assuming the types produced by
`builder_with_capacity`:

- `match_each_list_builder!` enumerates `ListBuilder<O>`.
- `match_each_listview_builder!` enumerates `ListViewBuilder<O, S>`.

`List::append_to_builder` tries `ListBuilder` (its non-canonical, directly-supplied
builder) and otherwise canonicalizes and dispatches, routing the canonical
`ListViewBuilder` case through `ListView::append_to_builder`, which tries
`ListViewBuilder`. Either encoding works with any unsigned offset/size types.
Other encodings that yield list dtypes (constant, dict, chunked, ...) already
route here through the canonical-dispatch fallback.

Signed-off-by: Robert <robert@spiraldb.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Mkrgj5SuJpaKBXK9jydtSc
@robert3005 robert3005 force-pushed the claude/nice-hypatia-g68zgt branch from 048b19b to 947a966 Compare June 27, 2026 20:53
robert3005 added a commit that referenced this pull request Jun 27, 2026
Rebased onto develop and made standalone (no longer stacked on #8609).

Instead of deprecating the hidden global session, expose it as
`vortex_array::legacy_session()` and add it to the central `clippy.toml`
`disallowed-methods` list (denied via `clippy::all`). New call sites are now
rejected by clippy; every current invocation is explicitly allowlisted with
`#[allow(clippy::disallowed_methods)]` on its enclosing function.

- `LEGACY_SESSION` static -> `legacy_session() -> &'static VortexSession`
  (the `LazyLock` is now a private static inside the accessor).
- All call sites renamed and allowlisted across vortex-array, the encodings,
  vortex-ffi, vortex-cuda, vortex-cxx, vortex-layout, vortex-python and
  vortex-web. develop has more call sites than #8609 (incl. test/builder code
  and crates #8609 had refactored away), all covered.

Verified with:
- cargo clippy -p vortex-array -p vortex-alp -p vortex-runend -p vortex-fsst
  -p vortex-fastlanes -p vortex-zstd -p vortex-ffi -p vortex-layout
  -p vortex-web-wasm -p vortex-cxx -p vortex-python --all-targets --all-features
  (0 disallowed-method hits, 0 warnings)
- cargo +nightly fmt --check (touched crates)

vortex-cuda could not be compiled here (CUDA toolchain unavailable); its two
call sites are allowlisted following the same pattern.

Signed-off-by: Robert Kruszewski <robert@spiraldb.com>

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018E5DywGCpq26ndoMHd1vyg
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/chore A trivial change

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Move logic into append_to_builder

3 participants