bench: run the full compress suite under --gpu-decompress - #9399
bench: run the full compress suite under --gpu-decompress#9399joseph-isaacs wants to merge 9 commits into
--gpu-decompress#9399Conversation
Drop the incremental GPU allow-list so --gpu-decompress runs every dataset in the compress suite, and include the airquality dataset on the GPU path. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
airquality and rplace come from the same pcodec bucket, so gate them consistently: the GPU suite runs every compress dataset. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Merging this PR will degrade performance by 12.14%
|
| Mode | Benchmark | BASE |
HEAD |
Efficiency | |
|---|---|---|---|---|---|
| ❌ | WallTime | words_gather_dispatch[1024] |
9 ns | 13 ns | -30.77% |
| ❌ | WallTime | words_gather_scalar[65536] |
8.3 µs | 9.4 µs | -11.83% |
| ❌ | Simulation | take[small_m/shuffled/primitive/nonnull/chunks=16384/indices=16] |
1 ms | 1.2 ms | -11.26% |
| ⚡ | WallTime | words_gather_dispatch[1024] |
33 ns | 30 ns | +10% |
Tip
Investigate this regression by commenting @codspeedbot fix this regression on this PR, or directly use the CodSpeed MCP with your agent.
Comparing claude/gpu-compress-benchmarks-x788xc (cf631cd) with develop (1a493b6)
Footnotes
-
46 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports. ↩
BENCHMARK FAILEDGPU Compression failed. Check the workflow run for details. |
The CUDA DateTimeParts executor dispatched with match_each_signed_integer_ptype, so it panicked with "Unsupported ptype u16" on the taxi benchmark. Compression picks the narrowest ptype per component, and the CPU decoder already accepts any integer ptype, so match the CPU behaviour and generate the kernel for every signed and unsigned integer width. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
RunEnd GPU decoding bailed when values carried Validity::Array. The error message claimed a CPU fallback, but execute_cuda can only fall back while all buffers are host-resident, so a device-resident scan (as in the GPU compression benchmark) failed outright. Add a runend_bool kernel that expands the per-run validity bitmap through the same run mapping as the values. Each thread owns a whole output byte so threads never race on bits within a byte. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The Public BI datasets are converted from CSV with the DuckDB CLI, which the CPU benchmark runner already installs. The GPU job never needed it while its suite was a single TPC-H dataset; now that it runs the full suite, Arade fails with "No such file or directory" when spawning duckdb. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The GPU compression benchmark failed on Euro2016 with "No CUDA kernel for encoding vortex.masked". A MaskedArray is a child array that carries no nulls of its own plus the validity bitmap that supplies them, so decode the child on the GPU, decode the mask on the GPU, and attach the mask to the result. Bail when the child itself carries a per-element validity bitmap: intersecting two device-resident bitmaps would need a CPU compute pass, and MaskedArray's own invariant makes that case unreachable for well-formed arrays. Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Canonical arrays keep validity as a separate child array, and into_host moved
only the data buffer, leaving a device-resident bitmap that CPU compute cannot
read ("unwrap_host called for Device allocation"). Any executor returning
Validity::Array from the GPU hit this; the MaskedExecutor test caught it.
Also call the executors directly in the nullable RunEnd tests. Going through
execute_cuda silently falls back to CPU for a host-resident array, so those
tests passed without ever running the new runend_bool kernel.
Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk>
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…tible vortex.sparse has no CUDA decode kernel, and only_cuda_compatible already excluded the integer and float sparse schemes. The string variant was missed, so Euro2016 failed the GPU compression benchmark with "No CUDA kernel for encoding vortex.sparse". Signed-off-by: Joe Isaacs <joe.isaacs@live.co.uk> Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Rationale for this change
--gpu-decompressincompress-benchwas gated behind an incremental allow-list thatcontained a single entry (
TPC-H l_comment canonical), so the GPU compression benchmarkjob only ever measured one dataset. This widens GPU coverage to the whole compress suite,
including the
airqualitydataset.Opened as a draft to let the GPU benchmark job tell us which datasets actually have
working CUDA decode kernels — that verification is exactly what the allow-list was
standing in for, and it cannot be done locally without a GPU.
What changes are included in this PR?
gpu_decompress_benchmarksallow-list frombenchmarks/compress-bench/src/main.rs.With no
--datasetsfilter,--gpu-decompressnow runs every dataset in the suite:taxi, the Public BI sets (Arade, Bimbo, CMSprovider, Euro2016, Food, HashTags), bothTPC-H
l_commentvariants, theStructListOfIntsvariants, plusairqualityandrplace.airquality/rplace. Note that the GPUpath now downloads both from pcodec's public S3 bucket on every run; the existing CPU-side
exclusion exists to avoid creating egress charges for pcodec, so this may want caching
rather than a permanent CI cost.
benchmarks/compress-bench/README.mdto match.What APIs are changed? Are there any user-facing changes?
No library API changes. The only behavior change is to the
compress-benchbenchmark binary:--gpu-decompressno longer filters the dataset list.Checks
cargo clippy -p compress-bench --all-targetspasses.cargo +nightly fmtcould not be run in this environment (the nightly toolchain ICEs);the edits are formatting-neutral but CI fmt is unverified.
draft's benchmark job is for.
Generated by Claude Code