Conversation
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
| pub const ID: &'static str = "vortex.alp"; | ||
|
|
||
| /// Returns the cached [`ArrayId`] for this encoding. | ||
| pub fn array_id() -> ArrayId { | ||
| static CACHED: std::sync::OnceLock<ArrayId> = std::sync::OnceLock::new(); | ||
| *CACHED.get_or_init(|| ArrayId::new(Self::ID)) | ||
| } |
There was a problem hiding this comment.
Does this not have an atomic? On read
There was a problem hiding this comment.
We could force vtables to hold onto an instance of their ID by having VTable::id(&Array) -> &ArrayId?
But it means no more ZSTs for vtables
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Benchmarks: FineWeb S3Verdict: No clear signal (low confidence) datafusion / vortex-file-compressed (0.933x ➖, 0↑ 0↓)
datafusion / vortex-compact (1.024x ➖, 0↑ 0↓)
datafusion / parquet (1.055x ➖, 0↑ 0↓)
duckdb / vortex-file-compressed (0.968x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.952x ➖, 0↑ 0↓)
duckdb / parquet (0.959x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: Statistical and Population GeneticsVerdict: No clear signal (low confidence) duckdb / vortex-file-compressed (1.064x ➖, 0↑ 2↓)
duckdb / vortex-compact (1.051x ➖, 0↑ 0↓)
duckdb / parquet (1.069x ➖, 0↑ 2↓)
Full attributed analysis
|
Benchmarks: Clickbench on NVMEVerdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (1.173x ❌, 9↑ 20↓)
datafusion / parquet (0.987x ➖, 10↑ 6↓)
duckdb / vortex-file-compressed (1.249x ❌, 8↑ 15↓)
duckdb / parquet (0.968x ➖, 3↑ 5↓)
duckdb / duckdb (0.982x ➖, 8↑ 5↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=1 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.924x ➖, 3↑ 0↓)
datafusion / vortex-compact (0.983x ➖, 1↑ 3↓)
datafusion / parquet (1.011x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed (0.916x ➖, 1↑ 0↓)
duckdb / vortex-compact (0.925x ➖, 0↑ 1↓)
duckdb / parquet (0.981x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: PolarSignals ProfilingVortex (geomean): 3.077x ❌ datafusion / vortex-file-compressed (3.077x ❌, 0↑ 10↓)
|
Benchmarks: TPC-H SF=1 on NVMEVerdict: Likely regression (environment too noisy confidence) datafusion / vortex-file-compressed (1.680x ❌, 0↑ 18↓)
datafusion / vortex-compact (1.507x ❌, 0↑ 19↓)
datafusion / parquet (0.998x ➖, 2↑ 0↓)
datafusion / arrow (0.994x ➖, 0↑ 1↓)
duckdb / vortex-file-compressed (1.086x ➖, 0↑ 6↓)
duckdb / vortex-compact (1.073x ➖, 0↑ 3↓)
duckdb / parquet (0.998x ➖, 3↑ 2↓)
duckdb / duckdb (0.995x ➖, 1↑ 0↓)
Full attributed analysis
|
Benchmarks: TPC-DS SF=1 on NVMEVerdict: No clear signal (low confidence) datafusion / vortex-file-compressed (1.253x ❌, 0↑ 58↓)
datafusion / vortex-compact (1.177x ❌, 0↑ 48↓)
datafusion / parquet (1.050x ➖, 0↑ 9↓)
duckdb / vortex-file-compressed (1.082x ➖, 2↑ 21↓)
duckdb / vortex-compact (1.057x ➖, 0↑ 7↓)
duckdb / parquet (1.030x ➖, 1↑ 4↓)
duckdb / duckdb (1.035x ➖, 1↑ 8↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=10 on S3Verdict: No clear signal (environment too noisy confidence) datafusion / vortex-file-compressed (0.938x ➖, 1↑ 0↓)
datafusion / vortex-compact (1.009x ➖, 1↑ 0↓)
datafusion / parquet (1.098x ➖, 0↑ 2↓)
duckdb / vortex-file-compressed (0.983x ➖, 0↑ 0↓)
duckdb / vortex-compact (0.913x ➖, 0↑ 0↓)
duckdb / parquet (0.925x ➖, 0↑ 0↓)
Full attributed analysis
|
Benchmarks: FineWeb NVMeVerdict: Likely regression (low confidence) datafusion / vortex-file-compressed (1.830x ❌, 0↑ 9↓)
datafusion / vortex-compact (1.385x ❌, 0↑ 9↓)
datafusion / parquet (1.151x ❌, 0↑ 9↓)
duckdb / vortex-file-compressed (1.320x ❌, 0↑ 9↓)
duckdb / vortex-compact (1.205x ❌, 0↑ 9↓)
duckdb / parquet (1.161x ❌, 0↑ 9↓)
Full attributed analysis
|
Benchmarks: TPC-H SF=10 on NVMEVerdict: Likely regression (low confidence) datafusion / vortex-file-compressed (1.212x ❌, 0↑ 9↓)
datafusion / vortex-compact (1.133x ❌, 0↑ 9↓)
datafusion / parquet (0.920x ➖, 5↑ 0↓)
datafusion / arrow (0.902x ➖, 11↑ 0↓)
duckdb / vortex-file-compressed (1.229x ❌, 0↑ 8↓)
duckdb / vortex-compact (1.070x ➖, 0↑ 6↓)
duckdb / parquet (0.970x ➖, 0↑ 0↓)
duckdb / duckdb (0.968x ➖, 1↑ 0↓)
Full attributed analysis
|
Merging this PR will degrade performance by 25.53%
Performance Changes
Comparing Footnotes
|
Signed-off-by: Nicholas Gates <nick@nickgates.com>
Polar Signals Profiling ResultsLatest Run
Powered by Polar Signals Cloud |
|
Lol yeah.... I'm not sure this will work unless we hold the Id on the vtable |
Benchmarks: CompressionVortex (geomean): 1.005x ➖ unknown / unknown (1.005x ➖, 0↑ 1↓)
|
Benchmarks: Random AccessVortex (geomean): 0.862x ✅ unknown / unknown (0.959x ➖, 10↑ 4↓)
|
Fixes #6823