Skip to content

Improve intersect_by_rank performance#7744

Merged
robert3005 merged 7 commits intodevelopfrom
rk/intersect-by-rank
May 8, 2026
Merged

Improve intersect_by_rank performance#7744
robert3005 merged 7 commits intodevelopfrom
rk/intersect-by-rank

Conversation

@robert3005
Copy link
Copy Markdown
Contributor

@robert3005 robert3005 commented May 1, 2026

Intersect by rank is an operation for performing filter on top of another filter

This pr improves
Dense on Dense masks: 10-18x
Sparse on Dense masks: 8x
Sparse on Sparse masks: 2x

Signed-off-by: Robert Kruszewski github@robertk.io

@robert3005
Copy link
Copy Markdown
Contributor Author

I will go over this tomorrow @joseph-isaacs I looked at #7098 and #7393 which both optimised slightly different cases of this function. I tried to combine the two.

@robert3005 robert3005 added the changelog/performance A performance improvement label May 1, 2026
@codspeed-hq
Copy link
Copy Markdown

codspeed-hq Bot commented May 1, 2026

Merging this PR will degrade performance by 23.81%

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

⚡ 24 improved benchmarks
❌ 24 regressed benchmarks
✅ 1160 untouched benchmarks

⚠️ Please fix the performance issues or acknowledge them on CodSpeed.

Performance Changes

Mode Benchmark BASE HEAD Efficiency
Simulation take_search[(0.005, 0.05)] 132 µs 168.3 µs -21.6%
Simulation take_search[(0.005, 0.1)] 247.5 µs 320.3 µs -22.72%
Simulation take_search[(0.005, 0.5)] 1.2 ms 1.5 ms -23.67%
Simulation take_search[(0.005, 1.0)] 2.3 ms 3.1 ms -23.81%
Simulation take_search[(0.01, 0.05)] 143.1 µs 179.4 µs -20.27%
Simulation take_search[(0.01, 0.1)] 268.4 µs 341.2 µs -21.33%
Simulation take_search[(0.01, 0.5)] 1.3 ms 1.6 ms -22.21%
Simulation take_search[(0.01, 1.0)] 2.5 ms 3.3 ms -22.34%
Simulation take_search[(0.1, 0.05)] 212.7 µs 249.1 µs -14.6%
Simulation take_search[(0.1, 0.1)] 385.9 µs 458.6 µs -15.87%
Simulation take_search[(0.1, 0.5)] 1.8 ms 2.2 ms -16.93%
Simulation take_search[(0.1, 1.0)] 3.5 ms 4.3 ms -17.09%
Simulation take_search_chunked[(0.005, 0.05)] 162.2 µs 193.2 µs -16.01%
Simulation take_search_chunked[(0.005, 0.1)] 307.3 µs 369.2 µs -16.76%
Simulation take_search_chunked[(0.005, 0.5)] 1.5 ms 1.8 ms -17.39%
Simulation take_search_chunked[(0.005, 1.0)] 2.9 ms 3.5 ms -17.48%
Simulation take_search_chunked[(0.01, 0.05)] 175.3 µs 206.2 µs -14.99%
Simulation take_search_chunked[(0.01, 0.1)] 332.1 µs 393.9 µs -15.7%
Simulation take_search_chunked[(0.01, 0.5)] 1.6 ms 1.9 ms -16.3%
Simulation take_search_chunked[(0.01, 1.0)] 3.2 ms 3.8 ms -16.38%
... ... ... ... ... ...

ℹ️ Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.


Comparing rk/intersect-by-rank (8578962) with develop (823991f)

Open in CodSpeed

@robert3005 robert3005 force-pushed the rk/intersect-by-rank branch 7 times, most recently from 3083876 to ea3104a Compare May 8, 2026 10:11
@robert3005 robert3005 marked this pull request as ready for review May 8, 2026 12:13
indices.iter().for_each(|&idx| buf.set(idx));
buf
pub fn from_indices(len: usize, indices: impl IntoIterator<Item = usize>) -> BitBufferMut {
let mut buffer = BufferMut::<u64>::zeroed(len.div_ceil(64));
Copy link
Copy Markdown
Contributor

@joseph-isaacs joseph-isaacs May 8, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add a todo to add a sized iterator variant, then you can switch over the ratio of len to indices

Comment thread vortex-mask/src/intersect_by_rank.rs
@joseph-isaacs
Copy link
Copy Markdown
Contributor

Add benchmark numbers to the desc

Comment thread vortex-mask/src/intersect_by_rank.rs Outdated
robert3005 added 6 commits May 8, 2026 13:38
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005 robert3005 force-pushed the rk/intersect-by-rank branch from 6121376 to 4e90444 Compare May 8, 2026 12:38
Signed-off-by: Robert Kruszewski <github@robertk.io>
@robert3005 robert3005 merged commit b212667 into develop May 8, 2026
61 of 62 checks passed
@robert3005 robert3005 deleted the rk/intersect-by-rank branch May 8, 2026 14:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog/performance A performance improvement

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants