perf: do not match each token in aggregation - #474
Conversation
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #474 +/- ##
==========================================
+ Coverage 70.68% 71.11% +0.43%
==========================================
Files 233 232 -1
Lines 18399 18417 +18
==========================================
+ Hits 13005 13098 +93
+ Misses 4416 4339 -77
- Partials 978 980 +2 ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
🔴 Performance DegradationSome benchmarks have degraded compared to the previous run. Show table
|
|
Nice, @dkharms Started benchmark
I'll post the statistics here once it finishes. Here is a list of helpful links: Have a great time! |
|
Nice, @dkharms Benchmark Show summary
Compare it against another run with |
|
@seqbenchbot compare 41da199f cf3cffd6 |
|
Nice, @dkharms Comparison of baseline Show comparison
See both runs overlaid (baseline aligned to comparison) on the Grafana dashboard. Have a great time! |
||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|
@seqbenchbot start bulk 0-skip-matching --duration=5m |
|
Nice, @dkharms Started benchmark
I'll post the statistics here once it finishes. Here is a list of helpful links: Have a great time! |
|
Nice, @dkharms Benchmark Show summary
Compare it against another run with |
Description
When getting tokens for aggregation, we scan each token and checking it against
*-- so basically, useless computations and IO (we have to load+unpack token blocks). I suggest to return range[firstTID; lastTID].Obviously, this optimization will have more impact on high-cardinality fields that span over several token blocks (like
remote_addror something).I will collect better measurements but for cold queries (flushed userspace cache and page cache) I got following results:
If you have used LLM/AI assistance please provide model name and full prompt: