Skip to content

[feature](query-cache) Support incremental merge for stale query cache entries#65482

Open
asdf2014 wants to merge 1 commit into
apache:masterfrom
asdf2014:query-cache-incremental-merge
Open

[feature](query-cache) Support incremental merge for stale query cache entries#65482
asdf2014 wants to merge 1 commit into
apache:masterfrom
asdf2014:query-cache-incremental-merge

Conversation

@asdf2014

Copy link
Copy Markdown
Member

What problem does this PR solve?

Related PR: N/A

Problem Summary:

For hourly batch-load workloads, every load bumps the tablet version of the
hot partition, so its query cache entries never hit again: each "last N days"
aggregation recomputes the whole hot partition every hour, and BE CPU spikes
during load windows.

This PR adds incremental merge for the query cache (experimental, default
off). When a cache entry's version falls behind, BE reuses it instead of
discarding it: it scans only the delta rowsets in
(cached_version, current_version], emits their partial aggregation state
side by side with the cached partial blocks so the existing upstream merge
aggregation combines both, and writes the merged entry back under the new
version. The execution plan is unchanged.

Full recompute vs incremental merge

Measured impact (80M-row duplicate-key table, 200k-row hourly appends,
group by a non-distribution column): a stale query drops from ~307 ms
(full recompute) to ~19 ms with incremental merge, about 16x and on
par with an exact hit (~17 ms); the cost no longer grows with the base data
size (8M rows: 48 ms full vs 19 ms incremental; 80M rows: 307 ms vs 19 ms).
Full numbers in the Verification section below.

Design highlights:

  1. Correctness algebra: S(v2) = S(v1) ⊎ Δ for append-only data plus the
    homomorphism partial(A ⊎ B) ≡ merge(partial(A), partial(B)). Every
    precondition guards one of the two properties; any violation falls back
    to a full recompute silently, so results are always correct.
  2. FE authorizes via a new optional thrift field
    (TQueryCacheParam.allow_incremental) only for non-finalize aggregations
    directly above the olap scan on an append-only index: DUP_KEYS, or
    merge-on-write UNIQUE_KEYS. Merge-on-read UNIQUE and AGG tables always
    fall back.
  3. BE re-validates per tablet at decision time (fallback reasons are visible
    in the profile): cloud mode, version order, the compaction threshold
    (query_cache_max_incremental_merge_count, default 8, 0 disables), keys
    type, delta capturability on the version graph, delete predicates in the
    delta, and for merge-on-write tables a delete-bitmap window check that
    rejects loads rewriting pre-existing keys. A rare backfill therefore costs
    exactly one full recompute, which re-bases the entry, and the next
    pure-append load is incremental again.
  4. A fragment-level QueryCacheRuntime makes one idempotent decision per
    instance (HIT / INCREMENTAL / MISS), pins the entry and pre-captures the
    delta read source. This also fixes three pre-existing defects: the
    scan/cache-source double-lookup race (could write back an empty poisoned
    entry), row-binlog scans polluting the cache, and build_cache_key
    failures aborting the query instead of degrading to uncached execution.
  5. Entropy control: every merge appends the delta blocks to the entry, so
    after query_cache_max_incremental_merge_count merges the next query
    recomputes in full and compacts the entry; oversized entries keep the
    delta-scan benefit but skip the write back.

Verification:

  • FE UT: QueryCacheNormalizerTest 13/13 (8 assertions on the incremental
    authorization matrix: switch, plan shapes, DUP / MoW / MoR / AGG / nested
    aggregation).

  • BE UT: 34/34 across the decision layer, the incremental scenarios (MoW
    pure-append / history-rewrite / irrelevant bitmap entries, version gap,
    capture error via debug point, delete predicates) and the operator layer.
    llvm-cov: zero uncovered changed lines; query_cache.cpp at 100% line
    coverage.

  • Regression: query_cache_incremental passes on a real 1FE+1BE cluster;
    every step is checked against a cache-off baseline. BE metrics confirm the
    incremental path actually fires (stale_hit_total +40 across the suite,
    fallbacks exactly at the three designed spots).

  • Benchmark (80M-row DUP table, 200k-row hourly appends, group by a
    non-distribution column, 5 rounds each):

    scenario latency (median)
    no cache, full scan ~446 ms
    exact hit ~17 ms
    stale + incremental merge (this PR) ~19 ms
    stale + full recompute (switch off) ~307 ms

    A stale query becomes as cheap as an exact hit, and its cost no longer
    grows with the base data size (8M rows: 48ms full vs 19ms incremental;
    80M rows: 307ms vs 19ms).

Release note

Add experimental incremental merge for the query cache: a stale entry can be
reused by scanning only the delta rowsets and merging them with the cached
partial aggregation state. Controlled by the session variable
enable_query_cache_incremental (default off) and the BE config
query_cache_max_incremental_merge_count (default 8).

Check List (For Author)

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

…e entries

Hourly batch loads keep bumping the version of the hot partition, so its
query cache entries never hit again and every "last N days" aggregation
recomputes the whole partition each hour. This change lets BE reuse a
stale entry instead of discarding it: scan only the delta rowsets in
(cached_version, current_version], emit their partial aggregation state
side by side with the cached partial blocks (the upstream merge
aggregation combines both), and write the merged entry back under the
new version.

Correctness rests on two properties: append-only snapshots decompose as
S(v2) = S(v1) + delta, and partial aggregation states merge
homomorphically. Every precondition guards one of them, and any
violation falls back to a full recompute silently:

- FE only sets TQueryCacheParam.allow_incremental (new optional field 8)
  when the experimental session switch enable_query_cache_incremental is
  on, the cache point is a non-finalize aggregation directly above the
  olap scan node, and the selected index is append-only: DUP_KEYS, or
  merge-on-write UNIQUE_KEYS.
- BE re-validates per tablet at decision time: local storage mode,
  version order, the compaction threshold
  (query_cache_max_incremental_merge_count, BE config, default 8), keys
  type, delta capturability, no delete predicates in the delta, and for
  merge-on-write tables a delete-bitmap window check that rejects any
  load that rewrote pre-existing keys, so an occasional backfill costs
  exactly one full recompute and re-bases the entry.

A new fragment-level QueryCacheRuntime makes one idempotent decision
(HIT / INCREMENTAL / MISS) per instance, pins the entry and pre-captures
the delta read source. Centralizing the decision also fixes three
pre-existing defects: the scan/cache-source double-lookup race that
could write back an empty poisoned entry, row-binlog scans polluting
the cache, and build_cache_key failures aborting the whole query
instead of degrading to uncached execution.

Observability: profile fields HitCacheStale, IncrementalDeltaVersions
and IncrementalFallbackReason, plus three BE metrics
(query_cache_stale_hit_total, query_cache_incremental_fallback_total,
query_cache_write_back_total).

Benchmark (80M-row duplicate-key table, 200k-row hourly appends,
group by a non-distribution column): a stale query drops from ~307ms
(full recompute) to ~19ms with incremental merge, on par with an exact
hit, and the cost no longer grows with the base data size.
@hello-stephen

Copy link
Copy Markdown
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@morningman morningman self-assigned this Jul 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants