perf(cache-proxy): parallel peer fills, origin hedge, tighter probe timeout - #1047
Open
EDsCODE wants to merge 1 commit into
Open
perf(cache-proxy): parallel peer fills, origin hedge, tighter probe timeout#1047EDsCODE wants to merge 1 commit into
EDsCODE wants to merge 1 commit into
Conversation
…imeout Phase-timing trails from production cold scans showed 100% of block-request serving time inside FetchFromPeers (median 714ms cold / 237ms steady-state per request, vs 8ms for local hits), with 16% of cold requests burning the full 1s has-probe timeout, and 2-block requests costing exactly twice the single-block peer round. Three changes, each aimed at one measured cost: - peerHasTimeout 1s -> 150ms: healthy peers answer probes in single-digit ms while the origin path costs ~200ms, so waiting longer for the fleet's slowest "no" only delays a faster fallback. - Peer fills for a request's missing blocks now run concurrently (bounded at 8) instead of one FetchFromPeers round per block in sequence; phase 1 consumes results in order, so miss-run coalescing and single-flight keys are unchanged. - A wait budget (400ms, shared absolute deadline across the request's fills) hedges stalled fills to the coalesced origin fetch instead of pinning the request on the 30s body-transfer timeout; the fill keeps running in the background and still populates the cache. New counter cache_proxy_peer_fill_hedged_total and Served. field blocks_hedged track how often that happens. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Test Impact PlanDeterministic summary of how this PR changes tests, CI runners, and coverage-risk signals. Summary
Signals
Coverage risk: neutral or increased No coverage-reduction warnings detected. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Per-request phase timing (#1045) localized essentially all block-request serving time to
FetchFromPeers: cold-scan trails showedpeer_msaccounting for ~100% ofdur_ms(median 714ms post-deploy, 237ms steady state per request, versus 8ms for a pure local hit), with 16% of cold requests burning the full 1s has-probe timeout and 2-block requests costing exactly two sequential peer rounds.Changes
Three independent changes, each aimed at one measured cost:
peerHasTimeout1s → 150ms.FetchFromPeerswaits for every peer's "no" before giving up, so the slowest peer in the fleet gates every cold fill. Healthy peers answer probes in single-digit ms while the origin path costs ~200ms — once a probe has gone unanswered for ~150ms, waiting longer only delays a faster fallback.FetchFromPeersround per block in sequence. Phase 1 consumes each block's result in order, so miss-run coalescing, single-flight keys, and thelastIdxclamp logic are unchanged. The fill check precedes theHascheck so a block landed by our own prefetch is attributed topeer, notlocal.New observability:
cache_proxy_peer_fill_hedged_totalcounter and ablocks_hedgedfield on theServed.log line.Testing
TestServeBlockAlignedPeerFillsRunConcurrently: peer/cache/gethandlers gate on all three of a request's blocks being fetched at once against a closed origin — a regression to sequential fills can never open the gate and fails loudly instead of assembling the response.TestServeBlockAlignedHedgesSlowPeerToOrigin: a peer that claims the block but stalls the transfer; asserts the request completes from origin within the budget and the hedge counter increments.-race(the pre-existingTestHandleConnectLogsOpenAndCloserace reproduces on unmodified main and is untouched).