Skip to content

fix(archiver): serve log-by-tag queries from read-only snapshots - #25315

Open
spalladino wants to merge 1 commit into
spl/faster-get-private-logs-by-tagsfrom
spl/log-store-read-only-snapshots
Open

fix(archiver): serve log-by-tag queries from read-only snapshots#25315
spalladino wants to merge 1 commit into
spl/faster-get-private-logs-by-tagsfrom
spl/log-store-read-only-snapshots

Conversation

@spalladino

@spalladino spalladino commented Aug 25, 2026

Copy link
Copy Markdown
Contributor

Stack

Part of A-1817. Bottom to top, each PR targets the branch below it:

Context

Fixes A-1817.

#25254 kept tag log queries inside db.transactionAsync to preserve snapshot consistency, which on lmdb-v2 means routing them through the store's single serial writer queue: a query waits for every queued write (notably checkpoint ingestion) to commit before it even starts. #25280 added readOnlyTransaction, which gives the same one-snapshot guarantee without touching the writer queue, so the log store can have consistency and not queue.

Approach

  • LogStore's read paths (getPrivateLogsByTags, getPublicLogsByTags, getPrivateLogsForBlock, getPublicLogsForBlock) switch from db.transactionAsync to db.readOnlyTransaction. The inner reads already resolve the ambient transaction through acquireReadTx, so these are one-line swaps. addLogs and deleteLogs are untouched.

  • The class JSDoc no longer justifies staying on the writer queue; it documents the snapshot instead.

  • TAG_SCAN_CONCURRENCY drops from 8 to 2. Cursors bound to one read-only snapshot serialize on that snapshot's mutex in C++, so a wide pool only hides the per-cursor round trip. Measured on a seeded store (20 blocks x 4 txs x 5 private logs, 100-tag query, 30 iterations after warmup, two runs):

    TAG_SCAN_CONCURRENCY avg latency avg with 5 queries in flight
    1 7.51 / 7.32 ms 26.3 / 26.9 ms
    2 4.72 / 5.10 ms 26.9 / 27.8 ms
    4 4.89 / 5.00 ms 27.9 / 27.8 ms
    8 4.93 / 5.08 ms 27.9 / 28.2 ms
    16 5.15 ms 29.3 ms

    Going from 1 to 2 buys a third of the latency; everything above 2 is flat uncontended and marginally worse under concurrency, while holding more cursors.

  • Each open snapshot holds one of the LMDB reader slots, so MAX_READERS is no longer hardcoded in kv-store/src/lmdb-v2/factory.ts: it comes from DataStoreConfig.dataStoreMaxReaders (DATA_STORE_MAX_READERS), defaulting to the previous 16.

Testing

log_store_write_contention now asserts the read does not wait for the queued writes. With 5 x 50ms writes queued in front of it:

  • on transactionAsync (before): baseline 10.56ms, contended 259.92ms — fails
  • on readOnlyTransaction (after): baseline 11.40ms, contended 8.39ms — passes

The bound is contendedMs < baselineMs + queuedWriteMs / 2, leaving 125ms of jitter headroom so it only trips when the read is genuinely serialized.

The log store's read paths now run inside `db.readOnlyTransaction` instead of
`db.transactionAsync`. They still see one consistent snapshot for the
`referenceBlock` reorg check and every per-tag scan, but no longer go through
the store's single serial writer queue, so a query does not wait for queued
writes (notably block ingestion) to commit.

Only the read paths move; `addLogs` and `deleteLogs` stay on
`transactionAsync`.

Cursors bound to one snapshot serialize on that snapshot's mutex natively, so
`TAG_SCAN_CONCURRENCY` drops from 8 to 2: on a seeded store a 100-tag query
averages 7.4ms at 1, 4.9ms at 2, and 5.0ms at 4 and 8, and with five queries in
flight the wider pools are marginally slower.

Each open snapshot holds one LMDB reader slot, so the reader table size is now
configurable via `DATA_STORE_MAX_READERS` / `DataStoreConfig.dataStoreMaxReaders`
instead of being hardcoded, keeping the previous 16 as the default.
@spalladino
spalladino force-pushed the spl/log-store-read-only-snapshots branch from ddb0e53 to fa7156a Compare August 25, 2026 21:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant