feat(kv-store): expose read-only lmdb transactions - #25280
Open
spalladino wants to merge 1 commit into
Open
Conversation
`store.readOnlyTransaction(cb)` opens a real LMDB read transaction and keeps it open for the whole callback, so every read inside sees one snapshot. Readers do not go through the writer queue, so a write can commit while the callback runs without the callback observing it. The transaction is propagated through an AsyncLocalStorage, so container reads (`map.getAsync`, `entriesAsync`, ...) inside the callback hit the snapshot without being handed the transaction explicitly. Nested calls reuse the enclosing transaction, matching how `transactionAsync` handles recursion. Each open snapshot consumes an LMDB reader slot, so it acquires from the same semaphore as cursors; cursors bound to a snapshot skip acquisition since they reuse its slot, which the store now tracks per cursor id to avoid over-releasing on close. `readOnlyTransaction` is added to `AztecAsyncKVStore`; the other backends have no snapshot of their own that outlives an operation, so they delegate to their regular transaction, which gives the callback a consistent view. On the native side this adds START_READ_TX / CLOSE_READ_TX messages and an optional txId on GET and START_CURSOR, so the JS side can hold one LMDB read transaction open across many reads and iterations. LMDBStore gets a get() overload that reads against a caller-supplied read transaction. LMDBStoreWrapper keeps a registry of read transactions mirroring the cursor registry; because a read transaction must never be used by two threads at once, each one carries a mutex that every get and every cursor bound to it locks. Cursors record that mutex so advance_cursor serializes against sibling cursors and gets on the same snapshot.
spalladino
force-pushed
the
spl/kv-store-read-only-tx
branch
from
August 25, 2026 19:34
7b2bbb8 to
3a922c5
Compare
spalladino
changed the base branch from
merge-train/spartan-v5
to
spl/faster-block-ingestion
August 25, 2026 19:34
This was referenced Aug 25, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stack
Part of A-1817. Bottom to top, each PR targets the branch below it:
spl/faster-block-ingestion->merge-train/spartan-v5— cheaper checkpoint ingestion, so writes hold the store for less timespl/kv-store-read-only-tx->spl/faster-block-ingestion— addsreadOnlyTransactionplustxIdon the nativeGET/START_CURSOR<- this PRspl/batched-kv-gets->spl/kv-store-read-only-tx— addsgetMany/getManyAsync; depends on feat(kv-store): expose read-only lmdb transactions #25280 fortxIdonGetRequestspl/faster-get-private-logs-by-tags->spl/batched-kv-gets— faster tag scans; depends on fix(archiver): batch multi-key point reads into one LMDB round trip #25282 forgetManyAsyncon the log store's per-block readsspl/log-store-read-only-snapshots->spl/faster-get-private-logs-by-tags— moves the log store's reads onto snapshots; depends on feat(kv-store): expose read-only lmdb transactions #25280 forreadOnlyTransactionContext
AztecLMDBStoreV2only exposestransactionAsync, which opens a WRITE transaction serialized through the writer queue. The TSReadTransactiongives no snapshot consistency: everyGETmessage opens and aborts a throwaway LMDB read tx in C++, so two consecutive reads can straddle a commit. LMDB supports many concurrent readers that never block the writer, but that capability was not exposed to TS.Approach
START_READ_TX/CLOSE_READ_TXmessages to the node addon protocol, plus an optionaltxIdonGETandSTART_CURSOR. The C++ wrapper keeps a registry of live read transactions (mirroring the existing cursor registry), each guarded by a mutex since an LMDB read tx must not be used concurrently across the libuv pool threads; cursors opened against a shared tx serialize on that same mutex.lmdblib::LMDBStore::getgains an overload that reads through a caller-provided read transaction instead of opening its own.store.readOnlyTransaction(callback)opens one C++ read tx and propagates it viaAsyncLocalStorage, so plain container reads (map.getAsync, iteration) inside the callback hit the snapshot automatically. Nested calls reuse the ambient read or write tx, matchingtransactionAsyncsemantics.maxReaders - 1semaphore; cursors created inside a snapshot skip slot acquisition since they share the tx's reader slot.API changes
AztecAsyncKVStoregainsreadOnlyTransaction<T>(callback: () => Promise<T>): Promise<T>: runs the callback against a consistent read-only snapshot without blocking concurrent writers. The sqlite-opfs, indexeddb, and v1 lmdb implementations delegate to their existing transaction machinery (consistent view, no snapshot concurrency).