Skip to content

TraceDB: Snapshot-backed state for the trace baker#3360

Open
Kbhat1 wants to merge 5 commits intopr/trace-baker-mainfrom
pr/trace-snapshot-main
Open

TraceDB: Snapshot-backed state for the trace baker#3360
Kbhat1 wants to merge 5 commits intopr/trace-baker-mainfrom
pr/trace-snapshot-main

Conversation

@Kbhat1
Copy link
Copy Markdown
Contributor

@Kbhat1 Kbhat1 commented May 1, 2026

Describe your changes and provide context

  • Adds optional snapshot-backed trace baking so the baker can replay from in-memory memiavl state instead of SS-pebble.
  • Refcounts memiavl snapshots and releases trace leases through geth's existing StateReleaseFunc, avoiding GC finalizers.
  • Opt-in via trace_bake_use_snapshot; falls back when the backend cannot provide a snapshot.

Testing performed to validate your change

  • Verified on node
  • Unit tests
  • Long-running node on mainnet
  • go test ./sei-db/state_db/sc/memiavl -run 'TreeCopy|Snapshot' -count=1

@Kbhat1 Kbhat1 force-pushed the pr/trace-snapshot-main branch 4 times, most recently from 1764fc9 to c173794 Compare May 1, 2026 15:42
@Kbhat1 Kbhat1 changed the base branch from main to pr/trace-baker-main May 1, 2026 15:46
@Kbhat1 Kbhat1 changed the title Snapshot-backed state for the trace baker TraceDB: Snapshot-backed state for the trace baker May 1, 2026
@Kbhat1 Kbhat1 force-pushed the pr/trace-baker-main branch from 7b8a363 to af0de0f Compare May 1, 2026 15:59
@Kbhat1 Kbhat1 force-pushed the pr/trace-snapshot-main branch 2 times, most recently from 2412434 to e0a56bd Compare May 1, 2026 18:22
@Kbhat1 Kbhat1 force-pushed the pr/trace-baker-main branch from af0de0f to 6981a66 Compare May 1, 2026 19:05
@Kbhat1 Kbhat1 force-pushed the pr/trace-snapshot-main branch from e0a56bd to c5e3d21 Compare May 1, 2026 19:07
@Kbhat1 Kbhat1 force-pushed the pr/trace-baker-main branch from 6981a66 to fe9ec89 Compare May 1, 2026 21:02
@Kbhat1 Kbhat1 force-pushed the pr/trace-snapshot-main branch from c5e3d21 to 4bfc441 Compare May 1, 2026 21:04
@sei-protocol sei-protocol deleted a comment from github-actions Bot May 4, 2026
@Kbhat1 Kbhat1 force-pushed the pr/trace-baker-main branch from ae34e85 to f861506 Compare May 4, 2026 21:03
@Kbhat1 Kbhat1 force-pushed the pr/trace-snapshot-main branch from 4bfc441 to cf01744 Compare May 4, 2026 21:05
@github-actions
Copy link
Copy Markdown

github-actions Bot commented May 4, 2026

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedMay 5, 2026, 10:55 PM

@Kbhat1 Kbhat1 force-pushed the pr/trace-baker-main branch from 7c210c4 to 27dc9b0 Compare May 4, 2026 21:40
Stacks on the trace baker PR. Captures an O(1) memiavl snapshot of
the SC tree at EndBlock and serves trace re-execution from in-RAM
state instead of SS-pebble.

memiavl: refcount *Snapshot. Tree.Copy() Acquires; Snapshot.Close
unmaps only on the final release. Without this a held copy was a
use-after-munmap waiting to happen — the background snapshot rewrite
calls Tree.ReplaceWith → snapshot.Close mid-flight, segfaulting any
held copy. The internal rewrite goroutine also drops its clone's
ref so the refcount can reach zero.

Committer interface gains Copy(). memiavl delegates to *DB.Copy.
composite returns nil when flatkv is engaged so the snapshot path
silently falls back.

storev2 rootmulti adds SnapshotSCStore + CacheMultiStoreFromCommitter.

EVM keeper: TraceSnapshotStore (bounded by-height map) and EndBlock
capture keyed by snapshot.Version() (= H-1 at EndBlock(H)).

App: SnapshotAwareRPCContextProvider builds the sdk.Context directly
from the snapshot CMS to skip the throwaway CacheMultiStoreWithVersion
that CreateQueryContext would otherwise make.

Configurable via [evm]:
  trace_bake_use_snapshot     (default false)
  trace_bake_snapshot_window  (default 64)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Kbhat1 Kbhat1 force-pushed the pr/trace-snapshot-main branch from cf01744 to c28c23f Compare May 4, 2026 21:41
@codecov
Copy link
Copy Markdown

codecov Bot commented May 4, 2026

Codecov Report

❌ Patch coverage is 44.14414% with 124 lines in your changes missing coverage. Please review.
✅ Project coverage is 59.28%. Comparing base (27dc9b0) to head (784b639).
⚠️ Report is 23 commits behind head on pr/trace-baker-main.

Files with missing lines Patch % Lines
app/app.go 25.00% 26 Missing and 4 partials ⚠️
sei-cosmos/storev2/rootmulti/store.go 0.00% 21 Missing ⚠️
sei-db/state_db/sc/composite/store.go 0.00% 16 Missing ⚠️
sei-db/state_db/sc/memiavl/store.go 0.00% 13 Missing ⚠️
evmrpc/simulate.go 71.05% 4 Missing and 7 partials ⚠️
x/evm/keeper/trace_snapshot.go 79.62% 6 Missing and 5 partials ⚠️
evmrpc/config/config.go 0.00% 4 Missing and 2 partials ⚠️
sei-db/state_db/sc/memiavl/db.go 14.28% 6 Missing ⚠️
x/evm/keeper/abci.go 0.00% 2 Missing and 1 partial ⚠️
x/evm/keeper/keeper.go 25.00% 3 Missing ⚠️
... and 2 more
Additional details and impacted files

Impacted file tree graph

@@                   Coverage Diff                   @@
##           pr/trace-baker-main    #3360      +/-   ##
=======================================================
- Coverage                59.31%   59.28%   -0.03%     
=======================================================
  Files                     2099     2100       +1     
  Lines                   173370   173676     +306     
=======================================================
+ Hits                    102826   102970     +144     
- Misses                   61652    61789     +137     
- Partials                  8892     8917      +25     
Flag Coverage Δ
sei-chain-pr 64.89% <44.14%> (+2.27%) ⬆️
sei-db 70.41% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
evmrpc/tracers.go 63.12% <ø> (ø)
sei-db/state_db/sc/memiavl/tree.go 88.20% <100.00%> (+0.12%) ⬆️
evmrpc/server.go 87.79% <60.00%> (-0.67%) ⬇️
sei-db/state_db/sc/memiavl/snapshot.go 63.23% <84.61%> (+0.63%) ⬆️
x/evm/keeper/abci.go 55.55% <0.00%> (-1.59%) ⬇️
x/evm/keeper/keeper.go 54.54% <25.00%> (-0.36%) ⬇️
evmrpc/config/config.go 70.43% <0.00%> (-3.88%) ⬇️
sei-db/state_db/sc/memiavl/db.go 66.71% <14.28%> (+0.04%) ⬆️
evmrpc/simulate.go 79.22% <71.05%> (-1.74%) ⬇️
x/evm/keeper/trace_snapshot.go 79.62% <79.62%> (ø)
... and 4 more
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Kbhat1 and others added 2 commits May 5, 2026 14:23
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Kbhat1 and others added 2 commits May 5, 2026 18:45
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant