Skip to content

Fix panic during state sync#3638

Open
cody-littley wants to merge 1 commit into
mainfrom
cjl/fix-STO-572
Open

Fix panic during state sync#3638
cody-littley wants to merge 1 commit into
mainfrom
cjl/fix-STO-572

Conversation

@cody-littley

Copy link
Copy Markdown
Contributor

Describe your changes and provide context

Fixes an issue where a node could be forced to panic during a state sync

@cursor

cursor Bot commented Jun 24, 2026

Copy link
Copy Markdown

PR Summary

High Risk
This closes an exploitable panic on the state-sync restore path (malicious snapshots), though the behavioral change is confined to error handling in restore when SS is enabled.

Overview
Fixes a state-sync denial-of-service where malformed flatkv snapshot data could crash a syncing node. The SS import path used to run ssStore.Import in a goroutine and panic on any error; a peer could supply leaves that fail convertFlatKVNodes (e.g. keys without a module prefix) and take the process down.

restore now runs SS import through errgroup: import errors are collected after the snapshot read loop via ssGroup.Wait() and returned as restoreErr instead of panicking. Forwards to ssImporter use a select on ssCtx.Done() so if SS import stops draining the channel, the main loop does not block forever when the buffer fills.

Adds TestFlatKVMalformedSnapshotReturnsErrorNotPanic, which feeds a crafted flatkv snapshot stream and asserts Restore returns an error without crashing the test process.

Reviewed by Cursor Bugbot for commit 87b5344. Bugbot is set up for automated code reviews on this repo. Configure here.

@github-actions

github-actions Bot commented Jun 24, 2026

Copy link
Copy Markdown

The latest Buf updates on your PR. Results from workflow Buf / buf (pull_request).

BuildFormatLintBreakingUpdated (UTC)
✅ passed✅ passed✅ passed✅ passedJun 24, 2026, 7:08 PM

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 87b5344. Configure here.

}
if err := ssGroup.Wait(); err != nil && restoreErr == nil {
restoreErr = fmt.Errorf("ss import failed during restore: %w", err)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SS version set after import failure

Medium Severity

Replacing the SS import panic with ssGroup.Wait() lets restore continue after a failed ssStore.Import and still call SetEarliestVersion and SetLatestVersion at the restore height. The caller gets a non-nil error, but the state store can report it is synced to that height despite a failed or partial import.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit 87b5344. Configure here.

@codecov

codecov Bot commented Jun 24, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 88.88889% with 1 line in your changes missing coverage. Please review.
✅ Project coverage is 58.15%. Comparing base (197fe21) to head (87b5344).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
sei-cosmos/storev2/rootmulti/store.go 88.88% 1 Missing ⚠️
Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #3638      +/-   ##
==========================================
- Coverage   59.03%   58.15%   -0.89%     
==========================================
  Files        2252     2176      -76     
  Lines      185720   176896    -8824     
==========================================
- Hits       109645   102878    -6767     
+ Misses      66258    64928    -1330     
+ Partials     9817     9090     -727     
Flag Coverage Δ
sei-chain-pr 66.71% <88.88%> (?)
sei-db 70.41% <ø> (ø)
sei-db-state-db ?

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
sei-cosmos/storev2/rootmulti/store.go 68.69% <88.88%> (+0.66%) ⬆️

... and 76 files with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant