Skip to content

fix(dash-spv): avoid filter sync getting stuck on the first new block after a restart#835

Open
xdustinface wants to merge 2 commits into
devfrom
fix/filters-tip-boundary-stall
Open

fix(dash-spv): avoid filter sync getting stuck on the first new block after a restart#835
xdustinface wants to merge 2 commits into
devfrom
fix/filters-tip-boundary-stall

Conversation

@xdustinface

@xdustinface xdustinface commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

Fixes #824. When a fully synced client restarted and then a new block arrived, the block's filter was downloaded but never processed. The client stayed one block behind forever and never reported synced again.

  • The restart path left the filter download state uninitialized. The next block then triggered a re-download of every filter from height 1, and the in-order store loop got stuck waiting for a batch that could never come. start_download now anchors the store and processing cursors and parks the pipeline at the frontier before returning early on an already-synced wallet.
  • The reconnect path after losing all peers had the same gap. The already-synced branch of start_sync now delegates to start_download instead of duplicating that logic, which also downloads the boundary filter body when the restored filter header tip is ahead of the stored bodies.
  • A new block now always gets a processing batch, even while an older rescan is still draining.
  • New tests: unit regressions for both restart paths (verified to fail without the fix), a reconnect test with filter headers ahead of stored filter bodies, and a dashd integration test that mines a block right after a fully synced restart. That integration test is the one that exposed the real bug.

Summary by CodeRabbit

  • Bug Fixes
    • Improved restart behavior so sync resumes from the correct frontier after a fully synced or partially synced restart.
    • Fixed boundary block handling during historical rescan, helping new headers trigger the expected follow-up work even while batches are draining.
    • Reduced the chance of stale progress tracking after startup, which helps avoid missed or delayed filter processing.
  • Tests
    • Added regression coverage for synced restarts, filter-header restoration, and boundary-block processing during rescan.

@coderabbitai

coderabbitai Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Restart and syncing filter paths now anchor scan cursors from the stored filter tip, create lookahead batches even while active batches exist, and cover the restart/rescan boundary cases with updated regression tests.

Changes

Filters commit boundary fix

Layer / File(s) Summary
Expose internal cursors
dash-spv/src/sync/filters/manager.rs
next_batch_to_store, processing_height, and is_idle are module-visible for in-module assertions and control flow.
Anchor download frontier in start_download
dash-spv/src/sync/filters/manager.rs
start_download derives download_start from the stored filter tip, anchors the cursors before early return, and initializes the pipeline at the computed frontier.
Anchor cursors on restart
dash-spv/src/sync/filters/sync_manager.rs
start_sync updates the filter header tip for the fully synced restart path and delegates to start_download instead of initializing the pipeline directly.
Lookahead and boundary regressions
dash-spv/src/sync/filters/manager.rs, dash-spv/tests/dashd_sync/tests_restart.rs
handle_new_filter_headers always creates lookahead batches while syncing behind the tip, and the tests cover the restart and rescan boundary cases, including the #824 boundary-block scenario.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
    participant Client
    participant SyncManager
    participant FiltersManager
    participant Pipeline
    participant Peer

    Client->>SyncManager: start_sync()
    SyncManager->>SyncManager: set filter header tip to stored_filters_tip
    SyncManager->>FiltersManager: start_download()
    FiltersManager->>FiltersManager: compute download_start from stored tip
    FiltersManager->>Pipeline: init/park at download_start

    Peer-->>FiltersManager: new filter headers advance tip
    FiltersManager->>FiltersManager: handle_new_filter_headers()
    FiltersManager->>Pipeline: try_create_lookahead_batches()
Loading

Possibly related PRs

  • dashpay/rust-dashcore#816: Both PRs adjust start_download in the same area to avoid leaving scan cursors at a stale frontier after restart.

Suggested labels: ready-for-review

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes the main restart sync bug being fixed.
Linked Issues check ✅ Passed The changes anchor restart cursors and force boundary batch creation, matching #824’s restart/reconnect gap and stalled committed_height requirements.
Out of Scope Changes check ✅ Passed All code and tests are scoped to filter-sync restart/reconnect boundary handling for #824, with no unrelated functional changes.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
✨ Finishing Touches
📝 Generate docstrings
  • Create stacked PR
  • Commit on current branch
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch fix/filters-tip-boundary-stall

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🧹 Nitpick comments (1)
dash-spv/src/sync/filters/sync_manager.rs (1)

90-97: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Cursor anchoring on the fully-synced restart path is correct.

Setting next_batch_to_store/processing_height to stored_filters_tip + 1 here mirrors the frontier anchoring now done in manager.rs::start_download, closing the reconnect-after-disconnect variant of the #824 wedge. Directly validated by test_start_sync_synced_restart_then_boundary_block_commits in manager.rs.

Since this "anchor cursors to the frontier" pattern now exists in two places (here and start_download's early return), consider extracting a small shared private helper (e.g. fn anchor_frontier(&mut self, height: u32)) to reduce the chance of a third restart path drifting out of sync with this fix in the future, similar to how this bug originated from two independently-evolving restart paths.

♻️ Possible shared helper (illustrative)
+    /// Anchor the store/processing cursors at `height` so a block arriving
+    /// before the next explicit batch creation doesn't wedge the in-order
+    /// store loop or create a lookahead batch from a stale frontier.
+    fn anchor_frontier(&mut self, height: u32) {
+        self.next_batch_to_store = height;
+        self.processing_height = height;
+    }
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dash-spv/src/sync/filters/sync_manager.rs` around lines 90 - 97, The cursor
anchoring logic is correct, but it is duplicated between the fully-synced
restart path in sync_manager.rs and the frontier-anchoring early return in
manager.rs::start_download. Extract that shared “anchor cursors to the frontier”
behavior into a small private helper such as anchor_frontier on the relevant
manager type, and call it from both places so next_batch_to_store and
processing_height stay synchronized and future restart paths cannot drift apart.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@dash-spv/src/sync/filters/sync_manager.rs`:
- Around line 90-97: The cursor anchoring logic is correct, but it is duplicated
between the fully-synced restart path in sync_manager.rs and the
frontier-anchoring early return in manager.rs::start_download. Extract that
shared “anchor cursors to the frontier” behavior into a small private helper
such as anchor_frontier on the relevant manager type, and call it from both
places so next_batch_to_store and processing_height stay synchronized and future
restart paths cannot drift apart.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: b33ee98d-66cc-45a1-83b7-8bea96bbde52

📥 Commits

Reviewing files that changed from the base of the PR and between a8a0968 and 9b5a8af.

📒 Files selected for processing (3)
  • dash-spv/src/sync/filters/manager.rs
  • dash-spv/src/sync/filters/sync_manager.rs
  • dash-spv/tests/dashd_sync/tests_restart.rs

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 2, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
dash-spv/src/sync/filters/sync_manager.rs (1)

85-97: 🎯 Functional Correctness | 🟠 Major | ⚡ Quick win

Start downloading when persisted filter headers are already ahead.

If a restarted manager restores filter_header_tip_height = 101 from storage while stored_filters_tip == committed_height == 100, this branch anchors cursors but then falls through to WaitForEvents at Line 110. No new FilterHeadersStored event is guaranteed, so the boundary filter body is never requested and sync can remain one block behind.

Proposed fix
         if stored_filters_tip > 0 && stored_filters_tip == self.progress.committed_height() {
             self.progress.update_filter_header_tip_height(stored_filters_tip);
             // Initialize the pipeline at the current tip. On full disconnect in-flight state gets
             // reset, so we need to initialize the pipeline otherwise it would re-queue from height 1.
             self.filter_pipeline.init(stored_filters_tip + 1, stored_filters_tip);
@@
             self.next_batch_to_store = stored_filters_tip + 1;
             self.processing_height = stored_filters_tip + 1;
+
+            if self.progress.filter_header_tip_height() > stored_filters_tip {
+                let mut events = vec![SyncEvent::SyncStart {
+                    identifier: self.identifier(),
+                }];
+                events.extend(self.start_download(requests).await?);
+                return Ok(events);
+            }
+
             // Only emit SyncComplete if we've also reached the chain tip

Please add a regression where FiltersManager::new restores filter headers past the stored filter tip before start_sync runs. As per coding guidelines, Write unit tests for new functionality.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@dash-spv/src/sync/filters/sync_manager.rs` around lines 85 - 97, The restart
path in FiltersManager::start_sync leaves the manager in WaitForEvents even when
update_filter_header_tip_height has restored filter_header_tip_height ahead of
stored_filters_tip, so the boundary body download is never triggered. Update the
start_sync logic to detect this persisted-ahead state and immediately begin
requesting the missing filter bodies instead of relying on a future
FilterHeadersStored event. Add a regression test around FiltersManager::new and
start_sync that restores filter headers past the stored tip and verifies the
manager starts downloading the next boundary block.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@dash-spv/src/sync/filters/sync_manager.rs`:
- Around line 85-97: The restart path in FiltersManager::start_sync leaves the
manager in WaitForEvents even when update_filter_header_tip_height has restored
filter_header_tip_height ahead of stored_filters_tip, so the boundary body
download is never triggered. Update the start_sync logic to detect this
persisted-ahead state and immediately begin requesting the missing filter bodies
instead of relying on a future FilterHeadersStored event. Add a regression test
around FiltersManager::new and start_sync that restores filter headers past the
stored tip and verifies the manager starts downloading the next boundary block.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0d8b4240-e176-4579-9228-a607341c5687

📥 Commits

Reviewing files that changed from the base of the PR and between 9b5a8af and a45445f.

📒 Files selected for processing (3)
  • dash-spv/src/sync/filters/manager.rs
  • dash-spv/src/sync/filters/sync_manager.rs
  • dash-spv/tests/dashd_sync/tests_restart.rs
🚧 Files skipped from review as they are similar to previous changes (1)
  • dash-spv/tests/dashd_sync/tests_restart.rs

coderabbitai[bot]
coderabbitai Bot previously approved these changes Jul 2, 2026
@codecov

codecov Bot commented Jul 2, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 97.88360% with 4 lines in your changes missing coverage. Please review.
✅ Project coverage is 73.57%. Comparing base (b0a909c) to head (5a1fdf2).

Files with missing lines Patch % Lines
dash-spv/src/sync/filters/manager.rs 97.88% 4 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##              dev     #835      +/-   ##
==========================================
+ Coverage   73.44%   73.57%   +0.13%     
==========================================
  Files         324      324              
  Lines       72577    72758     +181     
==========================================
+ Hits        53306    53534     +228     
+ Misses      19271    19224      -47     
Flag Coverage Δ
core 76.94% <ø> (ø)
ffi 47.97% <ø> (ø)
rpc 20.00% <ø> (ø)
spv 90.65% <97.88%> (+0.38%) ⬆️
wallet 72.61% <ø> (ø)
Files with missing lines Coverage Δ
dash-spv/src/sync/filters/sync_manager.rs 100.00% <ø> (ø)
dash-spv/src/sync/filters/manager.rs 97.68% <97.88%> (+0.14%) ⬆️

... and 5 files with indirect coverage changes

@github-actions github-actions Bot added the ready-for-review CodeRabbit has approved this PR label Jul 2, 2026
… after a restart

Fixes #824. A fully synced client that restarted, or reconnected after losing all peers, came back with its filter download state uninitialized. The next new block then re-requested every filter from height 1 and wedged the in-order store loop, so `committed_height` froze one block below the tip and the client never reported synced again.

`start_download` now anchors `next_batch_to_store` and `processing_height` and parks the pipeline at the download frontier before its already-synced early return. The already-synced reconnect branch of `start_sync` delegates to `start_download` instead of duplicating that logic, which also downloads the boundary filter body when the restored filter header tip is ahead of the stored bodies. `handle_new_filter_headers` now gives a new boundary block a processing batch even while a historical rescan is still draining `active_batches`.

Covered by unit regression tests for both restart paths, a reconnect test with filter headers ahead of stored filter bodies, and a dashd integration test that mines a block right after a fully synced restart.
@xdustinface xdustinface force-pushed the fix/filters-tip-boundary-stall branch from a45445f to f2b4c18 Compare July 2, 2026 08:55
@github-actions github-actions Bot removed the ready-for-review CodeRabbit has approved this PR label Jul 2, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@dash-spv/src/sync/filters/sync_manager.rs`:
- Around line 87-89: The SyncManager::start_download path is overwriting a newer
filter-header tip with stored_filters_tip when committed_height matches, which
can regress progress and skip the missing boundary body request. Update the
logic in SyncManager::start_download (or the surrounding filter-tip update path)
to preserve the already-known frontier by taking the max of the existing
filter_header_tip_height and stored_filters_tip before calling
update_filter_header_tip_height, so the later early-return check does not
short-circuit the needed download.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: b7db2055-4106-4e78-b7c2-0916f1539034

📥 Commits

Reviewing files that changed from the base of the PR and between a45445f and f2b4c18.

📒 Files selected for processing (3)
  • dash-spv/src/sync/filters/manager.rs
  • dash-spv/src/sync/filters/sync_manager.rs
  • dash-spv/tests/dashd_sync/tests_restart.rs
🚧 Files skipped from review as they are similar to previous changes (2)
  • dash-spv/tests/dashd_sync/tests_restart.rs
  • dash-spv/src/sync/filters/manager.rs

Comment thread dash-spv/src/sync/filters/sync_manager.rs
@github-actions github-actions Bot added the ready-for-review CodeRabbit has approved this PR label Jul 2, 2026
@xdustinface xdustinface requested a review from ZocoLini July 2, 2026 10:54
@github-actions github-actions Bot added ready-for-review CodeRabbit has approved this PR and removed ready-for-review CodeRabbit has approved this PR labels Jul 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-review CodeRabbit has approved this PR

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Compact-filter committed_height freezes one block below tip after client restart, leaving SyncProgress::is_synced() permanently false

3 participants