Skip to content

Fix store checkpoint ordering#40

Open
mattfaltyn wants to merge 2 commits into
bitcoinresearchkit:mainfrom
mattfaltyn:agent/fix-store-checkpoint-order
Open

Fix store checkpoint ordering#40
mattfaltyn wants to merge 2 commits into
bitcoinresearchkit:mainfrom
mattfaltyn:agent/fix-store-checkpoint-order

Conversation

@mattfaltyn

@mattfaltyn mattfaltyn commented Jul 19, 2026

Copy link
Copy Markdown

Summary

  • persist Fjall mutations before publishing completed-height checkpoints
  • keep background checkpoint tasks synchronized with the live store metadata
  • persist vectors before the ordered store commit
  • recover from the lower shared checkpoint if a crash leaves backends uneven
  • preserve the public Stores::take_all_pending_ingests batching API
  • cover dropped, successful, failed, aggregate, reopen, and uneven-tip cases

Root cause

Store::take_pending_ingest previously wrote its completed-height metadata
before the returned deferred ingest closure ran. A crash or ingest failure in
that window left a checkpoint claiming a height was complete even though its
transaction, address, or UTXO mutations were missing. Restart then trusted the
checkpoint and skipped the missing work.

This change treats the height as the final commit marker. Forward commits now
ingest all pending mutations, persist Fjall with SyncData, and only then
publish checkpoints. The background indexer persists vector data before
running that ordered store commit. Rollback uses the conservative inverse:
persist the lowered checkpoints first, then persist destructive removals.

If a crash leaves vectors ahead of store checkpoints, startup now selects the
lower shared durable tip. Store rollback consumes the still-ahead vector
history before vector truncation, then normal replay resumes without a full
rebuild. The former public batching method remains available as a compatibility
wrapper around the single ordered task.

Impact

Interrupted indexing can no longer leave store metadata ahead of durable Fjall
data. Data or vectors may lead a checkpoint after a failure, which is safe
because BRK recovers from the shared checkpoint and replays the affected
height.

Validation

  • cargo test -p brk_store -p brk_indexer --lib
  • cargo test --workspace
  • cargo check -p brk_indexer --all-targets
  • cargo clippy -p brk_store -p brk_indexer --lib --tests --no-deps -- -D warnings
  • cargo package --workspace --allow-dirty --no-verify
  • rustfmt --check on the changed store files and
    crates/brk_indexer/src/stores.rs
  • git diff --check

All commands above pass. A full-file rustfmt check of
crates/brk_indexer/src/lib.rs still reports an unchanged pre-existing
formatting difference outside this PR's hunks.

The repository's unchanged JavaScript/Python release harnesses were also
attempted. They are currently broken on main: the generated-client tests use
the removed series.prices path, and the Python mempool-compat suite imports
untracked _lib.py and _endpoints.py helpers. Neither suite exercises the
changed Rust persistence flow.

Fixes #39

@mattfaltyn
mattfaltyn marked this pull request as ready for review July 19, 2026 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Store checkpoints can advance before deferred Fjall ingests complete

1 participant