fix(crdt): make flush cost proportional to what changed

[mkhairi]

Summary

Commits 1–3 of the shape agreed in NodeDB-Lab/nodedb#230: flush no longer treats a collection's full snapshot as a cheap value to rewrite on every tick. An idle store now does no snapshot work, and a store under sustained writes pays per flush in new operations rather than in document size.

Commit 4 of that plan — separating local restore from peer admission — is nodedb-crdt code and follows as a separate PR on the nodedb repo. This branch contains no changes there, and that one contains none here.

Changes

fix(crdt): export a collection's snapshot only when it has changed

CrdtEngine records the frontier each collection had when its snapshot was last written; flush() exports only collections whose oplog_version_vector() has moved. The mark is recorded after the batch commits, so a failed write leaves the collection dirty and is retried rather than skipped. compact_history and compact_at_version rewrite the document without advancing the frontier, so they drop the marks they invalidate. crdt_snapshot_export_count() exposes the export count so the property can be asserted directly instead of inferred from a timing.

feat(crdt): write incremental updates between snapshot checkpoints

Between checkpoints a flush exports the operations since the last persisted frontier under loro_delta:<collection>:<seq>, and rewrites the base snapshot only once accumulated updates reach a quarter of it. A checkpoint deletes the updates it supersedes in the same batch, so no restore replays operations the new base already contains. Restore replays the updates in key order after importing the base and seeds the checkpoint accounting from what it found, so the first flush after an open does not rewrite a base that is already current. An update failing its CRC32C check is an error rather than a warning: opening without it would silently roll the collection back to the checkpoint.

fix(core): let a slow flush delay its next tick instead of bursting

Tokio replays missed ticks immediately by default, so a flush that outlasted its own interval was followed by the ticks it missed with no gap between them, each taking the crdt lock. Both periodic tasks now measure the next period from the end of the previous pass. This is already the WASM behaviour, so it changes native only.

Design notes

A separate key space for the durability updates. The issue points at crdt:delta:{mutation_id} and restore_pending_deltas_incremental as the existing incremental path. Those entries are the sync outbox: they are deleted when Origin acknowledges them, not when a base snapshot absorbs them. Using that queue as the durability log would drop the state of every row Origin had already acknowledged. Hence loro_delta:<collection>:<seq> in Namespace::LoroState, replayed on open and deleted by the checkpoint that contains it. Happy to rename the keys if a different scheme fits better.

Half of the lock work is not in this branch. Bounding the tick cadence is here; moving the export itself out from under self.crdt is not. It needs a CrdtState handle that can be exported while the engine lock is released, and _single_owner: PhantomData<Cell<()>> makes sharing one a compile error by design. That is nodedb-crdt's contract to change, so it belongs with the decision about how such a handle should be given out rather than with a workaround here.

Tests

nodedb-lite/tests/crdt_flush_dirty_tracking.rs, all counter assertions rather than timings, so they fail for the reason they name on any machine:

• an idle store performs zero snapshot exports across 8 flushes
• a write made after a flush is still persisted and survives a reopen
• 63 flushes, each with one small write behind it, perform zero additional snapshot exports
• updates written between checkpoints replay on open rather than rolling back to the checkpoint

Each was confirmed to fail with the fix disabled.

A fourth test — file size within a bound of live-state size — was written and dropped. At test scale pagedb reclaims the superseded pages when nothing pins them, so 128 full rewrites left the file under 64 KB and the test passed against the unfixed code. A test that goes green on the bug is worse than no test; the export counter is the form that discriminates.

Validation

• cargo fmt --all — clean
• cargo clippy --workspace --all-targets -- -D warnings — clean
• cargo nextest run -p nodedb-lite — 24 failures, all reproducing on unmodified main: 4 in auto_flush (reopen storage: already open, pagedb releasing its advisory writer lock asynchronously) and 20 in sync_interop_* (connect to Origin pgwire: password missing, no Origin available in this environment)

Acceptance

The preserved store that reproduces the original fault needs commit 4 before it will open, so it cannot exercise this branch yet. It is kept, and I will report the result once that PR lands.

[mkhairi]

CI here stops at dependency resolution, before reaching this branch's code:

error: failed to select a version for the requirement `nodedb-array = "^0.5"`
candidate versions found which didn't match: 0.4.0, ...

The workspace pins all fourteen nodedb-* crates at 0.5; crates.io still has them at 0.4.0. It reproduces on a clean checkout of main with no .cargo/config.toml, so it isn't specific to this branch — the workspace resolves locally only through that file's path patches, and it's gitignored. Rebasing onto 789122b didn't change it.

Locally on the rebased branch: fmt and clippy --workspace --all-targets -D warnings clean, the four new tests pass, and the full suite has 30 failures against 40 on unmodified main — both dominated by tests needing an Origin and by pagedb's advisory lock releasing asynchronously.