Release readiness (full rerun on the reactive + cold-start main)#15
Merged
Conversation
Final pre-public-release pass closing the gaps surfaced by the readiness review. No behavioral regressions; the full CI gate (fmt, clippy -D warnings, tests, doctests, cargo doc -D warnings, cargo package) stays green. Engine / API - chain_id is now configurable instead of hard-defaulting to Arbitrum (42161). Add `EvmCacheBuilder::chain_id` (recommended) and `EvmCache::set_chain_id`; when unset, infer from the provider via `eth_chainId` and fall back to 1 (Ethereum mainnet). The inference also upgrades the bare `new`/`at_block` constructors. Resolution order: explicit builder value > disk CacheConfig > inferred > 1. (Breaking: the in-memory default changed from 42161 to inferred/1.) - events: `ingest_logs` now tracks `skipped_accounts` so a third-party decoder emitting cold `StateUpdate::Account` patches stays visible to reorg purge and freshness, matching the "every category" contract. Documentation accuracy - Narrow the freshness `Validation::Confirmed`/`Corrected` docs: reconcile covers volatile storage slots only, not account balance/nonce/code. Record the scope in KNOWN_ISSUES. - create3: relabel the factory as the ZeframLou/Solmate `CREATE3Factory` (LiFi-deployed) and disclaim that it is NOT pcaversaccio's CreateX. - access_list: drop the AMM (V2/V3 slot-selection) framing from the module and struct docs; the builder applies no per-entry selection. - Fix the StateDiff skip-accessor docs (all four skip categories), the cold definition (NotExisting overlay accounts), the cancellation/`# Panics`/empty `token_deltas` notes on the freshness optimistic loop, the `load_binary_state` read-error logging, and the overlay `block_hash`/ZERO behavior. Add KNOWN_ISSUES entries for BLOCKHASH-zero, unbounded `derived_slots`, deep-reorg under-purge, and >= 2^255 transfer magnitude. - Remove the never-shipped `SimulationErrorKind` deprecated alias. Benchmarks / packaging / community - Add a README "## Performance" section with measured COW-vs-deep-clone `create_snapshot` and overlay-reuse numbers, methodology, and reproduction. - Cargo.toml: exclude the internal phase specs and CI workflow from the package. - Add SECURITY.md and a dated CHANGELOG 0.1.0 section. Tests - Offline Multicall3 `aggregate3` coverage (input order + allowFailure) by etching the real Multicall3 runtime (new fixture), and a `TxConfig.access_list` EIP-2929 gas-accounting test. New `tests/builder_chain_id.rs` pins the chain_id order. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The README "Performance" section led with a copy-on-write `create_snapshot` vs `create_snapshot_deep_clone` table — an internal implementation A/B a first-time reader never chooses between. Replace it with the outcomes a searcher actually cares about, each measured against the fork-per-candidate loop they'd otherwise write. Four pillars, two of them exact CI-pinned integer counts: - **Data-fetch minimization (headline).** New `examples/fetch_minimization_counted.rs` and `tests/fetch_minimization.rs`: 500 candidates over a shared 8-slot hot set fetch 8 slots once (zero during fan-out) vs 4,000 fork-per-candidate — ~500x fewer RPC reads, a deterministic machine-independent count. - **Candidate-fan-out throughput.** New `benches/fanout.rs`: snapshot-once + overlay-per-candidate vs a full independent fork per candidate; per-candidate cost falls as N grows (~545x at N=128 over a 32k-slot fork). - **Reactive sync.** `tests/event_pipeline.rs` pins that `ingest_logs` keeps hot slots fresh with zero RPC fetches/block; `examples/reactive_cache.rs` prints it (0 during ingest, sampled reconcile as the honesty backstop). - **Optimistic latency-hiding.** Promote the existing `benches/freshness.rs` latency arms (time-to-result vs fetch-then-sim) in the README, labeled with the modeled 50ms delay. Honesty guardrails: both shared-hot-set (~Nx) and disjoint (1x, no win) cases are stated; wall-clock ratios are labeled machine-dependent and separated from the exact integer counts; the injected 50ms is called out as modeled, not measured. The internal COW-vs-deep-clone cost model moves to `docs/INTERNALS.md` (the `simulation` bench keeps it as a regression guard). `benches/fanout.rs` is registered in Cargo.toml. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
An adversarial review of the benchmark reframe flagged a framing gap: the candidate-throughput baseline (`fork_per_candidate`) is `create_snapshot_deep_clone`, an in-memory deep copy that does NO fetching — so the ~545× throughput speedup is snapshot-construction/isolation cost, not RPC avoidance. The "fair baseline" box implied the baseline cold-fetches, conflating the throughput win (②) with the separate fetch-count win (①). Disambiguate in the README Performance section: ① measures the fetch cost (a cold cache per candidate re-fetches), ② measures the isolation/CPU cost (a full independent clone per candidate, no RPC) — the two are separate and not double-counted. All measured numbers are unchanged and verified accurate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Visuals for the value-prop performance story: - assets/fetch_amplification.svg — the headline: fetches stay flat at 8 while fork-per-candidate grows linearly to 4,000 over 500 candidates (~500x). - assets/fanout_throughput.svg — per-candidate cost (log scale) falls 41→4 µs as N grows while fork-per-candidate stays flat near 2.1 ms (~545x at N=128). Both are self-contained dark-card SVGs (no external CSS) so they render on either GitHub theme; embedded via absolute raw-main URLs so they also resolve on crates.io after merge. - Mermaid state-stack/control-plane flowchart replacing the ASCII diagram in "Core concepts" (lib.rs keeps the ASCII for docs.rs), plus a sequence diagram of the optimistic verify-and-rerun loop showing time-to-result decoupled from RPC validation. Numbers match the measured benchmarks already cited in the Performance section. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
… plainly A red-team of the benchmark baselines (steelmanning what a competent revm / foundry-fork-db user actually writes) found the headline multipliers were beating a strawman, not a real peer: - ~500x fewer fetches: a shared foundry-fork-db SharedBackend caches + dedups, so it also fetches each hot slot once within a block (its cache hit path is backend.rs:204-242). Within-block ratio is ~1x, not 500x. - ~545x throughput: the baseline (create_snapshot_deep_clone per candidate) is an O(total-state) copy nobody writes; checkpoint/revert isolation (the crate's own primitive) is O(touched), so single-threaded the gap is ~1x. - ~3,800x latency: ~99.97% of it is the injected 50ms sleep the baseline elects to pay; a competent loop doesn't block on a fetch before acting. I also re-measured the parallelism I expected to be the real win: it is only ~1.2x on these micro-sims (revm is allocation-bound per call, so it contends rather than scaling with cores) — so I do not headline a core-count multiplier either. Rewrites the README "Performance" section to say all this plainly (a candor table showing ~1x where it is ~1x) and lead with what genuinely holds: cross-block freshness (0 RPC fetches/block — foundry-fork-db's cache is not block-keyed), point-in-time consistency, modest-but-real parallel fan-out, and the act-then-validate control plane. benches/fanout.rs now measures sequential vs parallel (the honest comparison), not deep-clone-per-candidate. The two charts that visualized the inflated numbers are removed; a new cross_block_freshness.svg visualizes the one defensible quantitative win. The fetch example prints the shared-backend caveat. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Second readiness pass, now over the combined main (post-#14, which added the reactive runtime, cold-start, and the live AlloySubscriber). #13's release work (chain_id builder, SECURITY.md, dated CHANGELOG, Cargo exclude, honest benchmark reframe, doc sweep, fetch-minimization + builder tests) is carried forward via cherry-pick; this commit closes the new-surface findings from the rerun review: Doc accuracy (the merge left stale claims): - README "Maturity" note: was "live subscription driving remains consumer-provided" while the AlloySubscriber ships — rewritten to reflect the default-enabled live WS subscriber + cold-start, with the honest remaining caveats. - KNOWN_ISSUES "event-driven sync" entry updated the same way; added a note on the bounded reconnect dedup window and the partial reactive integration-test coverage. - Documented cold_start (absent everywhere): README bullet, lib.rs module map, CHANGELOG entry (incl. the serial-account / first-failure-abort cost). - Repositioned "Why it exists" toward the out-of-the-box reactive tooling. Config/contract doc fixes: - hook_backpressure: documented as reserved/no-op (dispatch is synchronous). - journal_depth: documented as load-bearing for reorg rollback (not "reserved"). - StorageBatchFetchFn: documented the one-tuple-per-requested-slot contract that verify_slots / cold-start rely on. - "scaffold" doc comments on the fully-implemented Resync/Reorg/Error reports reworded to describe what they actually report. - Pending-tx Unsupported message: made transport-neutral (was "polling mode"). Code: - reactive-ws: best-effort install of rustls' ring CryptoProvider at AlloySubscriber init (avoids a wss:// "no process-level CryptoProvider" panic), with a Cargo comment explaining why the rustls dep exists. Tests: - Added a positive InputSource::Backfill assertion (the dedup test only covered the suppression side); pins the README's backfill-marking claim. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- reactive: correct the `journal_depth` and `ReorgReport` docs — a reorg deeper than `journal_depth` (or any reorg when `journal_depth = 0`) recovers only the resident span; aged-out blocks are neither rolled back nor purged and the freshness loop is the backstop. Emit a `tracing::warn!` when a reorg references a block no longer in the journal so the under-recovery is observable, not silent. - docs: add a KNOWN_ISSUES entry for the journal_depth recovery horizon; refresh ROADMAP to reflect the shipped reactive runtime + AlloySubscriber + cold_start (new Phase 6 section, corrected the stale "consumer-provided WS/reorg" claims, reframed remaining-work toward 1.0). - examples: add the offline `reactive_runtime` example (handler -> StateUpdate -> apply with 0 RPC, then journaled reorg rollback); list it and fetch_minimization_counted in the README example tables. - benchmarks: the README fan-out figure now names the committed 64–1,024 sweep instead of an unrun 4,096-candidate batch. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- lib.rs: hoist the primary entry points to the crate root for discoverability —
cache::{EvmCache, EvmCacheBuilder, EvmSnapshot, EvmOverlay, TxConfig,
CallSimulationResult} and (cfg reactive) reactive::{ReactiveRuntime,
ReactiveHandler, ReactiveConfig}. Purely additive; the module paths still work.
- examples/cold_start.rs: offline discover-then-verify working-set warming over
the MockERC20 fixture — a ColdStartPlanner discovers the slots a balanceOf call
touches, then verifies + injects them via run_cold_start, printing the report.
- cold_start: add a module-level doctest (planner + run_cold_start call shape) and
a runnable ColdStartPlan doctest.
- README: list the cold_start example in the offline table.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add `CallTracer`, a revm `Inspector` that reconstructs the call-frame tree (`CallTrace`/`CallKind`/`CallStatus`) of a simulation via the call/call_end/create/create_end hooks (no opcode/step or SLOAD/SSTORE tracing). Add `InspectorStack<A, B>`, a composing inspector that fans out every hook to two inner inspectors. Add the public, inspector-generic `EvmOverlay::call_raw_with_inspector`, which attaches any `Inspector` (honoring `TxConfig`/`commit`) and returns the raw `ExecutionResult` (revert/halt is `Ok`, not `Err`) plus the inspector. New types live in `src/tracing.rs` and are re-exported at the crate root. Makes the manager-authored acceptance tests in `tests/call_tracer.rs` pass (C1 single frame, C2 nested subcall, C3 revert attribution, C4 stack composition). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add an ordered multi-transaction bundle execution primitive over cumulative overlay state, with a revert policy and miner/coinbase payment accounting. - New `bundle` module: BundleTx, BundleOptions, RevertPolicy (Atomic / AllowReverts), GasAccounting (Mainnet / Raw), TxOutcome, BundleResult; re-exported at the crate root. - `EvmOverlay::simulate_bundle`: one outer + per-tx inner checkpoints on a single overlay/EVM (no per-tx rebuild). Atomic aborts roll the whole bundle back; AllowReverts roll back only whitelisted-index reverts and continue. Mainnet accounting subtracts the burned base fee (Σ gas_used×basefee over the committed txs) from the beneficiary delta since the simulator disables the base fee and revm credits the full gas price; Raw returns the bare delta. All saturating. - `EvmCache::simulate_bundle`: snapshots internally and runs on a transient overlay; never mutates the cache. - `EvmCache::set_basefee(U256)`: installs the block base fee into the next snapshot so offline caches can exercise Mainnet accounting. Tests: manager-authored tests/bundle_simulation.rs (AB1-AB6) pass verbatim, plus agent-authored tests/bundle_simulation_extra.rs (empty bundle, AllowReverts atomic-fallback, commit=false isolation, cache non-mutation) and bundle.rs unit tests for the type defaults. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- examples/bundle_simulation.rs and examples/call_tracer.rs: runnable offline demos of simulate_bundle (cumulative state, revert policy, coinbase payment) and CallTracer/InspectorStack (nested call-frame tree + transfer capture in one pass). - fix: drop GasAccounting (Mainnet/Raw). revm's reward_beneficiary already credits only the priority fee and burns the base fee in-EVM (EIP-1559), so the beneficiary delta IS the honest miner payment; the Mainnet mode was subtracting the base fee a second time (under-reporting whenever basefee > 0). coinbase_payment is now the bare delta. AB4/AB5 + agent extras + the example updated to the correct semantics. - docs: ROADMAP Phase 7 (bundle sim + tracer shipped); README example rows; KNOWN_ISSUES note on AllowReverts reverted-tx gas; CHANGELOG + spec correction. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Add a 'bundle' entry to the lib.rs module map (it had 'tracing' but not 'bundle') and 'Bundle simulation' + 'Call-frame tracing' bullets to the README 'What it provides today' list, so the Phase 6 capabilities are discoverable from the crate-level docs, not just the example table. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Supersedes #13. After #14 landed the reactive runtime + cold-start + live
AlloySubscriberonmain, this is the full release-readiness pass re-run over the combined surface (#13 hardened the older events-API line, which has been superseded). Branched off the post-#14 main; #13's work is carried forward by cherry-pick, then the new surface is hardened. Full gate green (fmt, clippy-D warnings, 43 test binaries / 0 failures, 28 doctests,cargo doc -D warnings), and adversarially reviewed.Carried forward from #13 (cherry-picked, auto-merged cleanly)
chain_id builder + provider inference +
set_chain_id(replaces the Arbitrum-42161default);SECURITY.md; dated0.1.0CHANGELOG + link refs;Cargo.tomlexcludeof internal phase specs; the honest benchmark reframe (drops the strawman ~500×/~545×/~3,800× multipliers, states parity vs a competent shared-backend/checkpoint-revert baseline, leads with cross-block freshness + thecross_block_freshness.svgchart + mermaid diagrams); the create3/access_list/state_update doc sweep;fetch_minimization+builder_chain_idtests; removed the never-shippedSimulationErrorKindalias;docs/INTERNALS.md.New-surface hardening (this rerun)
Doc accuracy — the merge left two stale claims, both fixed: the README "Maturity" note (said live transport was consumer-provided while
AlloySubscriberships) and the matching KNOWN_ISSUES entry. Documentedcold_starteverywhere it was missing (README bullet, lib.rs module map, CHANGELOG, including the serial-account/first-failure-abort cost). Repositioned "Why it exists" toward the out-of-the-box reactive tooling.Config/contract docs —
hook_backpressuremarked reserved/no-op (dispatch is synchronous);journal_depthdocumented as load-bearing for reorg rollback;StorageBatchFetchFnone-tuple-per-slot contract documented; "scaffold" comments on the fully-implemented Resync/Reorg/Error reports reworded; transport-neutral pending-txUnsupportedmessage.Code — best-effort install of rustls'
ringCryptoProvideratAlloySubscriberinit (avoids awss://"no process-level CryptoProvider" panic), with a Cargo comment explaining therustlsdep.Tests — added the positive
InputSource::Backfillassertion (the existing test only covered dedup suppression). Remaining reactive integration-coverage gaps (subscriber→runtime composition, block-header ingest,Decodedreport,EventDecoderHandler, custom pending-tx routing) are tracked as honest follow-ups in KNOWN_ISSUES.Review verdict
No soundness blockers. The reactive runtime's
!Senddiscipline is correct, reorg recovery is pinned by state-equivalence tests, and the README's WS claims match the code. Findings were doc-accuracy/coverage/hygiene, addressed above.🤖 Generated with Claude Code