diff --git a/CHANGELOG.md b/CHANGELOG.md index bb33fe2..c8ff08b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,11 +12,51 @@ surface freezes at 1.0. ## [Unreleased] +### Added + +- **Bundle simulation + coinbase accounting (Phase 6 Track A+B).** New + `EvmOverlay::simulate_bundle` (and the cache-side convenience + `EvmCache::simulate_bundle`, which snapshots internally and never mutates the + cache) apply an ordered sequence of `Call`-kind transactions over **cumulative** + block state on a single overlay — transaction `i` observes the committed writes + of `0..i`. A `RevertPolicy` chooses between `Atomic` (any revert rolls the whole + bundle back) and `AllowReverts(indices)` (whitelisted reverts roll back only + their own transaction and execution continues). `BundleResult.coinbase_payment` + reports the miner payment as the block beneficiary's balance delta — the honest + priority fee plus any direct coinbase tips; revm already burns the base-fee + portion in-EVM (EIP-1559), so no base-fee correction is applied. New public types + live in the `bundle` module and are re-exported at the crate root: `BundleTx`, + `BundleOptions`, `RevertPolicy`, `TxOutcome`, `BundleResult`. +- **`EvmCache::set_basefee(U256)`** installs a block base fee (the `BASEFEE` + opcode) that propagates into the next `create_snapshot()`, so offline caches + with no fetched header can exercise base-fee-aware (`Mainnet`) bundle accounting. +- Call-frame tracing (`tracing` module): `CallTracer`, a `revm::Inspector` that + reconstructs the call-frame tree (`CallTrace`) of a simulation — top-level call + plus nested `CALL`/`STATICCALL`/`DELEGATECALL`/`CALLCODE` and `CREATE`/`CREATE2` + frames, each with from/to/value/input/gas/output/`CallStatus`/depth/subcalls. + Implemented via the `call`/`call_end`/`create`/`create_end` hooks (no + opcode/step or `SLOAD`/`SSTORE` tracing). Re-exported at the crate root as + `CallTracer`, `CallTrace`, `CallKind`, `CallStatus`. +- `InspectorStack`, a composing `revm::Inspector` that fans out every hook + to two inner inspectors so, e.g., a `CallTracer` and a `TransferInspector` + capture independently in one pass. Re-exported at the crate root. +- `EvmOverlay::call_raw_with_inspector`: a public, inspector-generic single-call + seam that attaches any `revm::Inspector` (honoring `TxConfig` and `commit`) and + returns the raw `ExecutionResult` (a revert/halt is `Ok`, not `Err`) alongside + the inspector for the caller to read. + +## [0.1.0] - 2026-06-25 + This is the first release line. It captures the work done across the pre-release development phases (see [`docs/ROADMAP.md`](docs/ROADMAP.md)). ### Changed +- **Breaking:** the in-memory chain ID is no longer hard-defaulted to Arbitrum + (`42161`). When no chain ID is set explicitly and no disk `CacheConfig` is + supplied, the cache now infers it from the provider (`eth_chainId`) and falls + back to `1` (Ethereum mainnet) only if that query fails. Set it explicitly with + the new `EvmCacheBuilder::chain_id` / `EvmCache::set_chain_id`. - **Breaking:** extracted the old in-crate AMM adapter surface before public release. Protocol-specific storage layouts, protocol metadata, injector helpers, tick snapshots, and protocol log decoders belong in `evm-amm-state`; @@ -33,12 +73,27 @@ pre-release development phases (see [`docs/ROADMAP.md`](docs/ROADMAP.md)). ### Added +- **Honest performance section & benchmarks.** The README "Performance" section + is framed against a *competent* baseline (a shared `foundry-fork-db` + `SharedBackend` + `checkpoint`/`revert` isolation), not a naive + fork-per-candidate strawman. It states plainly that within-block fetch count, + single-threaded CPU, and time-to-result are ~1× against that baseline, and + leads with the genuinely-unique wins: cross-block freshness (0 RPC fetches/block, + pinned in `tests/event_pipeline.rs` — `foundry-fork-db`'s cache is not + block-keyed), parallel `Send` fan-out (`benches/fanout.rs`, a modest measured + ~1.2× on micro-sims), point-in-time consistency, and the act-then-validate + control plane (`benches/freshness.rs`). Adds `examples/fetch_minimization_counted.rs` + and `tests/fetch_minimization.rs` (the fetch-once mechanic, with the + shared-backend caveat stated). The internal copy-on-write snapshot cost model + moved to [`docs/INTERNALS.md`](docs/INTERNALS.md). - **Forked EVM cache** (`cache::EvmCache`) backed by `foundry-fork-db` with lazy RPC loading and on-disk persistence for accounts, storage, bytecode, and immutable metadata. - **`EvmCacheBuilder`** — a fluent constructor (`EvmCache::builder(provider)`) subsuming the positional `with_cache` / `from_backend` constructors, with - block pin, EVM spec, cache-config, and shared-memory-capacity configuration. + block pin, EVM spec, cache-config, chain-ID, and shared-memory-capacity + configuration. `EvmCacheBuilder::chain_id` sets the `CHAINID` opcode value + explicitly (recommended); `EvmCache::set_chain_id` sets it post-construction. - **Snapshots and overlays** — `create_snapshot()` produces an immutable, `Send + Sync` `EvmSnapshot`; `EvmOverlay` is a cheap per-simulation clone for isolated parallel evaluation. @@ -164,7 +219,21 @@ pre-release development phases (see [`docs/ROADMAP.md`](docs/ROADMAP.md)). resync requests. Reversible storage-slot changes are rolled back in reverse apply order; account/code changes and prior purge effects conservatively fall back to targeted purge updates because `StateDiff` does not carry enough data - to reconstruct those cache entries exactly. Generic core. + to reconstruct those cache entries exactly. Recovery is bounded to the + configured `ReactiveConfig::journal_depth` (default 64): a reorg deeper than + that — or any reorg when `journal_depth = 0` — recovers only the blocks still + resident in the journal and emits a `tracing::warn!` for the under-recovered + span (the freshness loop is the backstop; see `docs/KNOWN_ISSUES.md`). + Demonstrated offline in `examples/reactive_runtime.rs`. Generic core. +- **Cold-start** (`cold_start` module, default-enabled / reactive-gated) — + declarative warming of a working set of accounts and storage slots into the + cache in one batched pass via `EvmCache::run_cold_start` / + `execute_cold_start_round` and a `ColdStartPlanner` (discover slots through a + view-call, then verify them through the `StorageBatchFetchFn`), returning a + structured `ColdStartRunReport` (`ColdStartConfig`/`Plan`/`Results`/ + `RoundSummary`/`Step`/`RoundOutcome`/`Error`, with per-slot `SlotOutcome`s). The + account-warming phase is serial (one fetch per declared account) and aborts the + round on the first account failure; storage verify/probe are batched. Generic core. - **`StateUpdate::SlotMasked`** (`state_update`, Phase 4) — a cold-aware read-modify-write *masked* slot write (`new = (old & !mask) | (value & mask)`) with the `StateUpdate::slot_masked` constructor, so a pure decoder can update @@ -277,7 +346,7 @@ pre-release development phases (see [`docs/ROADMAP.md`](docs/ROADMAP.md)). layouts. - Simulation entry points that distinguish failure modes return `SimulationResult` (`Result`), separating decoded reverts, - EVM halts, and host errors. `SimulationErrorKind` remains as a deprecated alias. + EVM halts, and host errors. ### Fixed @@ -369,4 +438,5 @@ pre-release development phases (see [`docs/ROADMAP.md`](docs/ROADMAP.md)). - `EvmCache` requires a multi-thread tokio runtime for any RPC-touching path. - See [`docs/KNOWN_ISSUES.md`](docs/KNOWN_ISSUES.md) for current limitations. -[Unreleased]: https://github.com/KaiCode2/evm-fork-cache/commits/main +[Unreleased]: https://github.com/KaiCode2/evm-fork-cache/compare/v0.1.0...HEAD +[0.1.0]: https://github.com/KaiCode2/evm-fork-cache/releases/tag/v0.1.0 diff --git a/Cargo.toml b/Cargo.toml index f491eca..f995dd8 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -11,6 +11,12 @@ readme = "README.md" repository = "https://github.com/KaiCode2/evm-fork-cache" documentation = "https://docs.rs/evm-fork-cache" +# Keep the published crate lean: the per-phase development specs are internal +# planning artifacts (they reference an adapter surface extracted before release), +# and the CI workflow has no consumer value. The consumer-facing ROADMAP.md and +# KNOWN_ISSUES.md under docs/ are still shipped. +exclude = ["docs/phase-*-spec.md", ".github/"] + # Standalone workspace root: keeps this crate from being absorbed by any # ancestor-directory workspace and gives it its own Cargo.lock. [workspace] @@ -38,6 +44,12 @@ anyhow = "1.0.98" bincode = "1.3" foundry-fork-db = "0.22" revm = { version = "34.0", features = ["std", "serde", "optional_eip3607", "optional_no_base_fee", "optional_balance_check"] } +# Pulled by `reactive-ws` only to select rustls' `ring` crypto provider for the +# alloy WebSocket (wss://) TLS path. If a transitive dep also enabled aws-lc-rs, +# rustls 0.23 would have no unambiguous process default and a wss:// connection +# would panic ("no process-level CryptoProvider available"); the AlloySubscriber +# installs the ring provider best-effort at WS init. Do not remove without +# re-checking the TLS provider story. rustls = { version = "0.23", default-features = false, optional = true } serde = { version = "1.0.228", features = ["derive"] } thiserror = "2.0" @@ -71,6 +83,10 @@ harness = false name = "simulation" harness = false +[[bench]] +name = "fanout" +harness = false + [[bench]] name = "freshness" harness = false @@ -87,6 +103,14 @@ harness = false name = "reactive_alloy_amm_live_probe" required-features = ["reactive"] +[[example]] +name = "reactive_runtime" +required-features = ["reactive"] + +[[example]] +name = "cold_start" +required-features = ["reactive"] + # RPC-gated real-contract benchmarks. Skipped (not failed) when RPC_URL is unset, # so `cargo bench` stays offline by default. [[bench]] diff --git a/README.md b/README.md index 3ba5e9e..1b86146 100644 --- a/README.md +++ b/README.md @@ -28,20 +28,26 @@ around three capabilities that target exactly this workload: 1. **Cheap parallel fan-out** — freeze state once into an immutable snapshot, hand a cheap `Arc` clone to each task, and run many isolated simulations in parallel. No task can observe another's writes. -2. **Targeted state sync** — refresh or purge *specific* accounts and storage - slots in place (no RPC on the hot path), so hot contract state stays correct - without re-forking. +2. **Reactive, event-driven state sync** — keep hot state correct *from the + chain's own logs* with no RPC on the hot path: a default-enabled reactive + runtime decodes events into targeted writes, invalidates/resyncs what it can't + derive, and recovers from reorgs — driven **out of the box** by a live + WebSocket subscriber (or your own transport), and seeded by protocol-neutral + **cold-start** that warms a working set into the cache in one batched pass. 3. **Freshness as a first-class concept** — the engine tracks what it can trust, for how long, and verifies the rest. The optimistic verify-and-rerun loop hides RPC latency: act on speculative results immediately, get a `Confirmed` or `Corrected` verdict when the background validation lands. > **Maturity.** This crate is **pre-1.0** and under active development against a -> [phased roadmap](docs/ROADMAP.md). Capabilities (1) and (3) above are -> implemented today. Capability (2) has the targeted writer primitives, the -> event-to-state reader pipeline, and a default-enabled reactive handler runtime; -> live network subscription driving remains consumer-provided. The public API -> still changes between minor versions — see [Stability](#stability). +> [phased roadmap](docs/ROADMAP.md). All three capabilities ship today: copy-on-write +> snapshots + overlays (1); a default-enabled reactive runtime with a live +> `AlloySubscriber` (WebSocket `subscribe_logs`/`subscribe_blocks`/`subscribe_pending_transactions`, +> exponential-backoff reconnect, and `get_logs` backfill) plus protocol-neutral +> cold-start (2); and the optimistic verify-and-rerun loop (3). Honest remaining +> transport work: full block bodies, full pending-transaction hydration, and +> arbitrary historical backfill are follow-ups. The public API still changes +> between minor versions — see [Stability](#stability). ## What it provides today @@ -50,6 +56,12 @@ around three capabilities that target exactly this workload: - **Snapshots and overlays** — `create_snapshot()` produces an immutable, `Send + Sync` point-in-time view; each `EvmOverlay` is a cheap clone that simulates in isolation, ideal for parallel candidate evaluation. +- **Bundle simulation** — `simulate_bundle` applies an ordered sequence of + transactions over cumulative block state (each transaction sees the previous + one's writes), with an `Atomic` / `AllowReverts(indices)` revert policy and + coinbase/miner-payment accounting (the beneficiary balance delta — priority fee + plus direct tips; the base fee is burned in-EVM per EIP-1559). This is the shape + a searcher evaluates a candidate set (victim + backrun, sandwich) with. - **Freshness control plane** — a four-layer model (classification, observation, policy, mechanism) plus an optimistic verify-and-rerun execution loop with deferred validation. See the [`freshness`](src/freshness.rs) module. @@ -87,10 +99,19 @@ around three capabilities that target exactly this workload: `reactive-polling` feature. Full block bodies, full pending transaction hydration, and arbitrary historical backfill remain explicit follow-up transport work. +- **Cold-start** — declaratively warm a working set of accounts and storage slots + into the cache in one batched pass via `EvmCache::run_cold_start` and a + `ColdStartPlanner` (discover slots via a view-call, then verify them), returning + a structured `ColdStartRunReport`. This is how a consumer adopts a working set + (pools, feeds) into the fork before going reactive. (Reactive-gated.) - **ERC20 helpers** — balances, allowances, decimals, and controlled balance mutation (including automatic balance-slot discovery) for simulations. - **Transfer-inspector simulation** that reports per-token balance deltas straight from the `Transfer` event stream, no extra pre/post balance queries. +- **Call-frame tracing** — `CallTracer` reconstructs the nested `CALL`/`CREATE` + frame tree of a simulation (from/to/value/gas/status/subcalls); `InspectorStack` + composes it with transfer capture (or any `revm::Inspector`) in a single pass, + driven through `EvmOverlay::call_raw_with_inspector`. - **Access-list tooling** — `StorageAccessList` captures the EIP-2929 warm-access touch set; helpers build an EIP-2930 access list and estimate whether attaching one is profitable on an L2. @@ -111,9 +132,14 @@ use std::sync::Arc; use alloy_eips::BlockId; use alloy_provider::{ProviderBuilder, network::AnyNetwork}; use alloy_primitives::{Address, Bytes}; +use alloy_sol_types::sol; use evm_fork_cache::cache::EvmCache; use revm::primitives::hardfork::SpecId; +sol! { + function balanceOf(address account) external view returns (uint256); +} + # async fn example() -> anyhow::Result<()> { let provider = ProviderBuilder::new() .network::() @@ -127,6 +153,11 @@ let mut cache = EvmCache::builder(Arc::new(provider)) .build() .await; +let token = Address::repeat_byte(0x22); +let owner = Address::repeat_byte(0x33); +let balance = cache.call_sol(token, balanceOfCall { account: owner })?; +println!("owner balance: {balance}"); + let from = Address::ZERO; let to = Address::repeat_byte(0x11); let calldata = Bytes::new(); @@ -152,16 +183,20 @@ println!( ## Core concepts The state stack flows bottom-to-top; reads flow up and the fork DB lazily fetches -misses from RPC: - -``` -EvmOverlay × N isolated, Send simulations (cheap Arc clones) - ▲ clone × N -EvmSnapshot immutable, point-in-time, Send + Sync - ▲ create_snapshot() -EvmCache lazy RPC fetch + local state cache + targeted writes/purge - ▲ lazy fetch -RPC provider +misses from RPC. The event-log path writes hot state in with **no RPC** (the +reactive-sync control plane): + +```mermaid +flowchart BT + RPC["RPC provider"] -->|"lazy fetch · once"| CACHE + LOGS["on-chain event logs"] -.->|"decode → write · 0 RPC"| CACHE + CACHE["EvmCache · !Send
fetch · cache · targeted writes/purge"] -->|"create_snapshot()"| SNAP + SNAP["EvmSnapshot · Send + Sync
immutable · Arc · point-in-time"] -->|"cheap Arc clone × N"| OV + OV["EvmOverlay × N · Send
isolated parallel simulations"] + classDef hot fill:#102a17,stroke:#3fb950,color:#e6edf3; + classDef cool fill:#0d1f2d,stroke:#388bfd,color:#e6edf3; + class SNAP,OV hot; + class RPC,CACHE,LOGS cool; ``` - **`EvmCache`** owns the mutable fork: it fetches, caches, persists, and applies @@ -176,7 +211,30 @@ The [`freshness`](src/freshness.rs) module layers a freshness controller on top: classify each address/slot (`Pinned` / `Volatile` / `ValidThrough`), observe how often slots change, pick what to verify each cycle with a `FreshnessPolicy`, and run the optimistic loop that returns speculative results immediately and a -`Confirmed`/`Corrected`/`Unverified` verdict asynchronously. +`Confirmed`/`Corrected`/`Unverified` verdict asynchronously. Time-to-actionable-result +is gated on local simulation, not on the RPC validation that runs behind it: + +```mermaid +sequenceDiagram + autonumber + participant S as Search loop + participant C as FreshnessController + participant V as Background validator + participant R as RPC + S->>C: run(candidate sims) + C->>C: snapshot + run optimistic sims + C-->>S: SpeculativeSim — optimistic results (~µs) + Note over S: act on speculative results now + C->>V: spawn (Send data only) + V->>R: verify volatile read-set (~L ms) + R-->>V: fresh values + alt nothing the sims read changed + V-->>S: validate().await → Confirmed + else a read slot changed + V->>V: re-run only the affected sims + V-->>S: Corrected { results, changed } + end +``` ## Examples @@ -203,6 +261,11 @@ and inject all state directly: | `freshness_multi_sim` | Advanced | Many sims with selective re-run, plus classification and `ValidThrough` aging. | | `state_update_apply` | Advanced | Apply a mixed `StateUpdate` batch (`Slot`/`Account`/`Purge`) and inspect the returned `StateDiff`. | | `reactive_cache` | Advanced | Decode ERC-20 `Transfer` logs into `StateUpdate`s, ingest a block, reconcile drift, and purge on a reorg. | +| `reactive_runtime` | Advanced | Drive the `ReactiveRuntime`: a handler turns a log into a `StateUpdate` (0 RPC), then a reorg triggers automatic journaled rollback. | +| `cold_start` | Advanced | Warm a working set with `run_cold_start`: discover the slots a view-call touches, then authoritatively verify + inject them. | +| `bundle_simulation` | Advanced | `simulate_bundle`: ordered txs over cumulative state, `Atomic` vs `AllowReverts`, and coinbase-payment accounting. | +| `call_tracer` | Advanced | `CallTracer` reconstructs a nested call-frame tree; `InspectorStack` composes it with transfer capture in one pass. | +| `fetch_minimization_counted` | Advanced | Count real RPC fetches to show the fetch-once-then-0-per-block mechanic across a fan-out. | **RPC examples** fork real mainnet state. Set `RPC_URL` to an Ethereum RPC endpoint (they print instructions and exit if it is unset): @@ -255,26 +318,98 @@ println!("installed {} bytes at {}", etched.code_size, etched.target_address); # } ``` +## Performance & honest trade-offs + +It is easy to post huge multipliers against a *naive* loop (a fresh cold fork per +candidate that re-fetches everything and deep-clones to isolate). That is **not** +the loop a competent revm user writes. Measured against a **competent baseline** — +one shared [`foundry-fork-db`] `SharedBackend` (which caches and deduplicates +fetches) plus `checkpoint`/`revert` isolation on a single fork — this crate is +**roughly at parity on raw within-block speed**, and we say so plainly: + +| Axis | vs a competent shared-backend / checkpoint-revert loop | +|---|---| +| RPC reads **within one block** | **~1×** — a shared `SharedBackend` also fetches each hot slot once | +| Single-threaded per-candidate CPU | **~1×** — `checkpoint`/`revert` isolation is as cheap as an overlay | +| Time-to-result vs *blocking* validation | not a fair comparison — a competent loop doesn't block on a fetch before acting | + +The value of this crate is **not** a within-block speed multiplier. It is +correctness, cross-block freshness, and a structured control plane the bare +primitives don't give you: + +**① Cross-block freshness — the one quantitative win (exact, CI-pinned integer).** +`foundry-fork-db`'s cache is **not block-keyed**: re-pinning to a new block does +not invalidate cached slots, so a refresh-by-refetch loop must re-read every slot +that changed *each block* to stay correct. Decoding the block's logs into targeted +writes keeps that hot state correct with **0 RPC fetches/block** — pinned in +[`tests/event_pipeline.rs`](tests/event_pipeline.rs). Sampled `reconcile()` +re-reads a fraction to catch drift (the honesty backstop). See +[`reactive_cache`](examples/reactive_cache.rs). + +

+ Cumulative RPC slot reads over 8 blocks: refresh-by-refetch climbs linearly to 64 while event-driven writes stay flat at 8 (warmed once, then 0 per block) +

+ +> Honest caveat: an equally sophisticated peer running their *own* log +> subscription + delta applier also reaches 0 fetches/block. The crate's +> contribution is the packaged, cold-aware, reorg-safe, reconcilable vocabulary — +> not an unreachable number. + +**② Parallel fan-out — available, modest, workload-dependent.** +`create_snapshot()` is an immutable `Send + Sync` view; cloning the `Arc` hands +each thread its own overlay, so candidates fan out across cores — which a single +mutable fork cannot do. The measured speedup is honest and modest: **~1.2×** across +the 64–1,024-candidate sweep (`cargo bench --bench fanout`) on a 10-core M1 Pro, +because these micro-sims are bound by per-candidate allocation, not EVM compute. +The ratio scales with both core count and per-candidate compute weight. Heavier +candidates (real txs doing +substantial execution) parallelize better; trivial ones barely. We don't headline a +core-count multiplier we can't reproduce. `cargo bench --bench fanout`; +[`parallel_overlays`](examples/parallel_overlays.rs). + +**③ Point-in-time consistency.** Every overlay reads one frozen, consistent block +state. A lazily-filled shared backend can interleave reads taken at slightly +different moments unless carefully pinned; the snapshot removes that class of bug. + +**④ Act-then-validate control plane (structure, not speed).** Run optimistically +against current state, return immediately, and validate the volatile read-set in +the background — re-running *only* the sims whose slots changed (`rerun_count`), +with `Confirmed`/`Corrected`/`Unverified` verdicts and block-pinned validation. A +searcher who simply acts on warm state is equally fast; the value is the **safe, +selective re-run**, not a latency multiplier. See +[`freshness_optimistic`](examples/freshness_optimistic.rs). + +> [!NOTE] +> **Methodology & candor.** Offline (mocked provider, state injected — no +> network). We deliberately do **not** lead with the headline multipliers a naive +> baseline would produce (~500× fewer reads, ~545× throughput, ~3,800× latency): +> all three collapse toward ~1× against a competent `SharedBackend` + +> `checkpoint`/`revert` loop — which is the very primitive this crate wraps. The +> "0 fetches/block" integer is real and CI-pinned (`cargo test --test +> event_pipeline`); the parallel-fan-out ratio is a Criterion median on an Apple +> M1 Pro, read as a ratio not an absolute. Live-RPC checks live behind the +> `RPC_URL` gate. + ## Benchmarks -Criterion benchmarks live in [`benches/`](benches). The offline benches exercise -the current hot paths at a range of cache sizes, including the Phase 5 -copy-on-write snapshot implementation and retained deep-clone baselines where -useful for A/B comparison: +Criterion benchmarks live in [`benches/`](benches) and run fully offline (mocked +provider) so they are reproducible: | Bench | Measures | | --- | --- | -| `simulation` | `create_snapshot` across cache sizes (100 → 10k accounts), overlay fan-out, `call_raw` throughput, sequential bundle execution, batched storage injection. | -| `freshness` | The optimistic loop end-to-end (CPU and latency-hiding), `verify_slots` at scale (1 → 1000 slots), and multi-sim fan-out. | +| `fanout` | **Parallel fan-out (②).** N candidates **sequential vs across cores** over one shared snapshot — the parallelism a live mutable fork can't do. | +| `freshness` | **Act-then-validate (④).** The optimistic loop CPU cost, selective re-run, and the latency-hiding shape (vs a baseline that elects to block). `verify_slots` at scale; multi-sim fan-out. | +| `event_pipeline` | **Cross-block freshness (①).** `ingest_logs` decode+apply throughput (1 → 1000 logs), `reorg_to` purge; the 0-fetch/block property is pinned in `tests/event_pipeline.rs`. | | `state_update` | `apply_updates` throughput across batch sizes (1 → 1000 `Slot`s) and per-variant apply cost (`Slot` vs `Account` vs `Purge`). | -| `event_pipeline` | Per-decoder cost (ERC-20 `Transfer`, generic slot marker), `ingest_logs` decode+apply throughput (1 → 1000 logs), and `reorg_to` purge cost. | +| `simulation` | Hot-path micro-benches and snapshot-implementation regression guards (`create_snapshot` vs the deep-clone reference — an internal cost model, see [`docs/INTERNALS.md`](docs/INTERNALS.md)). | | `access_list` | Touch-set merge and EIP-2930 list construction. | -| `revert_decoding` | Built-in and custom revert decoding, including decoder dispatch with many registered errors. | +| `revert_decoding` | Built-in (`Error`/`Panic`) and custom-error revert decoding, and decoder dispatch over a registered custom error. | | `create3` | CREATE3 address derivation. | ```sh cargo bench # all offline benches -cargo bench --bench simulation # one suite +cargo bench --bench fanout # one suite ``` The `rpc_mainnet` bench runs against **live mainnet state** to validate diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..bb9a9de --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,48 @@ +# Security Policy + +`evm-fork-cache` is a forked-EVM simulation engine used in search / MEV / +backtesting pipelines, where a correctness bug can have direct financial +consequences for downstream users. Security reports are taken seriously. + +## Supported versions + +This crate is **pre-1.0** and under active development. Only the latest published +`0.x` release line receives security fixes; there is no back-porting to older +`0.x` versions before 1.0. See [`CHANGELOG.md`](CHANGELOG.md) for what shipped in +each release. + +| Version | Supported | +| ------- | --------- | +| latest `0.x` | ✅ | +| older `0.x` | ❌ | + +## Reporting a vulnerability + +**Please do not open a public issue for security-sensitive reports.** + +Report privately via GitHub's +[private vulnerability reporting](https://docs.github.com/en/code-security/security-advisories/guidance-on-reporting-and-writing-information-about-vulnerabilities/privately-reporting-a-security-vulnerability) +on this repository: open the **Security** tab → **Report a vulnerability**. If +that channel is unavailable, open a minimal public issue asking for a private +contact (without disclosing details) and a maintainer will follow up. + +Please include: + +- the crate version, Rust version, and feature flags; +- a description of the issue and its impact; +- a minimal reproduction if possible. + +You can expect an initial acknowledgement within a few business days. Once a fix +is available it will be released and the advisory published, crediting the +reporter unless anonymity is requested. + +## Scope and known limitations + +This crate exposes deliberate, documented escape hatches that **bypass its safety +invariants** — notably `unchecked_blockchain_db()` / `unchecked_backend()` (which +sidestep the copy-on-write snapshot invalidation funnel) and the freshness model's +documented reconciliation scope (storage slots only, not account-level state). +These behaviors and their correct usage are described in +[`docs/KNOWN_ISSUES.md`](docs/KNOWN_ISSUES.md). Misuse of a documented escape hatch +is a usage error, not a vulnerability; a way to violate a documented invariant +*without* using an escape hatch is in scope. When in doubt, report it. diff --git a/assets/cross_block_freshness.svg b/assets/cross_block_freshness.svg new file mode 100644 index 0000000..24fcef0 --- /dev/null +++ b/assets/cross_block_freshness.svg @@ -0,0 +1,62 @@ + + Cumulative RPC slot reads as blocks advance + When a hot set of ~8 slots mutates each block, a refresh-by-refetch loop re-reads them every block (cumulative reads climb linearly), while event-driven writes keep state fresh with zero re-fetches after the first block (flat at 8). + + + + Cross-block freshness — cumulative RPC slot reads + a ~8-slot hot set mutates each block; foundry-fork-db's cache is not block-keyed, so staying correct means re-reading + + + + + + + + + + + + 0 + 16 + 32 + 48 + 64 + + cumulative RPC slot reads + + + 1 + 2 + 3 + 4 + 5 + 6 + 7 + 8 + + block + + + + + 64 + + + + + + 8 + + + + refresh by re-fetch = ~8 reads / block + + event-driven writes = 0 reads / block (warm once) + + + 0 re-fetches / block + decode the block's logs → targeted writes + + Within ONE block a shared backend also fetches once (~1×); this is the cross-block win. A peer subscribing to logs themselves also reaches 0. + diff --git a/benches/fanout.rs b/benches/fanout.rs new file mode 100644 index 0000000..a99acad --- /dev/null +++ b/benches/fanout.rs @@ -0,0 +1,160 @@ +//! **Pillar 1 — parallel fan-out throughput.** +//! +//! The honest, defensible win of the snapshot/overlay model is **parallelism**. +//! A live mutable fork (one `revm` EVM isolated with `checkpoint`/`revert`) can +//! only evaluate candidates *sequentially* — it cannot be shared mutably across +//! threads. `create_snapshot()` produces an immutable `Send + Sync` view; cloning +//! the `Arc` hands each worker thread its own cheap `EvmOverlay`, so the same N +//! candidates fan out across cores. +//! +//! This bench therefore compares like-for-like execution, sequential vs parallel, +//! over the SAME warm in-memory snapshot (no RPC on either side): +//! +//! - **`sequential`** — N candidates one after another on overlays from one +//! snapshot. This is also where a competent single-threaded baseline lands: a +//! single fork isolated with `checkpoint`/`revert` per candidate is O(touched) +//! and runs in the same per-candidate range, so single-threaded the snapshot +//! model is **~1×** — not a throughput win. We do not claim one. +//! - **`parallel`** — the same N candidates split across +//! `available_parallelism()` worker threads, each driving its own overlays from +//! an `Arc::clone` of the shared snapshot. +//! +//! The win is the `parallel/sequential` ratio. Honest result: it is **modest** +//! (~1.2× on a 10-core M1 Pro for these workloads), because even at 48 ops per +//! candidate the cost is dominated by per-call revm allocation, which contends on +//! the allocator rather than scaling with cores. Heavier, compute-bound candidates +//! parallelize better; we report what we measure and do not claim a core-count +//! multiplier. Wall-clock and machine-dependent — read the ratio. The internal +//! copy-on-write snapshot-construction cost model (COW vs deep clone) lives in +//! `benches/simulation.rs` / `docs/INTERNALS.md`, not here. + +use std::hint::black_box; +use std::sync::Arc; +use std::thread; + +use alloy_primitives::{Address, Bytes, U256, hex}; +use alloy_provider::RootProvider; +use alloy_provider::network::AnyNetwork; +use alloy_rpc_client::RpcClient; +use alloy_sol_types::{SolCall, sol}; +use alloy_transport::mock::Asserter; +use criterion::{BenchmarkId, Criterion, Throughput, criterion_group, criterion_main}; +use evm_fork_cache::cache::{EvmCache, EvmOverlay}; +use revm::context::result::ExecutionResult; +use revm::state::{AccountInfo, Bytecode}; +use tokio::runtime::Runtime; + +const MOCK_ERC20_RUNTIME_HEX: &str = include_str!("../fixtures/mock_erc20_runtime.hex"); +const BALANCE_SLOT: u64 = 3; + +sol! { + interface MockERC20 { + function balanceOf(address account) returns (uint256); + } +} + +/// A warm cache holding a MockERC20 with one funded owner (state in memory, no RPC). +fn warm_cache(rt: &Runtime) -> (EvmCache, Address, Address, Bytes) { + let provider = RootProvider::::new(RpcClient::mocked(Asserter::new())); + let mut cache = rt.block_on(EvmCache::new(Arc::new(provider))); + + let token = Address::repeat_byte(0xAA); + let owner = Address::repeat_byte(0xBB); + let runtime = Bytecode::new_raw(Bytes::from( + hex::decode(MOCK_ERC20_RUNTIME_HEX.trim()).unwrap(), + )); + let code_hash = runtime.hash_slow(); + cache.db_mut().insert_account_info( + token, + AccountInfo { + balance: U256::ZERO, + nonce: 0, + code: Some(runtime), + code_hash, + account_id: None, + }, + ); + cache + .insert_mapping_storage_slot(token, U256::from(BALANCE_SLOT), owner, U256::from(1_000u64)) + .unwrap(); + + let calldata = Bytes::from(MockERC20::balanceOfCall { account: owner }.abi_encode()); + (cache, token, owner, calldata) +} + +fn bench_candidate_fanout(c: &mut Criterion) { + let rt = Runtime::new().unwrap(); + let (mut cache, token, owner, calldata) = warm_cache(&rt); + black_box(cache.create_snapshot()); // warm the memoized base once + + let workers = thread::available_parallelism() + .map(|n| n.get()) + .unwrap_or(8); + + // Each candidate runs a non-trivial simulation (many reads), modeling a real + // candidate tx that touches dozens of slots. Even so, the per-candidate cost + // stays largely allocation-bound (revm allocates per call), so the measured + // parallel speedup is modest (~1.2×) rather than core-count — reported + // honestly, not tuned until it looks good. + const OPS_PER_CANDIDATE: usize = 48; + let run_candidate = |overlay: &mut EvmOverlay| { + for _ in 0..OPS_PER_CANDIDATE { + let result = overlay.call_raw(owner, token, calldata.clone()).unwrap(); + debug_assert!(matches!(result, ExecutionResult::Success { .. })); + black_box(result); + } + }; + + let mut group = c.benchmark_group("candidate_fanout"); + for &n in &[64usize, 256, 1_024] { + group.throughput(Throughput::Elements(n as u64)); + + // Single-threaded: N candidates one after another. The competent + // single-thread baseline (one fork, checkpoint/revert per candidate) lands + // here too — single-threaded the snapshot model is ~1×, not a win. + group.bench_with_input(BenchmarkId::new("sequential", n), &n, |b, &n| { + b.iter(|| { + let snapshot = cache.create_snapshot(); + for _ in 0..n { + let mut overlay = EvmOverlay::new(snapshot.clone(), None); + run_candidate(&mut overlay); + } + }) + }); + + // Parallel: the same N candidates across `workers` threads, each driving + // its own overlays from an Arc::clone of the shared immutable snapshot — + // which a single mutable fork cannot do. + group.bench_with_input(BenchmarkId::new("parallel", n), &n, |b, &n| { + b.iter(|| { + let snapshot = cache.create_snapshot(); + let per = n.div_ceil(workers); + thread::scope(|s| { + for start in (0..n).step_by(per) { + let end = (start + per).min(n); + let snapshot = snapshot.clone(); + let calldata = calldata.clone(); + s.spawn(move || { + for _ in start..end { + let mut overlay = EvmOverlay::new(snapshot.clone(), None); + for _ in 0..OPS_PER_CANDIDATE { + let result = + overlay.call_raw(owner, token, calldata.clone()).unwrap(); + debug_assert!(matches!( + result, + ExecutionResult::Success { .. } + )); + black_box(result); + } + } + }); + } + }); + }) + }); + } + group.finish(); +} + +criterion_group!(benches, bench_candidate_fanout); +criterion_main!(benches); diff --git a/docs/INTERNALS.md b/docs/INTERNALS.md new file mode 100644 index 0000000..30468cb --- /dev/null +++ b/docs/INTERNALS.md @@ -0,0 +1,46 @@ +# Internals — the copy-on-write snapshot cost model + +> This is an implementation note, not a value-prop benchmark. For the numbers a +> consumer cares about (fetch minimization, candidate throughput, reactive sync, +> optimistic latency) see the **Performance** section of the [README](../README.md). + +`create_snapshot()` is the operation a search loop runs once per block before +fanning candidates out. Phase 5 replaced its original O(total state) deep clone +with a two-tier copy-on-write view: the cold `BlockchainDb` index (layer 2) is +flattened once into an immutable, `Arc`-shared base (per-account storage shared by +`Arc`), memoized across snapshots and rebuilt copy-on-write only for changed +addresses; each snapshot then folds just the hot CacheDB delta (layer 1). Reads +stay O(1) and bit-for-bit identical to the deep clone (pinned by the differential +gate in `tests/cow_snapshot.rs`). + +The retained `create_snapshot_deep_clone()` is kept as (a) the read-equivalence +reference and (b) the A/B regression baseline, so a future change can't silently +regress `create_snapshot` back toward O(total state). The `create_snapshot` group +in `benches/simulation.rs` measures both. + +Indicative A/B (Apple M1 Pro, `aarch64-apple-darwin`, Criterion medians; offline, +state injected directly — read the ratio, not the absolute): + +| Cache size (accounts × slots) | Deep clone | COW `create_snapshot` | Ratio | +|:-----------------------------:|:----------:|:---------------------:|:-----:| +| 100 × 8 | 53 µs | 2.1 µs | ~25× | +| 1,000 × 8 | 791 µs | 19 µs | ~41× | +| 2,000 × 16 | 3.2 ms | 52 µs | ~61× | +| 5,000 × 16 | 9.5 ms | 113 µs | ~84× | +| 10,000 × 16 | 16.5 ms | 214 µs | ~77× | + +The deep clone copies every account and slot on every snapshot (O(total state)); +the COW snapshot shares the memoized base and folds only the hot delta, so its +cost tracks *changed* state, not *total* state. This is what makes the +[per-block fan-out](../README.md#performance) cheap — but it is an internal cost +model, which is why it lives here rather than in the README headline. + +**Residual cost (honest):** a COW snapshot is not free. When layer 2 is unchanged +it still pays an O(accounts) length-scan of the layer-2 maps plus an O(layer-1) +fold of the hot delta; a full rebuild (first snapshot, or after `set_block`) is +still O(total state). The decisions and the full cost model are in +[`phase-5-spec.md`](phase-5-spec.md) and [`ROADMAP.md`](ROADMAP.md), and the +residual is recorded in [`KNOWN_ISSUES.md`](KNOWN_ISSUES.md). + +Reproduce: `cargo bench --bench simulation` (the `create_snapshot` and +`resnapshot_hot_loop` groups). diff --git a/docs/KNOWN_ISSUES.md b/docs/KNOWN_ISSUES.md index da9bfeb..d69baa0 100644 --- a/docs/KNOWN_ISSUES.md +++ b/docs/KNOWN_ISSUES.md @@ -106,6 +106,18 @@ surface was moved out of this crate. ## Limitations by design / roadmap +- **Freshness reconciles storage slots only, not account-level state.** The + optimistic verify-and-rerun loop (`FreshnessController::run`) builds its verify + set exclusively from the volatile storage *slots* in each sim's read set; it + never re-fetches or diffs an account's native balance, nonce, or bytecode. + Consequently `Validation::Confirmed` guarantees only that *no volatile storage + slot the sims read had changed* — a sim whose result depends on a + `BALANCE`/`SELFBALANCE` (or nonce/code) that moved on-chain without a + co-changing storage slot can still be reported `Confirmed`. If account-level + state matters to a sim, classify the account `Pinned` and keep it fresh via + event-driven writes (`apply_update`/`apply_updates`), or reconcile it out of + band. The verdict types and the `freshness` module docs state this scope; a + future revision may extend reconcile to touched accounts. - **Solidity `Panic(uint256)` codes above `u64::MAX` decode as `Unknown`.** `decode_solidity_panic` drops out-of-range codes rather than exposing a lossy `u64`. Real compiler-emitted panic codes are single-byte constants, so this is @@ -114,7 +126,47 @@ surface was moved out of this crate. reads `from`/`to` from indexed topics and `value` from the first 32 data bytes. Non-standard or packed `Transfer` encodings may parse incorrectly or be skipped. A self-transfer where `from == to` nets to zero for that owner; this is - documented at the call site. + documented at the call site. Transfer **values ≥ 2^255** are reinterpreted as + negative when accumulated as a signed `I256` delta (`I256::from_raw`), so a + token emitting such a value would corrupt the reconstructed delta. Real ERC-20 + supplies are far below 2^255, so this is unreachable for honest tokens; a + malicious token can misreport balances by other means regardless. +- **`BLOCKHASH` resolves to ZERO in ext-db-less overlays.** Snapshots do not track + block hashes (`block_hashes` is always empty — the live cache does not track + them either), so an `EvmOverlay` built without an `ext_db` (e.g. the freshness + validator's internal overlays) returns `B256::ZERO` for the `BLOCKHASH` opcode. + A simulation of a contract that reads `BLOCKHASH` through such an overlay sees + ZERO rather than the real historical hash. +- **`EventPipeline::derived_slots` grows unbounded without reorgs.** Every + event-derived `(address, slot)` is retained for sampled reconciliation and is + only pruned by `reorg_to`. In long-running steady-state ingestion (no reorgs) + the set grows monotonically (~52 bytes/slot). The field doc says "seen so far"; + a future revision may bound it to the reorg ring's horizon. +- **Reorgs deeper than `ReorgConfig::depth` cannot fully purge.** The touched- + address ring is bounded to `depth` blocks; `reorg_to(n)` only purges addresses + still in the ring, so state touched solely in blocks that have already aged out + of the horizon is silently left un-purged (no error). Size `depth` above the + deepest reorg you expect to handle. +- **Reactive runtime reorgs deeper than `ReactiveConfig::journal_depth` recover + only the resident span.** The runtime journals each canonical block's effects in + a ring capped at `journal_depth` (default 64). A reorg deeper than that — or any + reorg when `journal_depth = 0` — rolls back / purges only the blocks still in the + journal; effects from aged-out blocks are **neither rolled back nor purged**, and + the freshness/validation loop is the backstop for that span. This is not silent: + the runtime emits a `tracing::warn!` when a reorg references a block no longer in + the journal. Set `journal_depth` above the deepest reorg you intend to recover + precisely. (The full conservative-purge fallback for aged-out blocks is a tracked + follow-up, not a known defect.) +- **Bundle `coinbase_payment` excludes the gas of `AllowReverts` transactions that + actually revert.** `simulate_bundle` rolls a reverting whitelisted tx back to its + inner checkpoint, which also undoes the gas that tx charged to the beneficiary. So + `coinbase_payment` reflects only the kept (successful) txs' priority fees + direct + tips — the honest miner *receipt* for those txs. On real mainnet a reverted bundle + tx still consumes gas and pays the miner, so for precise searcher *cost* accounting + (miner payment minus the gas you spend on failed attempts) you must add the + reverted txs' gas yourself (it is in `per_tx[i].gas_used`). Only relevant under + `AllowReverts` with a tx that actually reverts; the `Atomic` path is unaffected. + A future revision may surface reverted-tx gas cost directly. - **Copy-on-write snapshots (Phase 5, Pillar A) — done.** `create_snapshot()` is no longer an O(total state) deep clone. The cold `BlockchainDb` index (layer 2) is flattened once into an internal, immutable, `Arc`-shared base (per-account @@ -161,15 +213,28 @@ surface was moved out of this crate. `evm-amm-state` or downstream crates. This crate provides the generic `StateUpdate` writer vocabulary, `EventDecoder`/`DecoderRegistry`, the ERC-20 decoder, and `EventPipeline` orchestration. -- **Event-driven sync (roadmap Pillar B) — reader/writer halves done; live WS - transport is not.** The Phase 3 **writer half** (`StateUpdate` + - `apply_update`/`apply_updates`) and the Phase 4 **reader half** (the `events` - module: `EventDecoder`/`DecoderRegistry`, the ERC-20 adapter, and the - `EventPipeline` with `ingest_logs`/`reorg_to`/`reconcile`) are implemented. - What is **not** shipped is a concrete production WS transport: the async - `events::drive`/`LogSource` convenience is generic over a log source and is - exercised only by the offline example feeding an in-memory source; wiring it to - a live `subscribe_logs`/WS provider (and detecting reorgs from block-hash - mismatches) is left to the consumer. +- **Event-driven sync (roadmap Pillar B) — shipped, with bounded live transport.** + The Phase 3 **writer half** (`StateUpdate` + `apply_update`/`apply_updates`), the + Phase 4 **reader half** (the `events` module), the **reactive runtime** + (`reactive`: handler routing, canonical dedup/ordering, resync-through-fetcher, + depth-bounded journaled reorg recovery), and a **live `AlloySubscriber`** + (WebSocket `subscribe_logs`/`subscribe_blocks`/`subscribe_pending_transactions` + with exponential-backoff reconnect and `get_logs` backfill) all ship. Honest + remaining transport limits: **full block bodies**, **full pending-transaction + hydration**, and **arbitrary historical backfill** are not implemented (the + subscriber returns a typed `SubscriberError::Unsupported` for non-hash pending + interests); reconnect backfills inclusively from the last-seen block and relies + on the bounded `dedupe_window` to suppress overlap, so a reconnect after more + than `dedupe_window` matching logs can re-emit already-processed logs as + `Backfill` records (the runtime's canonical dedup catches most). Account-field + resync is unsupported until a provider-neutral account fetch callback exists. +- **Reactive integration-test coverage is partial.** The reactive runtime, the + subscriber, and reorg recovery are well covered individually (reorg rollback is + pinned by state-equivalence assertions; reconnect/backfill/dedup by inline unit + tests), but several public paths lack dedicated/integration coverage: composing + `AlloySubscriber` output into `ReactiveRuntime::ingest_batch` end-to-end, the + block-header ingest path, the `ReactiveReport::Decoded` shape, the + `EventDecoderHandler` adapter, and custom pending-tx matcher/route-key routing. + These are tracked follow-ups, not known defects. - **Recent toolchain.** MSRV 1.88 and edition 2024 are intentional and CI-enforced; consumers on older toolchains are not supported. diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index c0c8384..3b558e0 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -1,7 +1,8 @@ # evm-fork-cache — engineering roadmap -> Status: living document. Last updated during Phase 1 public-release hardening -> after Phases 0-5 landed. +> Status: living document. Last updated after Phase 6 — the provider-neutral +> reactive runtime, the live `AlloySubscriber` transport, and cold-start — landed +> on top of the Phase 0-5 core. ## Vision @@ -15,11 +16,15 @@ backtesting. The moat is three capabilities working together: 3. **Freshness as a first-class concept** — the engine knows what it can trust, for how long, and purges the rest. -Today the crate implements the Phase 0-5 core: copy-on-write snapshots and -overlays, the freshness control plane, targeted state-update writers, and the -event-to-state reader pipeline. The remaining gap is operational integration -around that pipeline, especially a production WebSocket/log subscription -transport and application-specific reorg policy wiring. +Today the crate implements all three pillars end-to-end: copy-on-write snapshots +and overlays, the freshness control plane, targeted state-update writers, the +event-to-state reader pipeline, and — as of Phase 6 — a provider-neutral reactive +runtime with journaled reorg recovery, a live WebSocket `AlloySubscriber` +transport, declarative cold-start warming, and — as of Phase 7 — multi-transaction +bundle simulation with coinbase accounting and a call-frame tracer. The remaining +work toward 1.0 is breadth on those primitives (`Create`-kind bundle txs, opcode +tracing) and transport depth (full block bodies, full pending-tx hydration, +arbitrary historical backfill), not the core control plane. ## Target architecture @@ -39,8 +44,9 @@ RPC node Event-driven sync ← WS logs · new block the fork DB lazily fetches misses from RPC. - **Control plane (right):** decoded logs drive event-derived targeted writes (e.g. a V3 `Swap` → `slot0`) and purges stale state directly into the fork DB, - without RPC on the hot path. The reader/writer pipeline is shipped; production - WS subscription and block-hash reorg detection are consumer-provided. + without RPC on the hot path. The reader/writer pipeline, the reactive runtime + that drives it (journaled parent-hash reorg detection and recovery), and a live + WebSocket subscription transport (`AlloySubscriber`) all ship in-crate. ### The three pillars @@ -76,11 +82,13 @@ RPC node Event-driven sync ← WS logs · new block | **1** | Engine seam: typed errors, configurable tx/block env, hot-path benches, builder, protocol-adapter extraction path. | **Done** (`phase-1-engine-seam`) | | **2** | Freshness core (Pillar C): `Validity` + `FreshnessRegistry`; observation tracker; policies; optimistic verify-and-rerun loop. | **Done** (`phase-2-freshness`) | | **3** | State-update primitives (Pillar B.1): `StateUpdate` + targeted writers; refold `inject_*`; surface state-diff output. | **Done** (`phase-3-state-updates`) | -| **4** | Event pipeline + adapters (Pillar B.2): `EventDecoder` trait, ERC-20 + V3 adapters, ingest/reorg/reconcile pipeline. | **Done** (`phase-4-event-pipeline`) | +| **4** | Event pipeline + adapters (Pillar B.2): `EventDecoder` trait, ERC-20 decoder, ingest/reorg/reconcile pipeline. (The in-crate V3 adapter built here was later extracted to `evm-amm-state`; the core stays protocol-neutral.) | **Done** (`phase-4-event-pipeline`) | | **5** | COW snapshots (Pillar A): structural sharing; overlay buffer reuse. | **Done** (`phase-5-cow-snapshots`) | +| **6** | Reactive runtime + live transport: provider-neutral `ReactiveRuntime` / `ReactiveHandler`, journaled depth-bounded reorg recovery, the WebSocket `AlloySubscriber`, and declarative `cold_start` warming. | **Done** (`cold-start-sync`) | +| **7** | Bundle simulation + call tracing: `EvmOverlay::simulate_bundle` (ordered cumulative-state txs, `RevertPolicy`, coinbase-payment accounting) and a `CallTracer` (call-frame tree) + composable `InspectorStack`. | **Done** (`phase-6-bundle-sim`) | -Cross-cutting remaining work: call tracer Inspector, full no-provider build split, -and production event-transport integrations. +Cross-cutting remaining work: `Create`-kind / state-override bundles, opcode-level +tracing, and a full no-provider build split. --- @@ -446,16 +454,89 @@ fold cost model is recorded in `KNOWN_ISSUES.md`. --- +## Phase 6 — reactive runtime + live transport (detailed) + +Builds the operational layer that drives Pillar B and closes the "consumer wires +the transport themselves" gap from the original roadmap: a provider-neutral +runtime that turns subscription inputs into validated cache mutations, a live +WebSocket transport, and a declarative cold-start warmer. Landed on +`cold-start-sync`. + +### Locked decisions + +1. **Pure handlers, runtime-owned mutation.** `ReactiveHandler::handle` is a pure + function of `(context, input, &dyn StateView)` returning a `HandlerOutcome` of + declarative `ReactiveEffect`s (`StateUpdate`, `Invalidate`, `Resync`, + `Speculative`, `Hook`). The runtime — never the handler — validates the effect + set (rejecting conflicting writes and canonical mutations from pending inputs) + and applies canonical cache mutations. This keeps the `!Send` cache discipline + intact: handlers carry no cache handle and dispatch is synchronous. +2. **Journaled, depth-bounded reorg recovery.** The runtime journals each + canonical block's applied effects in a `VecDeque` capped at + `ReactiveConfig::journal_depth` (default 64). A removed log or a parent-hash + discontinuity drains the dropped blocks and recovers them: reversible + slot writes are rolled back to their exact prior values (LIFO), while accounts + whose balance/nonce/code moved are promoted to a `PurgeScope::Account` so the + next read re-fetches clean state. Hash-pinned resyncs from dropped blocks are + canceled. A `ReactiveReport::Reorg` describes precisely what was rolled back + and purged. See the runnable [`reactive_runtime`](../examples/reactive_runtime.rs) + example. +3. **Live transport behind a trait.** `EventSubscriber` is the transport seam; + `AlloySubscriber` is the in-crate implementation over Alloy pubsub — + `subscribe_logs` / `subscribe_blocks` / `subscribe_pending_transactions`, + exponential-backoff reconnect with a bounded dedupe window, and `get_logs` + backfill from the last-seen block on reconnect. WebSocket TLS uses rustls' ring + provider (`reactive-ws`, default); an HTTP polling transport is opt-in + (`reactive-polling`). +4. **Declarative cold-start.** `EvmCache::run_cold_start` drives a + `ColdStartPlanner` (a bounded discover-then-verify loop) to warm a working set + of accounts/slots into the cache in batched passes before going reactive, + returning a structured `ColdStartRunReport`. + +### Known limitations (tracked in `docs/KNOWN_ISSUES.md`) + +- Reorgs deeper than `journal_depth` (or any depth when `journal_depth = 0`) + recover only the blocks still resident in the journal; effects from aged-out + blocks are neither rolled back nor purged, and the freshness/validation loop is + the backstop. `journal_depth` must exceed the deepest reorg you intend to + recover from. +- The subscriber surfaces pending-transaction **hashes** only (no full pending-tx + hydration), no full block bodies, and backfill is anchored at the last-seen + block (no arbitrary historical backfill). Account-field resync targets return a + typed `Unsupported` error. Hook dispatch is synchronous on the ingest thread; + `hook_backpressure` is reserved for a future async dispatcher. + +### Acceptance — met + +`cargo fmt --check`, `clippy --all-targets -- -D warnings`, `cargo test`, +`RUSTDOCFLAGS=-D warnings cargo doc`, `cargo bench --no-run`. Landed: `src/reactive/` +(runtime, registry/router, journaling + reorg recovery, `EventSubscriber` + +`AlloySubscriber`), `src/cold_start/` (planner/driver/plan/results), the +`reactive_cache` / `reactive_runtime` / `reactive_alloy_amm_live_probe` examples, +`benches/event_pipeline.rs`, and `tests/reactive_*` + `tests/cold_start.rs`. + +--- + ## Remaining work toward 1.0 -1. **Production event transport.** The crate ships the generic `events::drive` - convenience, `LogSource` trait, synchronous `EventPipeline`, reorg purge, and - sampled reconciliation. It does not ship a concrete production WS provider, - block-hash reorg detector, or backfill/resubscribe strategy; consumers wire - those pieces to their provider stack. -2. **Snapshot consistency point in continuous ingestion.** Applications that run +1. **Bundle-simulation breadth (Phase 7 — core shipped).** `EvmOverlay::simulate_bundle` + now evaluates an ordered tx sequence over cumulative state with a revert policy + and coinbase-payment accounting, and a `CallTracer` reconstructs the call-frame + tree (see Phase 7 below). The remaining breadth: `Create`-kind bundle txs, a + builder-style state-override bundle, opcode/step-level tracing, and tightening + reverted-tx gas accounting under `AllowReverts` (a reverted tx's gas is rolled + back with its checkpoint, so it is not counted toward the searcher's cost — + tracked in `docs/KNOWN_ISSUES.md`). +2. **Transport depth.** The live `AlloySubscriber` ships log/block/pending-hash + subscriptions, exponential-backoff reconnect, `get_logs` backfill, and + journaled parent-hash reorg recovery. The remaining transport gaps are full + block bodies, full pending-transaction hydration (today only pending-tx + hashes), arbitrary historical backfill beyond the last-seen-block anchor, and + account-field (balance/nonce/code) resync — all tracked in + `docs/KNOWN_ISSUES.md`. +3. **Snapshot consistency point in continuous ingestion.** Applications that run a live event loop should snapshot at block boundaries or behind their own generation guard so simulations do not observe a partially applied block. -3. **Full no-provider build split.** The dependency graph still includes +4. **Full no-provider build split.** The dependency graph still includes provider/RPC crates. A later `rpc` feature can make those optional for pure offline users. diff --git a/docs/phase-6-bundle-sim-spec.md b/docs/phase-6-bundle-sim-spec.md new file mode 100644 index 0000000..844ce4f --- /dev/null +++ b/docs/phase-6-bundle-sim-spec.md @@ -0,0 +1,270 @@ +# Phase 6 — bundle simulation, coinbase accounting, call tracing (spec) + +> Status: implementation spec. Two parallel implementation tracks (A+B and C), +> each developed in an isolated worktree, integrated and reviewed by the manager. +> 0.2 feature work; separate from the release-readiness line. + +## Goal + +Lift the engine from an isolated single-call evaluator to an MEV-bundle simulator: +apply an **ordered sequence of transactions over cumulative block state**, account +for **miner (coinbase) payment**, and expose a **call-frame tracer**. All three are +additive — they build on machinery that already exists: + +- Cumulative commit: `EvmOverlay` `commit=true` + per-call `journaled_state.checkpoint()` + (`src/cache/overlay.rs`). Chaining committing calls on one overlay yields cumulative state. +- Inspector seam: `build_evm_with_inspector_local(…)` → `evm.inspect_one_tx(tx)` + is generic over any revm `Inspector` (`src/cache/overlay.rs:320`). `TransferInspector` + (`src/inspector.rs`) is the working model. +- Coinbase: `evm.block.beneficiary = coinbase` is already set from the snapshot + (`src/cache/overlay.rs:355`); the beneficiary accrues payment during execution. + +Reuse: `TxConfig` (value/gas_limit/gas_price/nonce/access_list), `SimError` / +`SimulationResult`, the checkpoint/commit path, and the `TransferInspector` pattern. + +## Non-goals (out of scope for Phase 6) + +- Full block building (mempool, tx ordering/auction). A "bundle" is an ordered tx + sequence on cumulative state — nothing more. +- `Create`-kind bundle txs (Call only for now; note as a follow-up). +- Opcode/step-level tracing, SLOAD/SSTORE capture, per-frame gas attribution beyond + total gas. The tracer ships call frames only. +- Builder-style state-override bundles. + +The crate must remain **protocol-neutral**: no AMM/protocol-specific logic in `src/`. + +--- + +## Track A + B — bundle execution + coinbase accounting + +> **Correction (post-implementation).** This spec assumed `disable_base_fee = true` +> makes revm credit the *full* `gas_price × gas_used` to the beneficiary, requiring +> a `GasAccounting::{Mainnet, Raw}` knob to subtract the burned base fee. That is +> wrong: revm's `reward_beneficiary` already credits only the priority fee +> (`(effective_gas_price − basefee) × gas_used`) under EIP-1559 and burns the base +> fee in-EVM (`disable_base_fee` only affects tx validation, not the reward). The +> shipped implementation therefore drops `GasAccounting` entirely and reports +> `coinbase_payment` as the bare beneficiary delta (the honest payment). The +> `GasAccounting` parts of the API sketch below are superseded. + +### Public API (new module `src/bundle.rs`, re-exported at crate root) + +```rust +/// One transaction in a bundle. `Call`-kind only for Phase 6. +#[derive(Clone, Debug)] +pub struct BundleTx { + pub from: Address, + pub to: Address, + pub calldata: Bytes, + pub tx: TxConfig, // value/gas_limit/gas_price/nonce/access_list (reused) +} +impl BundleTx { + pub fn new(from: Address, to: Address, calldata: Bytes) -> Self; // tx = default + pub fn with_config(from: Address, to: Address, calldata: Bytes, tx: TxConfig) -> Self; +} + +/// How miner payment is computed from the beneficiary balance delta. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Default)] +pub enum GasAccounting { + /// Base-fee-aware (honest profit): payment = beneficiary_delta − Σ(gas_usedᵢ × basefee). + /// Mirrors what a builder actually nets (base fee is burned on mainnet). + #[default] + Mainnet, + /// Raw beneficiary balance delta (base fee NOT subtracted). For diagnostics. + Raw, +} + +/// What happens when a bundle tx reverts. +#[derive(Clone, Debug)] +pub enum RevertPolicy { + /// Any tx revert/halt reverts the WHOLE bundle and sets `succeeded = false`. + Atomic, + /// The listed tx indices may revert without aborting the bundle; their state + /// effects are rolled back individually, later txs still execute. + AllowReverts(Vec), +} +impl Default for RevertPolicy { fn default() -> Self { RevertPolicy::Atomic } } + +#[derive(Clone, Debug, Default)] +pub struct BundleOptions { + pub revert_policy: RevertPolicy, // default Atomic + pub gas_accounting: GasAccounting, // default Mainnet + pub commit: bool, // default false (evaluate, don't persist) +} + +#[derive(Clone, Debug)] +pub struct TxOutcome { + pub result: ExecutionResult, // revm enum (Success/Revert/Halt) + pub gas_used: u64, + pub reverted: bool, // true if Revert or Halt + pub logs: Vec, +} + +#[derive(Clone, Debug)] +pub struct BundleResult { + pub per_tx: Vec, // one per executed tx (length == txs.len() unless Atomic aborted early) + pub coinbase_payment: U256, // per gas_accounting; saturating + pub gas_used: u64, // total across executed txs + pub succeeded: bool, // false iff Atomic aborted on a revert +} + +impl EvmOverlay { + /// Apply `txs` in order against this overlay with cumulative state. + pub fn simulate_bundle(&mut self, txs: &[BundleTx], opts: &BundleOptions) + -> SimulationResult; +} +impl EvmCache { + /// Convenience: snapshot self, run the bundle on a fresh overlay. + /// (Does not mutate the cache even when `opts.commit` is true — commit applies + /// to the transient overlay; document this clearly.) + pub fn simulate_bundle(&mut self, txs: &[BundleTx], opts: &BundleOptions) + -> SimulationResult; +} +``` + +### Behavioral requirements + +1. **Ordered cumulative state.** Tx `i` observes the committed writes of txs `0..i`. + Implement with one outer bundle checkpoint and one inner checkpoint per tx, on a + single overlay/EVM (do NOT rebuild a fresh overlay per tx). +2. **Revert policy.** + - `Atomic`: on the first tx that reverts/halts, revert the whole bundle to the + outer checkpoint, set `succeeded = false`, and stop (per_tx ends at the failing + tx, whose `reverted = true`). `coinbase_payment = 0` and state is unchanged. + - `AllowReverts(idxs)`: a revert at an index in `idxs` rolls back only that tx + (inner checkpoint revert), records `reverted = true`, and continues; a revert at + an index NOT in `idxs` behaves like `Atomic`. `succeeded = true` if the bundle + ran to completion. +3. **Coinbase / payment accounting.** Capture `beneficiary` balance before the bundle + and after the last committed tx. `Raw` returns the delta. `Mainnet` returns + `delta.saturating_sub(Σ gas_usedᵢ × basefee)`, where `basefee` is the snapshot/overlay + block base fee (0 if unset). This subtracts the burned base fee so the figure is the + honest priority-fee + direct-coinbase-transfer payment. Use saturating arithmetic. +4. **Base-fee control for honest accounting + testability.** Add a way to set the block + base fee on the cache/overlay (e.g. `EvmCacheBuilder::basefee(U256)` / + `EvmCache::set_basefee(U256)` propagated into `EvmSnapshot`), since offline caches + have no fetched header. Required so Mainnet accounting is exercisable. +5. **Commit semantics.** `commit = true` folds the bundle's cumulative state into the + overlay's dirty layer (and is observable by subsequent overlay calls). `commit = false` + reverts the outer checkpoint so the overlay is unchanged. `EvmCache::simulate_bundle` + always runs on a transient overlay (cache is not mutated). +6. **Isolation / safety.** A failed bundle (Atomic abort) never leaves partial state. + Errors (tx-env build failure, transact DB error) return `SimError`, not panic. + +### Acceptance criteria (Track A+B) + +- AB1 cumulative state: a 2-transfer bundle nets correctly; tx 2 sees tx 1's write. +- AB2 atomic revert: a bundle whose 2nd tx reverts → `succeeded == false`, owner state + unchanged from pre-bundle, `per_tx[1].reverted == true`. +- AB3 allow-reverts: same bundle with `AllowReverts([1])` → `succeeded == true`, + tx 0's effect persists, tx 1 rolled back. +- AB4 direct coinbase payment: a tx sending `value` to the beneficiary with `gas_price = 0` + yields `coinbase_payment == value` (both Raw and Mainnet, since gas credit is 0). +- AB5 mainnet vs raw gas: with a set base fee and a tx priced at `gas_price ≥ basefee`, + `Mainnet == Raw − gas_used×basefee` and `Mainnet < Raw`. +- AB6 commit: `commit=true` on an overlay persists cumulative state to the next call; + `commit=false` leaves it isolated. + +--- + +## Track C — call tracer + generalized inspector seam + +### Public API (new module `src/tracing.rs`, re-exported at crate root) + +```rust +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum CallKind { Call, StaticCall, DelegateCall, CallCode, Create, Create2 } + +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum CallStatus { Success, Revert, Halt } + +#[derive(Clone, Debug)] +pub struct CallTrace { + pub kind: CallKind, + pub from: Address, + pub to: Address, // created address for Create* + pub value: U256, + pub input: Bytes, + pub gas_used: u64, + pub output: Bytes, + pub status: CallStatus, + pub depth: usize, + pub subcalls: Vec, +} + +/// revm Inspector that builds a call-frame tree. +#[derive(Clone, Debug, Default)] +pub struct CallTracer { /* frame stack + completed root */ } +impl CallTracer { + pub fn new() -> Self; + /// The root frame after a transact (None if nothing executed). + pub fn into_trace(self) -> Option; + pub fn root(&self) -> Option<&CallTrace>; +} +impl revm::Inspector for CallTracer where INTR: InterpreterTypes { … } +``` + +### Generalized inspector seam + +`build_evm_with_inspector_local` is already generic. Expose a **public**, +inspector-generic simulate entry on `EvmOverlay` so callers can attach a `CallTracer` +(or any `Inspector`) instead of being limited to the hardcoded `TransferInspector`: + +```rust +impl EvmOverlay { + /// Run a single call with a caller-supplied inspector; returns the raw + /// ExecutionResult and hands the inspector back for the caller to read. + pub fn call_raw_with_inspector(&mut self, from: Address, to: Address, + calldata: Bytes, tx: &TxConfig, inspector: I, commit: bool) + -> SimulationResult<(ExecutionResult, I)> + where I: Inspector<…>; +} +``` + +Plus a small composing inspector so a tracer AND transfer tracking can run together: + +```rust +/// Runs two inspectors over the same execution. +pub struct InspectorStack(pub A, pub B); +impl Inspector for InspectorStack where … { /* fan out every hook */ } +``` + +### Behavioral requirements + +1. The tracer captures the full call-frame tree: top-level call + nested CALL/STATICCALL/ + DELEGATECALL/CREATE frames, each with from/to/value/input/output/gas_used/status/depth/subcalls. +2. Revert attribution: a frame that reverts has `status == Revert` (Halt → `Halt`); its + parent records it as a subcall regardless. +3. `InspectorStack` fans out every `Inspector` hook to both inner inspectors so a + `CallTracer` + `TransferInspector` produce their independent results in one pass. +4. `call_raw_with_inspector` honors `commit` exactly like `simulate_with_transfer_tracking`. + +### Acceptance criteria (Track C) + +- C1 single frame: a top-level call to a contract yields a root `CallTrace` with the + right from/to/input and `status == Success`. +- C2 nested calls: a contract that calls another contract yields a root with a `subcalls` + entry for the inner call (correct depth + to-address). +- C3 revert attribution: a call into a reverting contract yields a frame with + `status == Revert`. +- C4 composition: `InspectorStack` over a token transfer + yields BOTH a non-empty trace AND the expected `TokenTransfer`. + +--- + +## Integration (manager-owned) + +- A+B owns `src/bundle.rs` + `EvmOverlay::simulate_bundle` (one new method in `overlay.rs`) + + `EvmCache::simulate_bundle` + base-fee setter + crate-root re-exports. +- C owns `src/tracing.rs` + `EvmOverlay::call_raw_with_inspector` (one new method in + `overlay.rs`) + `InspectorStack` + crate-root re-exports. +- Shared edit surfaces: `overlay.rs` (each adds ONE localized method — keep them adjacent + to the existing `simulate_with_transfer_tracking` to ease the merge), `lib.rs` (each adds + its re-export block), `CHANGELOG.md`. The manager resolves these additive conflicts. + +## Verification + +`cargo fmt --check`, `cargo clippy --all-targets --all-features -- -D warnings`, +`cargo test --all-features`, doctests, `RUSTDOCFLAGS=-D warnings cargo doc --no-deps`. +Manager-authored acceptance tests: `tests/bundle_simulation.rs` (A+B), +`tests/call_tracer.rs` (C). No new production dependencies. Offline only (mocked provider). diff --git a/examples/bundle_simulation.rs b/examples/bundle_simulation.rs new file mode 100644 index 0000000..ba67cb7 --- /dev/null +++ b/examples/bundle_simulation.rs @@ -0,0 +1,188 @@ +//! Bundle simulation + coinbase accounting (Phase 6 Track A+B). +//! +//! An MEV bundle is an *ordered* sequence of transactions evaluated over +//! **cumulative** state — each transaction sees the writes of the ones before it — +//! with the miner's payment accounted at the end. This is the shape a searcher +//! evaluates (victim + backrun, sandwich front/back), which a single isolated +//! `call` cannot express. This example shows three things: +//! +//! 1. **A bundle over cumulative state**: two dependent token transfers plus a +//! native coinbase tip, with per-transaction outcomes and the miner payment. +//! 2. **The revert policy**: the same shape with a failing transaction, run +//! `Atomic` (the whole bundle reverts) vs `AllowReverts` (the failure is rolled +//! back individually and the rest still stands). +//! 3. **Base-fee-aware payment**: with a base fee set, the coinbase payment is the +//! priority fee only — revm burns the base fee in-EVM (EIP-1559), so the figure +//! is the honest miner payment automatically. +//! +//! Fully offline (mocked provider, injected state). Run with: +//! +//! ```sh +//! cargo run --example bundle_simulation +//! ``` + +#[path = "support/mock.rs"] +mod mock; + +use alloy_primitives::{Address, Bytes, U256}; +use alloy_sol_types::SolCall; +use anyhow::Result; +use evm_fork_cache::cache::EvmOverlay; +use evm_fork_cache::{BundleOptions, BundleTx, RevertPolicy, TxConfig}; +use revm::context::result::ExecutionResult; +use revm::state::AccountInfo; + +fn transfer(to: Address, amount: u64) -> Bytes { + Bytes::from( + mock::MockERC20::transferCall { + to, + amount: U256::from(amount), + } + .abi_encode(), + ) +} + +fn token_balance(overlay: &mut EvmOverlay, token: Address, owner: Address) -> Result { + let cd = Bytes::from(mock::MockERC20::balanceOfCall { account: owner }.abi_encode()); + match overlay.call_raw(owner, token, cd)? { + ExecutionResult::Success { output, .. } => Ok( + mock::MockERC20::balanceOfCall::abi_decode_returns(&output.into_data())?, + ), + other => anyhow::bail!("balanceOf failed: {other:?}"), + } +} + +#[tokio::main(flavor = "multi_thread")] +async fn main() -> Result<()> { + let mut cache = mock::offline_cache().await?; + let token = Address::repeat_byte(0x11); + let alice = Address::repeat_byte(0x22); + let bob = Address::repeat_byte(0x33); + let carol = Address::repeat_byte(0x44); + // Address::ZERO is the default block beneficiary (the "coinbase"). + mock::install_default_account(&mut cache, Address::ZERO); + for a in [alice, bob, carol] { + mock::install_default_account(&mut cache, a); + } + mock::install_mock_erc20(&mut cache, token); + cache.insert_mapping_storage_slot( + token, + U256::from(mock::MOCK_ERC20_BALANCE_SLOT), + alice, + U256::from(1_000u64), + )?; + + // ---- 1. A bundle over cumulative state, with a coinbase tip ---- + // tx0: alice -> bob 500 tx1: bob -> carol 200 (only valid after tx0) + // tx2: alice tips the coinbase 750 wei (a direct miner payment). + let tip = U256::from(750u64); + let bundle = vec![ + BundleTx::new(alice, token, transfer(bob, 500)), + BundleTx::new(bob, token, transfer(carol, 200)), + BundleTx::with_config( + alice, + Address::ZERO, + Bytes::new(), + TxConfig { + value: tip, + gas_price: Some(0), + ..Default::default() + }, + ), + ]; + + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + let result = overlay.simulate_bundle( + &bundle, + &BundleOptions { + commit: true, + ..Default::default() + }, + )?; + + println!("=== 1. cumulative bundle (Atomic) ==="); + println!( + " succeeded: {} total gas: {}", + result.succeeded, result.gas_used + ); + for (i, o) in result.per_tx.iter().enumerate() { + println!(" tx{i}: reverted={} gas={}", o.reverted, o.gas_used); + } + println!(" coinbase payment: {} wei", result.coinbase_payment); + println!( + " final balances -> alice: {} bob: {} carol: {} (cumulative: tx1 saw tx0)", + token_balance(&mut overlay, token, alice)?, + token_balance(&mut overlay, token, bob)?, + token_balance(&mut overlay, token, carol)?, + ); + + // ---- 2. Revert policy: Atomic vs AllowReverts ---- + // A bundle whose 2nd tx reverts (unknown selector). + let with_failure = vec![ + BundleTx::new(alice, token, transfer(bob, 100)), + BundleTx::new(alice, token, Bytes::from(vec![0xde, 0xad, 0xbe, 0xef])), + ]; + let snapshot = cache.create_snapshot(); + + let mut atomic = EvmOverlay::new(snapshot.clone(), None); + let r = atomic.simulate_bundle( + &with_failure, + &BundleOptions { + commit: true, + ..Default::default() + }, + )?; + println!("\n=== 2. revert policy ==="); + println!( + " Atomic: succeeded={} bob balance after: {} (whole bundle rolled back)", + r.succeeded, + token_balance(&mut atomic, token, bob)?, + ); + + let mut allow = EvmOverlay::new(snapshot, None); + let r = allow.simulate_bundle( + &with_failure, + &BundleOptions { + revert_policy: RevertPolicy::AllowReverts(vec![1]), + commit: true, + }, + )?; + println!( + " AllowReverts: succeeded={} bob balance after: {} (tx0 kept, tx1 rolled back)", + r.succeeded, + token_balance(&mut allow, token, bob)?, + ); + + // ---- 3. Base-fee-aware payment under a base fee ---- + // Fund a searcher's native balance and price a tx above the base fee. + let searcher = Address::repeat_byte(0x55); + cache.db_mut().insert_account_info( + searcher, + AccountInfo { + balance: U256::from(10u64).pow(U256::from(20u64)), + ..Default::default() + }, + ); + let basefee: u128 = 1_000_000_000; // 1 gwei, burned on mainnet + let priority: u128 = 2_000_000_000; // 2 gwei, the miner's actual cut + cache.set_basefee(U256::from(basefee)); + let priced = vec![BundleTx::with_config( + searcher, + token, + Bytes::from(mock::MockERC20::balanceOfCall { account: searcher }.abi_encode()), + TxConfig { + gas_price: Some(basefee + priority), + ..Default::default() + }, + )]; + + let result = cache.simulate_bundle(&priced, &BundleOptions::default())?; + println!("\n=== 3. base-fee-aware payment (basefee 1 gwei, gas_price 3 gwei) ==="); + println!(" gas used: {}", result.gas_used); + println!( + " coinbase payment: {} wei (= gas_used × 2 gwei priority; the 1 gwei base fee is burned, not paid)", + result.coinbase_payment, + ); + + Ok(()) +} diff --git a/examples/call_tracer.rs b/examples/call_tracer.rs new file mode 100644 index 0000000..c41af67 --- /dev/null +++ b/examples/call_tracer.rs @@ -0,0 +1,151 @@ +//! Call-frame tracing + composable inspectors (Phase 6 Track C). +//! +//! `CallTracer` is a `revm::Inspector` that reconstructs the **call-frame tree** of +//! a simulation — the top-level call plus every nested CALL/STATICCALL/CREATE — so +//! you can see who called whom, with what gas, and which frame reverted. It +//! attaches through the inspector-generic [`EvmOverlay::call_raw_with_inspector`], +//! and [`InspectorStack`] composes it with other inspectors (here the +//! [`TransferInspector`]) in a single pass. +//! +//! This example traces a real Multicall3 `aggregate3` (etched offline) fanning out +//! to a token, then composes a tracer with transfer-capture over an ERC-20 send. +//! +//! Fully offline (mocked provider, etched bytecode). Run with: +//! +//! ```sh +//! cargo run --example call_tracer +//! ``` + +#[path = "support/mock.rs"] +mod mock; + +use alloy_primitives::{Address, Bytes, U256, hex}; +use alloy_sol_types::SolCall; +use anyhow::Result; +use evm_fork_cache::cache::EvmCache; +use evm_fork_cache::inspector::TransferInspector; +use evm_fork_cache::multicall::{IMulticall3, MULTICALL3_ADDRESS}; +use evm_fork_cache::{CallTrace, CallTracer, EvmOverlay, InspectorStack, TxConfig}; +use revm::state::{AccountInfo, Bytecode}; + +const MULTICALL3_RUNTIME_HEX: &str = include_str!("../fixtures/multicall3_runtime.hex"); + +/// Etch runtime bytecode at `addr` and mark its storage local (an offline contract). +fn install_runtime(cache: &mut EvmCache, addr: Address, runtime_hex: &str) { + let code = Bytecode::new_raw(Bytes::from( + hex::decode(runtime_hex.trim()).expect("valid hex"), + )); + let code_hash = code.hash_slow(); + cache.db_mut().insert_account_info( + addr, + AccountInfo { + code: Some(code), + code_hash, + ..Default::default() + }, + ); + cache + .db_mut() + .replace_account_storage(addr, Default::default()) + .expect("mark storage local"); +} + +fn balance_of(owner: Address) -> Bytes { + Bytes::from(mock::MockERC20::balanceOfCall { account: owner }.abi_encode()) +} + +/// Print a call-frame tree, indented by depth. +fn print_frame(t: &CallTrace) { + let pad = " ".repeat(t.depth + 1); + println!( + "{pad}{:?} {} → {} [{:?}, gas {}]", + t.kind, t.from, t.to, t.status, t.gas_used + ); + for sub in &t.subcalls { + print_frame(sub); + } +} + +#[tokio::main(flavor = "multi_thread")] +async fn main() -> Result<()> { + let mut cache = mock::offline_cache().await?; + let token = Address::repeat_byte(0x11); + let caller = Address::repeat_byte(0x22); + let alice = Address::repeat_byte(0x33); + let bob = Address::repeat_byte(0x44); + mock::install_default_account(&mut cache, Address::ZERO); + mock::install_default_account(&mut cache, caller); + mock::install_default_account(&mut cache, alice); + mock::install_default_account(&mut cache, bob); + mock::install_mock_erc20(&mut cache, token); + install_runtime(&mut cache, MULTICALL3_ADDRESS, MULTICALL3_RUNTIME_HEX); + cache.insert_mapping_storage_slot( + token, + U256::from(mock::MOCK_ERC20_BALANCE_SLOT), + alice, + U256::from(1_000u64), + )?; + + // ---- 1. Trace a nested Multicall3 aggregate3 -> token ---- + let calls = vec![ + IMulticall3::Call3 { + target: token, + allowFailure: false, + callData: balance_of(alice), + }, + IMulticall3::Call3 { + target: token, + allowFailure: false, + callData: balance_of(bob), + }, + ]; + let calldata = Bytes::from(IMulticall3::aggregate3Call { calls }.abi_encode()); + + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + let (_result, tracer) = overlay.call_raw_with_inspector( + caller, + MULTICALL3_ADDRESS, + calldata, + &TxConfig::default(), + CallTracer::new(), + false, + )?; + + println!("=== 1. call-frame tree: aggregate3 fanning out to the token ==="); + if let Some(root) = tracer.root() { + print_frame(root); + println!(" (root has {} subcall(s))", root.subcalls.len()); + } + + // ---- 2. Compose a tracer with transfer capture in one pass ---- + let calldata = Bytes::from( + mock::MockERC20::transferCall { + to: bob, + amount: U256::from(250u64), + } + .abi_encode(), + ); + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + let (_result, stack) = overlay.call_raw_with_inspector( + alice, + token, + calldata, + &TxConfig::default(), + InspectorStack(CallTracer::new(), TransferInspector::new()), + false, + )?; + let InspectorStack(tracer, transfers) = stack; + + println!("\n=== 2. InspectorStack: trace + transfer capture, one pass ==="); + if let Some(root) = tracer.root() { + println!(" traced top-level call to {} ({:?})", root.to, root.status); + } + for t in &transfers.transfers { + println!( + " transfer captured: {} → {} value {} (token {})", + t.from, t.to, t.value, t.token + ); + } + + Ok(()) +} diff --git a/examples/cold_start.rs b/examples/cold_start.rs new file mode 100644 index 0000000..2a0f74f --- /dev/null +++ b/examples/cold_start.rs @@ -0,0 +1,158 @@ +//! Cold-start working-set warming: discover-then-verify. +//! +//! Before going reactive, a searcher warms a *working set* — the accounts and +//! storage slots their strategy reads — into the fork so simulations don't pay an +//! RPC round-trip per slot. The hard part is that you often don't know the exact +//! slots a contract uses. Cold-start solves this declaratively over a bounded, +//! planner-driven loop: +//! +//! 1. **Discover** — run a read-only view-call and capture the `(address, slot)` +//! pairs it touches. Here `balanceOf(owner)` SLOADs the hashed balance slot, so +//! the discover phase learns the slot without us hardcoding its layout. +//! 2. **Verify** — authoritatively re-fetch those discovered slots through the +//! batched [`StorageBatchFetchFn`] and inject the fresh values into the cache. +//! +//! The driver performs all IO; the [`ColdStartPlanner`] stays pure (it is handed +//! only a read-only [`StateView`] and the round's results). Runs fully offline +//! against a mocked provider, with a stub fetcher standing in for the chain. +//! +//! Run with: +//! +//! ```sh +//! cargo run --example cold_start +//! ``` + +#[path = "support/mock.rs"] +mod mock; + +use std::sync::Arc; + +use alloy_eips::BlockId; +use alloy_primitives::{Address, Bytes, U256, keccak256}; +use alloy_sol_types::{SolCall, SolValue, sol}; +use anyhow::Result; +use evm_fork_cache::cache::StorageBatchFetchFn; +use evm_fork_cache::events::StateView; +use evm_fork_cache::{ + ColdStartCall, ColdStartConfig, ColdStartPlan, ColdStartPlanner, ColdStartResults, + ColdStartStep, +}; + +sol! { + interface MockERC20 { + function balanceOf(address account) returns (uint256); + } +} + +/// The hashed `balanceOf[owner]` slot for the MockERC20 fixture (mapping at slot 3). +fn balance_slot(owner: Address) -> U256 { + let key = keccak256((owner, U256::from(mock::MOCK_ERC20_BALANCE_SLOT)).abi_encode()); + U256::from_be_bytes(key.0) +} + +/// Warms a token's working set in two rounds: round 1 *discovers* the slots a +/// `balanceOf(owner)` call touches, round 2 *verifies* (re-fetches + injects) them. +struct WorkingSetWarmer { + token: Address, + owner: Address, + discovered: Vec<(Address, U256)>, + verified_round: bool, +} + +impl ColdStartPlanner for WorkingSetWarmer { + fn initial_plan(&mut self, _state: &dyn StateView) -> ColdStartPlan { + // Round 1: discover. A read-only `balanceOf` view-call; `restrict_to` + // keeps only the token's own slots in the captured access list. + ColdStartPlan { + discover: vec![ColdStartCall { + from: self.owner, + to: self.token, + calldata: Bytes::from( + MockERC20::balanceOfCall { + account: self.owner, + } + .abi_encode(), + ), + restrict_to: Some(vec![self.token]), + }], + ..Default::default() + } + } + + fn on_results(&mut self, results: &ColdStartResults, _state: &dyn StateView) -> ColdStartStep { + if self.verified_round { + return ColdStartStep::Done; + } + // Collect every slot the discover call touched, then verify them next round. + for call in &results.discovered { + self.discovered.extend(call.access.slots.iter().copied()); + } + self.verified_round = true; + ColdStartStep::Continue(ColdStartPlan { + verify: self.discovered.clone(), + ..Default::default() + }) + } +} + +#[tokio::main(flavor = "multi_thread")] +async fn main() -> Result<()> { + let mut cache = mock::offline_cache().await?; + let token = Address::repeat_byte(0x11); + let owner = Address::repeat_byte(0x22); + // Pre-install the coinbase (Address::ZERO) so the view-call's gas accounting + // doesn't trigger a lazy RPC fetch against the mocked (offline) provider. + mock::install_default_account(&mut cache, Address::ZERO); + mock::install_default_account(&mut cache, owner); + mock::install_mock_erc20(&mut cache, token); + + let slot = balance_slot(owner); + + // A stub fetcher standing in for the chain: the owner's true on-chain balance + // is 1_000_000; everything else reads as a genuine zero. + let fetcher: StorageBatchFetchFn = Arc::new( + move |requests: Vec<(Address, U256)>, _block: Option| { + requests + .into_iter() + .map(|(a, s)| { + let value = if (a, s) == (token, slot) { + U256::from(1_000_000u64) + } else { + U256::ZERO + }; + (a, s, Ok(value)) + }) + .collect() + }, + ); + cache.set_storage_batch_fetcher(fetcher); + + println!( + "before cold start: balanceOf(owner) = {} (slot uncached, reads 0)", + mock::balance_of(&mut cache, token, owner)? + ); + + let mut planner = WorkingSetWarmer { + token, + owner, + discovered: Vec::new(), + verified_round: false, + }; + let report = cache.run_cold_start(&mut planner, ColdStartConfig::default())?; + + println!("\n=== cold-start report ==="); + println!(" rounds executed: {}", report.rounds); + println!(" slots discovered: {}", report.discovered_slots); + println!( + " slots verified: {} requested, {} changed + injected", + report.verified_slots, report.changed_slots + ); + println!(" fetch failures: {}", report.failed_slots); + + println!( + "\nafter cold start: balanceOf(owner) = {} (warmed from the fetcher; reads are now local)", + mock::balance_of(&mut cache, token, owner)? + ); + + Ok(()) +} diff --git a/examples/fetch_minimization_counted.rs b/examples/fetch_minimization_counted.rs new file mode 100644 index 0000000..e160970 --- /dev/null +++ b/examples/fetch_minimization_counted.rs @@ -0,0 +1,167 @@ +//! **Pillar 2 — data-fetch minimization (the headline number).** +//! +//! A searcher evaluating N candidate transactions against the *same* recent block +//! reads the same hot working set (a pool's slots, a handful of token balances) +//! over and over. The naive loop — a fresh fork/cache per candidate — cold-fetches +//! that working set *every* candidate. `evm-fork-cache` fetches it **once**, freezes +//! it into a snapshot, and fans every candidate out over cheap in-memory overlays +//! that hold no RPC backend, so the fan-out adds **zero** further fetches. +//! +//! This example counts real fetcher invocations (an `AtomicUsize`-wrapped +//! [`StorageBatchFetchFn`]) and prints the exact integers. The count is +//! deterministic and machine-independent — it is a literal tally of RPC reads +//! avoided, not a wall-clock measurement. +//! +//! Run with: `cargo run --example fetch_minimization_counted` + +#[path = "support/mock.rs"] +mod mock; + +use std::sync::Arc; +use std::sync::atomic::{AtomicUsize, Ordering}; + +use alloy_eips::BlockId; +use alloy_primitives::{Address, Bytes, U256, keccak256}; +use alloy_sol_types::{SolCall, SolValue}; +use anyhow::Result; +use evm_fork_cache::cache::{EvmCache, EvmOverlay, StorageBatchFetchFn}; +use revm::context::result::ExecutionResult; +use revm::state::AccountInfo; + +/// Candidate transactions evaluated against the head this block. +const N_CANDIDATES: usize = 500; +/// Distinct storage slots in the shared hot working set (token balance slots). +const WORKING_SET: usize = 8; + +/// An owner address derived from an index. +fn owner(i: usize) -> Address { + let mut bytes = [0u8; 20]; + bytes[12..20].copy_from_slice(&(i as u64 + 1).to_be_bytes()); + Address::from(bytes) +} + +/// `keccak256(abi.encode(owner, MOCK_ERC20_BALANCE_SLOT))` — the balance slot. +fn balance_slot(owner: Address) -> U256 { + U256::from_be_bytes( + keccak256((owner, U256::from(mock::MOCK_ERC20_BALANCE_SLOT)).abi_encode()).0, + ) +} + +/// An `AtomicUsize`-counting batch fetcher: tallies every requested slot and +/// returns a canned non-zero balance, standing in for the RPC backend. +fn counting_fetcher(counter: Arc) -> StorageBatchFetchFn { + Arc::new( + move |requests: Vec<(Address, U256)>, _block: Option| { + counter.fetch_add(requests.len(), Ordering::Relaxed); + requests + .into_iter() + .map(|(addr, slot)| (addr, slot, Ok(U256::from(1_000u64)))) + .collect() + }, + ) +} + +/// Build an offline cache with a MockERC20 installed at `token` (storage left +/// non-cleared so warmed layer-2 slots are readable) plus a counting fetcher. +async fn cache_with_counter(token: Address) -> Result<(EvmCache, Arc)> { + let mut cache = mock::offline_cache().await?; + let runtime = mock::mock_erc20_runtime(); + let code_hash = runtime.hash_slow(); + cache.db_mut().insert_account_info( + token, + AccountInfo { + balance: U256::ZERO, + nonce: 1, + code: Some(runtime), + code_hash, + account_id: None, + }, + ); + let counter = Arc::new(AtomicUsize::new(0)); + cache.set_storage_batch_fetcher(counting_fetcher(counter.clone())); + Ok((cache, counter)) +} + +fn balance_of_calldata(account: Address) -> Bytes { + Bytes::from(mock::MockERC20::balanceOfCall { account }.abi_encode()) +} + +#[tokio::main(flavor = "multi_thread")] +async fn main() -> Result<()> { + let token = Address::repeat_byte(0x42); + let working_set: Vec<(Address, U256)> = (0..WORKING_SET) + .map(|i| (token, balance_slot(owner(i)))) + .collect(); + + // ── evm-fork-cache: fetch the working set ONCE, then fan out ────────────── + let (mut cache, counter) = cache_with_counter(token).await?; + + // Warm the hot working set in a single batch (the prefetch a searcher does + // once per block). Every slot is fetched exactly once. + cache.verify_slots(&working_set)?; + let warmup_fetches = counter.load(Ordering::Relaxed); + + // Freeze the warmed state, then fan N candidates out over cheap overlays. + let snapshot = cache.create_snapshot(); + counter.store(0, Ordering::Relaxed); // measure only the fan-out from here + + for c in 0..N_CANDIDATES { + // Each candidate is an isolated simulation reading the shared hot set. + // The overlay holds no RPC backend (`None`), so it cannot fetch — every + // read is served from the frozen snapshot in memory. + let mut overlay = EvmOverlay::new(snapshot.clone(), None); + for i in 0..WORKING_SET { + let result = + overlay.call_raw(owner(c % WORKING_SET), token, balance_of_calldata(owner(i)))?; + debug_assert!(matches!(result, ExecutionResult::Success { .. })); + } + } + let fanout_fetches = counter.load(Ordering::Relaxed); + + // ── Vanilla baseline: a fresh cold cache per candidate ─────────────────── + // Each independent cache cold-fetches the working set it reads. We measure + // the per-candidate cost on one real cold cache; it then repeats N times. + let (mut cold, cold_counter) = cache_with_counter(token).await?; + cold.verify_slots(&working_set)?; + let per_candidate_fetches = cold_counter.load(Ordering::Relaxed); + let vanilla_total = per_candidate_fetches * N_CANDIDATES; + + let crate_total = warmup_fetches + fanout_fetches; + + println!( + "Workload: {N_CANDIDATES} candidate txs, each reading a shared {WORKING_SET}-slot hot set\n" + ); + println!("evm-fork-cache"); + println!(" warm-up fetches (once): {warmup_fetches}"); + println!( + " fan-out fetches ({N_CANDIDATES} sims): {fanout_fetches} <- overlays read the frozen snapshot, no backend" + ); + println!(" TOTAL slots fetched: {crate_total}"); + println!("\nVanilla fork-per-candidate"); + println!(" fetches per candidate: {per_candidate_fetches}"); + println!( + " TOTAL slots fetched: {vanilla_total} ({N_CANDIDATES} x {per_candidate_fetches})" + ); + println!( + "\n=> {}x fewer reads than a NAIVE un-shared loop ({} slots avoided)", + vanilla_total / crate_total.max(1), + vanilla_total - crate_total + ); + println!( + "\nHonest caveat: this beats only the *naive* baseline that builds a fresh cold\n\ + cache per candidate. A competent searcher sharing ONE foundry-fork-db\n\ + SharedBackend across candidates ALSO fetches each slot once within a block\n\ + (its cache dedups) — so within a single block this is ~1x, not {}x.\n\ + The durable win is CROSS-block: as blocks advance and the hot set mutates,\n\ + event-driven writes (see the reactive_cache example) keep it fresh with 0\n\ + re-fetches, where a refetch loop must re-read the changed slots every block.", + vanilla_total / crate_total.max(1), + ); + + assert_eq!( + warmup_fetches, WORKING_SET, + "warm-up fetches the working set once" + ); + assert_eq!(fanout_fetches, 0, "the fan-out adds zero fetches"); + Ok(()) +} diff --git a/examples/freshness_multi_sim.rs b/examples/freshness_multi_sim.rs index 15a8780..c837b12 100644 --- a/examples/freshness_multi_sim.rs +++ b/examples/freshness_multi_sim.rs @@ -29,7 +29,7 @@ use alloy_eips::BlockId; use alloy_primitives::{Address, Bytes, U256, keccak256}; use alloy_sol_types::{SolCall, SolValue}; use anyhow::Result; -use evm_fork_cache::cache::StorageBatchFetchFn; +use evm_fork_cache::cache::{SimStatus, StorageBatchFetchFn}; use evm_fork_cache::freshness::{ AlwaysVerify, FreshnessController, FreshnessRegistry, SimRequest, Validation, Validity, }; @@ -133,7 +133,7 @@ async fn main() -> Result<()> { let optimistic_ok: Vec = sim .optimistic() .iter() - .map(|r| !r.logs.is_empty()) + .map(|r| matches!(r.status, SimStatus::Success)) .collect(); println!("optimistic (against the snapshot): {optimistic_ok:?} (all succeed)"); @@ -146,7 +146,10 @@ async fn main() -> Result<()> { for c in &changed { println!(" sender slot {} : {} -> {}", c.slot, c.old, c.new); } - let corrected_ok: Vec = results.iter().map(|r| !r.logs.is_empty()).collect(); + let corrected_ok: Vec = results + .iter() + .map(|r| matches!(r.status, SimStatus::Success)) + .collect(); println!("corrected results: {corrected_ok:?}"); // Exactly the second sim flipped success -> revert; the others are diff --git a/examples/freshness_optimistic.rs b/examples/freshness_optimistic.rs index 42fa191..d4b8c5d 100644 --- a/examples/freshness_optimistic.rs +++ b/examples/freshness_optimistic.rs @@ -74,11 +74,20 @@ async fn main() -> Result<()> { ); cache.set_storage_batch_fetcher(fetcher); - // Classification: the balance slot is volatile (default), and slot 6 (a - // would-be immutable like `token0`) is pinned so it is never re-verified. + // Classification: by default every slot is volatile (re-verified); we pin + // slot 6 (a would-be immutable, constructor-set config value) so it is never + // re-verified. let mut registry = FreshnessRegistry::new(); registry.pin_slot(token, U256::from(6)); + // Show the classification before the registry is moved into the controller: + // an unpinned slot is volatile, the pinned one is not (so it is skipped). + println!( + "classification — slot 0 volatile? {} | slot 6 (pinned) volatile? {}", + registry.is_volatile(token, U256::from(0), 0), + registry.is_volatile(token, U256::from(6), 0), + ); + let mut controller = FreshnessController::new(registry, AlwaysVerify); // A non-committing `transfer(recipient, 100)` evaluation sim. diff --git a/examples/reactive_cache.rs b/examples/reactive_cache.rs index e498ad1..45544c3 100644 --- a/examples/reactive_cache.rs +++ b/examples/reactive_cache.rs @@ -21,6 +21,7 @@ mod mock; use std::collections::HashMap; use std::sync::Arc; +use std::sync::atomic::{AtomicUsize, Ordering}; use alloy_eips::BlockId; use alloy_primitives::{Address, Bytes, Log, U256, keccak256}; @@ -77,6 +78,25 @@ async fn main() -> Result<()> { let mut controller = FreshnessController::new(FreshnessRegistry::new(), AlwaysVerify); + // Install a counting fetcher up front so we can show that *ingest* performs no + // RPC reads (it decodes logs into writes), while *reconcile* below samples a + // few slots on purpose. It returns the (deliberately drifted) chain value for + // bob so the reconcile step has something to correct. + let fetches = Arc::new(AtomicUsize::new(0)); + let counter = fetches.clone(); + let fresh: HashMap<(Address, U256), U256> = + HashMap::from([((token, bob_slot), U256::from(260))]); + let fetcher: StorageBatchFetchFn = Arc::new( + move |requests: Vec<(Address, U256)>, _block: Option| { + counter.fetch_add(requests.len(), Ordering::Relaxed); + requests + .into_iter() + .map(|(a, s)| (a, s, Ok(fresh.get(&(a, s)).copied().unwrap_or(U256::ZERO)))) + .collect() + }, + ); + cache.set_storage_batch_fetcher(fetcher); + let block = 100u64; let digest = pipeline.ingest_logs( &mut cache, @@ -85,6 +105,11 @@ async fn main() -> Result<()> { ); println!("=== ingested block {} ===", digest.block); + println!( + " RPC slot fetches during ingest: {} <- a poller would refetch {} touched slot(s)/block", + fetches.load(Ordering::Relaxed), + digest.touched_slots.len(), + ); println!( " decoded {} log(s) -> {} slot change(s), {} skipped", digest.decoded_logs, @@ -108,20 +133,13 @@ async fn main() -> Result<()> { digest.touched_slots.len() ); - let fresh: HashMap<(Address, U256), U256> = - HashMap::from([((token, bob_slot), U256::from(260))]); - let fetcher: StorageBatchFetchFn = Arc::new( - move |requests: Vec<(Address, U256)>, _block: Option| { - requests - .into_iter() - .map(|(a, s)| (a, s, Ok(fresh.get(&(a, s)).copied().unwrap_or(U256::ZERO)))) - .collect() - }, - ); - cache.set_storage_batch_fetcher(fetcher); - + fetches.store(0, Ordering::Relaxed); // measure reconcile's sampling cost let report = pipeline.reconcile(&mut cache, &[(token, bob_slot)])?; println!("\n=== reconcile (sampled {} slot) ===", report.checked); + println!( + " RPC slot fetches during reconcile: {} <- the sampled honesty backstop", + fetches.load(Ordering::Relaxed), + ); if report.mismatched.is_empty() { println!(" no drift: event-derived state matches chain"); } else { @@ -137,8 +155,12 @@ async fn main() -> Result<()> { mock::balance_of(&mut cache, token, bob)? ); + // `EventPipeline::new` uses the default `ReorgConfig`, whose + // `PurgeScope::AllStorage` clears the storage of every address touched after + // the new head — so bob's balance slot reads back as uncached (`None`) and the + // next access would re-fetch it from RPC. let purge = pipeline.reorg_to(&mut cache, 99); - println!("\n=== reorg to block 99 ==="); + println!("\n=== reorg to block 99 (default scope: AllStorage) ==="); println!( " purged {} address(es); bob balance slot now cached as {:?}", purge.purged.len(), diff --git a/examples/reactive_runtime.rs b/examples/reactive_runtime.rs new file mode 100644 index 0000000..0b3210e --- /dev/null +++ b/examples/reactive_runtime.rs @@ -0,0 +1,324 @@ +//! Reactive runtime end-to-end (Pillar 2): handler → `StateUpdate` → apply, plus +//! journaled reorg rollback — the part a searcher would otherwise hand-roll. +//! +//! The reactive runtime is the crate's headline differentiator over bare `revm`: +//! `revm` executes, but it does not keep your forked state correct as the chain +//! moves. This example shows the full loop with no RPC and no manual bookkeeping: +//! +//! 1. Register a provider-neutral [`ReactiveHandler`] that turns a matching log +//! into a targeted [`StateUpdate`]. The key insight is that **events already +//! carry the post-state** — here the new value rides in the log's `data`, so we +//! *decode-and-write* it straight into a slot instead of re-fetching from RPC. +//! 2. Feed a canonical block's log through [`ReactiveRuntime::ingest_batch`]; the +//! runtime validates the handler's effects and applies them to the cache with +//! **0 RPC fetches** (counted below). +//! 3. Re-deliver that same log as `removed` (a reorg). The runtime unwinds the +//! journaled write back to its prior value automatically and emits a +//! [`ReactiveReport::Reorg`] describing exactly what it rolled back. +//! +//! A [`ReactiveHook`] observes the applied reports out-of-band (metrics, logging). +//! +//! Runs fully offline against a mocked provider — no network, no RPC key. +//! +//! Run with: +//! +//! ```sh +//! cargo run --example reactive_runtime +//! ``` + +#[path = "support/mock.rs"] +mod mock; + +use std::sync::atomic::{AtomicUsize, Ordering}; +use std::sync::{Arc, Mutex}; + +use alloy_eips::BlockId; +use alloy_network::Ethereum; +use alloy_primitives::{Address, B256, Bytes, Log as PrimitiveLog, U256, keccak256}; +use alloy_rpc_types_eth::{Filter, Log}; +use anyhow::Result; +use evm_fork_cache::StateUpdate; +use evm_fork_cache::cache::StorageBatchFetchFn; +use evm_fork_cache::events::StateView; +use evm_fork_cache::reactive::{ + AppliedReport, BlockRef, ChainStatus, HandlerError, HandlerId, HandlerOutcome, HookSignal, + InputSource, LogInterest, ReactiveConfig, ReactiveContext, ReactiveEffect, ReactiveHandler, + ReactiveHook, ReactiveInput, ReactiveInputBatch, ReactiveInputRecord, ReactiveInterest, + ReactiveReport, ReactiveRuntime, ReportTag, RouteKeySpec, StateEffectQuality, +}; + +/// A storage slot standing in for any piece of hot state whose new value an event +/// already carries (e.g. a pool's packed price word after a swap). +const TARGET_SLOT: u64 = 7; + +/// A protocol-neutral handler. Every matching log carries the post-state value in +/// its `data`, so we decode it and write it straight into `slot` — no RPC. This is +/// the "events already carry the post-state" idea that the reactive runtime exists +/// to package safely (validation, journaling, reorg recovery). +struct PostStateWriter { + emitter: Address, + slot: U256, +} + +impl ReactiveHandler for PostStateWriter { + fn id(&self) -> HandlerId { + HandlerId::new("post-state-writer") + } + + fn interests(&self) -> Vec { + // Subscribe to every log emitted by `emitter`. `provider_filter` is what a + // live transport (e.g. the bundled `AlloySubscriber`) would push upstream; + // `route_key` lets the runtime fan many emitters out to the right handler. + vec![ReactiveInterest::Logs(LogInterest { + provider_filter: Filter::new().address(self.emitter), + local_matcher: None, + route_key: Some(RouteKeySpec::EmitterAddress), + })] + } + + fn handle( + &self, + _ctx: &ReactiveContext, + input: &ReactiveInput, + _state: &dyn StateView, + ) -> Result { + let ReactiveInput::Log(log) = input else { + return Ok(HandlerOutcome::empty(StateEffectQuality::NoStateEffect)); + }; + + // The new value is the 32-byte log payload. A real decoder would pull it + // from the event's ABI; the runtime does not care how — it only sees the + // resulting `StateUpdate`. + let data = &log.inner.data.data; + if data.len() != 32 { + return Ok(HandlerOutcome::empty(StateEffectQuality::NoStateEffect)); + } + let value = U256::from_be_slice(data); + + Ok(HandlerOutcome { + effects: vec![ + ReactiveEffect::StateUpdate(StateUpdate::slot(log.address(), self.slot, value)), + // A hook signal is an out-of-band notification (metrics, logging); + // it never mutates the cache. + ReactiveEffect::Hook(HookSignal { + namespace: "demo".into(), + kind: "slot.write".into(), + labels: vec![ReportTag::new("slot", self.slot.to_string())], + payload: None, + }), + ], + // `ExactFromInput`: the value is authoritative from the event itself, + // not a guess that needs reconciling. + quality: StateEffectQuality::ExactFromInput, + tags: vec![], + }) + } +} + +/// An out-of-band observer of applied reports. Hooks run synchronously on the +/// ingest thread, so they must be cheap and non-blocking. +#[derive(Default)] +struct AppliedLog { + writes: Arc>>, + signals: Arc>>, +} + +impl ReactiveHook for AppliedLog { + fn on_report(&self, report: Arc>) { + if let ReactiveReport::Applied(AppliedReport { + diff, hook_signals, .. + }) = report.as_ref() + { + self.writes + .lock() + .unwrap() + .extend(diff.slots.iter().map(|change| change.new)); + self.signals.lock().unwrap().extend( + hook_signals + .iter() + .map(|signal| format!("{}:{}", signal.namespace, signal.kind)), + ); + } + } +} + +/// Build a log from `emitter` carrying a 32-byte big-endian `value` in its data. +fn value_log( + emitter: Address, + topic: B256, + value: U256, + block: &BlockRef, + log_index: u64, + removed: bool, +) -> Log { + Log { + inner: PrimitiveLog::new_unchecked( + emitter, + vec![topic], + Bytes::copy_from_slice(&value.to_be_bytes::<32>()), + ), + block_hash: Some(block.hash), + block_number: Some(block.number), + block_timestamp: block.timestamp, + transaction_hash: Some(B256::repeat_byte(0xfe)), + transaction_index: Some(0), + log_index: Some(log_index), + removed, + } +} + +fn block_ref(number: u64, hash: u8, parent: u8) -> BlockRef { + BlockRef { + number, + hash: B256::repeat_byte(hash), + parent_hash: Some(B256::repeat_byte(parent)), + timestamp: Some(1_700_000_000 + number), + } +} + +/// A canonical-inclusion context for a log at `block` / `log_index`. +fn included(block: BlockRef, log_index: u64) -> ReactiveContext { + ReactiveContext { + chain_id: Some(1), + source: InputSource::Batch, + chain_status: ChainStatus::Included { + block: block.clone(), + confirmations: 0, + }, + block: Some(block), + transaction_index: Some(0), + log_index: Some(log_index), + } +} + +/// A reorg context: `dropped` is the block that fell off the canonical chain. +fn reorged(dropped: BlockRef, log_index: u64) -> ReactiveContext { + ReactiveContext { + chain_id: Some(1), + source: InputSource::Batch, + chain_status: ChainStatus::Reorged { + dropped_from: dropped.clone(), + }, + block: Some(dropped), + transaction_index: Some(0), + log_index: Some(log_index), + } +} + +/// Wrap one (input, context) pair into a single-record batch. +fn one(input: ReactiveInput, ctx: ReactiveContext) -> ReactiveInputBatch { + ReactiveInputBatch::new(vec![ReactiveInputRecord::new(input, ctx)]) +} + +#[tokio::main(flavor = "multi_thread")] +async fn main() -> Result<()> { + let mut cache = mock::offline_cache().await?; + let emitter = Address::repeat_byte(0x42); + mock::install_mock_erc20(&mut cache, emitter); + + let slot = U256::from(TARGET_SLOT); + // Seed a known prior value so the reorg rollback has something to restore. + cache + .db_mut() + .insert_account_storage(emitter, slot, U256::from(10))?; + + // A counting fetcher so we can *prove* that ingest applies writes with no RPC. + // `ingest_batch` decodes logs into writes; if it ever reached for RPC, this + // counter would catch it. + let fetches = Arc::new(AtomicUsize::new(0)); + let counter = fetches.clone(); + let fetcher: StorageBatchFetchFn = Arc::new( + move |requests: Vec<(Address, U256)>, _block: Option| { + counter.fetch_add(requests.len(), Ordering::Relaxed); + requests + .into_iter() + .map(|(a, s)| (a, s, Ok(U256::ZERO))) + .collect() + }, + ); + cache.set_storage_batch_fetcher(fetcher); + + let hook = Arc::new(AppliedLog::default()); + let mut runtime = ReactiveRuntime::::new(ReactiveConfig::default()); + runtime.register_handler(Arc::new(PostStateWriter { emitter, slot }))?; + runtime.register_hook(hook.clone())?; + + let topic = keccak256(b"PostState(uint256)"); + println!( + "slot {} starts at {:?} (the on-chain value before any event)", + TARGET_SLOT, + cache.cached_storage_value(emitter, slot) + ); + + // 1) Canonical block 100: the log carries the new value (20). Decode-and-write. + let canonical = block_ref(100, 0x64, 0x63); + fetches.store(0, Ordering::Relaxed); + let report = runtime.ingest_batch( + &mut cache, + one( + ReactiveInput::Log(value_log( + emitter, + topic, + U256::from(20), + &canonical, + 0, + false, + )), + included(canonical.clone(), 0), + ), + )?; + println!("\n=== block {} ingested ===", canonical.number); + println!( + " applied {} update(s) with {} RPC fetch(es) <- events carry the post-state", + report.applied.len(), + fetches.load(Ordering::Relaxed), + ); + println!( + " slot {} = {:?}", + TARGET_SLOT, + cache.cached_storage_value(emitter, slot) + ); + + // 2) Reorg: the same log is re-delivered as `removed`. The runtime unwinds the + // journaled write back to the prior value — no RPC, no manual undo log. + let report = runtime.ingest_batch( + &mut cache, + one( + ReactiveInput::Log(value_log( + emitter, + topic, + U256::from(20), + &canonical, + 0, + true, + )), + reorged(canonical.clone(), 0), + ), + )?; + let reorg = report + .reports + .iter() + .find_map(|report| match report.as_ref() { + ReactiveReport::Reorg(reorg) => Some(reorg), + _ => None, + }) + .expect("a removed log emits a reorg report"); + println!("\n=== reorg: block {} dropped ===", canonical.number); + println!(" rolled back {} write(s):", reorg.rollback_updates.len()); + for change in &reorg.rollback_diff.slots { + println!(" slot rolled {} -> {}", change.old, change.new); + } + println!( + " slot {} restored to {:?} <- automatic, journaled recovery", + TARGET_SLOT, + cache.cached_storage_value(emitter, slot) + ); + + println!( + "\nhook observed: writes={:?}, signals={:?}", + hook.writes.lock().unwrap(), + hook.signals.lock().unwrap() + ); + + Ok(()) +} diff --git a/fixtures/README.md b/fixtures/README.md index ce6f067..60703de 100644 --- a/fixtures/README.md +++ b/fixtures/README.md @@ -43,3 +43,19 @@ jq -r '.deployedBytecode.object' out/MockERC20.sol/MockERC20.json \ jq -r '.bytecode.object' out/MockERC20.sol/MockERC20.json \ | sed 's/^0x//' > fixtures/mock_erc20_creation.hex ``` + +## `Multicall3` + +- `multicall3_runtime.hex` — the canonical [Multicall3](https://github.com/mds1/multicall) + deployed (runtime) bytecode. Multicall3 lives at the same address + (`0xcA11bde05977b3631167028862bE2a173976CA11`) on virtually every EVM chain, so + etching this bytecode there lets the offline [`../tests/multicall.rs`](../tests/multicall.rs) + suite exercise the real `aggregate3` build/execute/decode path (input-order + results and `allowFailure` semantics) with no network. + +### Regenerating + +```sh +cast code 0xcA11bde05977b3631167028862bE2a173976CA11 --rpc-url "$RPC_URL" \ + | sed 's/^0x//' > fixtures/multicall3_runtime.hex +``` diff --git a/fixtures/multicall3_runtime.hex b/fixtures/multicall3_runtime.hex new file mode 100644 index 0000000..9013a03 --- /dev/null +++ b/fixtures/multicall3_runtime.hex @@ -0,0 +1 @@ +6080604052600436106100f35760003560e01c80634d2301cc1161008a578063a8b0574e11610059578063a8b0574e1461025a578063bce38bd714610275578063c3077fa914610288578063ee82ac5e1461029b57600080fd5b80634d2301cc146101ec57806372425d9d1461022157806382ad56cb1461023457806386d516e81461024757600080fd5b80633408e470116100c65780633408e47014610191578063399542e9146101a45780633e64a696146101c657806342cbb15c146101d957600080fd5b80630f28c97d146100f8578063174dea711461011a578063252dba421461013a57806327e86d6e1461015b575b600080fd5b34801561010457600080fd5b50425b6040519081526020015b60405180910390f35b61012d610128366004610a85565b6102ba565b6040516101119190610bbe565b61014d610148366004610a85565b6104ef565b604051610111929190610bd8565b34801561016757600080fd5b50437fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff0140610107565b34801561019d57600080fd5b5046610107565b6101b76101b2366004610c60565b610690565b60405161011193929190610cba565b3480156101d257600080fd5b5048610107565b3480156101e557600080fd5b5043610107565b3480156101f857600080fd5b50610107610207366004610ce2565b73ffffffffffffffffffffffffffffffffffffffff163190565b34801561022d57600080fd5b5044610107565b61012d610242366004610a85565b6106ab565b34801561025357600080fd5b5045610107565b34801561026657600080fd5b50604051418152602001610111565b61012d610283366004610c60565b61085a565b6101b7610296366004610a85565b610a1a565b3480156102a757600080fd5b506101076102b6366004610d18565b4090565b60606000828067ffffffffffffffff8111156102d8576102d8610d31565b60405190808252806020026020018201604052801561031e57816020015b6040805180820190915260008152606060208201528152602001906001900390816102f65790505b5092503660005b8281101561047757600085828151811061034157610341610d60565b6020026020010151905087878381811061035d5761035d610d60565b905060200281019061036f9190610d8f565b6040810135958601959093506103886020850185610ce2565b73ffffffffffffffffffffffffffffffffffffffff16816103ac6060870187610dcd565b6040516103ba929190610e32565b60006040518083038185875af1925050503d80600081146103f7576040519150601f19603f3d011682016040523d82523d6000602084013e6103fc565b606091505b50602080850191909152901515808452908501351761046d577f08c379a000000000000000000000000000000000000000000000000000000000600052602060045260176024527f4d756c746963616c6c333a2063616c6c206661696c656400000000000000000060445260846000fd5b5050600101610325565b508234146104e6576040517f08c379a000000000000000000000000000000000000000000000000000000000815260206004820152601a60248201527f4d756c746963616c6c333a2076616c7565206d69736d6174636800000000000060448201526064015b60405180910390fd5b50505092915050565b436060828067ffffffffffffffff81111561050c5761050c610d31565b60405190808252806020026020018201604052801561053f57816020015b606081526020019060019003908161052a5790505b5091503660005b8281101561068657600087878381811061056257610562610d60565b90506020028101906105749190610e42565b92506105836020840184610ce2565b73ffffffffffffffffffffffffffffffffffffffff166105a66020850185610dcd565b6040516105b4929190610e32565b6000604051808303816000865af19150503d80600081146105f1576040519150601f19603f3d011682016040523d82523d6000602084013e6105f6565b606091505b5086848151811061060957610609610d60565b602090810291909101015290508061067d576040517f08c379a000000000000000000000000000000000000000000000000000000000815260206004820152601760248201527f4d756c746963616c6c333a2063616c6c206661696c656400000000000000000060448201526064016104dd565b50600101610546565b5050509250929050565b43804060606106a086868661085a565b905093509350939050565b6060818067ffffffffffffffff8111156106c7576106c7610d31565b60405190808252806020026020018201604052801561070d57816020015b6040805180820190915260008152606060208201528152602001906001900390816106e55790505b5091503660005b828110156104e657600084828151811061073057610730610d60565b6020026020010151905086868381811061074c5761074c610d60565b905060200281019061075e9190610e76565b925061076d6020840184610ce2565b73ffffffffffffffffffffffffffffffffffffffff166107906040850185610dcd565b60405161079e929190610e32565b6000604051808303816000865af19150503d80600081146107db576040519150601f19603f3d011682016040523d82523d6000602084013e6107e0565b606091505b506020808401919091529015158083529084013517610851577f08c379a000000000000000000000000000000000000000000000000000000000600052602060045260176024527f4d756c746963616c6c333a2063616c6c206661696c656400000000000000000060445260646000fd5b50600101610714565b6060818067ffffffffffffffff81111561087657610876610d31565b6040519080825280602002602001820160405280156108bc57816020015b6040805180820190915260008152606060208201528152602001906001900390816108945790505b5091503660005b82811015610a105760008482815181106108df576108df610d60565b602002602001015190508686838181106108fb576108fb610d60565b905060200281019061090d9190610e42565b925061091c6020840184610ce2565b73ffffffffffffffffffffffffffffffffffffffff1661093f6020850185610dcd565b60405161094d929190610e32565b6000604051808303816000865af19150503d806000811461098a576040519150601f19603f3d011682016040523d82523d6000602084013e61098f565b606091505b506020830152151581528715610a07578051610a07576040517f08c379a000000000000000000000000000000000000000000000000000000000815260206004820152601760248201527f4d756c746963616c6c333a2063616c6c206661696c656400000000000000000060448201526064016104dd565b506001016108c3565b5050509392505050565b6000806060610a2b60018686610690565b919790965090945092505050565b60008083601f840112610a4b57600080fd5b50813567ffffffffffffffff811115610a6357600080fd5b6020830191508360208260051b8501011115610a7e57600080fd5b9250929050565b60008060208385031215610a9857600080fd5b823567ffffffffffffffff811115610aaf57600080fd5b610abb85828601610a39565b90969095509350505050565b6000815180845260005b81811015610aed57602081850181015186830182015201610ad1565b81811115610aff576000602083870101525b50601f017fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe0169290920160200192915050565b600082825180855260208086019550808260051b84010181860160005b84811015610bb1578583037fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe001895281518051151584528401516040858501819052610b9d81860183610ac7565b9a86019a9450505090830190600101610b4f565b5090979650505050505050565b602081526000610bd16020830184610b32565b9392505050565b600060408201848352602060408185015281855180845260608601915060608160051b870101935082870160005b82811015610c52577fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffa0888703018452610c40868351610ac7565b95509284019290840190600101610c06565b509398975050505050505050565b600080600060408486031215610c7557600080fd5b83358015158114610c8557600080fd5b9250602084013567ffffffffffffffff811115610ca157600080fd5b610cad86828701610a39565b9497909650939450505050565b838152826020820152606060408201526000610cd96060830184610b32565b95945050505050565b600060208284031215610cf457600080fd5b813573ffffffffffffffffffffffffffffffffffffffff81168114610bd157600080fd5b600060208284031215610d2a57600080fd5b5035919050565b7f4e487b7100000000000000000000000000000000000000000000000000000000600052604160045260246000fd5b7f4e487b7100000000000000000000000000000000000000000000000000000000600052603260045260246000fd5b600082357fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffff81833603018112610dc357600080fd5b9190910192915050565b60008083357fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffe1843603018112610e0257600080fd5b83018035915067ffffffffffffffff821115610e1d57600080fd5b602001915036819003821315610a7e57600080fd5b8183823760009101908152919050565b600082357fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffc1833603018112610dc357600080fd5b600082357fffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffffa1833603018112610dc357600080fdfea2646970667358221220bb2b5c71a328032f97c676ae39a1ec2148d3e5d6f73d95e9b17910152d61f16264736f6c634300080c0033 diff --git a/src/access_list.rs b/src/access_list.rs index 4bb44d0..39d495b 100644 --- a/src/access_list.rs +++ b/src/access_list.rs @@ -1,9 +1,14 @@ -//! Selective EIP-2930 access list builder. +//! EIP-2930 access list builder with L2 profitability accounting. //! -//! Builds an access list containing only well-known, low-entropy storage slots -//! (V3 slot0/liquidity, V2 reserves) whose serialized form is mostly zero bytes -//! and cheap to post as L1 data. Skips keccak-derived mapping keys (tick bitmap, -//! tick info) which are 32 random bytes and expensive on L1. +//! The caller supplies the addresses and storage slots to include; this module +//! decides *whether attaching the list is profitable*, not *which slots are +//! interesting* (it carries no protocol-specific slot knowledge). The trade-off +//! is purely economic: pre-declaring an account/slot warms it (cheaper EIP-2929 +//! execution) but costs L1 data to post the list. Slots whose serialized form is +//! mostly zero bytes (e.g. small, low-entropy values) are cheap to post; dense, +//! high-entropy 32-byte keys are expensive on L1 — so the value of a given entry +//! depends on its bytes, which is why the include/exclude decision is left to the +//! caller and the profitability check below. //! //! On L2, automatically disables itself when L1 fees rise high enough that the //! L1 data cost exceeds the L2 execution savings. Arbitrum uses `ArbGasInfo` @@ -89,10 +94,13 @@ sol! { } } -/// A selective EIP-2930 access list built from the execution plan. +/// An EIP-2930 access list builder with L2 profitability accounting. /// -/// Only includes entries where the L2 execution savings (100 gas each) -/// are likely to exceed the L1 data posting cost of the serialized entry. +/// The caller decides which addresses/slots to add (via +/// [`add_address`](Self::add_address) / [`add_storage_key`](Self::add_storage_key)); +/// the builder itself applies no per-entry selection. The `into_access_list_*` +/// finalizers decide whether attaching the *whole* list is profitable, comparing +/// its L1 data-posting cost against the L2 warm-access savings. pub struct SmartAccessList { items: Vec, } diff --git a/src/bundle.rs b/src/bundle.rs new file mode 100644 index 0000000..fe0652b --- /dev/null +++ b/src/bundle.rs @@ -0,0 +1,189 @@ +//! Multi-transaction **bundle simulation** over cumulative block state, plus +//! **coinbase / miner-payment accounting** (Phase 6 Track A+B). +//! +//! A *bundle* is an ordered sequence of `Call`-kind transactions applied to a +//! single overlay so that transaction `i` observes the committed writes of +//! transactions `0..i` — the minimal primitive an MEV searcher needs to value a +//! candidate set of transactions as a unit. It is intentionally *not* a block +//! builder: there is no mempool, ordering auction, or `Create`-kind support (see +//! the Phase 6 spec non-goals). +//! +//! The execution itself lives in [`EvmOverlay::simulate_bundle`] (and the +//! cache-side convenience [`EvmCache::simulate_bundle`]); this module owns the +//! public vocabulary those methods speak. +//! +//! # Coinbase accounting +//! +//! [`BundleResult::coinbase_payment`] is the block beneficiary's balance delta +//! across the bundle — the honest miner payment. Under EIP-1559 (London+, which +//! this engine runs by default) revm credits the beneficiary only the **priority +//! fee** (`(effective_gas_price − basefee) × gas_used`) and burns the base-fee +//! portion in-EVM, so the delta already excludes the base fee. It also captures +//! any **direct value transfers to the beneficiary** (an explicit coinbase tip). +//! So `coinbase_payment = Σ priority_feeᵢ × gas_usedᵢ + direct coinbase tips`, +//! over the transactions whose effects are kept. Set the base fee with +//! [`EvmCache::set_basefee`](crate::cache::EvmCache::set_basefee) to model a +//! non-zero base fee (a higher base fee lowers the priority fee, and thus the +//! payment, for a fixed `gas_price`). All arithmetic is saturating. +//! +//! [`EvmCache::simulate_bundle`]: crate::cache::EvmCache::simulate_bundle +//! [`EvmOverlay::simulate_bundle`]: crate::cache::EvmOverlay::simulate_bundle + +use alloy_primitives::{Address, Bytes, Log, U256}; +use revm::context::result::ExecutionResult; + +use crate::cache::TxConfig; + +/// One transaction in a bundle. +/// +/// `Call`-kind only for Phase 6 (`Create`/`Create2` bundle transactions are a +/// documented follow-up). The [`tx`](BundleTx::tx) field reuses the existing +/// [`TxConfig`] vocabulary (`value` / `gas_limit` / `gas_price` / `nonce` / +/// `access_list`), so a bundle transaction can carry native value, be +/// gas-bounded, or pre-warm an EIP-2930 access list exactly like a single +/// [`call_raw_with_access_list_with`](crate::cache::EvmOverlay::call_raw_with_access_list_with) +/// call. +#[derive(Clone, Debug)] +pub struct BundleTx { + /// Sender of the call (the `from` / caller address). + pub from: Address, + /// Call target (`Call`-kind only for Phase 6). + pub to: Address, + /// ABI-encoded calldata for the call. + pub calldata: Bytes, + /// Per-transaction environment overrides (value / gas / nonce / access list). + pub tx: TxConfig, +} + +impl BundleTx { + /// A bundle transaction with a default [`TxConfig`] (zero value, default + /// gas/nonce, no access list). + pub fn new(from: Address, to: Address, calldata: Bytes) -> Self { + Self { + from, + to, + calldata, + tx: TxConfig::default(), + } + } + + /// A bundle transaction carrying an explicit [`TxConfig`] — e.g. to send + /// native `value` (a direct coinbase transfer), set a `gas_price`, or + /// pre-warm an access list. + pub fn with_config(from: Address, to: Address, calldata: Bytes, tx: TxConfig) -> Self { + Self { + from, + to, + calldata, + tx, + } + } +} + +/// What happens when a bundle transaction reverts (or halts). +/// +/// Defaults to [`Atomic`](RevertPolicy::Atomic). +#[derive(Clone, Debug, Default)] +pub enum RevertPolicy { + /// Any transaction revert/halt reverts the **whole** bundle to the outer + /// checkpoint and sets [`BundleResult::succeeded`] to `false`. Execution + /// stops at the failing transaction. + #[default] + Atomic, + /// The listed transaction indices may revert without aborting the bundle: + /// their state effects are rolled back **individually** (inner checkpoint + /// revert) and later transactions still execute. A revert at an index *not* + /// in the list behaves like [`Atomic`](RevertPolicy::Atomic). + AllowReverts(Vec), +} + +/// Options controlling a bundle simulation. +#[derive(Clone, Debug, Default)] +pub struct BundleOptions { + /// Revert handling for individual transactions. Default + /// [`Atomic`](RevertPolicy::Atomic). + pub revert_policy: RevertPolicy, + /// Whether the bundle's cumulative state is folded into the overlay's dirty + /// layer (`true`) or reverted so the overlay is left unchanged (`false`). + /// Default `false` (evaluate, don't persist). + pub commit: bool, +} + +/// Outcome of a single transaction executed within a bundle. +#[derive(Clone, Debug)] +pub struct TxOutcome { + /// The raw revm [`ExecutionResult`] (`Success` / `Revert` / `Halt`). + pub result: ExecutionResult, + /// Gas consumed by this transaction. + pub gas_used: u64, + /// `true` if this transaction reverted or halted. + pub reverted: bool, + /// Logs emitted by this transaction (empty for a revert/halt). + pub logs: Vec, +} + +/// Result of a bundle simulation. +#[derive(Clone, Debug)] +pub struct BundleResult { + /// One [`TxOutcome`] per executed transaction. Length equals `txs.len()` + /// unless an [`Atomic`](RevertPolicy::Atomic) bundle aborted early, in which + /// case it ends at the failing transaction. + pub per_tx: Vec, + /// Miner payment: the block beneficiary's balance delta across the kept + /// transactions (priority fee + direct coinbase tips; the base fee is already + /// excluded by revm — see the [module docs](self#coinbase-accounting)). + /// Saturating; `0` for an [`Atomic`](RevertPolicy::Atomic) bundle that aborted. + pub coinbase_payment: U256, + /// Total gas used across the executed transactions. + pub gas_used: u64, + /// `false` iff an [`Atomic`](RevertPolicy::Atomic) bundle aborted on a + /// revert/halt (or an `AllowReverts` bundle hit a revert at a non-whitelisted + /// index). + pub succeeded: bool, +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn revert_policy_defaults_to_atomic() { + assert!(matches!(RevertPolicy::default(), RevertPolicy::Atomic)); + } + + #[test] + fn bundle_options_default_is_evaluate_only_atomic() { + let opts = BundleOptions::default(); + assert!(matches!(opts.revert_policy, RevertPolicy::Atomic)); + assert!(!opts.commit, "default must not persist"); + } + + #[test] + fn bundle_tx_new_uses_default_tx_config() { + let from = Address::repeat_byte(0x01); + let to = Address::repeat_byte(0x02); + let tx = BundleTx::new(from, to, Bytes::from(vec![0xaa])); + assert_eq!(tx.from, from); + assert_eq!(tx.to, to); + assert_eq!(tx.calldata, Bytes::from(vec![0xaa])); + assert_eq!(tx.tx.value, U256::ZERO); + assert!(tx.tx.gas_price.is_none()); + assert!(tx.tx.access_list.is_none()); + } + + #[test] + fn bundle_tx_with_config_carries_value_and_gas_price() { + let tx = BundleTx::with_config( + Address::ZERO, + Address::ZERO, + Bytes::new(), + TxConfig { + value: U256::from(42u64), + gas_price: Some(7), + ..Default::default() + }, + ); + assert_eq!(tx.tx.value, U256::from(42u64)); + assert_eq!(tx.tx.gas_price, Some(7)); + } +} diff --git a/src/cache/binary_state.rs b/src/cache/binary_state.rs index ccfb3d0..369d433 100644 --- a/src/cache/binary_state.rs +++ b/src/cache/binary_state.rs @@ -114,13 +114,22 @@ pub fn save_binary_state(blockchain_db: &BlockchainDb, path: &Path) -> Result<() /// /// Returns `false` (rather than erroring) when `path` cannot be read or its /// contents fail the magic/version check or fail to decode as the expected -/// bincode layout; failures are logged at `warn` level. +/// bincode layout. A missing file (the normal cold-start case) is logged at +/// `debug`; an actual read error (e.g. permission denied) and any +/// magic/version/decode failure are logged at `warn`. pub fn load_binary_state(blockchain_db: &BlockchainDb, path: &Path) -> bool { let start = Instant::now(); let data = match std::fs::read(path) { Ok(d) => d, - Err(_) => return false, + Err(e) if e.kind() == std::io::ErrorKind::NotFound => { + debug!("No binary EVM state file found, starting fresh"); + return false; + } + Err(e) => { + warn!(error = %e, "Failed to read binary EVM state, starting fresh"); + return false; + } }; let Some(state) = versioned::decode::( diff --git a/src/cache/mod.rs b/src/cache/mod.rs index e58fbd3..28209b7 100644 --- a/src/cache/mod.rs +++ b/src/cache/mod.rs @@ -82,6 +82,13 @@ pub type RpcCallFn = Arc Result + Send + Sync>; /// snapshot was built from, so a concurrent [`EvmCache::set_block`] cannot make /// the deferred fetch read a *different* block than the snapshot it is compared /// against. +/// +/// **Contract:** an implementation must return **exactly one** result tuple per +/// requested `(address, slot)` (order does not matter). Callers — `verify_slots`, +/// `reconcile_slots`, and the cold-start verify/probe paths — derive their +/// per-slot outcomes from the returned tuples, so a fetcher that drops, dedups, +/// reorders-and-truncates, or duplicates entries breaks the "one outcome per +/// requested slot" guarantee those APIs document. pub type StorageBatchFetchFn = Arc< dyn Fn(Vec<(Address, U256)>, Option) -> Vec<(Address, U256, Result)> + Send @@ -294,6 +301,7 @@ pub struct EvmCacheBuilder

{ cache_config: Option, spec_id: SpecId, shared_memory_capacity: SharedMemoryCapacity, + chain_id: Option, } impl

EvmCacheBuilder

@@ -308,6 +316,7 @@ where cache_config: None, spec_id: SpecId::CANCUN, shared_memory_capacity: SharedMemoryCapacity::default(), + chain_id: None, } } @@ -337,6 +346,20 @@ where self } + /// Set the chain ID reported to simulations via the `CHAINID` opcode. + /// + /// **Recommended.** This is the explicit, authoritative way to set the chain + /// ID. If left unset, [`build`](Self::build) infers it from the provider + /// (`eth_chainId`), falling back to `1` (Ethereum mainnet) only if that query + /// fails. A disk [`cache_config`](Self::cache_config) also carries a + /// `chain_id` (which additionally namespaces the on-disk cache directory); + /// when both are set, the value passed here wins for the `CHAINID` opcode, so + /// keep them consistent. + pub fn chain_id(mut self, chain_id: u64) -> Self { + self.chain_id = Some(chain_id); + self + } + /// Enable disk-backed caching with the given configuration. /// /// Supplying a [`CacheConfig`] turns on persistence of EVM state, bytecodes, @@ -362,15 +385,26 @@ where } /// Build the [`EvmCache`], fetching the pinned block's header for context. + /// + /// If a chain ID was not set via [`chain_id`](Self::chain_id), it is inferred + /// from the provider (`eth_chainId`); see [`chain_id`](Self::chain_id) for the + /// full resolution order. pub async fn build(self) -> EvmCache { - EvmCache::with_cache_capacity( + let explicit_chain_id = self.chain_id; + let mut cache = EvmCache::with_cache_capacity( self.provider, self.block, self.cache_config, self.spec_id, self.shared_memory_capacity, ) - .await + .await; + // An explicit builder value is authoritative for the `CHAINID` opcode and + // overrides both the inferred value and any `cache_config` chain id. + if let Some(chain_id) = explicit_chain_id { + cache.set_chain_id(chain_id); + } + cache } } @@ -985,15 +1019,32 @@ impl EvmCache { }, ); + // Resolve the chain ID reported to simulations (the `CHAINID` opcode). A + // disk `CacheConfig` is authoritative (its `chain_id` also namespaces the + // on-disk cache directory); otherwise infer it from the provider via + // `eth_chainId`, falling back to 1 (Ethereum mainnet) only if that query + // fails. Resolved before `provider` is moved into the backend below. + // Prefer setting it explicitly through `EvmCacheBuilder::chain_id`. + let chain_id = match cache_config.as_ref() { + Some(cfg) => cfg.chain_id, + None => match provider.get_chain_id().await { + Ok(id) => id, + Err(e) => { + debug!( + error = %e, + "Failed to infer chain ID from provider; defaulting to 1 (Ethereum mainnet). Set it explicitly via EvmCacheBuilder::chain_id." + ); + 1 + } + }, + }; + // Spawn the backend handler on a background task let backend = SharedBackend::spawn_backend(provider, blockchain_db.clone(), Some(block_id)).await; let db = CacheDB::new(backend.clone()); - // Extract chain_id from cache config if available, default to Arbitrum - let chain_id = cache_config.as_ref().map(|c| c.chain_id).unwrap_or(42161); - // Resolve the shared-memory pre-allocation. For `Auto` we size from the // amount of layer-2 chain state actually loaded (post-filter), so a large // bincode state file yields a larger buffer; `Fixed` ignores the count. @@ -1073,6 +1124,13 @@ impl EvmCache { /// /// Useful when you want to share a backend between multiple caches /// (e.g. parallel simulation threads). + /// + /// **Shared pinned block.** A `SharedBackend` owns a single pinned fork + /// height. Calling [`set_block`](Self::set_block) / `repin_to_block` on *any* + /// cache built from the same backend re-pins the RPC fork height for **all** + /// of them. Sibling caches sharing one backend should agree on a block and not + /// re-pin independently; build separate backends if they must fork at + /// different heights. pub fn from_backend( backend: SharedBackend, blockchain_db: BlockchainDb, @@ -2236,6 +2294,17 @@ impl EvmCache { self.chain_id } + /// Set the chain ID reported to simulations via the `CHAINID` opcode. + /// + /// Prefer setting this at construction through + /// [`EvmCacheBuilder::chain_id`]. This setter exists for cases where the + /// chain ID must change after construction. It takes effect on the next + /// [`create_snapshot`](Self::create_snapshot) / `build_evm`; existing + /// snapshots and overlays keep the chain ID captured when they were created. + pub fn set_chain_id(&mut self, chain_id: u64) { + self.chain_id = chain_id; + } + /// Take a low-level, same-thread snapshot of the CacheDB overlay for /// in-place restore. /// @@ -2785,6 +2854,21 @@ impl EvmCache { self.basefee = basefee; } + /// Set the block base fee (the `BASEFEE` opcode) for subsequent simulations, + /// propagated into the next [`create_snapshot`](Self::create_snapshot). + /// + /// Offline caches built over a mocked provider have no fetched block header, + /// so the base fee is unset (and the `BASEFEE` opcode reads `0`). Use this to + /// install one explicitly — it determines the priority fee + /// (`gas_price − basefee`) credited to the beneficiary, and thus the + /// `coinbase_payment` a [`simulate_bundle`](Self::simulate_bundle) reports. + /// + /// The cache stores the base fee as a `u64` (matching the block header and the + /// `EvmSnapshot` field), so a `U256` larger than `u64::MAX` is saturated. + pub fn set_basefee(&mut self, basefee: U256) { + self.basefee = Some(basefee.saturating_to::()); + } + /// Override the block beneficiary (the `COINBASE` opcode) for subsequent /// simulations. /// @@ -3026,6 +3110,79 @@ impl EvmCache { self.call_raw_with(from, to, calldata, commit, &TxConfig::default()) } + /// Execute a non-committing typed Solidity call from [`Address::ZERO`]. + /// + /// This is the typed equivalent of encoding a [`SolCall`], passing it to + /// [`call_raw`](Self::call_raw) with `commit = false`, and decoding the + /// successful return data with [`SolCall::abi_decode_returns`]. + /// + /// ```no_run + /// # use alloy_primitives::Address; + /// # use alloy_sol_types::sol; + /// # use evm_fork_cache::cache::EvmCache; + /// # fn example(cache: &mut EvmCache, token: Address, owner: Address) -> anyhow::Result<()> { + /// sol! { + /// function balanceOf(address account) external view returns (uint256); + /// } + /// + /// let balance = cache.call_sol(token, balanceOfCall { account: owner })?; + /// # let _ = balance; + /// # Ok(()) + /// # } + /// ``` + pub fn call_sol(&mut self, to: Address, call: C) -> Result + where + C: SolCall, + { + self.call_sol_from(Address::ZERO, to, call) + } + + /// Execute a non-committing typed Solidity call from an explicit sender. + /// + /// Uses the default [`TxConfig`], so native value, gas limit/price, nonce, + /// and access list are left at the same defaults as [`call_raw`](Self::call_raw). + pub fn call_sol_from(&mut self, from: Address, to: Address, call: C) -> Result + where + C: SolCall, + { + self.call_sol_with_commit(from, to, call, &TxConfig::default(), false) + } + + /// Execute a non-committing typed Solidity call with explicit tx overrides. + /// + /// This is the typed equivalent of [`call_raw_with`](Self::call_raw_with) + /// with `commit = false`. + pub fn call_sol_with( + &mut self, + from: Address, + to: Address, + call: C, + tx: &TxConfig, + ) -> Result + where + C: SolCall, + { + self.call_sol_with_commit(from, to, call, tx, false) + } + + /// Execute a typed Solidity call and commit its state changes. + /// + /// This is the typed equivalent of [`call_raw_with`](Self::call_raw_with) + /// with `commit = true`; the call's state changes are persisted through the + /// same path as the raw committing API before the return data is decoded. + pub fn transact_sol( + &mut self, + from: Address, + to: Address, + call: C, + tx: &TxConfig, + ) -> Result + where + C: SolCall, + { + self.call_sol_with_commit(from, to, call, tx, true) + } + /// Execute a call with explicit transaction-environment overrides /// ([`TxConfig`]): native `value`, gas limit/price, nonce, and an input /// access list. This is the entry point for value-bearing and gas-bounded @@ -3791,6 +3948,38 @@ impl EvmCache { self.simulate_with_transfer_tracking(from, to, calldata, owner, tokens, commit) } + /// Simulate an ordered transaction **bundle** over cumulative block state, + /// with a revert policy and coinbase/miner-payment accounting (Phase 6 + /// Track A+B). + /// + /// This is a convenience wrapper: it snapshots the cache and runs the bundle + /// on a fresh transient [`EvmOverlay`] via + /// [`EvmOverlay::simulate_bundle`](crate::cache::EvmOverlay::simulate_bundle), + /// which carries the full semantics (ordered cumulative state, the + /// [`RevertPolicy`](crate::bundle::RevertPolicy), and coinbase accounting). + /// + /// The cache itself is **never** mutated — even when `opts.commit` is `true`. + /// `commit` controls only whether the bundle's cumulative state is folded + /// into the transient overlay (and is therefore moot here, since that overlay + /// is dropped when this call returns). Snapshot the cache yourself and drive + /// [`EvmOverlay::simulate_bundle`] directly when you need the committed + /// overlay state to outlive the call (e.g. to chain a follow-up read). + /// + /// # Errors + /// + /// Returns [`SimError`] if a transaction environment cannot be built or revm + /// fails to transact. A transaction reverting is reported through the + /// per-transaction outcome and the revert policy, not as an error. + pub fn simulate_bundle( + &mut self, + txs: &[crate::bundle::BundleTx], + opts: &crate::bundle::BundleOptions, + ) -> SimulationResult { + let snapshot = self.create_snapshot(); + let mut overlay = EvmOverlay::new(snapshot, None); + overlay.simulate_bundle(txs, opts) + } + /// Deploy a contract via CREATE transaction and return the deployed address. /// /// The `creation_code` should include the init code with ABI-encoded constructor @@ -4120,6 +4309,57 @@ impl EvmCache { .map_err(|e| anyhow!("Failed to build tx env: {:?}", e)) } + fn call_sol_with_commit( + &mut self, + from: Address, + to: Address, + call: C, + tx: &TxConfig, + commit: bool, + ) -> Result + where + C: SolCall, + { + let calldata = Bytes::from(call.abi_encode()); + let result = self + .call_raw_with(from, to, calldata, commit, tx) + .with_context(|| { + format!( + "failed to execute Solidity call {} from {from:?} to {to:?}", + C::SIGNATURE + ) + })?; + Self::decode_sol_call_result::(from, to, result) + } + + fn decode_sol_call_result( + from: Address, + to: Address, + result: ExecutionResult, + ) -> Result + where + C: SolCall, + { + match result { + ExecutionResult::Success { output, .. } => { + let output = output.into_data(); + C::abi_decode_returns(&output).map_err(|error| { + anyhow!( + "failed to decode Solidity call {} return data from {from:?} to {to:?}: output_len={}, error: {:?}", + C::SIGNATURE, + output.len(), + error + ) + }) + } + other => Err(anyhow!( + "Solidity call {} from {from:?} to {to:?} did not succeed: {:?}", + C::SIGNATURE, + other + )), + } + } + fn erc20_balance_of_in_evm( evm: &mut CacheEvm<'_>, caller: Address, diff --git a/src/cache/overlay.rs b/src/cache/overlay.rs index 372f256..c5d11e1 100644 --- a/src/cache/overlay.rs +++ b/src/cache/overlay.rs @@ -17,6 +17,7 @@ use revm::{ use super::snapshot::EvmSnapshot; use super::{CallSimulationResult, SimStatus, TxConfig, unix_timestamp_secs_saturating}; use crate::access_set::StorageAccessList; +use crate::bundle::{BundleOptions, BundleResult, BundleTx, RevertPolicy, TxOutcome}; use crate::errors::{SimError, SimulationError, SimulationResult}; use crate::inspector::TransferInspector; @@ -494,6 +495,334 @@ impl EvmOverlay { outcome } + /// Run a single call with a caller-supplied [`Inspector`](revm::Inspector), + /// returning the raw [`ExecutionResult`] and handing the inspector back for the + /// caller to read. + /// + /// This is the inspector-generic public seam: where + /// [`Self::simulate_with_transfer_tracking`] hard-wires the + /// [`TransferInspector`], this accepts any + /// [`revm::Inspector`] — a [`CallTracer`](crate::tracing::CallTracer), an + /// [`InspectorStack`](crate::tracing::InspectorStack) composing several, or a + /// caller-defined one. It honors a full [`TxConfig`] (value/gas/nonce/access + /// list) exactly like [`Self::call_raw_with_access_list_with`] and recycles the + /// reusable shared-memory buffer like the other call methods. + /// + /// Unlike `simulate_with_transfer_tracking`, a revert or halt is **not** an + /// error: the raw [`ExecutionResult`] variant + /// ([`Success`](ExecutionResult::Success) / + /// [`Revert`](ExecutionResult::Revert) / [`Halt`](ExecutionResult::Halt)) is + /// returned as `Ok` so the inspector's captured frames (e.g. a reverted call + /// tree) remain observable. Only a tx-env build failure or a transact/database + /// error yields `Err`. + /// + /// On a successful transact the journaled changes are either committed into the + /// overlay's dirty layer (`commit == true`) or reverted (`commit == false`), + /// matching [`Self::simulate_with_transfer_tracking`]. On a revert/halt the + /// checkpoint is always reverted regardless of `commit`, so a failed call never + /// mutates this overlay. On a transact error the checkpoint is reverted too. + /// + /// # Errors + /// + /// Returns an error if the [`TxEnv`] cannot be built from `from`/`to`/`tx`, or + /// if revm fails to transact the call (e.g. a database error while loading + /// state). + /// + /// # Examples + /// + /// ```no_run + /// # use std::sync::Arc; + /// # use alloy_primitives::{Address, Bytes}; + /// # use evm_fork_cache::cache::{EvmOverlay, EvmSnapshot, TxConfig}; + /// # use evm_fork_cache::CallTracer; + /// # fn run(snapshot: Arc, to: Address) -> anyhow::Result<()> { + /// let mut overlay = EvmOverlay::new(snapshot, None); + /// let (result, tracer) = overlay.call_raw_with_inspector( + /// Address::ZERO, + /// to, + /// Bytes::new(), + /// &TxConfig::default(), + /// CallTracer::new(), + /// false, + /// )?; + /// let _ = result; + /// let _trace = tracer.into_trace(); + /// # Ok(()) + /// # } + /// ``` + pub fn call_raw_with_inspector( + &mut self, + from: Address, + to: Address, + calldata: Bytes, + tx: &TxConfig, + inspector: I, + commit: bool, + ) -> SimulationResult<(ExecutionResult, I)> + where + I: for<'a> revm::Inspector< + Context< + BlockEnv, + TxEnv, + CfgEnv, + &'a mut EvmOverlay, + Journal<&'a mut EvmOverlay>, + (), + >, + >, + { + let mut builder = TxEnv::builder() + .caller(from) + .kind(TxKind::Call(to)) + .data(calldata) + .value(tx.value); + if let Some(gas_limit) = tx.gas_limit { + builder = builder.gas_limit(gas_limit); + } + if let Some(gas_price) = tx.gas_price { + builder = builder.gas_price(gas_price); + } + if let Some(nonce) = tx.nonce { + builder = builder.nonce(nonce); + } + if let Some(access_list) = &tx.access_list { + builder = builder.access_list(access_list.clone()); + } + let tx_env = builder + .build() + .map_err(|e| SimError::Other(anyhow!("Failed to build tx env: {:?}", e)))?; + + // Recycle the reusable buffer (Pillar A.2); reclaimed after the EVM drops. + let buffer = Rc::new(RefCell::new(std::mem::take(&mut self.reusable_buffer))); + let local = LocalContext { + shared_memory_buffer: Rc::clone(&buffer), + precompile_error_message: None, + }; + + let outcome = { + let mut evm = self.build_evm_with_inspector_local(inspector, local); + + use revm::context_interface::JournalTr; + let checkpoint = evm.journaled_state.checkpoint(); + + match evm.inspect_one_tx(tx_env) { + Ok(result) => { + if commit && matches!(result, ExecutionResult::Success { .. }) { + evm.commit_inner(); + } else { + evm.journaled_state.checkpoint_revert(checkpoint); + } + // Hand the inspector back to the caller. + Ok((result, evm.inspector)) + } + Err(e) => { + evm.journaled_state.checkpoint_revert(checkpoint); + Err(SimError::Other(anyhow!("Failed to transact: {:?}", e))) + } + } + }; + + self.reclaim_buffer(buffer); + outcome + } + + /// Apply `txs` in order against this overlay over **cumulative** block state, + /// with a revert policy and coinbase/miner-payment accounting (Phase 6 + /// Track A+B). + /// + /// Each transaction observes the committed writes of the ones before it: + /// the bundle runs on a single overlay/EVM with one outer checkpoint plus a + /// per-transaction inner checkpoint, so it does **not** rebuild a fresh + /// overlay per transaction. See the [`bundle`](crate::bundle) module for the + /// public vocabulary ([`BundleTx`], [`BundleOptions`], [`RevertPolicy`], + /// [`TxOutcome`], [`BundleResult`]). + /// + /// # Revert policy + /// + /// - [`RevertPolicy::Atomic`]: the first transaction that reverts/halts + /// rolls the whole bundle back to the outer checkpoint, sets + /// `succeeded = false`, and stops (`per_tx` ends at the failing + /// transaction). `coinbase_payment` is `0` and the overlay is unchanged. + /// - [`RevertPolicy::AllowReverts`]: a revert at a whitelisted index rolls + /// back only that transaction (inner checkpoint) and execution continues; + /// a revert at a non-whitelisted index behaves like `Atomic`. + /// + /// # Coinbase accounting + /// + /// `coinbase_payment` is the block beneficiary's balance delta across the kept + /// transactions. Under EIP-1559 revm credits the beneficiary only the priority + /// fee (`(effective_gas_price − basefee) × gas_used`) and burns the base fee + /// in-EVM, so the delta is the honest miner payment (plus any direct coinbase + /// tips). Saturating. + /// + /// # Commit semantics + /// + /// `opts.commit == true` folds the bundle's cumulative state into this + /// overlay's dirty layer (observable by subsequent overlay calls); + /// `false` reverts the outer checkpoint so the overlay is unchanged. A + /// failed atomic bundle never leaves partial state regardless of `commit`. + /// + /// # Errors + /// + /// Returns [`SimError`] if a transaction environment cannot be built or revm + /// fails to transact (e.g. a database error). A transaction *reverting* is + /// not an error — it is reported through the per-transaction + /// [`TxOutcome`] and the revert policy. + pub fn simulate_bundle( + &mut self, + txs: &[BundleTx], + opts: &BundleOptions, + ) -> SimulationResult { + // Build every TxEnv up front so a build failure surfaces as an error + // before we touch the EVM/journal (and the borrow of `self` is clean). + let tx_envs: Vec = txs + .iter() + .map(|bt| { + let mut builder = TxEnv::builder() + .caller(bt.from) + .kind(TxKind::Call(bt.to)) + .data(bt.calldata.clone()) + .value(bt.tx.value); + if let Some(gas_limit) = bt.tx.gas_limit { + builder = builder.gas_limit(gas_limit); + } + if let Some(gas_price) = bt.tx.gas_price { + builder = builder.gas_price(gas_price); + } + if let Some(nonce) = bt.tx.nonce { + builder = builder.nonce(nonce); + } + if let Some(access_list) = &bt.tx.access_list { + builder = builder.access_list(access_list.clone()); + } + builder + .build() + .map_err(|e| SimError::Other(anyhow!("Failed to build tx env: {:?}", e))) + }) + .collect::>()?; + + // Resolve the beneficiary and read its pre-bundle balance before the + // mutable borrow of `self` by the EVM (the post-bundle delta is the miner + // payment; revm already burns the base fee per EIP-1559). + let beneficiary = self + .snapshot + .coinbase + .unwrap_or_else(|| revm::context::BlockEnv::default().beneficiary); + let pre_beneficiary_balance = self + .basic(beneficiary) + .map_err(|e| SimError::Other(anyhow!("Failed to load beneficiary: {:?}", e)))? + .map(|info| info.balance) + .unwrap_or(U256::ZERO); + + // Recycle the reusable buffer (Pillar A.2); reclaimed after the EVM drops. + let buffer = Rc::new(RefCell::new(std::mem::take(&mut self.reusable_buffer))); + let local = LocalContext { + shared_memory_buffer: Rc::clone(&buffer), + precompile_error_message: None, + }; + + let outcome = { + use revm::context_interface::JournalTr; + let mut evm = self.build_evm_with_local(local); + + // Outer checkpoint: the whole-bundle savepoint. + let outer = evm.journaled_state.checkpoint(); + + let mut per_tx: Vec = Vec::with_capacity(tx_envs.len()); + let mut total_gas: u64 = 0; + let mut aborted = false; + + 'bundle: for (idx, tx_env) in tx_envs.into_iter().enumerate() { + // Inner checkpoint: this transaction's savepoint. + let inner = evm.journaled_state.checkpoint(); + let result = match evm.transact_one(tx_env) { + Ok(result) => result, + Err(e) => { + // Host/transact error: undo this tx and the whole bundle, + // reclaim the buffer, and surface as SimError. + evm.journaled_state.checkpoint_revert(inner); + evm.journaled_state.checkpoint_revert(outer); + drop(evm); + self.reclaim_buffer(buffer); + return Err(SimError::Other(anyhow!("Failed to transact: {:?}", e))); + } + }; + + let gas_used = result.gas_used(); + let reverted = !result.is_success(); + let logs = result.logs().to_vec(); + total_gas = total_gas.saturating_add(gas_used); + + per_tx.push(TxOutcome { + result, + gas_used, + reverted, + logs, + }); + + if reverted { + let allowed = match &opts.revert_policy { + RevertPolicy::Atomic => false, + RevertPolicy::AllowReverts(idxs) => idxs.contains(&idx), + }; + if allowed { + // Roll back only this transaction; later txs still run. + evm.journaled_state.checkpoint_revert(inner); + continue 'bundle; + } else { + // Atomic abort: roll the whole bundle back and stop. + evm.journaled_state.checkpoint_revert(outer); + aborted = true; + break 'bundle; + } + } + // Successful tx: its effects stay journaled for the next tx. + } + + if aborted { + // State is reverted to the pre-bundle outer checkpoint regardless + // of `commit`; no payment. + BundleResult { + per_tx, + coinbase_payment: U256::ZERO, + gas_used: total_gas, + succeeded: false, + } + } else { + // Read the beneficiary's post-bundle balance from the journaled + // state (present iff it was touched) BEFORE commit/revert, since + // `commit_inner` finalizes (drains) the journal and an outer + // revert would undo the credit. + let post_beneficiary_balance = evm + .journaled_state + .state + .get(&beneficiary) + .map(|acct| acct.info.balance) + .unwrap_or(pre_beneficiary_balance); + // revm already excludes the base fee from the beneficiary credit + // (EIP-1559), so the delta is the honest miner payment. + let coinbase_payment = + post_beneficiary_balance.saturating_sub(pre_beneficiary_balance); + + if opts.commit { + evm.commit_inner(); + } else { + evm.journaled_state.checkpoint_revert(outer); + } + + BundleResult { + per_tx, + coinbase_payment, + gas_used: total_gas, + succeeded: true, + } + } + }; + + self.reclaim_buffer(buffer); + Ok(outcome) + } + /// Execute a non-committing call and return the result plus the touched /// [`StorageAccessList`]. /// @@ -745,6 +1074,11 @@ impl Database for EvmOverlay { if let Some(ref ext_db) = self.ext_db { return ext_db.block_hash_ref(number); } + // Snapshots never populate `block_hashes` (the live cache does not track + // block hashes), so without an `ext_db` the `BLOCKHASH` opcode resolves to + // ZERO. Overlays built internally (e.g. the freshness validator) pass + // `ext_db = None`; a contract that reads `BLOCKHASH` through such an + // overlay sees ZERO. Documented in docs/KNOWN_ISSUES.md. Ok(B256::ZERO) } } diff --git a/src/cold_start/mod.rs b/src/cold_start/mod.rs index 6747fb5..d94f3e9 100644 --- a/src/cold_start/mod.rs +++ b/src/cold_start/mod.rs @@ -27,6 +27,44 @@ //! //! Like the rest of the crate's RPC seams, cold-start fetching drives async work //! synchronously and must run on a **multi-thread** tokio runtime. +//! +//! # Example +//! +//! A planner that authoritatively warms a fixed working set of slots in one round. +//! (See `examples/cold_start.rs` for the runnable discover-then-verify version.) +//! +//! ``` +//! use alloy_primitives::{Address, U256}; +//! use evm_fork_cache::cache::EvmCache; +//! use evm_fork_cache::events::StateView; +//! use evm_fork_cache::{ +//! ColdStartConfig, ColdStartError, ColdStartPlan, ColdStartPlanner, ColdStartResults, +//! ColdStartStep, +//! }; +//! +//! struct WarmSlots { +//! slots: Vec<(Address, U256)>, +//! } +//! +//! impl ColdStartPlanner for WarmSlots { +//! fn initial_plan(&mut self, _state: &dyn StateView) -> ColdStartPlan { +//! // Round 1: re-fetch these slots authoritatively and inject the fresh values. +//! ColdStartPlan { verify: self.slots.clone(), ..Default::default() } +//! } +//! fn on_results(&mut self, _r: &ColdStartResults, _s: &dyn StateView) -> ColdStartStep { +//! ColdStartStep::Done +//! } +//! } +//! +//! // `run_cold_start` needs a live cache on a multi-thread runtime; this helper +//! // shows the call shape (it is never invoked, so the doctest needs no runtime). +//! # fn warm(cache: &mut EvmCache, slots: Vec<(Address, U256)>) -> Result<(), ColdStartError> { +//! let mut planner = WarmSlots { slots }; +//! let report = cache.run_cold_start(&mut planner, ColdStartConfig::default())?; +//! assert_eq!(report.rounds, 1); +//! # Ok(()) +//! # } +//! ``` mod config; mod driver; diff --git a/src/cold_start/plan.rs b/src/cold_start/plan.rs index 3120a38..8aef18b 100644 --- a/src/cold_start/plan.rs +++ b/src/cold_start/plan.rs @@ -18,6 +18,25 @@ use alloy_primitives::{Address, Bytes, U256}; /// - `accounts` are pre-seeded into the cache before discovery runs. /// - `discover` view-calls capture the `(address, slot)` pairs and accounts they /// touch. +/// +/// ``` +/// use alloy_primitives::{Address, Bytes, U256}; +/// use evm_fork_cache::{ColdStartCall, ColdStartPlan}; +/// +/// // Verify one known slot and discover more via a read-only view-call. +/// let plan = ColdStartPlan { +/// verify: vec![(Address::repeat_byte(0x11), U256::from(0))], +/// discover: vec![ColdStartCall { +/// from: Address::ZERO, +/// to: Address::repeat_byte(0x11), +/// calldata: Bytes::new(), +/// restrict_to: None, +/// }], +/// ..Default::default() +/// }; +/// assert_eq!(plan.verify.len(), 1); +/// assert_eq!(plan.discover.len(), 1); +/// ``` #[derive(Clone, Debug, Default)] pub struct ColdStartPlan { /// Slots to authoritatively re-fetch, classify, and inject when changed. diff --git a/src/create3.rs b/src/create3.rs index b7d7082..e3f2779 100644 --- a/src/create3.rs +++ b/src/create3.rs @@ -9,13 +9,20 @@ use alloy_primitives::{Address, B256, address, b256, keccak256}; -/// Address of the widely deployed universal CREATE3 factory (the CreateX / -/// CREATE3 factory implementation). +/// Address of the widely deployed `CREATE3Factory` (the ZeframLou / Solmate-style +/// implementation, deterministically deployed cross-chain — e.g. by LiFi). /// -/// This is the canonical cross-chain address at which the CreateX-style -/// CREATE3 factory has been deterministically deployed on many EVM networks. -/// The derivation in this module assumes the factory at this address uses the -/// CREATE3 proxy init code whose hash is `CREATE3_PROXY_INITCODE_HASH`. +/// This is the canonical cross-chain address at which this factory has been +/// deployed on many EVM networks. Its derivation salts as +/// `keccak256(abi.encodePacked(deployer, salt))` over the standard Solady/Solmate +/// CREATE3 proxy init code whose hash is `CREATE3_PROXY_INITCODE_HASH`, which is +/// what [`derive_universal_create3_address`] reproduces. +/// +/// **This is not pcaversaccio's CreateX** (`0xba5Ed0...28ba5Ed`). CreateX applies +/// a guarded-salt transformation (permissioned-deploy / cross-chain-redeploy +/// protection, chain-ID mixing) that differs from the plain +/// `keccak256(deployer, salt)` used here, so this module does **not** predict +/// CreateX deployment addresses. /// /// Callers must verify the factory is actually deployed at this address on /// their target chain before relying on a derived address: if the factory is @@ -23,7 +30,7 @@ use alloy_primitives::{Address, B256, address, b256, keccak256}; /// address will not correspond to any real deployment. pub const UNIVERSAL_CREATE3_FACTORY: Address = address!("93FEC2C00BfE902F733B57c5a6CeeD7CD1384AE1"); -// CREATE3 proxy initcode used by the universal factory implementation. +// Standard Solady/Solmate CREATE3 proxy initcode used by this factory. // keccak256(0x67363d3d37363d34f03d5260086018f3) const CREATE3_PROXY_INITCODE_HASH: B256 = b256!("21c35dbe1b344a2488cf3321d6ce542f8e9f305544ff09e4993a62319a497c1f"); diff --git a/src/errors.rs b/src/errors.rs index 46b9582..649ada7 100644 --- a/src/errors.rs +++ b/src/errors.rs @@ -654,13 +654,6 @@ impl From for SimError { } } -/// Deprecated alias for [`SimError`]. -#[deprecated( - since = "0.2.0", - note = "renamed to `SimError`; `Halt` is now a distinct variant" -)] -pub type SimulationErrorKind = SimError; - #[cfg(test)] mod tests { use super::*; diff --git a/src/events/mod.rs b/src/events/mod.rs index f4dbda8..0bfe210 100644 --- a/src/events/mod.rs +++ b/src/events/mod.rs @@ -297,6 +297,13 @@ impl EventPipeline { touched_addrs.insert(skip.address); self.note_touched_slot(&mut digest, skip.address, skip.slot); } + // Skipped (cold) `Account` patches carry an address but no slot, so + // track the address only — mirroring `skipped_balances`. A third-party + // decoder emitting `StateUpdate::Account` against a cold account would + // otherwise be invisible to reorg purge and freshness tracking. + for skip in &diff.skipped_accounts { + touched_addrs.insert(skip.address); + } digest.applied.merge(diff); } diff --git a/src/freshness.rs b/src/freshness.rs index 3241906..430b6ab 100644 --- a/src/freshness.rs +++ b/src/freshness.rs @@ -21,6 +21,18 @@ //! `clock.now()` as `now: u64` through the tracker, the policy, and //! [`FreshnessRegistry::is_volatile`]. //! +//! # Reconciliation scope +//! +//! The optimistic loop verifies only **volatile storage slots** in each sim's +//! read set. Account-level state — native balance, nonce, and bytecode — is +//! **not** re-fetched or diffed, so [`Validation::Confirmed`] means *"no volatile +//! storage slot the sims read had changed"*, not *"no account state changed"*. A +//! sim whose result depends on a `BALANCE`/`SELFBALANCE` (or nonce/code) that +//! moved on-chain without a co-changing storage slot in its read set can still be +//! reported `Confirmed`. If account-level state matters to a sim, mark the +//! account [`Validity::Pinned`] and keep it fresh via event-driven writes, or +//! reconcile it out of band. See [`Validation`] for the per-verdict note. +//! //! # Example //! //! Classification + policy selection, no network required: @@ -32,22 +44,25 @@ //! }; //! use evm_fork_cache::cache::SlotObservationTracker; //! -//! let pool = Address::repeat_byte(0x01); -//! let slot0 = U256::from(0); -//! let immutable = U256::from(6); // e.g. token0 +//! let contract = Address::repeat_byte(0x01); +//! let volatile_slot = U256::from(0); +//! let immutable_slot = U256::from(6); // e.g. a constructor-set config value //! //! let mut registry = FreshnessRegistry::new(); // default: Volatile -//! registry.pin_slot(pool, immutable); // never re-verified +//! registry.pin_slot(contract, immutable_slot); // never re-verified //! //! // `now` is in clock units (block number for the default BlockClock). //! let now = 100; -//! assert!(registry.is_volatile(pool, slot0, now)); -//! assert!(!registry.is_volatile(pool, immutable, now)); +//! assert!(registry.is_volatile(contract, volatile_slot, now)); +//! assert!(!registry.is_volatile(contract, immutable_slot, now)); //! //! // Policies pick which volatile candidates to verify this cycle. //! let obs = SlotObservationTracker::new(); -//! let candidates = [(pool, slot0)]; -//! assert_eq!(AlwaysVerify.select(&candidates, &obs, now), vec![(pool, slot0)]); +//! let candidates = [(contract, volatile_slot)]; +//! assert_eq!( +//! AlwaysVerify.select(&candidates, &obs, now), +//! vec![(contract, volatile_slot)] +//! ); //! assert!(NeverVerify.select(&candidates, &obs, now).is_empty()); //! ``` @@ -505,12 +520,27 @@ pub enum SlotFetch { } /// The deferred verdict on a [`SpeculativeSim`]'s optimistic results. +/// +/// # Reconciliation scope +/// +/// The verdict reflects only **volatile storage slots** in each sim's read set. +/// Account-level state — native balance, nonce, and bytecode — is **not** +/// re-verified, so a sim whose result depends on a `BALANCE`/`SELFBALANCE` (or +/// nonce/code) that changed on-chain *without* a co-changing storage slot in its +/// read set can still be reported [`Confirmed`](Validation::Confirmed). Classify +/// such accounts as [`Validity::Pinned`] and keep them fresh via event-driven +/// writes if their account-level state matters to your sims. See the module-level +/// docs for the full freshness contract. pub enum Validation { - /// Nothing the sims read had changed; the optimistic results are correct. + /// No **volatile storage slot** in the sims' read sets had changed, so the + /// optimistic results are trusted. Note this does *not* cover account-level + /// balance/nonce/code changes — see the [type-level scope](Validation). Confirmed, - /// At least one read slot changed. `results` is the optimistic set with the - /// affected sims re-run against the fresh values; `changed` lists the slots - /// that differed (also queued for flow-back into the cache). + /// At least one read storage slot changed. `results` is the optimistic set + /// with the affected sims re-run against the fresh values; `changed` lists the + /// slots that differed (also queued for flow-back into the cache). Only + /// storage slots are reconciled — account-level state is not (see the + /// [type-level scope](Validation)). Corrected { /// Optimistic results with the affected sims replaced by re-runs. results: Vec, @@ -605,12 +635,16 @@ impl SimRequest { /// cancel flag and aborts the background task. Cancellation is **cooperative and /// best-effort, not instantaneous**: `run_validator` is synchronous, so an abort /// cannot preempt it once it is running. Instead the validator checks the flag at -/// a few checkpoints — before fetching, and before recording observations or -/// queuing a correction — so a cancel observed at a checkpoint prevents the -/// remaining side effects. A validator already executing a synchronous step (e.g. -/// mid-fetch) completes that step before reaching the next checkpoint. The intent -/// is that a dropped speculation does not flow its corrections back into the -/// cache; it does not guarantee that an in-flight fetch is interrupted. +/// several checkpoints — before each fetch, and (on the first pass) after a fetch +/// returns but before it records observations or queues corrections — so a cancel +/// observed at a checkpoint skips the side effects downstream of it. A validator +/// already executing a synchronous step (e.g. mid-fetch) completes that step +/// before reaching the next checkpoint, and corrections accumulated up to a fetch +/// that completed in the *final* loop iteration may still be queued for flow-back +/// (the post-loop queue is not guarded by an immediately-preceding checkpoint). +/// The intent is that a dropped speculation stops doing further work and, in the +/// common case, does not flow corrections back into the cache; it does not +/// guarantee that an in-flight or just-completed fetch's correction is withheld. pub struct SpeculativeSim { optimistic: Vec, /// `Option` so `validate`/`into_optimistic` can take the handle and skip the @@ -624,6 +658,11 @@ pub struct SpeculativeSim { impl SpeculativeSim { /// The optimistic results, readable before validation completes. + /// + /// These (and the re-run results in [`Validation::Corrected`]) carry an + /// **empty `token_deltas`** map: the optimistic loop does not run transfer + /// tracking, so the signed per-token balance deltas populated by the + /// committing `simulate_call_with_balance_deltas` path are not available here. pub fn optimistic(&self) -> &[CallSimulationResult] { &self.optimistic } @@ -812,11 +851,13 @@ impl FreshnessController { /// [`SpeculativeSim`] immediately. /// /// # Panics - /// Spawns a background task whose (synchronous) fetcher uses - /// `tokio::task::block_in_place` internally, so it must run on a - /// **multi-thread** tokio runtime (`#[tokio::main(flavor = "multi_thread")]` - /// or `Builder::new_multi_thread()`). On a current-thread runtime - /// `block_in_place` panics, mirroring the [`EvmCache`] constructor note. + /// Must be called from within a tokio runtime: it calls `tokio::spawn` to + /// launch the background validator, which panics (`there is no reactor + /// running`) if no runtime is active. The spawned (synchronous) fetcher + /// additionally uses `tokio::task::block_in_place` internally, so the runtime + /// must be **multi-thread** (`#[tokio::main(flavor = "multi_thread")]` or + /// `Builder::new_multi_thread()`); on a current-thread runtime `block_in_place` + /// panics, mirroring the [`EvmCache`] constructor note. /// /// # Errors /// Returns an error if any optimistic simulation fails to execute against the diff --git a/src/lib.rs b/src/lib.rs index 55000e6..69cf2d0 100644 --- a/src/lib.rs +++ b/src/lib.rs @@ -57,9 +57,23 @@ //! - `reactive` — default-enabled provider-neutral handler runtime for logs, //! blocks, and pending transaction signals. Pure handlers emit `StateUpdate`s, //! invalidations, resync requests, speculative signals, and hooks; the runtime -//! validates and applies canonical cache mutations. +//! validates and applies canonical cache mutations, journals canonical block +//! effects for depth-bounded reorg recovery, and includes a live +//! `AlloySubscriber` (WebSocket) transport. +//! - `cold_start` — default-enabled (reactive-gated) declarative warming of a +//! working set of accounts/slots into the cache in one batched pass +//! (`EvmCache::run_cold_start` + `ColdStartPlanner`), returning a structured +//! `ColdStartRunReport`. +//! - [`bundle`] — multi-transaction bundle execution over cumulative block state +//! ([`EvmOverlay::simulate_bundle`](cache::EvmOverlay::simulate_bundle)): ordered +//! txs, an `Atomic`/`AllowReverts` revert policy, and coinbase/miner-payment +//! accounting. //! - [`inspector`] — an [`Inspector`](revm::Inspector) that captures ERC20 //! `Transfer` events to reconstruct balance deltas from a simulation. +//! - [`tracing`] — a call-frame [`Inspector`](revm::Inspector) +//! ([`CallTracer`] building a [`CallTrace`] tree) plus [`InspectorStack`] for +//! composing several inspectors over one pass, driven through +//! [`EvmOverlay::call_raw_with_inspector`](cache::EvmOverlay::call_raw_with_inspector). //! - [`multicall`] — batched read-only calls through Multicall3. //! - [`deploy`] / [`create3`] — contract deployment and CREATE3 address math. //! - [`prefetch_registry`] — two-stage storage-slot pre-warming. @@ -105,6 +119,7 @@ //! state over RPC. See the crate README for the full list. pub mod access_list; pub mod access_set; +pub mod bundle; pub mod cache; #[cfg(feature = "reactive")] pub mod cold_start; @@ -119,8 +134,17 @@ pub mod prefetch_registry; #[cfg(feature = "reactive")] pub mod reactive; pub mod state_update; +pub mod tracing; pub use access_set::StorageAccessList; +// Phase 6 Track A+B: bundle simulation + coinbase accounting public vocabulary. +pub use bundle::{BundleOptions, BundleResult, BundleTx, RevertPolicy, TxOutcome}; +// Primary entry points, hoisted to the crate root for discoverability. The +// fully-qualified module paths (`cache::EvmCache`, `reactive::ReactiveRuntime`, +// …) remain valid, so this is purely additive. +pub use cache::{ + CallSimulationResult, EvmCache, EvmCacheBuilder, EvmOverlay, EvmSnapshot, TxConfig, +}; #[cfg(feature = "reactive")] pub use cold_start::{ ColdStartCall, ColdStartCallResult, ColdStartConfig, ColdStartError, ColdStartPin, @@ -137,7 +161,10 @@ pub use freshness::{ FreshnessPolicy, FreshnessRegistry, NeverVerify, ObservationDriven, SimRequest, SlotChange, SlotFetch, SlotOutcome, SpeculativeSim, Validation, Validity, WallClock, }; +#[cfg(feature = "reactive")] +pub use reactive::{ReactiveConfig, ReactiveHandler, ReactiveRuntime}; pub use state_update::{ AccountChange, AccountPatch, PurgeRecord, PurgeScope, SkippedAccountPatch, SkippedBalanceDelta, SkippedDelta, SkippedMask, SlotDelta, StateDiff, StateUpdate, }; +pub use tracing::{CallKind, CallStatus, CallTrace, CallTracer, InspectorStack}; diff --git a/src/reactive/mod.rs b/src/reactive/mod.rs index 54cffaa..70d662c 100644 --- a/src/reactive/mod.rs +++ b/src/reactive/mod.rs @@ -812,9 +812,23 @@ impl SpeculativeId { /// Configuration for [`ReactiveRuntime`]. #[derive(Clone, Debug, PartialEq, Eq)] pub struct ReactiveConfig { - /// Hook backpressure policy for future async dispatchers. + /// Hook backpressure policy. **Reserved — currently has no effect.** Hook + /// dispatch is synchronous today (every report is delivered to every hook in + /// order), so this field is a no-op placeholder for a future async dispatcher. + /// Setting it to anything other than the default does not change behavior. pub hook_backpressure: HookBackpressure, - /// Reorg journal depth reserved for future rollback support. + /// Reorg journal depth: the number of recent canonical blocks whose effects + /// are journaled for rollback. This is **load-bearing** for reorg recovery: + /// only blocks still resident in the journal can be recovered. A reorg deeper + /// than `journal_depth` recovers the blocks still in the journal and leaves + /// the aged-out blocks' effects in place — they are **neither rolled back nor + /// purged**, so the freshness/validation loop is the only backstop for that + /// span. `0` disables journaling entirely: no reorg is rolled back or purged. + /// + /// Set `journal_depth` to exceed the deepest reorg you intend to recover + /// precisely. When a reorg references a block that is no longer in the journal, + /// the runtime emits a `tracing::warn!` so the under-recovery is observable + /// rather than silent. pub journal_depth: usize, } @@ -908,7 +922,9 @@ pub struct AppliedReport { pub _network: PhantomData, } -/// Resync reporting scaffold. +/// Report of the storage resync requests executed during an ingest cycle: the +/// requests considered, the authoritative updates built from successful fetches +/// (and their applied diff), and any targets that could not be resynced. #[derive(Clone, Debug, Default, PartialEq, Eq)] pub struct ResyncReport { /// Requests considered by the resync execution pass. @@ -960,7 +976,16 @@ pub struct BlockReport { pub _network: PhantomData, } -/// Reorg reporting scaffold. +/// Report of a detected reorg and the recovery it performed: the dropped +/// block(s) and inputs, the exact rollback updates applied for reversible dropped +/// effects, the conservative purge updates for irreversible ones, the canceled +/// hash-pinned resyncs, and why recovery ran. +/// +/// Recovery only covers blocks still resident in the journal. If a reorg runs +/// deeper than [`ReactiveConfig::journal_depth`], the aged-out blocks do not +/// appear here and their effects are neither rolled back nor purged (the runtime +/// logs a `tracing::warn!` in that case); the freshness/validation loop is the +/// backstop for that span. #[derive(Clone, Debug)] pub struct ReorgReport { /// First dropped block, when known. @@ -996,7 +1021,8 @@ pub enum ReorgReason { ParentMismatch, } -/// Error report scaffold. +/// Report of a non-fatal error surfaced during an ingest cycle, with the +/// associated input (when known) and a human-readable message. #[derive(Clone, Debug)] pub struct ReactiveErrorReport { /// Input associated with the error, when known. @@ -1522,9 +1548,11 @@ impl ReactiveRuntime { { self.drain_journal_after(parent_index) } else { + self.warn_under_recovery(block.number); self.drain_journal_from_number(block.number) } } else { + self.warn_under_recovery(block.number); self.drain_journal_from_number(block.number) }; @@ -1544,6 +1572,7 @@ impl ReactiveRuntime { { self.drain_journal_from(index) } else { + self.warn_under_recovery(dropped_block.number); self.drain_journal_from_number(dropped_block.number) }; @@ -1570,6 +1599,24 @@ impl ReactiveRuntime { self.recover_dropped_journals(cache, dropped, reason) } + /// Warn that a reorg references a block no longer resident in the journal, so + /// recovery is limited to the blocks still journaled — effects from aged-out + /// blocks are neither rolled back nor purged (the freshness/validation loop is + /// the backstop). Makes the under-recovery observable instead of silent. + fn warn_under_recovery(&self, reorg_number: u64) { + let oldest_journaled = self.journal.front().map(|entry| entry.block.number); + tracing::warn!( + reorg_block = reorg_number, + oldest_journaled = ?oldest_journaled, + journal_depth = self.config.journal_depth, + "reactive reorg recovery is incomplete: the reorged block is no longer \ + in the journal, so effects from blocks aged out of the journal are \ + neither rolled back nor purged (the freshness/validation loop is the \ + backstop). Increase ReactiveConfig::journal_depth to recover deeper \ + reorgs precisely." + ); + } + fn record_journal_input(&mut self, block: &BlockRef, input_ref: InputRef) { let entry = self.journal_entry_mut(block); if !entry.inputs.contains(&input_ref) { @@ -2621,9 +2668,25 @@ pub struct AlloySubscriber { _network: PhantomData, } +/// Best-effort installation of rustls' `ring` crypto provider as the process +/// default, so an `wss://` TLS handshake under `reactive-ws` does not panic with +/// "no process-level CryptoProvider available". Runs at most once and ignores the +/// error if a default provider is already installed (the host app may have set +/// its own). +#[cfg(feature = "reactive-ws")] +fn ensure_ring_crypto_provider() { + use std::sync::Once; + static INSTALL: Once = Once::new(); + INSTALL.call_once(|| { + let _ = rustls::crypto::ring::default_provider().install_default(); + }); +} + impl AlloySubscriber { /// Create a new Alloy subscriber. pub fn new(provider: P, mode: SubscriberMode, config: SubscriberConfig) -> Self { + #[cfg(feature = "reactive-ws")] + ensure_ring_crypto_provider(); Self { provider, mode, @@ -3444,6 +3507,39 @@ mod subscriber_helper_tests { InputSource::Subscription ); } + + #[test] + fn backfilled_logs_surface_with_backfill_source() { + // A backfilled log with no prior subscription duplicate is delivered as + // an `InputSource::Backfill` record (the positive side of the dedup test, + // pinning the README's "marking recovered records as InputSource::Backfill" + // claim — the only place that source is produced). + let provider = ProviderBuilder::new().connect_mocked_client(Asserter::new()); + let mut subscriber = AlloySubscriber::<_, Ethereum>::new( + provider, + SubscriberMode::PubSub, + SubscriberConfig::default(), + ); + subscriber.interests = vec![ReactiveInterest::Logs(LogInterest { + provider_filter: Filter::new() + .address(Address::repeat_byte(0x42)) + .event_signature(B256::repeat_byte(0x01)), + local_matcher: None, + route_key: None, + })]; + + subscriber.enqueue_event(SubscriberEvent::BackfilledLogs { + source_id: 0, + logs: vec![rpc_log(false)], + }); + + assert_eq!(subscriber.pending_records.len(), 1); + assert_eq!( + subscriber.pending_records[0].context.source, + InputSource::Backfill + ); + assert_eq!(subscriber.last_seen_log_blocks.get(&0), Some(&7)); + } } fn resolve_subscriber_transport( @@ -3532,7 +3628,7 @@ fn validate_supported_interests( if !config.hydrate_pending_transactions && interest.matches_hash_only() => {} ReactiveInterest::PendingTransactions(_) => { return Err(SubscriberError::Unsupported( - "AlloySubscriber polling mode currently supports pending transaction hash interests only", + "AlloySubscriber currently supports pending transaction hash interests only (full pending-tx hydration is unimplemented)", )); } ReactiveInterest::Blocks(interest) => match (transport, interest.mode) { diff --git a/src/state_update.rs b/src/state_update.rs index f67a474..8e688e9 100644 --- a/src/state_update.rs +++ b/src/state_update.rs @@ -207,9 +207,11 @@ pub enum StateUpdate { /// /// Read-modify-write: the current `AccountInfo::balance` is read, the /// [`SlotDelta`] applied, and the result written back through both layers - /// (nonce and code preserved). **Cold-aware** — "cold" here means the account - /// is absent from *both* layers (its balance is unknown). A `BalanceDelta` on a - /// cold account is not applied; it is surfaced in + /// (nonce and code preserved). **Cold-aware** — "cold" here means the + /// account's info is unknown to the EVM: absent from *both* layers, **or** + /// present in the overlay as revm `NotExisting` (which the internal account + /// read also treats as cold, mirroring `DbAccount::info()`). A `BalanceDelta` + /// on a cold account is not applied; it is surfaced in /// [`StateDiff::skipped_balances`] instead (so no default account is /// materialized to mask the real on-chain one — see the module docs). BalanceDelta { @@ -246,8 +248,9 @@ pub enum StateUpdate { /// Patch an already-known account's balance/nonce/code (partial — see /// [`AccountPatch`]). /// - /// Cold-aware: if the account is absent from **both** layers, the patch is not - /// applied and is surfaced in [`StateDiff::skipped_accounts`] as a + /// Cold-aware: if the account's info is unknown to the EVM (absent from + /// **both** layers, or present in the overlay as revm `NotExisting`), the patch + /// is not applied and is surfaced in [`StateDiff::skipped_accounts`] as a /// [`SkippedAccountPatch`]. Use [`StateUpdate::AccountUpsert`] when /// materializing a cold/default account is intentional. Account { @@ -474,7 +477,8 @@ pub enum PurgeScope { /// /// [`is_empty`](Self::is_empty) / [`len`](Self::len) are **changes-only**, so a /// cold-skipped update ([`SlotDelta`](StateUpdate::SlotDelta) / -/// [`BalanceDelta`](StateUpdate::BalanceDelta) / [`Account`](StateUpdate::Account)) +/// [`BalanceDelta`](StateUpdate::BalanceDelta) / +/// [`SlotMasked`](StateUpdate::SlotMasked) / [`Account`](StateUpdate::Account)) /// is invisible to them. After applying cold-aware updates, check /// [`has_skipped`](Self::has_skipped) (or inspect the `skipped_*` fields) — a cold /// target was dropped, not applied. @@ -511,28 +515,32 @@ pub struct StateDiff { impl StateDiff { /// Whether the diff recorded no change at all. /// - /// Changes-only: counts `slots` + `accounts` + `purged`. A skipped relative - /// update ([`skipped`](Self::skipped) / [`skipped_balances`](Self::skipped_balances)) - /// is informational metadata, not a change, so it does not affect this. + /// Changes-only: counts `slots` + `accounts` + `purged`. Any skipped update + /// (any of the `skipped_*` fields) is informational metadata, not a change, so + /// it does not affect this. pub fn is_empty(&self) -> bool { self.slots.is_empty() && self.accounts.is_empty() && self.purged.is_empty() } /// Total number of changed entries (slots + accounts + purges). /// - /// Changes-only: skipped relative updates are not counted (a skip is not a - /// change). See [`skipped_len`](Self::skipped_len) for the skip count. + /// Changes-only: skipped updates (any `skipped_*` field) are not counted (a + /// skip is not a change). See [`skipped_len`](Self::skipped_len) for the skip + /// count. pub fn len(&self) -> usize { self.slots.len() + self.accounts.len() + self.purged.len() } - /// Whether any relative update was skipped (slot **or** balance). + /// Whether any cold-aware update was skipped (relative slot, balance, masked + /// slot, **or** cold account patch). /// - /// `true` iff [`skipped`](Self::skipped) or - /// [`skipped_balances`](Self::skipped_balances) is non-empty. A cold-skipped + /// `true` iff any of [`skipped`](Self::skipped), + /// [`skipped_balances`](Self::skipped_balances), + /// [`skipped_masks`](Self::skipped_masks), or + /// [`skipped_accounts`](Self::skipped_accounts) is non-empty. A cold-skipped /// update produces no change, so it is invisible to - /// [`is_empty`](Self::is_empty) — callers applying relative updates should - /// check this to avoid silently dropping a balance update. + /// [`is_empty`](Self::is_empty) — callers applying cold-aware updates should + /// check this to avoid silently dropping an update. pub fn has_skipped(&self) -> bool { !self.skipped.is_empty() || !self.skipped_balances.is_empty() @@ -618,8 +626,9 @@ pub struct SkippedDelta { } /// A relative balance update ([`StateUpdate::BalanceDelta`]) that could not be -/// applied because the account is absent from **both** cache layers (its native -/// balance is unknown). +/// applied because the account's info is unknown to the EVM — absent from +/// **both** cache layers, or present in the overlay as revm `NotExisting` (so its +/// native balance is unknown). /// /// A delta against a cold account is skipped rather than applied (applying it /// against an assumed-zero balance would corrupt an unknown value, and @@ -662,7 +671,8 @@ pub struct SkippedMask { } /// An account patch ([`StateUpdate::Account`]) that could not be applied because -/// the account is absent from **both** cache layers. +/// the account's info is unknown to the EVM — absent from **both** cache layers, +/// or present in the overlay as revm `NotExisting`. /// /// A partial patch against a cold account is skipped rather than applied against /// [`AccountInfo::default`](revm::state::AccountInfo::default), because default diff --git a/src/tracing.rs b/src/tracing.rs new file mode 100644 index 0000000..8da83e7 --- /dev/null +++ b/src/tracing.rs @@ -0,0 +1,524 @@ +//! Call-frame tracing and a composing inspector seam. +//! +//! This module provides [`CallTracer`], a [`revm::Inspector`] that reconstructs +//! the **call-frame tree** of a simulation — the top-level call plus every nested +//! `CALL`/`STATICCALL`/`DELEGATECALL`/`CALLCODE` and `CREATE`/`CREATE2` frame — +//! without opcode/step-level tracing. Each [`CallTrace`] records the caller, the +//! callee (or created address), the call value, calldata, gas used, return data, +//! a [`CallStatus`], the call depth, and its child frames. +//! +//! It also provides [`InspectorStack`], a tiny composing inspector that fans out +//! every [`Inspector`] hook to two inner inspectors so, e.g., a `CallTracer` and a +//! [`TransferInspector`](crate::inspector::TransferInspector) can run in a single +//! pass and each produce its own independent result. +//! +//! Attach either via the inspector-generic +//! [`EvmOverlay::call_raw_with_inspector`](crate::cache::EvmOverlay::call_raw_with_inspector). +//! +//! # Calldata resolution caveat +//! +//! revm represents a frame's calldata as a [`CallInput`], which is either owned +//! [`CallInput::Bytes`] or a [`CallInput::SharedBuffer`] range into the EVM's +//! shared-memory scratch buffer. Resolving a `SharedBuffer` range back to bytes +//! requires the concrete EVM context (`ContextTr`), which this inspector — written +//! against the same fully-generic `CTX` as the existing +//! [`TransferInspector`](crate::inspector::TransferInspector) — deliberately does +//! not bind. The **top-level** call's calldata is always `CallInput::Bytes` (revm +//! builds it directly from the transaction), so a root frame's +//! [`input`](CallTrace::input) is always the real calldata. Nested calls whose +//! calldata is a `SharedBuffer` are recorded with an **empty** `input`; their +//! callee address, value, gas, status, and subcalls are captured faithfully. This +//! is a documented limitation, not a correctness bug: the tracer never fabricates +//! calldata it cannot resolve. + +use alloy_primitives::{Address, Bytes, Log, U256}; +use revm::Inspector; +use revm::interpreter::{ + CallInput, CallInputs, CallOutcome, CallScheme, CreateInputs, CreateOutcome, CreateScheme, + Interpreter, InterpreterTypes, +}; + +/// The kind of EVM frame a [`CallTrace`] represents. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +pub enum CallKind { + /// A `CALL`. + Call, + /// A `STATICCALL`. + StaticCall, + /// A `DELEGATECALL`. + DelegateCall, + /// A `CALLCODE`. + CallCode, + /// A `CREATE`. + Create, + /// A `CREATE2`. + Create2, +} + +impl CallKind { + /// Map a revm [`CallScheme`] (the opcode behind a message call) to a [`CallKind`]. + fn from_call_scheme(scheme: CallScheme) -> Self { + match scheme { + CallScheme::Call => CallKind::Call, + CallScheme::CallCode => CallKind::CallCode, + CallScheme::DelegateCall => CallKind::DelegateCall, + CallScheme::StaticCall => CallKind::StaticCall, + } + } + + /// Map a revm [`CreateScheme`] to a [`CallKind`]. + /// + /// `CreateScheme::Custom` (an internally-addressed create) is reported as + /// [`CallKind::Create`]. + fn from_create_scheme(scheme: CreateScheme) -> Self { + match scheme { + CreateScheme::Create => CallKind::Create, + CreateScheme::Create2 { .. } => CallKind::Create2, + CreateScheme::Custom { .. } => CallKind::Create, + } + } +} + +/// The terminal status of an EVM frame. +#[derive(Clone, Debug, PartialEq, Eq)] +pub enum CallStatus { + /// The frame returned normally (`STOP`/`RETURN`/`SELFDESTRUCT`). + Success, + /// The frame reverted (`REVERT`, or a revert-class condition). + Revert, + /// The frame halted on an exceptional error (e.g. out of gas, invalid opcode). + Halt, +} + +/// A single node in the call-frame tree captured by a [`CallTracer`]. +/// +/// One `CallTrace` is produced per EVM message call or contract creation. Child +/// frames (the calls/creates made *by* this frame) are nested under +/// [`subcalls`](Self::subcalls) in execution order. +#[derive(Clone, Debug)] +pub struct CallTrace { + /// Whether this frame is a call (and which kind) or a contract creation. + pub kind: CallKind, + /// The caller (the account initiating this frame). + pub from: Address, + /// The callee, or — for a `CREATE`/`CREATE2` — the created contract address. + pub to: Address, + /// The call value (wei). Always zero for `STATICCALL`. + pub value: U256, + /// The calldata (for a create, the init code). Empty when the calldata was a + /// shared-memory range this tracer could not resolve — see the + /// [module docs](crate::tracing#calldata-resolution-caveat). + pub input: Bytes, + /// Gas spent executing this frame. + pub gas_used: u64, + /// The frame's return data (for a successful create, the deployed code). + pub output: Bytes, + /// The terminal status of the frame. + pub status: CallStatus, + /// The frame's call depth (the top-level call is depth `0`). + pub depth: usize, + /// Child frames (calls/creates made by this frame), in execution order. + pub subcalls: Vec, +} + +/// A frame whose `*_end` hook has not yet fired (its output/gas/status are unset). +/// +/// While a frame is open its already-completed children accumulate in `subcalls`; +/// when the matching `call_end`/`create_end` fires it is finalized into a +/// [`CallTrace`] and attached to its parent (or installed as the root). +#[derive(Clone, Debug)] +struct PendingFrame { + kind: CallKind, + from: Address, + to: Address, + value: U256, + input: Bytes, + depth: usize, + subcalls: Vec, +} + +/// A [`revm::Inspector`] that builds a [`CallTrace`] tree from the call/create +/// frame hooks. +/// +/// Drive it through +/// [`EvmOverlay::call_raw_with_inspector`](crate::cache::EvmOverlay::call_raw_with_inspector), +/// then read the captured tree with [`root`](Self::root) or [`into_trace`](Self::into_trace). +/// +/// Only the call-frame hooks (`call`/`call_end`/`create`/`create_end`) are used; +/// no opcode/step, `SLOAD`/`SSTORE`, or per-opcode gas tracing is performed. +/// +/// ``` +/// use evm_fork_cache::CallTracer; +/// +/// let tracer = CallTracer::new(); +/// assert!(tracer.root().is_none()); // nothing executed yet +/// ``` +#[derive(Clone, Debug, Default)] +pub struct CallTracer { + /// Open frames, innermost last. A `call`/`create` pushes; the matching + /// `*_end` pops. + stack: Vec, + /// The finalized top-level frame, set when the outermost frame's `*_end` fires. + root: Option, +} + +impl CallTracer { + /// Create an empty tracer with no captured frames. + pub fn new() -> Self { + Self::default() + } + + /// The root (top-level) frame after a transact, or `None` if nothing executed. + pub fn root(&self) -> Option<&CallTrace> { + self.root.as_ref() + } + + /// Consume the tracer and return the root frame, or `None` if nothing executed. + pub fn into_trace(self) -> Option { + self.root + } + + /// Push a new open frame onto the stack at the current depth. + fn push_frame( + &mut self, + kind: CallKind, + from: Address, + to: Address, + value: U256, + input: Bytes, + ) { + let depth = self.stack.len(); + self.stack.push(PendingFrame { + kind, + from, + to, + value, + input, + depth, + subcalls: Vec::new(), + }); + } + + /// Finalize the innermost open frame into a [`CallTrace`] and attach it to its + /// parent (or install it as the root if it was the top-level frame). + /// + /// `to_override` lets the create hooks supply the created address, which is + /// not known until `create_end`. + fn pop_frame( + &mut self, + gas_used: u64, + output: Bytes, + status: CallStatus, + to_override: Option

, + ) { + let Some(pending) = self.stack.pop() else { + // Defensive: an unbalanced `*_end` with no matching open frame. revm + // pairs the hooks, so this should not happen; ignore rather than panic. + return; + }; + + let trace = CallTrace { + kind: pending.kind, + from: pending.from, + to: to_override.unwrap_or(pending.to), + value: pending.value, + input: pending.input, + gas_used, + output, + status, + depth: pending.depth, + subcalls: pending.subcalls, + }; + + if let Some(parent) = self.stack.last_mut() { + parent.subcalls.push(trace); + } else { + self.root = Some(trace); + } + } +} + +/// Resolve a frame's [`CallInput`] to owned bytes. +/// +/// Owned [`CallInput::Bytes`] are cloned directly. A [`CallInput::SharedBuffer`] +/// range cannot be resolved without the concrete EVM context, so it yields empty +/// bytes — see the [module docs](crate::tracing#calldata-resolution-caveat). +fn resolve_call_input(input: &CallInput) -> Bytes { + match input { + CallInput::Bytes(bytes) => bytes.clone(), + CallInput::SharedBuffer(_) => Bytes::new(), + } +} + +/// Maps a frame's terminal [`InstructionResult`](revm::interpreter::InstructionResult) +/// to a [`CallStatus`]. +fn status_from_result(result: revm::interpreter::InstructionResult) -> CallStatus { + if result.is_ok() { + CallStatus::Success + } else if result.is_revert() { + CallStatus::Revert + } else { + // Everything else is an exceptional halt (out of gas, invalid opcode, …). + CallStatus::Halt + } +} + +impl Inspector for CallTracer +where + INTR: InterpreterTypes, +{ + fn call(&mut self, _context: &mut CTX, inputs: &mut CallInputs) -> Option { + self.push_frame( + CallKind::from_call_scheme(inputs.scheme), + inputs.caller, + inputs.target_address, + inputs.call_value(), + resolve_call_input(&inputs.input), + ); + None + } + + fn call_end(&mut self, _context: &mut CTX, _inputs: &CallInputs, outcome: &mut CallOutcome) { + let status = status_from_result(*outcome.instruction_result()); + let gas_used = outcome.gas().spent(); + let output = outcome.output().clone(); + self.pop_frame(gas_used, output, status, None); + } + + fn create(&mut self, _context: &mut CTX, inputs: &mut CreateInputs) -> Option { + self.push_frame( + CallKind::from_create_scheme(inputs.scheme()), + inputs.caller(), + // The created address is not known until `create_end`; fill a + // placeholder now and override it on finalize. + Address::ZERO, + inputs.value(), + inputs.init_code().clone(), + ); + None + } + + fn create_end( + &mut self, + _context: &mut CTX, + _inputs: &CreateInputs, + outcome: &mut CreateOutcome, + ) { + let status = status_from_result(*outcome.instruction_result()); + let gas_used = outcome.gas().spent(); + let output = outcome.output().clone(); + self.pop_frame(gas_used, output, status, outcome.address); + } +} + +/// Runs two [`Inspector`]s over the same execution. +/// +/// Every [`Inspector`] hook is fanned out to both inner inspectors (`.0` first, +/// then `.1`) so each captures its own result independently in a single pass. The +/// canonical use is pairing a [`CallTracer`] with a +/// [`TransferInspector`](crate::inspector::TransferInspector) to obtain both a +/// call-frame trace and the ERC-20 transfers from one simulation: +/// +/// ```no_run +/// # use std::sync::Arc; +/// # use alloy_primitives::{Address, Bytes}; +/// # use evm_fork_cache::cache::{EvmOverlay, EvmSnapshot, TxConfig}; +/// # use evm_fork_cache::inspector::TransferInspector; +/// # use evm_fork_cache::{CallTracer, InspectorStack}; +/// # fn run(snapshot: Arc, from: Address, to: Address) -> anyhow::Result<()> { +/// let mut overlay = EvmOverlay::new(snapshot, None); +/// let (_result, stack) = overlay.call_raw_with_inspector( +/// from, +/// to, +/// Bytes::new(), +/// &TxConfig::default(), +/// InspectorStack(CallTracer::new(), TransferInspector::new()), +/// false, +/// )?; +/// let InspectorStack(tracer, transfer) = stack; +/// let _trace = tracer.into_trace(); +/// let _transfers = transfer.transfers; +/// # Ok(()) +/// # } +/// ``` +/// +/// For the `call`/`create` hooks — which return `Option<…Outcome>` to optionally +/// *override* the result — the first inspector that returns `Some` wins and the +/// second's hook is not called, mirroring revm's own composition of inspector +/// tuples. The observe-only inspectors here ([`CallTracer`], +/// [`TransferInspector`](crate::inspector::TransferInspector)) always return +/// `None`, so both always observe. +#[derive(Clone, Debug, Default)] +pub struct InspectorStack(pub A, pub B); + +impl Inspector for InspectorStack +where + INTR: InterpreterTypes, + A: Inspector, + B: Inspector, +{ + fn initialize_interp(&mut self, interp: &mut Interpreter, context: &mut CTX) { + self.0.initialize_interp(interp, context); + self.1.initialize_interp(interp, context); + } + + fn step(&mut self, interp: &mut Interpreter, context: &mut CTX) { + self.0.step(interp, context); + self.1.step(interp, context); + } + + fn step_end(&mut self, interp: &mut Interpreter, context: &mut CTX) { + self.0.step_end(interp, context); + self.1.step_end(interp, context); + } + + fn log(&mut self, context: &mut CTX, log: Log) { + self.0.log(context, log.clone()); + self.1.log(context, log); + } + + fn log_full(&mut self, interp: &mut Interpreter, context: &mut CTX, log: Log) { + self.0.log_full(interp, context, log.clone()); + self.1.log_full(interp, context, log); + } + + fn call(&mut self, context: &mut CTX, inputs: &mut CallInputs) -> Option { + self.0 + .call(context, inputs) + .or_else(|| self.1.call(context, inputs)) + } + + fn call_end(&mut self, context: &mut CTX, inputs: &CallInputs, outcome: &mut CallOutcome) { + self.0.call_end(context, inputs, outcome); + self.1.call_end(context, inputs, outcome); + } + + fn create(&mut self, context: &mut CTX, inputs: &mut CreateInputs) -> Option { + self.0 + .create(context, inputs) + .or_else(|| self.1.create(context, inputs)) + } + + fn create_end( + &mut self, + context: &mut CTX, + inputs: &CreateInputs, + outcome: &mut CreateOutcome, + ) { + self.0.create_end(context, inputs, outcome); + self.1.create_end(context, inputs, outcome); + } + + fn selfdestruct(&mut self, contract: Address, target: Address, value: U256) { + self.0.selfdestruct(contract, target, value); + self.1.selfdestruct(contract, target, value); + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn new_tracer_has_no_root() { + let tracer = CallTracer::new(); + assert!(tracer.root().is_none()); + assert!(tracer.into_trace().is_none()); + } + + #[test] + fn call_kind_maps_call_schemes() { + assert_eq!(CallKind::from_call_scheme(CallScheme::Call), CallKind::Call); + assert_eq!( + CallKind::from_call_scheme(CallScheme::StaticCall), + CallKind::StaticCall + ); + assert_eq!( + CallKind::from_call_scheme(CallScheme::DelegateCall), + CallKind::DelegateCall + ); + assert_eq!( + CallKind::from_call_scheme(CallScheme::CallCode), + CallKind::CallCode + ); + } + + #[test] + fn call_kind_maps_create_schemes() { + assert_eq!( + CallKind::from_create_scheme(CreateScheme::Create), + CallKind::Create + ); + assert_eq!( + CallKind::from_create_scheme(CreateScheme::Create2 { salt: U256::ZERO }), + CallKind::Create2 + ); + } + + #[test] + fn resolve_input_reads_owned_bytes_and_empties_shared_buffer() { + let owned = CallInput::Bytes(Bytes::from(vec![1, 2, 3])); + assert_eq!(resolve_call_input(&owned), Bytes::from(vec![1, 2, 3])); + + let shared = CallInput::SharedBuffer(0..8); + assert!(resolve_call_input(&shared).is_empty()); + } + + #[test] + fn status_mapping_classifies_results() { + use revm::interpreter::InstructionResult; + assert_eq!( + status_from_result(InstructionResult::Stop), + CallStatus::Success + ); + assert_eq!( + status_from_result(InstructionResult::Return), + CallStatus::Success + ); + assert_eq!( + status_from_result(InstructionResult::Revert), + CallStatus::Revert + ); + assert_eq!( + status_from_result(InstructionResult::OutOfGas), + CallStatus::Halt + ); + } + + /// A finalized child frame attaches to its open parent, and the parent + /// becomes the root when its own `*_end` fires. + #[test] + fn frames_nest_into_a_tree() { + let mut tracer = CallTracer::new(); + let root_addr = Address::repeat_byte(0x11); + let child_addr = Address::repeat_byte(0x22); + + tracer.push_frame( + CallKind::Call, + Address::ZERO, + root_addr, + U256::ZERO, + Bytes::from(vec![0xaa]), + ); + tracer.push_frame( + CallKind::StaticCall, + root_addr, + child_addr, + U256::ZERO, + Bytes::new(), + ); + // Inner frame ends first. + tracer.pop_frame(10, Bytes::new(), CallStatus::Success, None); + // Outer frame ends, becoming the root. + tracer.pop_frame(100, Bytes::from(vec![0xbb]), CallStatus::Success, None); + + let root = tracer.into_trace().expect("root frame"); + assert_eq!(root.to, root_addr); + assert_eq!(root.depth, 0); + assert_eq!(root.input, Bytes::from(vec![0xaa])); + assert_eq!(root.subcalls.len(), 1); + assert_eq!(root.subcalls[0].to, child_addr); + assert_eq!(root.subcalls[0].depth, 1); + assert_eq!(root.subcalls[0].kind, CallKind::StaticCall); + } +} diff --git a/tests/builder_chain_id.rs b/tests/builder_chain_id.rs new file mode 100644 index 0000000..cea0780 --- /dev/null +++ b/tests/builder_chain_id.rs @@ -0,0 +1,99 @@ +//! Offline tests for chain-ID resolution on [`EvmCacheBuilder`] / [`EvmCache`]. +//! +//! Resolution order (highest priority first): +//! 1. an explicit [`EvmCacheBuilder::chain_id`] value; +//! 2. a disk [`CacheConfig`]'s `chain_id` (also the on-disk namespace); +//! 3. the value inferred from the provider via `eth_chainId`; +//! 4. `1` (Ethereum mainnet) as a last-resort fallback when inference fails. +//! +//! All tests run fully offline over a mocked provider whose `eth_chainId` query +//! errors (empty `Asserter`), so the inference branch deterministically takes the +//! mainnet fallback — letting us assert the explicit / config / fallback rungs +//! without a network. + +use std::sync::Arc; +use std::time::{SystemTime, UNIX_EPOCH}; + +use alloy_provider::RootProvider; +use alloy_provider::network::AnyNetwork; +use alloy_rpc_client::RpcClient; +use alloy_transport::mock::Asserter; +use anyhow::Result; +use evm_fork_cache::cache::{CacheConfig, EvmCache, EvmCacheBuilder}; + +fn mock_provider() -> Arc> { + Arc::new(RootProvider::::new(RpcClient::mocked( + Asserter::new(), + ))) +} + +fn unique_cache_dir(tag: &str) -> std::path::PathBuf { + let nanos = SystemTime::now() + .duration_since(UNIX_EPOCH) + .unwrap() + .as_nanos(); + std::env::temp_dir().join(format!("evm_fork_cache_chainid_{tag}_{nanos}")) +} + +#[tokio::test(flavor = "multi_thread")] +async fn explicit_builder_chain_id_wins() -> Result<()> { + // 8453 = Base. The explicit builder value is authoritative. + let cache = EvmCacheBuilder::new(mock_provider()) + .chain_id(8453) + .build() + .await; + assert_eq!(cache.chain_id(), 8453); + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn unset_chain_id_falls_back_to_mainnet_when_inference_fails() -> Result<()> { + // No explicit value, no cache_config, and the mock provider's eth_chainId + // errors — so resolution lands on the mainnet (1) fallback, not Arbitrum. + let cache = EvmCacheBuilder::new(mock_provider()).build().await; + assert_eq!(cache.chain_id(), 1); + + // The bare `new` constructor takes the same inference path. + let via_new = EvmCache::new(mock_provider()).await; + assert_eq!(via_new.chain_id(), 1); + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn cache_config_chain_id_is_used_when_no_explicit_value() -> Result<()> { + // 10 = OP Mainnet. With a disk cache and no explicit builder value, the + // CacheConfig chain id is authoritative. + let dir = unique_cache_dir("config"); + let cfg = CacheConfig::new(&dir, 10, Default::default(), Default::default()); + let cache = EvmCacheBuilder::new(mock_provider()) + .cache_config(cfg) + .build() + .await; + assert_eq!(cache.chain_id(), 10); + let _ = std::fs::remove_dir_all(&dir); + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn explicit_builder_chain_id_overrides_cache_config() -> Result<()> { + let dir = unique_cache_dir("override"); + let cfg = CacheConfig::new(&dir, 10, Default::default(), Default::default()); + let cache = EvmCacheBuilder::new(mock_provider()) + .cache_config(cfg) + .chain_id(8453) + .build() + .await; + // Explicit builder value wins for the CHAINID opcode even when a config is set. + assert_eq!(cache.chain_id(), 8453); + let _ = std::fs::remove_dir_all(&dir); + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn set_chain_id_updates_after_construction() -> Result<()> { + let mut cache = EvmCacheBuilder::new(mock_provider()).build().await; + assert_eq!(cache.chain_id(), 1); + cache.set_chain_id(137); // Polygon + assert_eq!(cache.chain_id(), 137); + Ok(()) +} diff --git a/tests/bundle_simulation.rs b/tests/bundle_simulation.rs new file mode 100644 index 0000000..438dda8 --- /dev/null +++ b/tests/bundle_simulation.rs @@ -0,0 +1,302 @@ +//! Manager-authored red-green acceptance tests for Phase 6 Track A+B: +//! ordered multi-transaction bundle simulation over cumulative state, revert +//! policy, and coinbase/payment accounting. +//! +//! These describe the public contract before the implementation exists. The +//! implementation agent must make them pass WITHOUT weakening, skipping, or +//! rewriting them; if a test encodes a wrong assumption about EVM/mock behavior +//! (as opposed to the feature contract), surface it to the manager with a +//! justification rather than silently changing it. +//! +//! Fully offline (mocked provider, injected state). +#![cfg(feature = "reactive")] + +mod common; + +use alloy_primitives::{Address, Bytes, U256}; +use alloy_sol_types::SolCall; +use anyhow::Result; +use revm::context::result::ExecutionResult; +use revm::state::AccountInfo; + +use common::{ + MOCK_ERC20_BALANCE_SLOT, MockERC20, install_default_account, install_mock_erc20, setup_cache, +}; +use evm_fork_cache::{BundleOptions, BundleTx, EvmOverlay, RevertPolicy, TxConfig}; + +/// transfer(to, amount) calldata. +fn transfer_calldata(to: Address, amount: u64) -> Bytes { + Bytes::from( + MockERC20::transferCall { + to, + amount: U256::from(amount), + } + .abi_encode(), + ) +} + +/// Read balanceOf(owner) on `token` through an overlay (so we observe the +/// overlay's committed cumulative state, not the cache). +fn overlay_balance(overlay: &mut EvmOverlay, token: Address, owner: Address) -> Result { + let calldata = Bytes::from(MockERC20::balanceOfCall { account: owner }.abi_encode()); + match overlay.call_raw(owner, token, calldata)? { + ExecutionResult::Success { output, .. } => Ok( + MockERC20::balanceOfCall::abi_decode_returns(&output.into_data())?, + ), + other => anyhow::bail!("balanceOf failed: {other:?}"), + } +} + +/// A token with `alice` funded; returns (token, alice, bob, carol). +async fn token_with_funded_alice() +-> Result<(evm_fork_cache::EvmCache, Address, Address, Address, Address)> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x11); + let alice = Address::repeat_byte(0x22); + let bob = Address::repeat_byte(0x33); + let carol = Address::repeat_byte(0x44); + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, alice); + install_default_account(&mut cache, bob); + install_default_account(&mut cache, carol); + install_mock_erc20(&mut cache, token); + cache.insert_mapping_storage_slot( + token, + U256::from(MOCK_ERC20_BALANCE_SLOT), + alice, + U256::from(1_000u64), + )?; + Ok((cache, token, alice, bob, carol)) +} + +/// AB1 — ordered cumulative state: tx 2 (bob -> carol) only nets correctly if it +/// observed tx 1's write (alice -> bob). Final balances pin cumulative semantics. +#[tokio::test(flavor = "multi_thread")] +async fn bundle_applies_txs_over_cumulative_state() -> Result<()> { + let (mut cache, token, alice, bob, carol) = token_with_funded_alice().await?; + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + + let txs = vec![ + BundleTx::new(alice, token, transfer_calldata(bob, 100)), + BundleTx::new(bob, token, transfer_calldata(carol, 30)), + ]; + let result = overlay.simulate_bundle( + &txs, + &BundleOptions { + commit: true, + ..Default::default() + }, + )?; + + assert!(result.succeeded, "atomic bundle should succeed"); + assert_eq!(result.per_tx.len(), 2); + assert!(result.per_tx.iter().all(|o| !o.reverted)); + + // Cumulative: alice 1000-100=900, bob 0+100-30=70, carol 0+30=30. + assert_eq!( + overlay_balance(&mut overlay, token, alice)?, + U256::from(900u64) + ); + assert_eq!( + overlay_balance(&mut overlay, token, bob)?, + U256::from(70u64) + ); + assert_eq!( + overlay_balance(&mut overlay, token, carol)?, + U256::from(30u64) + ); + Ok(()) +} + +/// AB2 — atomic revert: the 2nd tx (bad selector -> revert) aborts the whole +/// bundle; nothing persists and `succeeded` is false. +#[tokio::test(flavor = "multi_thread")] +async fn atomic_bundle_reverts_whole_on_failure() -> Result<()> { + let (mut cache, token, alice, bob, _carol) = token_with_funded_alice().await?; + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + + let txs = vec![ + BundleTx::new(alice, token, transfer_calldata(bob, 100)), + // Unknown selector -> the contract reverts. + BundleTx::new(alice, token, Bytes::from(vec![0xde, 0xad, 0xbe, 0xef])), + ]; + let result = overlay.simulate_bundle( + &txs, + &BundleOptions { + revert_policy: RevertPolicy::Atomic, + commit: true, + }, + )?; + + assert!( + !result.succeeded, + "atomic bundle must fail when a tx reverts" + ); + assert!(result.per_tx.last().map(|o| o.reverted).unwrap_or(false)); + // The whole bundle rolled back: alice keeps her full balance. + assert_eq!( + overlay_balance(&mut overlay, token, alice)?, + U256::from(1_000u64) + ); + assert_eq!(overlay_balance(&mut overlay, token, bob)?, U256::ZERO); + Ok(()) +} + +/// AB3 — allow-reverts: the same bundle with the failing tx whitelisted; tx 0 +/// persists, tx 1 is rolled back individually, and the bundle succeeds. +#[tokio::test(flavor = "multi_thread")] +async fn allow_reverts_keeps_prior_effects() -> Result<()> { + let (mut cache, token, alice, bob, _carol) = token_with_funded_alice().await?; + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + + let txs = vec![ + BundleTx::new(alice, token, transfer_calldata(bob, 100)), + BundleTx::new(alice, token, Bytes::from(vec![0xde, 0xad, 0xbe, 0xef])), + ]; + let result = overlay.simulate_bundle( + &txs, + &BundleOptions { + revert_policy: RevertPolicy::AllowReverts(vec![1]), + commit: true, + }, + )?; + + assert!( + result.succeeded, + "bundle should succeed when the revert is allowed" + ); + assert!(!result.per_tx[0].reverted); + assert!(result.per_tx[1].reverted); + // tx 0 persisted, tx 1 rolled back. + assert_eq!( + overlay_balance(&mut overlay, token, alice)?, + U256::from(900u64) + ); + assert_eq!( + overlay_balance(&mut overlay, token, bob)?, + U256::from(100u64) + ); + Ok(()) +} + +/// AB4 — direct coinbase payment: a native-value transfer to the beneficiary +/// (Address::ZERO by default) with gas_price = 0 is captured as `coinbase_payment`. +#[tokio::test(flavor = "multi_thread")] +async fn direct_coinbase_payment_is_captured() -> Result<()> { + let mut cache = setup_cache().await?; + let searcher = Address::repeat_byte(0x22); + // Beneficiary defaults to Address::ZERO; install it so it is not lazily fetched. + install_default_account(&mut cache, Address::ZERO); + cache.db_mut().insert_account_info( + searcher, + AccountInfo { + balance: U256::from(10_000u64), + ..Default::default() + }, + ); + + let pay = U256::from(777u64); + let tx = BundleTx::with_config( + searcher, + Address::ZERO, // the default beneficiary + Bytes::new(), + TxConfig { + value: pay, + gas_price: Some(0), + ..Default::default() + }, + ); + + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + let result = overlay.simulate_bundle( + std::slice::from_ref(&tx), + &BundleOptions { + commit: true, + ..Default::default() + }, + )?; + assert!(result.succeeded); + assert_eq!(result.coinbase_payment, pay); + Ok(()) +} + +/// AB5 — base-fee-aware payment: with a base fee set and a tx priced above it, +/// `coinbase_payment` is the priority fee only (`gas_used × (gas_price − basefee)`). +/// revm burns the base-fee portion of the beneficiary credit (EIP-1559), so the +/// base fee is never paid to the miner — the delta already reflects that. +#[tokio::test(flavor = "multi_thread")] +async fn coinbase_payment_is_priority_fee_only() -> Result<()> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x11); + let caller = Address::repeat_byte(0x22); + install_default_account(&mut cache, Address::ZERO); + install_mock_erc20(&mut cache, token); + // Fund the caller's native balance so gas accounting is well-defined. + cache.db_mut().insert_account_info( + caller, + AccountInfo { + balance: U256::from(10u64).pow(U256::from(20u64)), + ..Default::default() + }, + ); + + let basefee: u128 = 1_000_000_000; // 1 gwei, burned + let priority: u128 = 2_000_000_000; // 2 gwei, the miner's cut + cache.set_basefee(U256::from(basefee)); + + let tx = BundleTx::with_config( + caller, + token, + Bytes::from(MockERC20::balanceOfCall { account: caller }.abi_encode()), + TxConfig { + gas_price: Some(basefee + priority), + ..Default::default() + }, + ); + + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + let result = overlay.simulate_bundle(std::slice::from_ref(&tx), &BundleOptions::default())?; + + assert!(result.gas_used > 0); + // Payment is the priority fee only; the base fee is burned, not paid. + assert_eq!( + result.coinbase_payment, + U256::from(result.gas_used) * U256::from(priority), + "coinbase payment must be gas_used * priority_fee (base fee excluded)" + ); + Ok(()) +} + +/// AB6 — commit semantics: commit=false leaves the overlay unchanged; commit=true +/// persists cumulative state to the next overlay call. +#[tokio::test(flavor = "multi_thread")] +async fn commit_flag_controls_overlay_persistence() -> Result<()> { + let (mut cache, token, alice, bob, _carol) = token_with_funded_alice().await?; + let snapshot = cache.create_snapshot(); + + // commit = false: overlay unchanged afterward. + let mut overlay = EvmOverlay::new(snapshot.clone(), None); + overlay.simulate_bundle( + &[BundleTx::new(alice, token, transfer_calldata(bob, 100))], + &BundleOptions { + commit: false, + ..Default::default() + }, + )?; + assert_eq!(overlay_balance(&mut overlay, token, bob)?, U256::ZERO); + + // commit = true: change visible to the next call on the same overlay. + let mut overlay = EvmOverlay::new(snapshot, None); + overlay.simulate_bundle( + &[BundleTx::new(alice, token, transfer_calldata(bob, 100))], + &BundleOptions { + commit: true, + ..Default::default() + }, + )?; + assert_eq!( + overlay_balance(&mut overlay, token, bob)?, + U256::from(100u64) + ); + Ok(()) +} diff --git a/tests/bundle_simulation_extra.rs b/tests/bundle_simulation_extra.rs new file mode 100644 index 0000000..2a8a968 --- /dev/null +++ b/tests/bundle_simulation_extra.rs @@ -0,0 +1,164 @@ +//! Agent-authored supplementary tests for Phase 6 Track A+B bundle simulation. +//! +//! These cover edge cases the manager acceptance suite +//! (`tests/bundle_simulation.rs`) does not pin: an empty bundle, the +//! `AllowReverts` atomic-fallback when a non-whitelisted index reverts, +//! `commit = false` leaving an `AllowReverts` bundle isolated, and the +//! `EvmCache::simulate_bundle` convenience never mutating the cache. +//! +//! Fully offline (mocked provider, injected state). +#![cfg(feature = "reactive")] + +mod common; + +use alloy_primitives::{Address, Bytes, U256}; +use alloy_sol_types::SolCall; +use anyhow::Result; +use revm::context::result::ExecutionResult; + +use common::{ + MOCK_ERC20_BALANCE_SLOT, MockERC20, install_default_account, install_mock_erc20, setup_cache, +}; +use evm_fork_cache::{BundleOptions, BundleTx, EvmCache, EvmOverlay, RevertPolicy}; + +fn transfer_calldata(to: Address, amount: u64) -> Bytes { + Bytes::from( + MockERC20::transferCall { + to, + amount: U256::from(amount), + } + .abi_encode(), + ) +} + +fn overlay_balance(overlay: &mut EvmOverlay, token: Address, owner: Address) -> Result { + let calldata = Bytes::from(MockERC20::balanceOfCall { account: owner }.abi_encode()); + match overlay.call_raw(owner, token, calldata)? { + ExecutionResult::Success { output, .. } => Ok( + MockERC20::balanceOfCall::abi_decode_returns(&output.into_data())?, + ), + other => anyhow::bail!("balanceOf failed: {other:?}"), + } +} + +async fn token_with_funded_alice() -> Result<(EvmCache, Address, Address, Address)> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x11); + let alice = Address::repeat_byte(0x22); + let bob = Address::repeat_byte(0x33); + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, alice); + install_default_account(&mut cache, bob); + install_mock_erc20(&mut cache, token); + cache.insert_mapping_storage_slot( + token, + U256::from(MOCK_ERC20_BALANCE_SLOT), + alice, + U256::from(1_000u64), + )?; + Ok((cache, token, alice, bob)) +} + +/// An empty bundle is a no-op success: zero outcomes, zero gas, zero payment. +#[tokio::test(flavor = "multi_thread")] +async fn empty_bundle_succeeds_as_noop() -> Result<()> { + let mut cache = setup_cache().await?; + install_default_account(&mut cache, Address::ZERO); + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + + let result = overlay.simulate_bundle(&[], &BundleOptions::default())?; + assert!(result.succeeded); + assert!(result.per_tx.is_empty()); + assert_eq!(result.gas_used, 0); + assert_eq!(result.coinbase_payment, U256::ZERO); + Ok(()) +} + +/// `AllowReverts` that does NOT whitelist the failing index behaves like +/// `Atomic`: the whole bundle rolls back and `succeeded == false`. +#[tokio::test(flavor = "multi_thread")] +async fn allow_reverts_non_whitelisted_index_aborts_atomically() -> Result<()> { + let (mut cache, token, alice, bob) = token_with_funded_alice().await?; + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + + let txs = vec![ + BundleTx::new(alice, token, transfer_calldata(bob, 100)), + // Reverts, but only index 0 is whitelisted -> atomic fallback. + BundleTx::new(alice, token, Bytes::from(vec![0xde, 0xad, 0xbe, 0xef])), + ]; + let result = overlay.simulate_bundle( + &txs, + &BundleOptions { + revert_policy: RevertPolicy::AllowReverts(vec![0]), + commit: true, + }, + )?; + + assert!(!result.succeeded); + assert_eq!(result.per_tx.len(), 2); + assert!(result.per_tx[1].reverted); + // Whole bundle rolled back: alice keeps her full balance, bob gets nothing. + assert_eq!( + overlay_balance(&mut overlay, token, alice)?, + U256::from(1_000u64) + ); + assert_eq!(overlay_balance(&mut overlay, token, bob)?, U256::ZERO); + Ok(()) +} + +/// `commit = false` with `AllowReverts` leaves the overlay unchanged even though +/// the bundle "succeeded" (the kept tx is rolled back by the outer revert). +#[tokio::test(flavor = "multi_thread")] +async fn allow_reverts_commit_false_is_isolated() -> Result<()> { + let (mut cache, token, alice, bob) = token_with_funded_alice().await?; + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + + let txs = vec![ + BundleTx::new(alice, token, transfer_calldata(bob, 100)), + BundleTx::new(alice, token, Bytes::from(vec![0xde, 0xad, 0xbe, 0xef])), + ]; + let result = overlay.simulate_bundle( + &txs, + &BundleOptions { + revert_policy: RevertPolicy::AllowReverts(vec![1]), + commit: false, + }, + )?; + + assert!(result.succeeded); + assert!(!result.per_tx[0].reverted); + assert!(result.per_tx[1].reverted); + // commit = false: even tx 0's effect is reverted out of the overlay. + assert_eq!( + overlay_balance(&mut overlay, token, alice)?, + U256::from(1_000u64) + ); + assert_eq!(overlay_balance(&mut overlay, token, bob)?, U256::ZERO); + Ok(()) +} + +/// `EvmCache::simulate_bundle` runs on a transient overlay and never mutates the +/// cache, even with `commit = true`. +#[tokio::test(flavor = "multi_thread")] +async fn cache_simulate_bundle_does_not_mutate_cache() -> Result<()> { + let (mut cache, token, alice, bob) = token_with_funded_alice().await?; + + let result = cache.simulate_bundle( + &[BundleTx::new(alice, token, transfer_calldata(bob, 100))], + &BundleOptions { + commit: true, + ..Default::default() + }, + )?; + assert!(result.succeeded); + assert_eq!(result.per_tx.len(), 1); + + // The cache is unchanged: a fresh snapshot still shows alice's full balance. + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + assert_eq!( + overlay_balance(&mut overlay, token, alice)?, + U256::from(1_000u64) + ); + assert_eq!(overlay_balance(&mut overlay, token, bob)?, U256::ZERO); + Ok(()) +} diff --git a/tests/cache_state.rs b/tests/cache_state.rs index cd663a4..b01c6fa 100644 --- a/tests/cache_state.rs +++ b/tests/cache_state.rs @@ -8,6 +8,7 @@ mod common; +use alloy_eips::eip2930::{AccessList, AccessListItem}; use alloy_primitives::{Address, B256, Bytes, I256, U256, keccak256}; use alloy_sol_types::{SolCall, SolValue}; use anyhow::{Context, Result}; @@ -119,6 +120,74 @@ async fn call_raw_with_carries_native_value() -> Result<()> { Ok(()) } +/// `TxConfig.access_list` is wired into EIP-2929/EIP-2930 gas accounting: declaring +/// the touched account + slot warms them for the call but adds the EIP-2930 +/// intrinsic cost (2400/address + 1900/key). For a `balanceOf` that reads the slot +/// once, the intrinsic outweighs the ~2000-gas warm SLOAD saving, so the declared +/// run costs *more* — what matters is that the field measurably changes gas, i.e. +/// it is not silently ignored. +#[tokio::test(flavor = "multi_thread")] +async fn tx_config_access_list_changes_eip2929_gas_accounting() -> Result<()> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x11); + let owner = Address::repeat_byte(0x22); + install_default_account(&mut cache, Address::ZERO); + install_mock_erc20(&mut cache, token); + + // Layer-1 CacheDB balance seed so `balanceOf` reads a real (non-zero) value. + let slot = + U256::from_be_bytes(keccak256((owner, U256::from(MOCK_ERC20_BALANCE_SLOT)).abi_encode()).0); + cache + .db_mut() + .insert_account_storage(token, slot, U256::from(123u64))?; + + let calldata = Bytes::from(MockERC20::balanceOfCall { account: owner }.abi_encode()); + + let gas_of = |result: ExecutionResult| match result { + ExecutionResult::Success { gas_used, .. } => gas_used, + other => panic!("balanceOf did not succeed: {other:?}"), + }; + + // No access list: the `balanceOf` SLOAD pays the cold (2100-gas) cost. + let gas_plain = gas_of(cache.call_raw_with( + Address::ZERO, + token, + calldata.clone(), + false, + &TxConfig::default(), + )?); + + // Declare (token, slot): the SLOAD is warmed, but the access list carries its + // own EIP-2930 intrinsic cost. + let tx = TxConfig { + access_list: Some(AccessList(vec![AccessListItem { + address: token, + storage_keys: vec![B256::from(slot)], + }])), + ..Default::default() + }; + let gas_with_list = gas_of(cache.call_raw_with(Address::ZERO, token, calldata, false, &tx)?); + + assert_ne!( + gas_plain, gas_with_list, + "TxConfig.access_list must affect gas accounting, not be ignored" + ); + // Net for a single warmed-once slot: +2400 (addr) + 1900 (key) − 2000 (warm + // SLOAD) = +2300. The access list raises gas for this single-access call. + assert!( + gas_with_list > gas_plain, + "single-access declaration costs more intrinsic than it saves: \ + with_list {gas_with_list} vs plain {gas_plain}" + ); + assert_eq!( + gas_with_list - gas_plain, + 2_300, + "expected the exact EIP-2930 intrinsic minus single warm-SLOAD saving" + ); + + Ok(()) +} + #[tokio::test(flavor = "multi_thread")] async fn simulation_reports_balance_deltas() -> Result<()> { let mut cache = setup_cache().await?; diff --git a/tests/call_tracer.rs b/tests/call_tracer.rs new file mode 100644 index 0000000..8c1f1ea --- /dev/null +++ b/tests/call_tracer.rs @@ -0,0 +1,202 @@ +//! Manager-authored red-green acceptance tests for Phase 6 Track C: the +//! call-frame tracer (`CallTracer`) and the generalized public inspector seam +//! (`EvmOverlay::call_raw_with_inspector` + `InspectorStack`). +//! +//! These describe the public contract before the implementation exists. The +//! implementation agent must make them pass WITHOUT weakening, skipping, or +//! rewriting them; if a test encodes a wrong assumption about EVM/mock behavior +//! (as opposed to the feature contract), surface it to the manager with a +//! justification rather than silently changing it. +//! +//! Fully offline (mocked provider, injected state). +#![cfg(feature = "reactive")] + +mod common; + +use alloy_primitives::{Address, Bytes, U256, hex}; +use alloy_sol_types::SolCall; +use anyhow::Result; +use revm::state::{AccountInfo, Bytecode}; + +use common::{ + MOCK_ERC20_BALANCE_SLOT, MockERC20, install_default_account, install_mock_erc20, setup_cache, +}; +use evm_fork_cache::inspector::TransferInspector; +use evm_fork_cache::multicall::{IMulticall3, MULTICALL3_ADDRESS}; +use evm_fork_cache::{CallStatus, CallTracer, EvmOverlay, InspectorStack, TxConfig}; + +const MULTICALL3_RUNTIME_HEX: &str = include_str!("../fixtures/multicall3_runtime.hex"); + +/// Etch runtime bytecode at `addr` and mark its storage local (offline contract). +fn install_runtime(cache: &mut evm_fork_cache::EvmCache, addr: Address, runtime_hex: &str) { + let code = Bytecode::new_raw(Bytes::from( + hex::decode(runtime_hex.trim()).expect("valid hex"), + )); + let code_hash = code.hash_slow(); + cache.db_mut().insert_account_info( + addr, + AccountInfo { + code: Some(code), + code_hash, + ..Default::default() + }, + ); + cache + .db_mut() + .replace_account_storage(addr, Default::default()) + .expect("mark storage local"); +} + +fn balance_of_calldata(owner: Address) -> Bytes { + Bytes::from(MockERC20::balanceOfCall { account: owner }.abi_encode()) +} + +/// C1 — a single top-level call yields a root frame with the right from/to/input +/// and a Success status. +#[tokio::test(flavor = "multi_thread")] +async fn tracer_captures_single_top_level_frame() -> Result<()> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x11); + let caller = Address::repeat_byte(0x22); + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, caller); + install_mock_erc20(&mut cache, token); + + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + let calldata = balance_of_calldata(caller); + let (_result, tracer) = overlay.call_raw_with_inspector( + caller, + token, + calldata.clone(), + &TxConfig::default(), + CallTracer::new(), + false, + )?; + + let root = tracer + .into_trace() + .expect("a top-level call produces a root frame"); + assert_eq!(root.from, caller); + assert_eq!(root.to, token); + assert_eq!(root.input, calldata); + assert_eq!(root.status, CallStatus::Success); + Ok(()) +} + +/// C2 — a Multicall3 `aggregate3` that targets the token produces a root frame +/// (caller -> Multicall3) with a nested subcall to the token. +#[tokio::test(flavor = "multi_thread")] +async fn tracer_captures_nested_subcalls() -> Result<()> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x11); + let caller = Address::repeat_byte(0x22); + let owner = Address::repeat_byte(0x33); + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, caller); + install_mock_erc20(&mut cache, token); + install_runtime(&mut cache, MULTICALL3_ADDRESS, MULTICALL3_RUNTIME_HEX); + + let calls = vec![IMulticall3::Call3 { + target: token, + allowFailure: false, + callData: balance_of_calldata(owner), + }]; + let calldata = Bytes::from(IMulticall3::aggregate3Call { calls }.abi_encode()); + + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + let (_result, tracer) = overlay.call_raw_with_inspector( + caller, + MULTICALL3_ADDRESS, + calldata, + &TxConfig::default(), + CallTracer::new(), + false, + )?; + + let root = tracer.into_trace().expect("root frame"); + assert_eq!(root.to, MULTICALL3_ADDRESS); + assert!( + root.subcalls.iter().any(|c| c.to == token), + "the aggregate3 frame should contain a subcall to the token, got {:#?}", + root.subcalls + ); + Ok(()) +} + +/// C3 — a call into a contract that reverts (unknown selector) is attributed as a +/// reverted frame. +#[tokio::test(flavor = "multi_thread")] +async fn tracer_attributes_reverts() -> Result<()> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x11); + let caller = Address::repeat_byte(0x22); + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, caller); + install_mock_erc20(&mut cache, token); + + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + let (_result, tracer) = overlay.call_raw_with_inspector( + caller, + token, + Bytes::from(vec![0xde, 0xad, 0xbe, 0xef]), // unknown selector -> revert + &TxConfig::default(), + CallTracer::new(), + false, + )?; + + let root = tracer.into_trace().expect("root frame"); + assert_eq!(root.status, CallStatus::Revert); + Ok(()) +} + +/// C4 — `InspectorStack` runs a `CallTracer` and a `TransferInspector` in one +/// pass: both produce their independent results. +#[tokio::test(flavor = "multi_thread")] +async fn inspector_stack_composes_tracer_and_transfer() -> Result<()> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x11); + let alice = Address::repeat_byte(0x22); + let bob = Address::repeat_byte(0x33); + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, alice); + install_default_account(&mut cache, bob); + install_mock_erc20(&mut cache, token); + cache.insert_mapping_storage_slot( + token, + U256::from(MOCK_ERC20_BALANCE_SLOT), + alice, + U256::from(1_000u64), + )?; + + let calldata = Bytes::from( + MockERC20::transferCall { + to: bob, + amount: U256::from(50u64), + } + .abi_encode(), + ); + + let mut overlay = EvmOverlay::new(cache.create_snapshot(), None); + let (_result, stack) = overlay.call_raw_with_inspector( + alice, + token, + calldata, + &TxConfig::default(), + InspectorStack(CallTracer::new(), TransferInspector::new()), + false, + )?; + + let InspectorStack(tracer, transfer) = stack; + assert!( + tracer.root().is_some(), + "tracer should have captured a frame" + ); + assert!( + transfer + .transfers + .iter() + .any(|t| t.from == alice && t.to == bob && t.token == token), + "transfer inspector should have captured the ERC-20 Transfer" + ); + Ok(()) +} diff --git a/tests/event_pipeline.rs b/tests/event_pipeline.rs index 5cadd13..bea625e 100644 --- a/tests/event_pipeline.rs +++ b/tests/event_pipeline.rs @@ -532,3 +532,79 @@ async fn reconcile_errors_without_fetcher() -> Result<()> { ); Ok(()) } + +/// Pillar 3 (reactive sync): ingesting a block's `Transfer` logs keeps the hot +/// balance slots correct with **zero** RPC fetches — the decode→write pipeline +/// never touches the fetcher, unlike a poller that re-reads every changed slot +/// each block. Sampled `reconcile` (which *does* fetch) is the honesty backstop. +#[tokio::test(flavor = "multi_thread")] +async fn ingest_keeps_state_fresh_with_zero_fetches() -> Result<()> { + use std::sync::atomic::{AtomicUsize, Ordering}; + + use alloy_eips::BlockId; + use common::{balance_of, install_default_account}; + use evm_fork_cache::cache::StorageBatchFetchFn; + + let token = Address::repeat_byte(0x33); + let owners: Vec
= (0..8u8).map(|i| Address::repeat_byte(0x40 + i)).collect(); + + let mut cache = setup_cache().await?; + install_mock_erc20(&mut cache, token); + install_default_account(&mut cache, Address::ZERO); // fee beneficiary + for o in &owners { + // Install each owner as an account (so it can be the balanceOf caller + // without a lazy RPC load) and seed its balance (layer 1, readable) so the + // deltas apply hot. + install_default_account(&mut cache, *o); + cache + .db_mut() + .insert_account_storage(token, mapping_slot(*o, 3), U256::from(1_000))?; + } + + // A counting fetcher: `ingest_logs` must never invoke it. + let fetches = Arc::new(AtomicUsize::new(0)); + let counter = fetches.clone(); + let f: StorageBatchFetchFn = + Arc::new(move |reqs: Vec<(Address, U256)>, _b: Option| { + counter.fetch_add(reqs.len(), Ordering::Relaxed); + reqs.into_iter() + .map(|(a, s)| (a, s, Ok(U256::from(1_000)))) + .collect() + }); + cache.set_storage_batch_fetcher(f); + + let mut registry = DecoderRegistry::new(); + registry.register(Arc::new(Erc20TransferDecoder::new(U256::from(3)))); + let mut pipeline = EventPipeline::new(registry); + + // A block of transfers around a ring (each owner sends 10 and receives 10 → + // balances net back to 1000), touching every owner's balance slot. + let logs: Vec = (0..owners.len()) + .map(|i| { + transfer_log( + token, + owners[i], + owners[(i + 1) % owners.len()], + U256::from(10), + ) + }) + .collect(); + + let digest = pipeline.ingest_logs(&mut cache, 100, &logs); + + assert_eq!( + fetches.load(Ordering::Relaxed), + 0, + "ingest decodes logs into writes and performs ZERO RPC fetches" + ); + assert_eq!(digest.decoded_logs, owners.len()); + assert!( + !digest.applied.has_skipped(), + "all hot balance deltas applied" + ); + // Correctness: state kept fresh from the logs alone equals the true post-state. + for o in &owners { + assert_eq!(balance_of(&mut cache, token, *o)?, U256::from(1_000)); + } + Ok(()) +} diff --git a/tests/fetch_minimization.rs b/tests/fetch_minimization.rs new file mode 100644 index 0000000..b622bcb --- /dev/null +++ b/tests/fetch_minimization.rs @@ -0,0 +1,128 @@ +//! Pins the Pillar 2 headline (data-fetch minimization) as a deterministic, +//! CI-guaranteed integer invariant: the crate fetches a shared hot working set +//! ONCE, and fanning N candidate simulations out over the frozen snapshot adds +//! ZERO further fetches — so a fork-per-candidate loop fetches `N x` as much. + +mod common; + +use std::sync::Arc; +use std::sync::atomic::{AtomicUsize, Ordering}; + +use alloy_eips::BlockId; +use alloy_primitives::{Address, Bytes, U256, keccak256}; +use alloy_sol_types::{SolCall, SolValue}; +use anyhow::Result; +use evm_fork_cache::cache::{EvmCache, EvmOverlay, StorageBatchFetchFn}; +use revm::context::result::ExecutionResult; +use revm::state::AccountInfo; + +use common::{MOCK_ERC20_BALANCE_SLOT, MockERC20, mock_erc20_runtime, setup_cache}; + +const WORKING_SET: usize = 8; +const N_CANDIDATES: usize = 500; +const SEEDED_BALANCE: u64 = 1_000; + +fn owner(i: usize) -> Address { + let mut bytes = [0u8; 20]; + bytes[12..20].copy_from_slice(&(i as u64 + 1).to_be_bytes()); + Address::from(bytes) +} + +fn balance_slot(owner: Address) -> U256 { + U256::from_be_bytes(keccak256((owner, U256::from(MOCK_ERC20_BALANCE_SLOT)).abi_encode()).0) +} + +fn counting_fetcher(counter: Arc) -> StorageBatchFetchFn { + Arc::new( + move |requests: Vec<(Address, U256)>, _block: Option| { + counter.fetch_add(requests.len(), Ordering::Relaxed); + requests + .into_iter() + .map(|(addr, slot)| (addr, slot, Ok(U256::from(SEEDED_BALANCE)))) + .collect() + }, + ) +} + +async fn cache_with_counter(token: Address) -> Result<(EvmCache, Arc)> { + let mut cache = setup_cache().await?; + let runtime = mock_erc20_runtime(); + let code_hash = runtime.hash_slow(); + cache.db_mut().insert_account_info( + token, + AccountInfo { + balance: U256::ZERO, + nonce: 1, + code: Some(runtime), + code_hash, + account_id: None, + }, + ); + let counter = Arc::new(AtomicUsize::new(0)); + cache.set_storage_batch_fetcher(counting_fetcher(counter.clone())); + Ok((cache, counter)) +} + +fn balance_of(overlay: &mut EvmOverlay, caller: Address, token: Address, account: Address) -> U256 { + let calldata = Bytes::from(MockERC20::balanceOfCall { account }.abi_encode()); + match overlay.call_raw(caller, token, calldata).unwrap() { + ExecutionResult::Success { output, .. } => { + MockERC20::balanceOfCall::abi_decode_returns(&output.into_data()).unwrap() + } + other => panic!("balanceOf failed: {other:?}"), + } +} + +#[tokio::test(flavor = "multi_thread")] +async fn fan_out_reuses_one_warmup_fetch_across_all_candidates() -> Result<()> { + let token = Address::repeat_byte(0x42); + let working_set: Vec<(Address, U256)> = (0..WORKING_SET) + .map(|i| (token, balance_slot(owner(i)))) + .collect(); + + let (mut cache, counter) = cache_with_counter(token).await?; + + // Warm the working set once. + cache.verify_slots(&working_set)?; + assert_eq!( + counter.load(Ordering::Relaxed), + WORKING_SET, + "warm-up fetches each working-set slot exactly once" + ); + + let snapshot = cache.create_snapshot(); + counter.store(0, Ordering::Relaxed); + + // Fan N candidates out; each reads the whole shared hot set from the snapshot. + for c in 0..N_CANDIDATES { + let mut overlay = EvmOverlay::new(snapshot.clone(), None); + for i in 0..WORKING_SET { + // Correctness: the warmed value is actually served (reuse works), and + // the read does not fetch. + assert_eq!( + balance_of(&mut overlay, owner(c % WORKING_SET), token, owner(i)), + U256::from(SEEDED_BALANCE) + ); + } + } + + assert_eq!( + counter.load(Ordering::Relaxed), + 0, + "the {N_CANDIDATES}-candidate fan-out adds zero fetches (overlays read the snapshot)" + ); + + // The fork-per-candidate baseline: one real cold cache fetches the whole + // working set, so N of them fetch N x as much. + let (mut cold, cold_counter) = cache_with_counter(token).await?; + cold.verify_slots(&working_set)?; + let per_candidate = cold_counter.load(Ordering::Relaxed); + assert_eq!(per_candidate, WORKING_SET); + assert_eq!( + per_candidate * N_CANDIDATES, + WORKING_SET * N_CANDIDATES, + "vanilla fork-per-candidate re-fetches the working set every candidate" + ); + + Ok(()) +} diff --git a/tests/multicall.rs b/tests/multicall.rs index 8ac1895..60e3dbc 100644 --- a/tests/multicall.rs +++ b/tests/multicall.rs @@ -1,23 +1,61 @@ //! Offline integration tests for the Multicall3 helpers. //! -//! The live `aggregate3` execution path requires the Multicall3 contract to be -//! deployed in the fork, which the RPC-gated `multicall_batch` example exercises. -//! These tests pin the network-free behavior: empty-batch short-circuits, the -//! result-decoding helpers, and the documented batch constants. +//! These pin the network-free behavior end to end: the empty-batch short-circuit, +//! the result-decoding helpers, the documented batch constants, and — by etching +//! the real Multicall3 runtime at its canonical address — the live `aggregate3` +//! build/execute/decode path, including input-order results and `allowFailure` +//! partial-result semantics. mod common; -use alloy_primitives::{Address, Bytes, U256}; +use alloy_primitives::{Address, Bytes, U256, hex, keccak256}; use alloy_sol_types::{SolCall, SolValue, sol}; use anyhow::Result; +use revm::state::{AccountInfo, Bytecode}; -use common::setup_cache; +use common::{MOCK_ERC20_BALANCE_SLOT, install_default_account, install_mock_erc20, setup_cache}; +use evm_fork_cache::cache::EvmCache; use evm_fork_cache::multicall::{ - IMulticall3, MAX_BATCH_SIZE, MulticallBatch, decode_result, execute_batched, try_decode_result, + IMulticall3, MAX_BATCH_SIZE, MULTICALL3_ADDRESS, MulticallBatch, decode_result, + execute_batched, try_decode_result, }; +/// Real Multicall3 deployed (runtime) bytecode, fetched from mainnet. Multicall3 +/// is deployed at the same address ([`MULTICALL3_ADDRESS`]) on virtually all EVM +/// chains, so etching it locally reproduces the real `aggregate3` behavior. +const MULTICALL3_RUNTIME_HEX: &str = include_str!("../fixtures/multicall3_runtime.hex"); + sol! { function getValue() external returns (uint256); + function balanceOf(address account) external returns (uint256); +} + +/// `keccak256(abi.encode(owner, balanceSlot))` — the MockERC20 balance mapping slot. +fn balance_slot(owner: Address) -> U256 { + U256::from_be_bytes(keccak256((owner, U256::from(MOCK_ERC20_BALANCE_SLOT)).abi_encode()).0) +} + +/// Etch deployed runtime bytecode at `addr` and mark its storage local, so an +/// unseeded slot reads as zero rather than hitting the mocked RPC backend. +fn install_runtime(cache: &mut EvmCache, addr: Address, runtime_hex: &str) { + let code = Bytecode::new_raw(Bytes::from( + hex::decode(runtime_hex.trim()).expect("valid runtime hex"), + )); + let code_hash = code.hash_slow(); + cache.db_mut().insert_account_info( + addr, + AccountInfo { + balance: U256::ZERO, + nonce: 1, + code: Some(code), + code_hash, + account_id: None, + }, + ); + cache + .db_mut() + .replace_account_storage(addr, Default::default()) + .expect("clear etched storage"); } /// An empty batch returns empty results without invoking the EVM, on all three @@ -95,3 +133,88 @@ fn decode_result_rejects_garbage_payload() { fn max_batch_size_constant() { assert_eq!(MAX_BATCH_SIZE, 200); } + +/// Etching the real Multicall3 runtime lets the live `aggregate3` path run fully +/// offline: results come back one-per-call in input order and decode to the +/// seeded values. +#[tokio::test(flavor = "multi_thread")] +async fn aggregate3_executes_offline_in_input_order() -> Result<()> { + let mut cache = setup_cache().await?; + install_runtime(&mut cache, MULTICALL3_ADDRESS, MULTICALL3_RUNTIME_HEX); + + let token = Address::repeat_byte(0x22); + let owner = Address::repeat_byte(0x33); + let other = Address::repeat_byte(0x44); + install_default_account(&mut cache, Address::ZERO); + install_mock_erc20(&mut cache, token); + // Seed the layer-1 CacheDB (the layer the EVM reads from) so the StorageCleared + // mock reports it; a backend-only seed would read as zero via the + // account-state-aware read path. + cache + .db_mut() + .insert_account_storage(token, balance_slot(owner), U256::from(5_000u64))?; + + let mut batch = MulticallBatch::new(); + batch.add_call(token, balanceOfCall { account: owner }, false); + batch.add_call(token, balanceOfCall { account: other }, false); + + let results = batch.execute(&mut cache)?; + assert_eq!(results.len(), 2, "one result per input call"); + assert!(results[0].success && results[1].success); + // Input order is preserved: owner first (5000), other second (0). + assert_eq!( + decode_result::(&results[0])?, + U256::from(5_000u64) + ); + assert_eq!(decode_result::(&results[1])?, U256::ZERO); + Ok(()) +} + +/// `allowFailure = true` lets a reverting call surface as `success = false` +/// alongside successful calls; `allowFailure = false` makes the whole batch revert +/// (an `Err`). `execute_tracked` captures the touched accounts. +#[tokio::test(flavor = "multi_thread")] +async fn aggregate3_allow_failure_partial_results_and_strict_revert() -> Result<()> { + let mut cache = setup_cache().await?; + install_runtime(&mut cache, MULTICALL3_ADDRESS, MULTICALL3_RUNTIME_HEX); + + let token = Address::repeat_byte(0x22); + let owner = Address::repeat_byte(0x33); + install_default_account(&mut cache, Address::ZERO); + install_mock_erc20(&mut cache, token); + cache + .db_mut() + .insert_account_storage(token, balance_slot(owner), U256::from(7u64))?; + + // An unknown selector reverts in the MockERC20 (no matching function). + let reverting = Bytes::from_static(&[0xde, 0xad, 0xbe, 0xef]); + + let mut batch = MulticallBatch::new(); + batch.add_call(token, balanceOfCall { account: owner }, false); + batch.add(token, reverting.clone(), true); // allowed to fail + let (results, access) = batch.execute_tracked(&mut cache)?; + + assert_eq!(results.len(), 2); + assert!(results[0].success); + assert_eq!( + decode_result::(&results[0])?, + U256::from(7u64) + ); + assert!( + !results[1].success, + "the reverting call surfaces as success = false, not an Err" + ); + assert!( + access.accounts.contains(&token), + "execute_tracked must capture the token account the inner call touched" + ); + + // The same revert with allow_failure = false fails the entire aggregate3. + let mut strict = MulticallBatch::new(); + strict.add(token, reverting, false); + assert!( + strict.execute(&mut cache).is_err(), + "a non-allowed revert reverts the whole batch" + ); + Ok(()) +} diff --git a/tests/typed_sol_call.rs b/tests/typed_sol_call.rs new file mode 100644 index 0000000..b8ab4ff --- /dev/null +++ b/tests/typed_sol_call.rs @@ -0,0 +1,223 @@ +//! Offline integration tests for typed `SolCall` helpers on `EvmCache`. + +mod common; + +use alloy_primitives::{Address, Bytes, U256, hex}; +use alloy_sol_types::sol; +use anyhow::Result; +use evm_fork_cache::cache::{EvmCache, TxConfig}; +use revm::state::{AccountInfo, Bytecode}; + +use common::{ + MOCK_ERC20_BALANCE_SLOT, MockERC20, balance_of, install_default_account, install_mock_erc20, + setup_cache, +}; + +sol! { + function who() external returns (address); + function paid() external payable returns (uint256); + function willRevert() external returns (uint256); + function malformed() external returns (uint256); +} + +fn install_runtime(cache: &mut EvmCache, addr: Address, runtime_hex: &str) -> Result<()> { + let code = Bytecode::new_raw(Bytes::from(hex::decode(runtime_hex)?)); + let code_hash = code.hash_slow(); + cache.db_mut().insert_account_info( + addr, + AccountInfo { + balance: U256::ZERO, + nonce: 1, + code: Some(code), + code_hash, + account_id: None, + }, + ); + cache + .db_mut() + .replace_account_storage(addr, Default::default())?; + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn call_sol_decodes_mock_erc20_balance() -> Result<()> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x11); + let owner = Address::repeat_byte(0x22); + let expected = U256::from(12_345u64); + + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, owner); + install_mock_erc20(&mut cache, token); + cache.insert_mapping_storage_slot( + token, + U256::from(MOCK_ERC20_BALANCE_SLOT), + owner, + expected, + )?; + + let balance = cache.call_sol(token, MockERC20::balanceOfCall { account: owner })?; + assert_eq!(balance, expected); + + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn call_sol_from_threads_msg_sender() -> Result<()> { + let mut cache = setup_cache().await?; + let sender = Address::repeat_byte(0x33); + let target = Address::repeat_byte(0x44); + + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, sender); + // CALLER; MSTORE(0); RETURN(0, 32) + install_runtime(&mut cache, target, "3360005260206000f3")?; + + let returned = cache.call_sol_from(sender, target, whoCall {})?; + assert_eq!(returned, sender); + + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn call_sol_with_threads_tx_value() -> Result<()> { + let mut cache = setup_cache().await?; + let sender = Address::repeat_byte(0x55); + let target = Address::repeat_byte(0x66); + let value = U256::from(98_765u64); + + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, sender); + // CALLVALUE; MSTORE(0); RETURN(0, 32) + install_runtime(&mut cache, target, "3460005260206000f3")?; + + let tx = TxConfig { + value, + ..Default::default() + }; + let returned = cache.call_sol_with(sender, target, paidCall {}, &tx)?; + assert_eq!(returned, value); + + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn call_sol_from_does_not_commit_state() -> Result<()> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0x77); + let owner = Address::repeat_byte(0x88); + let recipient = Address::repeat_byte(0x99); + let initial = U256::from(1_000u64); + let amount = U256::from(250u64); + + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, owner); + install_default_account(&mut cache, recipient); + install_mock_erc20(&mut cache, token); + cache.insert_mapping_storage_slot( + token, + U256::from(MOCK_ERC20_BALANCE_SLOT), + owner, + initial, + )?; + cache.insert_mapping_storage_slot( + token, + U256::from(MOCK_ERC20_BALANCE_SLOT), + recipient, + U256::ZERO, + )?; + + let ok = cache.call_sol_from( + owner, + token, + MockERC20::transferCall { + to: recipient, + amount, + }, + )?; + assert!(ok); + assert_eq!(balance_of(&mut cache, token, owner)?, initial); + assert_eq!(balance_of(&mut cache, token, recipient)?, U256::ZERO); + + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn transact_sol_commits_state() -> Result<()> { + let mut cache = setup_cache().await?; + let token = Address::repeat_byte(0xaa); + let owner = Address::repeat_byte(0xbb); + let recipient = Address::repeat_byte(0xcc); + let initial = U256::from(1_000u64); + let amount = U256::from(250u64); + + install_default_account(&mut cache, Address::ZERO); + install_default_account(&mut cache, owner); + install_default_account(&mut cache, recipient); + install_mock_erc20(&mut cache, token); + cache.insert_mapping_storage_slot( + token, + U256::from(MOCK_ERC20_BALANCE_SLOT), + owner, + initial, + )?; + cache.insert_mapping_storage_slot( + token, + U256::from(MOCK_ERC20_BALANCE_SLOT), + recipient, + U256::ZERO, + )?; + + let ok = cache.transact_sol( + owner, + token, + MockERC20::transferCall { + to: recipient, + amount, + }, + &TxConfig::default(), + )?; + assert!(ok); + assert_eq!(balance_of(&mut cache, token, owner)?, initial - amount); + assert_eq!(balance_of(&mut cache, token, recipient)?, amount); + + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn reverting_sol_call_reports_function_and_target() -> Result<()> { + let mut cache = setup_cache().await?; + let target = Address::repeat_byte(0xdd); + + install_default_account(&mut cache, Address::ZERO); + // REVERT(0, 0) + install_runtime(&mut cache, target, "60006000fd")?; + + let err = cache.call_sol(target, willRevertCall {}).unwrap_err(); + let message = format!("{err:#}"); + assert!(message.contains("willRevert()"), "{message}"); + assert!(message.contains(&format!("{target:?}")), "{message}"); + assert!(message.contains("did not succeed"), "{message}"); + assert!(message.contains("Revert"), "{message}"); + + Ok(()) +} + +#[tokio::test(flavor = "multi_thread")] +async fn malformed_return_data_reports_decode_context() -> Result<()> { + let mut cache = setup_cache().await?; + let target = Address::repeat_byte(0xee); + + install_default_account(&mut cache, Address::ZERO); + // STOP: succeeds with empty return data, which cannot decode as uint256. + install_runtime(&mut cache, target, "00")?; + + let err = cache.call_sol(target, malformedCall {}).unwrap_err(); + let message = format!("{err:#}"); + assert!(message.contains("malformed()"), "{message}"); + assert!(message.contains(&format!("{target:?}")), "{message}"); + assert!(message.contains("output_len=0"), "{message}"); + assert!(message.contains("failed to decode"), "{message}"); + + Ok(()) +}