feat: headless-host metrics and stress fleet#269
Draft
eugypalu wants to merge 159 commits into
Draft
Conversation
Adds the canonical testing module (api/testing.rs) and its v01/v02/versioned wiring used by the Rust host runtime and generated clients.
New crate defining the host syscall traits (storage, navigation, consent, permissions, ...) that host runtimes implement. Types are re-exported from truapi::versioned/v01 rather than redefined.
WASM host runtime that hosts implement: dispatcher, SCALE frames, subscription streams, chain runtime, host logic (sessions, SSO pairing, permissions, statement store, dotns) and the wasm bindings. Includes the committed generated dispatcher/wire-table under src/generated/.
…backs Extends the rustdoc-JSON code generator to emit the Rust dispatcher and wire table consumed by truapi-server, plus the TS host-callbacks adapter. Golden tests pin the emitted shapes.
New WASM-backed host runtime package embedding the Rust core, with web iframe and Web Worker entry points. Updates the @parity/truapi client (SCALE, sandbox, transport) and drops the obsolete explorer 0.3.2 codegen snapshot.
Updates CLAUDE.md/README, CI workflows, Makefile, deny.toml, changesets, and linguist attributes for generated code, and bumps the dotli submodule to the host integration that consumes the WASM runtime.
Adds the canonical testing module (api/testing.rs) and its v01/v02/versioned wiring used by the Rust host runtime and generated clients.
New crate defining the host syscall traits (storage, navigation, consent, permissions, ...) that host runtimes implement. Types are re-exported from truapi::versioned/v01 rather than redefined.
…backs Extends the rustdoc-JSON code generator to emit the Rust dispatcher and wire table consumed by truapi-server, plus the TS host-callbacks adapter. Golden tests pin the emitted shapes.
Publish the host runtime as @parity/truapi-host and move the dotli diagnosis harness into the playground repo.
…t-core-port # Conflicts: # rust/crates/truapi-codegen/tests/golden/host-callbacks-adapter.ts # rust/crates/truapi-codegen/tests/golden/host-callbacks.ts # rust/crates/truapi-codegen/tests/golden/worker-callbacks.ts # rust/crates/truapi-codegen/tests/golden_rust_emit.rs # rust/crates/truapi-server/src/lib.rs
Move the entire Bulletin TransactionStorage.store submission into
truapi-server. The core builds the extrinsic offline (subxt 0.50.2), signs
it with the wallet-delegated allowance key, dry-runs it, broadcasts, and
watches for inclusion over the existing chainHead runtime — replacing the
host signer-callback seam so the allowance secret never crosses the
host/FFI boundary.
Core:
- host_logic/extrinsic.rs: offline SubstrateConfig assembler with a
config-pinned genesis hash, an sr25519 Signer, and metadata / transaction
validity / events / header decoders.
- host_logic/bulletin.rs: store{data} construction signed with the allowance
key, with audited pallet/call-index pinning plus a canonical-bytes guard so
provider metadata cannot redirect the signature, and a memcpy call-data
encoder that avoids scale-encode's per-byte cost.
- runtime/bulletin_rpc.rs: serialized submit flow (ephemeral with_runtime
follow, metadata, nonce, validate_transaction dry-run, broadcast, single
event-loop inclusion watch gated on nonce advance, System.Events dispatch
check), typed error taxonomy, and broadcast stop on every exit.
- runtime.rs: Preimage::submit gates on bulletin availability before any
prompt and refreshes the allowance (Increase policy) with one retry on an
allowance rejection; lookup_subscribe verifies blake2_256(value)==key and
serves an in-core content-addressed cache.
- BulletinAllowanceKey is zeroized on drop; PreimageHost keeps only
lookup_preimage; both host configs gain an optional Bulletin genesis hash.
Codegen/TS: regenerate goldens (product wire unchanged) and drop the signer
bridge from the handwritten host worker. CI compiles the crate for
wasm32-unknown-unknown; .gitignore ignores the renamed wasm bundle path.
Bumps the hosts/dotli gitlink to the matching submodule commit.
Adversarial review of the watch loop found two defects: - A crafted or buggy chain provider could send a self-referential or cyclic `NewBlock` parent link. The nonce-advance ancestor walk had no visited-set guard and no `.await`, so such a link spun forever, freezing the worker and permanently holding the submit lock across all products. The walk is extracted into `ancestors_to_check`, which guards against self-parent and cyclic links with a visited set, and is unit-tested. - Blocks that failed the nonce gate were marked checked but never unpinned, leaking chainHead pins over the watch's lifetime and risking a false BroadcastUnverified once the server's pin limit was hit. They are now unpinned like body-negative blocks. Also log a warning when a host lookup value is downgraded to a miss for failing the blake2_256(value)==key integrity check.
…bler The signing-host role now builds and signs transactions locally instead of returning Unavailable, reusing host_logic/extrinsic.rs. Because ProductAccountTxPayload carries each extension's `extra` and `additional_signed` already SCALE-encoded in canonical order, assembly is a pure offline concatenation — no metadata, no RPC: - extrinsic.rs gains build_signed_extrinsic_v4 + v4_signer_payload and an Sr25519Signer::from_keypair constructor. Body = Compact(len) ++ 0x84 ++ MultiAddress::Id(signer) ++ MultiSignature::Sr25519(sig) ++ Σextra ++ call_data; signer payload = call_data ++ Σextra ++ Σadditional_signed, blake2_256 only when >256 bytes. Layout is byte-identical to subxt / frame-decode. - signing_host::create_transaction handles Product and LegacyAccount (with a fail-closed slot-zero key-match check); the product-facing entrypoint's caller-scoping, chain-submit permission, and user-confirmation gates already precede it. - Extrinsic V5 (tx_ext_version != 0) returns the new AuthorityError::NotSupported -> HostCreateTransactionError::NotSupported: V5 general carries the signature inside a VerifySignature extension, which cannot come from pre-encoded parts. Tested: v4 layout + signature verification, the >256 hashing boundary, extension order preservation, and Product/LegacyAccount success plus v5/mismatch/no-session rejections.
Resolve the headless-host conflicts, add statement-store allowance allocation for SSO, and refresh the headless diagnosis path.
Decode ring revisions correctly for Bulletin allowance claims. Add CLI-managed signer accounts, network presets, and script-capable host modes. Regenerate headless signing and pairing diagnosis reports with zero failures.
…ing-bot, per-VU metrics)
…pt mislabel, wire-layout test, doc/fleet cleanup)
eugypalu
force-pushed
the
eugenio/headless-host-stress
branch
from
July 17, 2026 11:04
c9d02de to
59e99a9
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The measurement layer for the headless host, plus a first stress fleet on top of it.
Both pieces live in
truapi-host-cli; no core changes.HostMetricRecord(category and op derived from the wire id, latency, outcome) to a JSONL sink. Recording is opt-in viaMETRICS_JSONL; unset, the recorder is a no-op and behaviour is unchanged. The outcome is decoded from the response frame, so a domain error counts as an error, not a clean dispatch.e2e/fleet.shstarts N virtual users on a ramp, each a pairing host on its own port running a flow script, paired against the signing-bot (POST /api/pair), which auto-provisions an attested user and signs per pairing. All VUs append to one shared JSONL, keyed byvuIndex.