Skip to content

feat: port Rust core runtime#104

Draft
pgherveou wants to merge 29 commits into
mainfrom
worktree-issue-96-rust-core-port
Draft

feat: port Rust core runtime#104
pgherveou wants to merge 29 commits into
mainfrom
worktree-issue-96-rust-core-port

Conversation

@pgherveou
Copy link
Copy Markdown
Collaborator

@pgherveou pgherveou commented May 17, 2026

What this PR ships

The shared-core SDK becomes real: one Rust core runs the TrUAPI protocol, and every host platform (web, Electron, iOS, Android) plus every product (the JS SPA inside the sandbox) consume the same auto-generated artifacts from that single source.

Concretely, after this PR a product engineer can:

import {
  AccountManagementClient,
  createMessagePortProvider,
  createTransport,
} from "@parity/truapi";

const transport = createTransport(createMessagePortProvider(port));
const accounts = new AccountManagementClient(transport);

const result = await accounts.accountGet({
  productAccountId: { dotNsIdentifier: "my-product.dot", derivationIndex: 0 },
});

…and the same call lands in the Rust core that's running inside the host, regardless of whether the host is dotli on web, Electron, an iOS app, or an Android app.

Runtime topology

   Product (iframe / WebView)              Host (web shell / Electron / iOS / Android)
 ┌───────────────────────┐               ┌────────────────────────────────────────┐
 │ @parity/truapi        │   wire bytes  │  ┌──────────────────────────────────┐  │
 │ generated TS client   │ ◄───────────► │  │   truapi-server (Rust core)      │  │
 │                       │  (transport)  │  └──────────────────────────────────┘  │
 └───────────────────────┘               │                  │                     │
                                         │           ┌──────┴──────┐              │
                                         │           │  bindings   │              │
                                         │           │  WASM       │ web/electron │
                                         │           │  UniFFI     │ ios/android  │
                                         │           └──────┬──────┘              │
                                         │                  │                     │
                                         │   ┌──────────────▼──────────────────┐  │
                                         │   │  HostCallbacks                  │  │
                                         │   │  implemented in JS / Swift /    │  │
                                         │   │  Kotlin by the host app         │  │
                                         │   └──────────────┬──────────────────┘  │
                                         │                  ▼                     │
                                         │             OS primitives              │
                                         └────────────────────────────────────────┘

Transports:

  • Web + Electron: MessageChannel. The product opens a port; the host hands it to its core via createMessagePortProvider(port).
  • iOS + Android: a localhost WebSocket bridge. The host calls core.startWsBridge() and forwards the resulting ws://127.0.0.1:<port>/?t=<token> URL to the product, which feeds it to @parity/truapi's createWebSocketProvider(url).

What gets generated, from one Rust source

The contract for every protocol domain (accounts, signing, chain access, chat, statements, payments, …) is a Rust trait in rust/crates/truapi/. Each method carries a stable #[wire(id = N)] annotation. From that single definition the codegen emits:

  • @parity/truapi, the TypeScript product client. Typed methods, typed errors, typed subscription items.
  • A Rust dispatcher (truapi-server/src/generated/) that the core uses to route incoming frames to host trait impls. Wire ids on the dispatcher and on the TS wire table are checked at build time to stay in lockstep.
  • A typed HostCallbacks TypeScript interface mirroring truapi-platform's Rust traits, for web hosts.
  • Kotlin and Swift HostCallbacks interfaces via UniFFI, for Android and iOS hosts.

The TS, Kotlin, and Swift surfaces are derived from Rust — they cannot drift.

        ┌─────────────────────────────────────┐
        │   Rust traits  (single source)      │
        │   truapi (protocol)                 │
        │   truapi-platform (host callbacks)  │
        └─┬───────────────┬────────────────┬──┘
          │               │                │
       codegen         used by          codegen + UniFFI
          │               │                │
          ▼               ▼                ▼
   ┌──────────────┐ ┌─────────────┐ ┌────────────────┐
   │ @parity/     │ │ Rust core   │ │ HostCallbacks  │
   │ truapi (TS)  │ │ runtime     │ │ TS / Kotlin /  │
   └──────────────┘ └─────────────┘ │ Swift          │
                                    └────────────────┘

A new RFC for a protocol method is a reviewable PR against the Rust traits. After it merges, codegen reruns, and every downstream artifact updates together.

The host surface (truapi-platform)

A host's only job is to fulfil the small set of capabilities the Rust core physically cannot do from its own process — open URLs, prompt the user, persist key/value pairs scoped to the origin, deliver push notifications, and open chain connections. That surface is:

  • Storage — scoped key/value persistence
  • Navigation — open URLs
  • Notifications — push notifications
  • Permissions — user permission prompts (device and remote-domain, kept as a two-call split per v0.1)
  • Features — feature-support probing
  • ChainProvider + JsonRpcConnection — chain RPC factory

Account management, signing, statement-store, and preimage flows live in the Rust core itself, backed by Storage for key material and ChainProvider for chain access. They are not part of the host surface. Their in-core implementations are tracked separately; until they land, the corresponding methods on the typed product client resolve to CallError::Unsupported.

What lands in this PR

Rust crates (new):

  • truapi-platform — the capability traits above.
  • truapi-server — the runtime: dispatcher, SCALE codec, subscription lifecycle, chain-runtime (chainHead v1 state machine on top of any JsonRpcConnection, plus an embedded smoldot provider behind a feature flag), localhost WebSocket bridge for native hosts, UniFFI surface for iOS/Android, wasm-bindgen surface for web/Electron with setActiveSession / clearActiveSession for cross-tab session handoff.
  • uniffi-bindgen-cli — workspace tooling to regenerate Kotlin and Swift bindings.

Host SDKs (new):

  • @parity/truapi-host-shared, @parity/truapi-host-web, @parity/truapi-host-electron — the JS host runtime split by integration surface. Pre-built WASM bundles ship with @parity/truapi-host-shared.
  • android/ — Kotlin host shell wrapping the UniFFI bindings (HostBridge, HostStorage, TrUAPIHostCore).
  • ios/TrUAPIHost/ — Swift Package with the matching Swift shell.

Codegen extensions:

  • Rust dispatcher emitter (TS wire-table parity verified by a workspace test).
  • TypeScript HostCallbacks emitter derived from truapi-platform rustdoc — so web hosts get the same typed surface UniFFI gives iOS/Android.

Architecture documentation under docs/design/:

  • dotli-rust-core-proposal.md — recommends running the Rust core inside dotli's stable-origin protocol iframe (host.dot.li), inside a SharedWorker so the core is one-per-browser and storage stays on a stable origin even when product CIDs change.
  • dotli-architecture-change.md — diagrams the transition.

What this PR explicitly does not ship

Scoped out and tracked separately:

  • Account / signing / statement-store / preimage in-core implementations. Their methods on the typed client resolve to Unsupported.
  • npm publish workflow for @parity/truapi-host-*. The packages build and test in CI; the release pipeline is a follow-up.
  • A SCALE-adapter layer that would let the typed generated HostCallbacks fully replace the byte-level WASM bridge glue in @parity/truapi-host-shared. Both surfaces are exported; the adapter is the next step.
  • Native host repos. The android/ and ios/ SDKs are ready to be consumed by Android and iOS host repos as vendored sources or git submodules, the same model hosts/dotli/ uses today.

How to verify

  • make check — full Rust + JS workspace verification.
  • cargo test --workspace --features ws-bridge — 178 tests including the WS-bridge round-trip and the Rust↔TS wire-table parity check.
  • cargo check -p truapi-server --target wasm32-unknown-unknown — the WASM bundle compiles.
  • ./scripts/codegen.sh then git diff — the committed generated TS, dispatcher, and host-callbacks artifacts are byte-identical to what fresh codegen produces.

Closes #96, #97, #98, #99, #100, #101, #102, #103.

Lands the Rust runtime layer behind the TrUAPI protocol: codegen-emitted
dispatcher, platform abstraction crate, server runtime with frame/transport/
dispatcher/subscription, runtime adapter from platform traits, host_logic
helpers, WS bridge for native WebView hosts, UniFFI native bridge, and
wasm-bindgen surface for Web Workers.

New crates:
- truapi-platform: Storage, Navigation, Notifications, Permissions (split
  device/remote callbacks), Features, ChainProvider, JsonRpcConnection,
  Accounts, Signing, StatementStore, Preimage; Platform super-trait.
- truapi-server: TrUApiCore, Dispatcher, Frame (with [requestId][disc][payload]
  envelope), Subscription lifecycle, PlatformRuntimeHost<P> adapter,
  host_logic/{dotns,features,permissions,session}, ws_bridge (feature-gated),
  native UniFFI bridge, wasm32 wasm-bindgen surface.
- uniffi-bindgen-cli: thin wrapper around uniffi::uniffi_bindgen_main()
  for regenerating Kotlin/Swift bindings.

Codegen:
- truapi-codegen --rust-output emits dispatcher.rs and wire_table.rs.
- Methods are keyed by snake_case(trait)_method so StatementStore::submit
  and Preimage::submit no longer collide.
- Responses wrap in SCALE Result discriminant byte ([0x00, ok] / [0x01, err])
  matching the @parity/truapi TS codec.
- CallContext is constructed with the actual message request_id.

truapi crate (additive only):
- Display impls on v01 and versioned permission enums for modal copy and
  storage key construction in host_logic.

Tests: 139 across the workspace (with ws-bridge feature), including WS
round-trip against a real tokio-tungstenite client, frame snapshot,
dotns scheme allowlist (rejects javascript:/file:/data:/vbscript:),
permission cache isolation, session broadcast, dispatcher error path
with Result discriminant.

Relates to #96. Remaining phases tracked in #97-#103.
pgherveou added 15 commits May 17, 2026 08:43
- #98: SCALE-encode + hex the canonical remote permission bundle for the
  storage key instead of joining slugs with separators. Domains containing
  '|', ',', or 'truapi:permissions:' can no longer alias a different
  bundle's grant.
- #99: golden_rust_emit tests use per-test tempdirs (no shared target/doc
  race) and panic loudly when nightly is unavailable instead of silently
  passing.
- #100: strip phase/migration narration from doc comments and README;
  switch Chain trait stubs from CallError::HostFailure to
  CallError::Unsupported.
- #102: replace ProtocolMessage::decode_error / call_error with free
  functions encode_decode_error / encode_call_error_payload returning
  Vec<u8>. Handlers now return Result<Vec<u8>, Vec<u8>>; the dispatcher
  owns envelope construction so a malformed empty-tag frame can no longer
  escape onto the wire.

3 new tests, 142 total. Workspace clean, clippy --all-features clean,
wasm32 target clean.

Relates to #96.
…subscription (#97)

Subscriptions previously parked one OS thread each via
std::thread::spawn(|| block_on(future)). On native this leaked a thread
per active stream; on wasm32 it would panic outright since wasm has no
threads.

The dispatcher now requires a Spawner (Arc<dyn Fn(BoxFuture<'static, ()>) + Send + Sync>)
provided at construction time. Each entry point picks the right executor:

- native.rs: shared futures::executor::ThreadPool (one pool per
  NativeTrUApiCore, default sized to CPU count). Falls back to the
  thread-per-subscription spawner if pool construction fails.
- wasm.rs: wasm_bindgen_futures::spawn_local.
- ws_bridge tests: thread_per_subscription_spawner() helper.

TrUApiCore::new and ::from_platform now take the spawner. Dispatcher::new
takes it too — no Default impl, since an empty Default would silently
regress the leak.

Added the `thread-pool` feature to the non-wasm32 `futures` dep.
Added subscription_uses_provided_spawner_not_native_thread test.
143 tests passing, wasm32 target clean.

Relates to #96.
+33 tests across 7 files covering review gaps:

Runtime delegation (core.rs, +13 tests): round-trip via TrUApiCore for
get_account, get_account_alias, create_account_proof, get_legacy_accounts,
get_user_id, sign_payload, sign_raw, LocalStorage read/write/clear,
push_notification, request_remote_permission, connection_status_subscribe.

Frame internals (frame.rs, +6): id_for_tag / tag_for_id known + unmapped,
compose_action round-trip across every FrameKind, IdFactory monotonic and
two-factory state isolation.

WS bridge (ws_bridge.rs + native.rs, +3): wrong_token_is_rejected_at_handshake,
drop_calls_stop_idempotently, start_ws_bridge_twice_returns_already_running.

Session pruning (session.rs, +2): clear_when_empty_is_silent_no_op,
dropped_subscriber_is_pruned.

Permission error paths (permissions.rs, +3): prompt_failure_collapses_to_denial,
corrupt_cache_entry_returns_none, storage_read_error_propagates.

Codegen negative paths (truapi-codegen/src/rust.rs, +6): wire_table rejects
request-with-subscription-id, subscription-with-request-id, missing IDs;
dispatcher rejects multi-param methods and non-named-root response types.

All via existing public APIs and test helpers — no production shims added.
176 total tests workspace-wide with ws-bridge feature.

Relates to #96.
Lands chainHead-v1 state machine and light-client integration:

- chain_runtime.rs (always compiled): ChainRuntime, RuntimeChainProvider,
  UnavailableChainProvider, json-rpc state machine + follow-event parsing
  into truapi::v01 RemoteChainHeadFollowItem variants. Spawner is threaded
  through to spawn the response loop, matching the dispatcher discipline.

- smoldot_provider/ (feature `smoldot`, off by default): SmoldotChainProvider,
  SmoldotJsonRpcConnection wrapping smoldot-light. Native + wasm32 platform
  glue (websocket transport on wasm). Bundled paseo + asset-hub-paseo
  chainspecs.

All 13 Chain trait methods are now routed through ChainRuntime instead of
returning CallError::Unsupported. PlatformRuntimeHost wraps the host's
ChainProvider into a RuntimeChainProvider via PlatformChainRuntimeProvider.

Tests: 186 passing with --features ws-bridge,smoldot (+10 since previous).
Smoldot adds ~36s to first compile; ~10s incremental. wasm32 target clean
with --features smoldot.

Closes #103 (Phase 4d portion). Relates to #96.
… JS, #103)

Three new npm packages under host-libs/js/ wrapping the truapi-server
WASM core for browser, worker, and electron host contexts.

@parity/host-shared:
- WasmRawCallbacks matching Rust JsBridge::from_js (navigateTo,
  devicePermission, remotePermission, featureSupported, getLegacyAccounts, ...)
- createWasmProvider / createNodeWasmProvider (lazy WASM load)
- Web Worker entrypoint + worker-protocol wire shape
- Re-exports createHostServer / toResponsePayload / toFlatResponsePayload
  from @parity/truapi-host

@parity/host-web:
- createIframeHost: embeds an iframe, handshakes a MessagePort via
  the `truapi-init` message, surfaces the host-side port via onPort.
- createWebWorkerProvider: bridges a Web Worker to a Provider.
- No legacy @novasamatech/host-api compat path.

@parity/host-electron:
- createElectronProvider({port}): wraps an Electron MessagePortMain
  as a Provider.

Pre-built WASM artifacts committed under host-libs/js/shared/dist/wasm/
(web + node targets, ~924 KB each, built with `wasm-pack --no-default-features`,
smoldot feature off in the shipped bundle).

Tests: 5 (shared) + 2 (web) + 2 (electron) = 9 passing via `node --test`.
tsc --noEmit clean across all three packages. No changes to truapi/src/api/*.

Relates to #96, #103.
Two thin language-idiomatic SDKs wrapping the UniFFI bindings emitted
from truapi-server, ready to be consumed by iOS/Android host repos.

host-libs/android/ (Kotlin):
- io.parity.truapi.{TrUAPIHostCore, HostBridge, HostStorage, CoreInbound,
  WebViewTransport} wrapping NativeTrUApiCore via UniFFI
- build.gradle.kts (Android library, JDK 17, JNA 5.14, kotlinx-coroutines)
- AndroidManifest, settings.gradle.kts, gradle.properties for standalone
  assembly; also includable as :host-libs:android from a parent project
- Generated UniFFI bindings under src/main/kotlin/generated/

host-libs/ios/TrUAPIHost/ (Swift Package):
- TrUAPIHost.{TrUAPIHostCore, HostBridge, HostStorageBackend, CoreInbound,
  WebViewTransport, LocalhostBridgeBootstrap}
- Package.swift exposing TrUAPIHost + truapi_serverFFI systemLibrary targets
- Generated UniFFI bindings under Sources/TrUAPIHost/truapi_server.swift
  and Sources/truapi_serverFFI/include/{truapi_serverFFI.h,module.modulemap}

cdylib built with --features ws-bridge so the native surface includes
startWsBridge/stopWsBridge for WebView hosts that prefer the localhost
WS rail.

HostBridge interface keeps the v0.1 device/remote permission split as
two named methods (no merged prompt_permission).

Bindings committed (consumers don't run Rust). 92K Kotlin (2467 LOC),
56K Swift (1744 LOC), 36K C header.

Relates to #96, #103.
Two pre-existing parser bugs in tests/wire_table_ts_parity.rs surfaced
once the TS wire-table.ts file appeared locally:

- parse_ts looked for `method: 'foo'` lines, but the TS codegen emits
  `export const FOO_BAR = { request: N, response: N }` shape.
  parse_ts now reads the const name and lowercases it.
- parse_rust used starts_with("WireKind::Subscription"), but the actual
  line is `kind: WireKind::Subscription {` — starts with "kind:".
  Switched to contains().

Wire tables now confirmed in lockstep across Rust and TS.
Makefile:
- `make wasm` rebuilds the WASM bundle under host-libs/js/shared/dist/wasm/
- `make uniffi` regenerates Kotlin + Swift bindings from a release cdylib
- build / test / check extended to install + test the three host-libs/js
  packages alongside the existing TS client and playground

docs/design/:
- Ported dotli-architecture-change.md and dotli-rust-core-proposal.md
  (the architectural rationale for the WASM-in-worker shared-core move),
  with @parity package names and current repo paths.

.github/workflows/host-libs.yml:
- wasm-bundle-check: rebuilds via `make wasm` and fails if the committed
  bundle drifts.
- host-libs-js-tests: builds the @parity/truapi-host + @parity/truapi
  deps, then runs the three host-libs/js test suites.
- Path-filtered to host-libs/** and rust/crates/truapi-server/**.

README.md + CLAUDE.md:
- Document the new truapi-platform / truapi-server / uniffi-bindgen-cli
  crates, the @parity/host-* packages, the iOS/Android shells, and the
  invariants (truapi crate canonical; types via versioned::*; pre-built
  WASM committed; make wasm / make uniffi to regenerate).

host-libs/js/*/README.md note the npm publish workflow is intentionally
deferred pending release-process discussion.

`make check` runs end-to-end clean.

Closes #97, #98, #99, #100, #101, #102, #103. Relates to #96.
…weaken wasm-bundle-check

- ws_bridge.rs: nightly clippy fires `result_large_err` because the
  handshake callback returns `Result<Response, ErrorResponse>` and
  tokio-tungstenite's `ErrorResponse` carries the full HTTP response
  (~136 bytes). The closure signature is dictated by 3rd-party API;
  silence at the function level with an explanatory comment.
- .github/workflows/host-libs.yml: the wasm-bundle-check job previously
  asserted the CI-rebuilt WASM was byte-identical to the committed
  bundle. wasm-pack output varies with the Rust toolchain and
  wasm-bindgen version, so cross-machine reproducibility isn't
  achievable. Weakened to "build succeeds + artifact files exist".
  The committed bundle stays as a consumer convenience.
Move host SDK SDKs out of the host-libs/ umbrella and rename the JS
packages so they share the @parity/truapi-host- prefix consistently
with @parity/truapi and @parity/truapi-host:

- host-libs/android → android (now an Android library module; standalone
  settings.gradle.kts + gradle.properties dropped — consumers include
  this dir from their own root settings)
- host-libs/ios → ios (keeps the inner TrUAPIHost/ wrapper, so the
  Swift Package root remains at ios/TrUAPIHost/Package.swift)
- host-libs/js/shared → js/packages/truapi-host-shared
  (@parity/host-shared → @parity/truapi-host-shared)
- host-libs/js/web → js/packages/truapi-host-web
  (@parity/host-web → @parity/truapi-host-web)
- host-libs/js/electron → js/packages/truapi-host-electron
  (@parity/host-electron → @parity/truapi-host-electron)
- host-libs/ removed entirely.

Also:

- Inter-package file: deps simplified (now siblings under js/packages/).
- Makefile WASM_DIST + uniffi --out-dir paths updated.
- CI workflow renamed host-libs.yml → host-packages.yml; path filters
  and step labels updated.
- README + CLAUDE layout + design docs + package READMEs scrubbed for
  stale host-libs references.
- chain_runtime::drop_follow_stream_sends_unfollow flake fixed by
  bumping the cleanup poll budget from 1s to 5s (was racing on loaded
  CI runners).

Workspace clippy + tests pass; 3 JS packages tsc + npm test pass.
…rker on host.dot.li

The AFTER diagram in dotli-architecture-change.md previously placed the
truapi-server WASM Worker on the dot.li main thread. That's wrong:

- dot.li (and *.dot.li catchall) → host build → user-visible shell
- <cid>.app.dot.li → sandbox build → product iframes (per-CID origin)
- host.dot.li → protocol build → stable-origin hidden iframe

A Worker spawned from dot.li has stable storage on that origin (good)
but cohabits with paint frames (bad). A Worker spawned from a product
iframe at <cid>.app.dot.li loses its storage on every CID update.

The protocol iframe at host.dot.li is the only origin that's both stable
and UI-free. Constructing a SharedWorker from there gives:

- IndexedDB on host.dot.li (stable across product CID updates)
- Cross-tab single core (replaces the existing BroadcastChannel glue
  for shared auth, folds the standalone smoldot SharedWorker into one)
- No protocol decoding on a paint thread

Updated:
- AFTER diagram in dotli-architecture-change.md now shows the three
  origins, the host shell as a port relay, and the SharedWorker hosting
  the WASM core with embedded smoldot.
- New "Origin model" section explains the nginx routing + why
  host.dot.li is the right script origin for the worker.
- Module-diff section clarifies @parity/truapi-host-{shared,web} roles.
- dotli-rust-core-proposal.md recommendation flipped from Option 1
  (per-tab worker) to Option 2 (SharedWorker), with a fallback note for
  engines without SharedWorker support.
- Migration considerations add the localStorage → IndexedDB migration
  required when state moves off the protocol-iframe main thread.
…t + typealiases

Apply the SDK-team vision (HostCallbacks = OS primitives the Rust core
can't do alone) and drop indirection that wasn't pulling its weight:

- Trim Platform to Storage + Navigation + Notifications + Permissions +
  Features + ChainProvider + JsonRpcConnection. Drop Accounts, Signing,
  StatementStore, Preimage — these belong in the Rust core itself,
  backed by Storage + chain runtime, not as platform traits.
- Drop #[async_trait]; use native `async fn -> impl Future + Send` in
  trait bodies. PlatformRuntimeHost<P> is generic over P: Platform so
  dyn-trait compatibility isn't required.
- Drop the 11 typealiases at the top of platform/src/lib.rs
  (StorageKey/StorageValue/StorageError/NavigateToError/...). Trait
  signatures use the canonical v01 names directly.
- Drop the `async-trait` and `parity-scale-codec` deps from
  truapi-platform/Cargo.toml.

Cascade:

- runtime.rs: remove Account/Signing/StatementStore/Preimage impls
  for PlatformRuntimeHost<P>; truapi::api::* default bodies return
  CallError::Unsupported, which is the right semantic until those
  flows land in-core.
- native.rs, wasm.rs: drop the 4 CallbackPlatform impls and the
  corresponding callback fields on HostCallbacks / JsBridge.
- host_logic/permissions.rs: PermissionsService<S, P> generic, peek()
  takes &impl Storage. HostLocalStorageReadError used directly.
- chain_runtime.rs, smoldot_provider/mod.rs: drop GenesisHash type
  alias use; switch to Vec<u8> / &[u8].
- All test stubs drop #[async_trait]; the 7 round-trip tests that
  asserted specific Domain returns from the removed Platform traits
  are deleted (their replacement would be a one-line Unsupported
  assertion, already covered by request_login_returns_unsupported).
- Regenerated UniFFI Kotlin + Swift bindings.

CLAUDE.md adds two guidelines:
1. Don't introduce typealias chains that just rename a public type
   from another crate.
2. Update README.md after every code change so the top-level docs
   reflect what the repo actually contains.

Verification:
- cargo +nightly fmt --check, clippy --all-features, test --features
  ws-bridge: clean
- 177 tests passing (was 184; 7 removed-trait round-trip tests
  deleted)
- wasm32 target builds clean
Products running in a WebView/WKWebView now connect to the Rust core
through the localhost WebSocket bridge instead of a base64-over-
JavascriptInterface (Android) / base64-over-WKScriptMessageHandler (iOS)
shim. The Rust ws_bridge already existed; this commit adds the
matching JS client and strips the obsolete native transport.

@parity/truapi:
- New `createWebSocketProvider(url, opts?)` producing a `Provider` over
  a binary WebSocket. Connects to the localhost endpoint the native
  shell prints via `startWsBridge`.

android/TrUAPIHost.kt:
- Drop `WebViewTransport`, `CoreInbound`, `bootstrapScript`, and all
  base64/JavascriptInterface plumbing.
- `TrUAPIHostCore` keeps its `receiveFromProduct` entrypoint for tests
  and alternative transports; the WS bridge feeds the core internally.

ios/TrUAPIHost.swift:
- Drop `WebViewTransport` (`WKScriptMessageHandler` + base64 path) and
  `CoreInbound` protocol. Drop the `WebKit` import.
- Keep `LocalhostBridgeBootstrap` — its purpose is exactly to publish
  the WS endpoint to the product page so it can call
  `createWebSocketProvider(url)`.

READMEs for both native shells now describe the WS-bridge flow:
1. Host calls `core.startWsBridge()` → gets port + token.
2. Inject the endpoint into the product page (Android: query string;
   iOS: `LocalhostBridgeBootstrap.script()` as a WKUserScript).
3. Product JS reads the URL and passes it to `createWebSocketProvider`.

@parity/truapi build + tests pass with the new export.
…tform (#18)

UniFFI already gives Kotlin (Android) and Swift (iOS) trait surfaces
straight from `truapi-platform`. Web hosts had hand-written TS types
in `@parity/truapi-host-shared/src/runtime.ts` that mirrored the Rust
traits manually — drift hazard. Codegen now covers TS too so all three
platforms track the same Rust source.

What's new:

- `truapi-codegen` gains a platform-crate rustdoc parser
  (`src/platform.rs`, ~470 LOC). Walks each `pub trait`, classifies
  method return types (`Unit` / `Result<...>` / `BoxStream` / `Box<dyn T>` /
  plain), records `impl Future + Send` returns as async. Detects the
  `Platform` super-trait (method-less, composed of other local traits)
  and preserves the composition order.
- New TS emitter `src/ts/host_callbacks.rs` (~250 LOC) emits one
  `export interface` per Rust capability trait plus a composite
  `HostCallbacks extends Storage, Navigation, ...` interface that
  mirrors `Platform`. Types resolve to existing imports from
  `@parity/truapi`.
- CLI flags `--platform-input` and `--platform-ts-output` wire the
  emitter into the workflow. Existing `--input`/`--output`/`--host-output`/
  `--rust-output` flags are unchanged.
- `scripts/codegen.sh` runs `cargo +nightly rustdoc -p truapi-platform`
  alongside the existing `-p truapi` step and emits to
  `js/packages/truapi-host-shared/src/generated/host-callbacks.ts`.

Consumer integration in `@parity/truapi-host-shared`:

- `src/generated/` and `dist/generated/` added to `.gitignore` (matches
  the `@parity/truapi` policy).
- `runtime.ts` re-exports the typed surface (`HostCallbacks`,
  `Storage`, `Navigation`, ...) from the generated file.
  `WasmRawCallbacks` stays in place — it's the byte-level WASM bridge
  surface and additionally covers core-internal account/sign/statement-
  store callbacks that aren't part of `truapi-platform`. A short
  doc-block makes the typed-vs-raw layering explicit. Bridging the two
  via a SCALE adapter is a deliberate follow-up.

Tests:
- `golden_host_callbacks_ts` integration test pairs rustdoc-of-both-
  crates with the codegen binary and diffs against a checked-in golden.
- 178 workspace tests passing (was 177; +1 for the new golden).
- wasm32 target builds clean.
- `npm install && npm run build && npm test` clean in
  `@parity/truapi-host-shared`.
…emitter

The new TS HostCallbacks emitter from #18 hand-chained 15 writeln! calls
to build the output. CLAUDE.md (this commit also) now requires codegen
modules to prefer indoc::writedoc! / formatdoc! over writeln! chains so
the emitted shape stays visible in source.

Switched host_callbacks.rs to:
- writedoc! for the header + import block (two multi-line writes)
- formatdoc! returning strings for trait interfaces, methods, and the
  super-interface; the parent function joins them
- render_jsdoc returns a String instead of writing through a sink

writeln! count: 15 → 2 (both inside writedoc! invocations themselves).
Golden test rebuilt clean; workspace tests all pass.
@pgherveou pgherveou marked this pull request as draft May 17, 2026 16:22
…ns, in PR descriptions

Adds a guideline that PR descriptions, issue comments, and similar
artifacts that survive after squash-merge should describe what the
system *does* after the change, not the diff between commits.
"Previously X, now Y" and "the old shim is gone" framing reads as
ephemeral history once the PR lands and the writer is no longer in
the picture. Commit messages remain the place for transition narrative;
they stay readable via `git log` after the squash.
pgherveou added 4 commits May 18, 2026 08:01
Bug fixes:
- Makefile uniffi target now writes Swift bindings through a tempdir
  and copies into the actual SwiftPM layout (Sources/TrUAPIHost/ for
  the .swift file, Sources/truapi_serverFFI/include/ for the C header
  and module.modulemap). Cdylib path picks .dylib on macOS.
- electron provider: onClose sets disposed=true and clears listener
  sets so subsequent postMessage no-ops cleanly and close listeners
  don't fire twice if the caller also calls dispose().
- web worker provider: init-failure paths call worker.terminate()
  before rejecting so a failed core init doesn't leak the worker.
- createWebSocketProvider: subscribe / subscribeClose use Set instead
  of Array+indexOf so double-registration of the same callback is
  idempotent (matches the other providers in the package).

Test coverage:
- New js/packages/truapi/test/ws-provider.test.mjs exercises the WS
  provider against a stubbed WebSocket constructor (queue+flush,
  fan-out, set semantics, close fan-out, post-after-close throws).

Doc + style fixes:
- docs/design/dotli-architecture-change.md describes featureSupported
  as a current callback with a planned removal, matching the platform
  trait and the host shells. The implementation-vs-doc contradiction
  is gone.
- Migration narrative stripped from runtime.rs, wasm.rs, native.rs,
  host_logic/features.rs, the Kotlin shell, and the Swift shell.
- Em-dashes swept from 18 PR-scope files (CLAUDE.md, both design
  docs, both native READMEs, the TS host packages, a few platform
  doc comments).
- create-iframe-host.ts comment now describes current behaviour
  without referencing the "legacy fallback".
- tests/object_safety.rs renamed to tests/bounds.rs with a docstring
  that matches what the test actually asserts (the trait isn't
  object-safe; the bound check is on Send + Sync + 'static).
- iOS: drop dead `_ = self.callbackRetainer` line; explain the
  retainer in plain prose instead of via a no-op suppression.
- Android: drop redundant @Suppress("unused") on a property that is
  read in init.

Deferred to follow-up issues (#105-#111).
…n artifact

The Android library is now consumed the same way the iOS Swift Package
is: a git tag in this monorepo. JitPack builds the artifact on demand
when a consumer pulls the tag; no Maven Central account, GPG signing,
or org-internal artifact server required.

Consumer integration:

    repositories {
        maven { url = uri("https://jitpack.io") }
    }
    dependencies {
        implementation("com.github.paritytech.truapi:truapi-android:0.1.0")
    }

Layout:

- `settings.gradle.kts` at the repo root declares `:truapi-android`
  pointing at `android/`.
- `build.gradle.kts` at the repo root pins the AGP + Kotlin plugin
  versions.
- `gradle.properties` carries the workspace Gradle settings.
- `jitpack.yml` tells JitPack which JDK to use and which Gradle task
  produces the publication.
- `android/build.gradle.kts` applies `maven-publish`, declares the
  `io.parity:truapi-host-android` publication with full POM metadata
  (name, description, licenses, SCM, developers), and exposes sources
  + javadoc jars. JNA is an `api` dependency so consumers don't need
  to redeclare it.
- `android/consumer-rules.pro` keeps `uniffi.truapi_server.*` and
  `io.parity.truapi.*` reachable through R8/ProGuard.

CI:

- `host-packages.yml` gets an `android-assemble` job that runs
  `gradle :truapi-android:assembleRelease` and
  `publishReleasePublicationToMavenLocal` on every PR touching
  `android/` or `truapi-server/`, then verifies the AAR + POM +
  sources jar landed in `~/.m2`.

Release flow: tag the commit `truapi-host-android@<version>` after
bumping `publicationVersion` in `android/build.gradle.kts`. JitPack
picks up the tag on first consumer fetch.

Native cdylib is the consumer's responsibility: cross-build
`libtruapi_server.so` per ABI with `cargo-ndk` and drop into
`src/main/jniLibs/<abi>/`. README documents the flow. Pre-built ABI
bundles inside the AAR are tracked as a follow-up.
…host/`

Layout becomes parallel and lowercase:

    android/
      truapi-host/      io.parity:truapi-host-android (Maven library)
    ios/
      truapi-host/      TrUAPIHost (SwiftPM package)

Leaves room for additional native packages alongside (e.g.
`android/widgets/`, `ios/something-else/`) without re-shaping the
top-level directory layout.

Gradle subproject renames from `:truapi-android` to `:truapi-host`; the
JitPack consumer coordinate becomes
`com.github.paritytech.truapi:truapi-host:<tag>`. The Maven artifactId
stays `truapi-host-android` (Maven sees no platform path context, so the
suffix earns its keep against a hypothetical future
`truapi-host-jvm`/etc.).

Path updates everywhere they're referenced: settings.gradle.kts,
build.gradle.kts at root, jitpack.yml, Makefile, host-packages.yml CI,
README.md, CLAUDE.md, uniffi-bindgen-cli README, android/truapi-host
README, .gitignore.

Also align with polkadot-app-android-v2 conventions:
- minSdk bumped to 29 (their floor).
- README documents JNA as a transitive (~1.5MB; they don't currently
  carry it).
- README's cross-build section leads with the
  `mozilla-rust-android-gradle` plugin recipe (their existing toolchain)
  and keeps `cargo-ndk` as the standalone option.
@pgherveou pgherveou changed the title feat: port Rust core runtime from truapi_next prototype feat: port Rust core runtime May 18, 2026
pgherveou added 2 commits May 29, 2026 14:54
Reconcile the Rust core runtime port with main's evolved truapi surface.

main realigned the canonical truapi protocol while this branch was open:
- RemotePermissionRequest carries a single `permission`, not a `permissions`
  bundle, so the runtime's permission storage-key canonicalization operates on
  one permission (domain lists still sorted for stable, injection-safe keys).
- GenericError is a plain struct `{ reason }`; the runtime constructs it
  directly.
- The protocol drops the JsonRpc capability, moves push notifications into a
  dedicated Notifications trait (send/cancel with host-assigned ids and
  scheduling per RFC 0019), and restructures Payment (request/response rename
  plus top-up and balance/status subscriptions) alongside a new CoinPayment
  capability.

The runtime tracks that surface: CoinPayment and the remaining
platform-unmodeled capabilities resolve to their default `unavailable` bodies,
and Notifications send/cancel return `unavailable` since the v0.1 platform
contract models neither notification ids, cancellation, nor scheduling. The
generated dispatcher, wire-table, codegen goldens, and TypeScript client are
regenerated from the merged trait surface; Rust↔TS wire ids agree.

The dotli submodule advances to main's pointer.
…kages

The `@parity/truapi*` packages form a build DAG (truapi → truapi-host →
truapi-host-shared → {truapi-host-web, truapi-host-electron}). Each package's
tsconfig is now a `composite` project that `references` the packages it
imports, and every `build` script runs `tsc -b`, so building any package first
builds its dependencies' `dist/*.d.ts` in topological order.

This makes a cold `npm ci` reliable: previously the workspace `prepare` builds
ran a bare `tsc` concurrently, so a dependent package (e.g. truapi-host-electron)
could compile against `@parity/truapi` before that sibling's `dist/` existed and
fail with `TS2307: Cannot find module '@parity/truapi'`. With `tsc -b` the
dependency is always built first, and TypeScript's `.tsbuildinfo` up-to-date
checks make the concurrent prepare invocations safe.
pgherveou and others added 6 commits May 29, 2026 15:35
Harden the Rust core runtime and host SDKs against the review of the core
port. No change to the wire protocol or the generated artifacts (codegen
regenerates byte-identically; the committed WASM bundle is rebuilt).

truapi-server:
- Subscriptions use generation-stamped reservation slots: a reused or raced
  request_id stops-and-replaces instead of leaking an unstoppable stream, and
  a _stop arriving before activation cancels the pending subscription.
  Unregistered methods answer Unsupported instead of leaving the caller to
  hang. Dead negotiated_version slot and interrupt API removed.
- chainHead-v1 runtime: single-flight follow setup (no duplicate or leaked
  remote subscriptions), the close-vs-insert race in request_value is closed
  so a caller can no longer hang forever, the pending follow-event buffer is
  bounded, and an orphaned remote follow is unfollowed when its local follow
  is torn down mid-setup.
- WS bridge: constant-time token comparison, bounded inbound message size,
  bounded outbound queue and connection count, documented loopback trust
  model.
- wasm surface registers a panic hook and surfaces non-boolean / non-string
  host returns as errors; permissions persist only genuine user decisions, so
  a transient prompt-callback error no longer locks a capability out; dotns
  classifies trailing-dot FQDNs and lowercases/de-dupes remote domains;
  navigate_to returns an error instead of panicking on a contract violation.
- smoldot provider is wired into the runtime behind its feature (with a
  platform fallback), with finite json-rpc ceilings, a distinguishable
  reconnect error, and Apache-2.0 attribution (SPDX headers +
  THIRD_PARTY_NOTICES.md).
- Tests: subscription-race regressions, malformed-frame drop, subscription
  lifecycle through the wire boundary, and a hardened Rust<->TS wire-table
  parity check (panics on unparseable ids, compares all four subscription
  ids, asserts a row-count floor).

codegen: Rust dispatcher/wire-table emitters use writedoc!/formatdoc!
(byte-identical output); TS host-callbacks handles bare trait-object returns.

JS hosts: worker-runtime faults now reach close subscribers; the WebSocket
provider is built on the shared close-once base provider; the electron
provider detaches port listeners on remote close; new tests for the node
WASM round-trip, WS error/non-binary/CLOSING paths, and the iframe
handshake/origin/dispose paths.

native: Android and iOS HostBridge document that callbacks fire on the
truapi-ws-bridge worker thread; the iOS bootstrap uses a JSONEncoder-based JS
string-literal escaper; Maven coordinates reconciled across build.gradle.kts,
jitpack.yml, and the README.

CI/docs: host-packages CI runs the host-shared smoke test against the
freshly built WASM bundle and triggers on truapi-platform changes; the
Makefile wasm target no longer hides the committed bundle; design docs and
the truapi-platform README are reconciled to the shipped trait set.
Native hosts only use the localhost WebSocket bridge, where the core writes
outbound frames straight to the socket, so the direct
receiveFromProduct/onCoreResponse delivery path was dead weight on the UniFFI
surface.

- truapi-server: remove HostCallbacks::on_core_response, NativeCallbackTransport,
  and NativeTrUApiCore::{receive_from_product, debug_smoke_feature_request_frame}.
- Regenerate the UniFFI Kotlin + Swift bindings to drop those symbols.
- Android + iOS shells: drop HostBridge.onCoreResponse and the
  receiveFromProduct / debugSmokeFeatureRequestFrame wrappers; trim the README
  examples.

Web (WasmTrUApiCore.emitFrame / receiveFromProduct) is unchanged — it is the
primary delivery path there — as is the in-process
TrUApiCore::receive_from_product convenience used by the tests.
Merge @parity/truapi-host-{shared,web,electron} into one package,
@parity/truapi-host-wasm, with tree-shakeable subpath entries:
  .               core: createWasmProvider, createNodeWasmProvider, createHostServer, types
  ./web           createIframeHost, createWebWorkerProvider
  ./electron      createElectronProvider
  ./node-runtime · ./worker-runtime · ./wasm/{web,node}

@parity/truapi-host (no shared core) is unchanged. The ./web and ./electron
entries are side-effect-free and statically pull no core/node-runtime/wasm at
the value level, so bundlers tree-shake them; the `sideEffects` field scopes
the genuinely-effectful worker entry and the wasm glue. The wasm subpath
exports carry a `types` condition for typed consumers.

Updates codegen.sh (--platform-ts-output + prettier + echoes), the Makefile,
host-packages.yml (path filters + jobs), the committed WASM bundle location,
README/CLAUDE.md, the design docs, and package-lock.json to the new package.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Port Rust core (truapi-server + host runtime SDKs) from truapi_next prototype

1 participant