feat(runtime): add Generic WASM and Python Reactor execution

[bkmashiro]

Summary

Add two opt-in wazero execution paths:

• Generic WASM for modules that implement Shimmy's memory / alloc / dispatch ABI.
• Python Reactor for prebuilt CPython/WASI runtimes with a trusted evaluator script.

The PR also includes a safe Python evaluator example, Linux HTTP end-to-end coverage, and a guided quick start.

What changed

• Added the generic profile, including request/response transfer through linear memory, prepared-memory reset, and explicit host path and environment allowlists.
• Added the python-reactor profile and its prepared instance pool.
• Added startup checks for the runtime artifact digest, ABI, imports, exports, and declared Python module capabilities.
• Python Reactor instances initialize once, then restore their full linear memory after every request.
• Discarded instances that grow memory during a request, since WebAssembly memory cannot shrink back to the prepared size.
• Returned request failures and timeouts before replacing discarded slots. Refill runs asynchronously and a failed refill leaves the pool closed rather than reusing a bad instance.
• Added examples/safe-eval-python with demo, stdin/stdout, unit test, and preview modes.
• Bounded stdout, stderr, traceback, and test details as they are written, including multibyte UTF-8 output.
• Added serve.sh and try.sh for starting a profile and exercising it over Shimmy's HTTP interface.

Runtime boundaries

The runtime manifest controls which Python modules an artifact may provide. The evaluator's AST import checks are useful diagnostics, but they are not the security boundary.

Python Reactor does not expose host filesystem access, networking, or process creation by default. Artifact hashes confirm that the deployed files match the manifest; provenance still depends on how the runtime artifact is built and distributed.

Testing

The test suite covers Generic WASM dispatch, capability allowlists, prepared-memory reset, and instance discard after memory growth.

The Linux E2E used the exact Producer base artifact and exercised artifact and ABI validation, Python Reactor over HTTP, all safe evaluator modes, timeout recovery, and the guided base-profile quick start.

A timed-out request returned HTTP 500 in 5.74 seconds. The discarded slot was replaced in the background and a later request completed successfully. Runtime artifacts and receipts were checked against their SHA-256 hashes before the run.

[Copilot]

Pull request overview

This PR adds two opt-in in-process WebAssembly execution paths to Shimmy (Generic WASM and Python Reactor) backed by wazero, including artifact/ABI validation, snapshot-based state reset, and accompanying end-to-end fixtures and onboarding docs/scripts.

Changes:

• Introduces a Generic WASM dispatcher that uses a memory/alloc/dispatch ABI and restores a prepared linear-memory snapshot between requests.
• Adds a Python Reactor dispatcher that validates a Producer artifact + manifest, supports multiple lifecycle modes (snapshot/single-use/fresh), and discards/refills instances after failures or memory drift.
• Adds an artifact checker CLI, Linux HTTP E2E scripts, and a safe-eval-python example (with tests + guided quick start) to demonstrate the Python Reactor profile.

Reviewed changes

Copilot reviewed 44 out of 46 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
tests/e2e/python-reactor/evaluator.py	Linux E2E evaluator fixture for Python Reactor HTTP verification.
scripts/e2e-safe-eval-python.sh	End-to-end script exercising safe-eval-python over HTTP, including timeout recovery.
scripts/e2e-python-reactor.sh	End-to-end script exercising Python Reactor HTTP path and snapshot reset.
README.md	Documents opt-in WASM/Python Reactor paths and clarifies sandboxing constraints (incl. Lambda).
internal/execution/wasm/testhelpers_test.go	Adds shared wazero/fixture helpers for WASM tests.
internal/execution/wasm/testdata/echo.wat	Adds WAT source for the echo ABI fixture used in WASM tests.
internal/execution/wasm/supervisor.go	Implements per-instance snapshot/restore with unhealthy marking on restore failure or memory growth.
internal/execution/wasm/snapshot.go	Adds SnapshotStrategy + FullMemcpyStrategy for linear-memory snapshotting.
internal/execution/wasm/snapshot_test.go	Unit tests for snapshot strategy behavior (roundtrip, drift, ownership, idempotence).
internal/execution/wasm/robustness_test.go	Robustness tests for malformed ABI modules (alloc/dispatch signature issues).
internal/execution/wasm/python_reactor_artifact.go	Inspects and verifies Python Reactor module shape against manifest contract.
internal/execution/wasm/python_preload_config_test.go	Tests default and validation for Python preload mode config.
internal/execution/wasm/pool.go	Adds generic pool drain helper with context cancellation to avoid deadlocks.
internal/execution/wasm/json_util.go	Adds JSON response parsing helper with bounded raw context in errors.
internal/execution/wasm/dispatcher.go	Adds Generic WASM dispatcher with pool management, shutdown coordination, and replacement logic.
internal/execution/wasm/dispatcher_test.go	Comprehensive tests for generic dispatcher behavior, concurrency, shutdown races, and memory-grow handling.
internal/execution/wasm/config.go	Extends WASM config for sandbox allowlists and Python Reactor options.
internal/execution/wasm/artifact_check.go	Adds artifact compilation + ABI validation entrypoint (generic + python-reactor).
internal/execution/wasm/artifact_check_test.go	Tests for artifact checker generic ABI validation behavior.
internal/execution/wasm/agent_python.go	Implements Python Reactor dispatcher with lifecycle modes, snapshot reset, discard/refill, and bounded diagnostics.
internal/execution/wasm/agent_python_test.go	Tests for manifest validation, request/response contracts, lifecycle behavior, and real-artifact compatibility hooks.
internal/execution/wasm/agent_python_protocol.go	Adds pinned consumer-side protocol/manifest parsing + request/response encoding/decoding.
internal/execution/wasm/agent_python_observer.go	Adds phase observer types and emission for lifecycle instrumentation.
internal/execution/wasm/agent_python_lifecycle_config_test.go	Tests lifecycle defaults and invalid lifecycle/capacity bounds.
internal/execution/wasm/adapter.go	Implements the generic alloc/dispatch adapter (host ↔ linear memory transfer).
internal/execution/supervisor/models.go	Adds wasm IO interface constant.
internal/execution/supervisor/config.go	Documents the new wasm interface behavior and selection contract.
internal/execution/dispatcher.go	Routes FUNCTION_INTERFACE=wasm to generic or python-reactor dispatchers based on FUNCTION_WASM_PROFILE.
go.sum	Adds wazero module checksums.
go.mod	Adds wazero dependency.
examples/safe-eval-python/try.sh	Guided HTTP driver for the safe-eval-python example requests.
examples/safe-eval-python/serve.sh	Launcher to validate artifact/manifest and start Shimmy configured for Python Reactor.
examples/safe-eval-python/safe_eval.py	Trusted evaluator implementing demo/io_test/unit_test/preview with bounded output handling.
examples/safe-eval-python/safe_eval_test.py	Unit tests for safe_eval behavior and output bounding.
examples/safe-eval-python/requests/unit-tests.json	Example request payload for unit-test mode.
examples/safe-eval-python/requests/sympy.json	Example request payload for SymPy profile behavior.
examples/safe-eval-python/requests/preview-blocked.json	Example request payload demonstrating preview rejection of blocked imports.
examples/safe-eval-python/requests/numpy-core.json	Example request payload for NumPy profile behavior.
examples/safe-eval-python/requests/io-tests-pass.json	Example request payload for passing IO tests (incl. hidden).
examples/safe-eval-python/requests/io-tests-fail.json	Example request payload for failing IO tests.
examples/safe-eval-python/requests/demo.json	Example request payload for demo mode.
examples/safe-eval-python/README.md	Full quick start + role separation + security boundary explanation for safe-eval-python.
docs/execution-paths.md	Adds end-user docs for Generic WASM and Python Reactor execution paths and verification steps.
cmd/shimmy-artifact-check/main.go	New CLI tool to validate WASM artifacts/manifests without starting Shimmy.
cmd/root.go	Updates CLI help for --interface / --command to include WASM usage.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.