Skip to content

FE-1341: Give experiment compute backends one interface - #9178

Draft
kube wants to merge 3 commits into
cf/fe-1237-place-token-capacityfrom
cf/fe-1341-swappable-experiment-backends
Draft

FE-1341: Give experiment compute backends one interface#9178
kube wants to merge 3 commits into
cf/fe-1237-place-token-capacityfrom
cf/fe-1341-swappable-experiment-backends

Conversation

@kube

@kube kube commented Aug 8, 2026

Copy link
Copy Markdown
Collaborator

🌟 What is the purpose of this PR?

Adds @hashintel/petrinaut-core/experiments: one interface both compute backends satisfy, and a registry that picks between them.

ExperimentsProvider currently hardcodes the choice. #9179 registers a backend against this interface instead of rewiring the provider.

Refactor only. No behaviour change, no changeset.

🔗 Related links

🚫 Blocked by

🔍 What does this change?

Both paths already produce a MonteCarloExperiment, and consumers drive one with no branching. That half is untouched. This adds four types:

Type What it is
ExperimentRequest What to compute, as serializable data. No worker factory, no ODE method, no signal, no callbacks: those describe how, and belong to backend construction or per-call options
ExperimentAssessment A backend's answer to "can you run this?", as a value. Blockers carry code, optional itemId, and an origin
ExperimentBackend assess settles the net without acquiring a device or worker pool, then returns an instantiate() closure carrying that work forward
selectExperimentBackend Walks registrations in preference order, records each refusal, treats a failed instantiation as a refusal

Blocker origins drive what a UI does:

Origin Action
model Edit the net
configuration Edit the experiment
environment Hide the option
capacity Retry, or use fewer runs
flowchart TD
  R[registrations] --> L[load] --> A{assess}
  A -->|blockers| N[record refusal] --> R
  A -->|eligible| I{instantiate}
  I -->|environment / capacity| N
  I -->|handle| D[run]
Loading

load is deferred, so a heavy backend stays out of a bundle that never uses it.

Adds content/experiments/backend-selection.mdx and the core.experiments layer.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

  • does not modify any publishable blocks or libraries, or modifications do not need publishing

📜 Does this require a change to the docs?

The changes in this PR:

  • require changes to docs which are made as part of this PR

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

  • do not affect the execution graph

⚠️ Known issues

One backend is registered here, so assess never refuses and the blocker types are unexercised until #9179.

Worker pools and GPUDevice are still per experiment. Reuse depends on the worker resetting frame and metric state; if it does not, results stay plausible rather than visibly wrong.

🐾 Next steps

Supply backends through a React context, so an embedder can register its own.

🛡 What tests cover this?

select-experiment-backend.test.ts: 9 cases covering preference order, deferred loading, refusal recording, instantiation failure, blocker ordering, unavailable backends, module load failure, and per-requirement request building.

❓ How to test this?

Run an experiment. Behaviour is unchanged.

@vercel

vercel Bot commented Aug 8, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
hash Ready Ready Preview Aug 15, 2026 1:13am
petrinaut Error Error Aug 15, 2026 1:13am
1 Skipped Deployment
Project Deployment Actions Updated (UTC)
hashdotdesign-tokens Ignored Ignored Preview Aug 15, 2026 1:13am

kube added 2 commits August 15, 2026 02:50
Choosing a backend was hardcoded in ExperimentsProvider. The runtime
half of swappability already worked — a backend produces a
MonteCarloExperiment and consumers drive one with no branching — so
this adds only the missing part: asking a backend whether it can run a
net, and choosing between backends when one declines.

A request is plain serializable data; anything describing how to
compute belongs to the backend's construction or to per-call options.
Refusal is a value carrying structured blockers with a code, an item
id and an origin, so a UI can attribute a problem to the item that
caused it. Assessment settles the net without acquiring a device or
worker pool and hands back an instantiate() closure.

Pure refactor: no user-visible behaviour changes. Registrations carry
a deferred load so a heavy backend need not enter the bundle.
Replaces em dashes and drops 'deliberately' in
backend-selection.mdx. No content change.
Rewrites the module docstrings in src/experiments to state what each
file is before why it is that way, and replaces em dashes. No code
change.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area/libs Relates to first-party libraries/crates/packages (area) type/eng > backend Owned by the @backend team type/eng > frontend Owned by the @frontend team

Development

Successfully merging this pull request may close these issues.

1 participant