FE-1408: Run a trial as multiple seeded simulations aggregated by mean

[kube]

🌟 What is the purpose of this PR?

Optimization trials currently run one simulation with the fixed execution.seed, so a stochastic net is scored on a single replicate. This PR makes the Petrinaut CLI run a trial's execution.seedsPerTrial seeded simulations — sequentially — and aggregate the per-seed objectives by mean, keeping the {objective} response shape so the Python service needs no logic changes. Simulator side of FE-1273; aggregation-method selection (FE-1277) and Optuna-side handling of per-seed values (FE-1281) are out of scope.

Parallelising the seeded runs is deliberately not done here: it will come through the shared experiment-backend interface (FE-1341/#9178) so the CLI and the editor share one worker story instead of a CLI-specific pool.

Stack #9226: FE-1410 (contract) → FE-1411 (async-safe protocol) → this PR → arch-docs manual → FE-1270 (Python bindings) → FE-1412 (Python client support).

🔗 Related links

• FE-1408 (internal) — this PR
• FE-1273 (internal) — parent: M seeded runs per Optuna step
• FE-1341 (internal) — the shared experiment-backend interface future parallelism builds on
• FE-1429 (internal) — follow-up: run these replicates through that backend
• FE-1277 (internal) — follow-up: user-selectable aggregation

🔍 What does this change?

@hashintel/petrinaut-cli only:

• optimization.evaluate runs the trial's seeds sequentially and returns their mean objective plus a replicates: [{ seed, objective }] array; optimization.describe reports study.seedsPerTrial. A trial's wall-clock time grows linearly with seedsPerTrial.
• Seeds derive once per study: replicate 0 keeps execution.seed (M=1 stays bit-identical to today, and one editor run still reproduces a trial replicate), replicate i uses the exported Monte Carlo deriveRunSeed(seed, i). Every trial reuses the same list → common random numbers across Optuna steps.
• A new built-CLI smoke test spawns the real bundled dist/cli.js and checks the full describe/evaluate exchange, including a two-seed trial.

Pre-Merge Checklist 🚀

🚢 Has this modified a publishable library?

This PR:

• modifies a workspace but not a publishable library (@hashintel/petrinaut-cli is private; the @hashintel/petrinaut-core changeset landed with FE-1410)

📜 Does this require a change to the docs?

The changes in this PR:

• require changes to docs which are made as part of this PR
  • OPTIMIZATION_INTEGRATION.md and the CLI README document the field, seed derivation, and response shape. The in-app user guide is untouched: the UI never sets seedsPerTrial yet, so in-app behaviour is unchanged until FE-1273's UI half lands.

🕸️ Does this require a change to the Turbo Graph?

The changes in this PR:

• affected the execution graph, and the turbo.json's have been updated to reflect this
  • petrinaut-cli's test:unit now depends on its own build, so the built-CLI smoke test always runs against a fresh bundle in CI.

🐾 Next steps

• FE-1412 (top of stack): Python client support — the per-response timeout scales with seedsPerTrial, which sequential execution makes essential.
• Parallel execution of the seeded runs via the shared experiment backend once FE-1341 lands — tracked as FE-1429.
• FE-1277 / FE-1281 / FE-1273 UI half: aggregation selection, Optuna-side per-seed handling, drawer controls.

🛡 What tests cover this?

• optimization.test.ts — mean aggregation, per-seed replicates, identical seed list across trials (CRN), non-finite replicate rejection, deriveTrialSeeds range/stability/uniqueness.
• transports.test.ts — end-to-end multi-seed evaluate over stdio, seedsPerTrial in describe.
• built-cli.test.ts — spawns the real bundled CLI and checks the full describe/evaluate exchange with two seeds.

❓ How to test this?

1. turbo run test:unit --filter @hashintel/petrinaut-cli
2. Manual: add "seedsPerTrial": 4 to execution in libs/@hashintel/petrinaut-cli/examples/supply-chain-profit-optimization.json, yarn workspace @hashintel/petrinaut-cli build, then node libs/@hashintel/petrinaut-cli/dist/cli.js serve --optimization <manifest> --stdio and send an optimization.evaluate request.
3. Confirm the response carries the mean objective plus four replicates, and that repeating the request returns identical values.

🤖 Generated with Claude Code

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
hash	Ready	Preview	Aug 18, 2026 2:08am

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
hashdotdesign-tokens	Ignored	Preview	Aug 18, 2026 2:08am
petrinaut	Skipped		Aug 18, 2026 2:08am

[cursor]

PR Summary

Medium Risk
Changes the optimization evaluate path and trial latency scales with seedsPerTrial, but default behavior stays single-seed and the change is covered by unit, transport, and built-CLI tests.

Overview
Petrinaut CLI optimization trials can now score stochastic models with execution.seedsPerTrial (default 1). Each optimization.evaluate runs that many simulations sequentially with a deterministic seed list (first run uses execution.seed; later runs use the same Monte Carlo derivation as experiments), reusing the same seeds across trials for common random numbers.

The returned objective is the mean of per-seed objectives (online averaging to avoid overflow). When seedsPerTrial > 1, the response also includes replicates: [{ seed, objective }]; optimization.describe exposes study.seedsPerTrial. Invalid/non-finite objectives fail the trial before remaining seeds run.

Docs (OPTIMIZATION_INTEGRATION.md, README) describe the contract and timeout implications. test:unit now depends on build so a new built-cli.test.ts smoke test exercises the shipped bundle end-to-end.

^{Reviewed by Cursor Bugbot for commit 1e8fcb6. Bugbot is set up for automated code reviews on this repo. Configure here.}

[Copilot]

Pull request overview

Adds deterministic multi-seed optimization trials to the Petrinaut CLI, aggregating replicate objectives by mean.

Changes:

• Runs sequential simulations using reusable derived seeds.
• Returns mean objectives and per-seed replicate results.
• Adds protocol, bundle, and unit coverage plus documentation.

Reviewed changes

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
turbo.json	Builds CLI before unit tests.
optimization.ts	Implements seeded trial evaluation and aggregation.
optimization.test.ts	Tests seeds, aggregation, and validation.
transports.test.ts	Tests multi-seed stdio responses.
built-cli.test.ts	Adds bundled CLI smoke test.
README.md	Documents multi-seed evaluation.
OPTIMIZATION_INTEGRATION.md	Documents protocol and seed derivation.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (1)

libs/@hashintel/petrinaut-cli/src/runtime/optimization.ts:275

• The online update still overflows for opposite-signed finite values: [Number.MAX_VALUE, -Number.MAX_VALUE] computes -MAX_VALUE - MAX_VALUE on the second iteration and returns -Infinity, although the mean is 0. The transports then serialize that objective as null (src/commands/stdio.ts:57). Use a sign-aware update that avoids both overflowing sums and differences, and cover this opposite-sign boundary case.

      const objective = replicates.reduce(
        (mean, replicate, index) =>
          mean + (replicate.objective - mean) / (index + 1),

[Copilot]

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated no new comments.

Suppressed comments (1)

libs/@hashintel/petrinaut-cli/src/runtime/optimization.ts:275

• This online-mean formula still overflows for finite objectives with opposite signs: [Number.MAX_VALUE, -Number.MAX_VALUE] makes replicate.objective - mean equal -Infinity, so the protocol returns a non-finite aggregate (serialized as null) instead of the correct mean 0. Use a same-sign incremental update and a weighted form for opposite signs, and cover this boundary case alongside the existing overflow test.

      const objective = replicates.reduce(
        (mean, replicate, index) =>
          mean + (replicate.objective - mean) / (index + 1),