diff --git a/CHANGELOG.md b/CHANGELOG.md index cc30f34f..512a0c0b 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,24 @@ ## Unreleased +## 0.124.0 + +### The analyst can be a tool-equipped agent, and graphs watch their workers + +An `analyzes` edge may now name a graph NODE as its analyst (`analyst: ''`). +On each matching settle, `runGraph` spawns that node's pinned profile as a real WORKER through the same spawn machinery every worker uses (`Scope.spawn` + the `makeWorkerAgent` seam): its task is the edge's registry directive plus the settled worker's tool-trace evidence, its spend reserves from the graph's one conserved budget, its node is journaled and traced like any worker, and its settle OUTPUT is the findings — published and routed per `to` exactly like registry-analyst findings, with the same ledger rows and canonicalization. +Oracle doctrine holds structurally: an analyst node with a delegates edge pointing at it is refused, the driver cannot spawn it (`spawn_agent` still rejects non-worker nodes), an id living in both the registry and the nodes is refused as ambiguous, and an analyzes edge OVER an analyst node is refused because it would silently never fire. + +- `AnalyzeOnSettleRoute` gains `agent?: AgentProfile` — the coordination-layer form of the node analyst, usable by direct `supervise()` callers; lens routes still require the `analysts` registry, agent routes do not. +- `WorkerSpawnContext` gains `analyst?: string`, the runtime-authored marker a node-pinning `makeWorkerAgent` reads to admit an analyst run it would refuse as a driver-authored spawn. +- An analyst run's settlement never enters the settled-worker ledger or the finalizer and never re-fires the analyst hook, so an analyst cannot cascade onto itself; a refused analyst spawn publishes `{ analystSpawnRefused }` and a failed run `{ analystRunFailed }` as findings — observable, never silent. + +`RunGraphOptions` gains `watchWorkers` (mirroring `SuperviseOptions.watchWorkers`): the online detector panel now runs under `runGraph` with no leaf-seam wiring, raising `finding` events on the coordination bus the moment a live worker loops or error-storms. +`examples/graphs/watchdog-steer.ts` now uses the passthrough, and the new `examples/graphs/analyst-agent-review.ts` shows a tool-equipped reviewer node analyzing an implementer. +The kernel entry additionally exports the `WorkerWatchOptions` type. + +Known limit: durable-run resume does not yet compose with analyst-node graphs — an analyst spawned by a prior process settles as an ordinary worker on resume (documented at the in-flight map in `coordination.ts`). + ## 0.123.0 ### Current shared contracts and honest CLI accounting diff --git a/docs/api/index.md b/docs/api/index.md index f7277903..429e0ef5 100644 --- a/docs/api/index.md +++ b/docs/api/index.md @@ -8403,7 +8403,7 @@ work: it returns the committed result on `prior` (see `SpawnOpts.key`). ###### agent -[`Agent`](runtime.md#agent-1)\<`unknown`, `C`\> \| (() => [`Agent`](runtime.md#agent-1)\<`unknown`, `C`\>) +[`Agent`](runtime.md#agent-2)\<`unknown`, `C`\> \| (() => [`Agent`](runtime.md#agent-2)\<`unknown`, `C`\>) ###### task @@ -8608,7 +8608,7 @@ live `RootHandle` (the Q2 substrate the chat/pi-viz client later consumes). ###### root -[`Agent`](runtime.md#agent-1)\<`Task`, `Out`\> +[`Agent`](runtime.md#agent-2)\<`Task`, `Out`\> ###### task diff --git a/docs/api/mcp.md b/docs/api/mcp.md index 5f89741e..7df447e7 100644 --- a/docs/api/mcp.md +++ b/docs/api/mcp.md @@ -4103,8 +4103,11 @@ Analyst lenses run AUTOMATICALLY when a worker settles `done` (the analyst-on-se [AnalyzeOnSettleRoute](runtime.md#analyzeonsettleroute) generalizes the DESTINATION: findings can be delivered to a named live WORKER (wrapped in the route's directive, through the same authorized steer machinery a driver steer uses) instead of being hardwired to the spawning driver, and `over` - restricts which settled workers feed the lens. Omit/empty = no auto-analysis (default; the - driver can still run lenses on demand via `run_analyst`). Requires `analysts`. + restricts which settled workers feed the lens. A route carrying `agent` generalizes the + ANALYST itself: a tool-equipped agent spawned as a worker whose settle output is the + findings (see [AnalyzeOnSettleRoute.agent](runtime.md#agent)). Omit/empty = no auto-analysis (default; + the driver can still run lenses on demand via `run_analyst`). Lens routes require + `analysts`; agent routes do not. ##### maxLiveWorkers? @@ -4133,7 +4136,7 @@ Max wall-clock ms a single `await_event` call may block waiting on a live worker ##### watchWorkers? -> `readonly` `optional` **watchWorkers?**: [`WorkerWatchOptions`](#workerwatchoptions) +> `readonly` `optional` **watchWorkers?**: [`WorkerWatchOptions`](runtime.md#workerwatchoptions) OPT-IN: run the ONLINE detector panel over each spawned worker's live tool trace and raise a `finding` on the bus the moment a detector fires — so the driver learns "this worker is @@ -4165,27 +4168,6 @@ Omit/empty = fresh ledger (every run that is not a resume). *** -### WorkerWatchOptions - -Online-detector wiring for spawned workers (`CoordinationToolsOptions.watchWorkers`). - -#### Properties - -##### detectors? - -> `readonly` `optional` **detectors?**: readonly `StreamingDetector`[] - -Detector panel; omit for the default stuck-loop + error-streak pair. - -##### maxFindingsPerWorker? - -> `readonly` `optional` **maxFindingsPerWorker?**: `number` - -Raise at most this many findings per worker, so one pathological worker cannot flood the - driver's inbox with the same signal every span. Default 3; `<= 0` = unlimited. - -*** - ### CoordinationTools The supervisor-side toolbox returned by [createCoordinationTools](#createcoordinationtools): the MCP tool @@ -7861,3 +7843,9 @@ Re-exports [MakeWorkerAgent](runtime.md#makeworkeragent) ### WorkerSpawnContext Re-exports [WorkerSpawnContext](runtime.md#workerspawncontext) + +*** + +### WorkerWatchOptions + +Re-exports [WorkerWatchOptions](runtime.md#workerwatchoptions) diff --git a/docs/api/primitive-catalog.md b/docs/api/primitive-catalog.md index c57f71e2..3a16b688 100644 --- a/docs/api/primitive-catalog.md +++ b/docs/api/primitive-catalog.md @@ -7,7 +7,7 @@ # Primitive catalog — the never-stale anti-reinvention inventory -> **GENERATED** from `@tangle-network/agent-runtime@0.123.1` and `@tangle-network/agent-eval@0.142.2` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`. +> **GENERATED** from `@tangle-network/agent-runtime@0.124.0` and `@tangle-network/agent-eval@0.142.2` by `scripts/gen-primitive-catalog.mjs`. Do NOT hand-edit — run `pnpm run docs:api`. This is the mechanical companion to the JUDGMENT in `canonical-api.md` (§2 decision table + §1.5 AgentProfile law): that doc says WHICH primitive to reach for and what NOT to build; this catalog proves WHAT exists. Per-symbol signatures + `file:line` live in the per-module pages under `docs/api/`. ## 1. agent-runtime — own public surface @@ -510,7 +510,7 @@ Import from `@tangle-network/agent-runtime/intelligence` — 166 exports. ### Execution kernel — recursive atom, supervision, executors, round-synchronous loop -Import from `@tangle-network/agent-runtime/kernel` — 716 exports. +Import from `@tangle-network/agent-runtime/kernel` — 717 exports. | Symbol | Kind | Summary | |---|---|---| @@ -985,6 +985,7 @@ Import from `@tangle-network/agent-runtime/kernel` — 716 exports. | `WorkerSteerRequest` | interface | One durable down-leg request appended to a worker's inbox file. | | `WorkerToolTraceArtifact` | interface | Bytes stored under `WorkerTraceEvidence.traceRef`. | | `WorkerTraceSeamCarrier` | interface | What the two readers below need off an `ExecutorContext` — its seam bag, and nothing else. | +| `WorkerWatchOptions` | interface | Online-detector wiring for spawned workers (`CoordinationToolsOptions.watchWorkers`). | | `WorktreeCommandResult` | interface | Outcome of one verification command run in the worktree (test or typecheck). | | `WorktreeHarnessResult` | interface | The canonical result of one worktree-harness run, projected by each port to its own shape. | | `WorktreeProfileMaterializationReceipt` | interface | Proof of the profile inputs delivered before the worker process started. | diff --git a/docs/api/runtime.md b/docs/api/runtime.md index 314c8de3..6248383f 100644 --- a/docs/api/runtime.md +++ b/docs/api/runtime.md @@ -1036,7 +1036,25 @@ graph expressed at the coordination layer; the finding is ALWAYS also published > `readonly` **kind**: `string` -The analyst lens id (resolved against the `analysts` registry). +The analyst id: a lens id resolved against the `analysts` registry, or — when `agent` is + present — the AGENT analyst's stable identity carried on its finding/steer events (it need + not exist in any registry). + +##### agent? + +> `readonly` `optional` **agent?**: `AgentProfile` + +Make this analyst a tool-equipped AGENT instead of a registry lens: on each matching settle +the runtime spawns this profile as a WORKER through the SAME spawn machinery a driver spawn +uses (`Scope.spawn` + the run's `makeWorkerAgent` seam) — so its spend reserves from the +conserved pool, its node is journaled and traced like any worker, and a node-pinning seam +sees the spawn context marker (`WorkerSpawnContext.analyst`). Its task is `directive` plus +the settled worker's tool-trace evidence; its settle OUTPUT is the findings, published as a +`finding` event (same canonicalization) and delivered per `to` exactly like registry-analyst +findings. A settle that failed publishes `{ analystRunFailed }`; a spawn the pool or fences +refuse publishes `{ analystSpawnRefused }` — observable, never a silent drop. An agent +analyst's settlement never enters the settled-worker ledger, never feeds the finalizer, and +never re-fires the analyst-on-settle hook (no analyst-on-analyst cascade by construction). ##### to? @@ -1053,7 +1071,10 @@ Deliver the findings to this live worker, named by its PROFILE NAME (the stable > `readonly` `optional` **directive?**: `string` Standing instruction wrapped around the findings on a routed delivery — what the recipient - should DO with the analysis. Omit = the bare findings JSON. + should DO with the analysis. For an AGENT analyst (`agent` set) it is instead the analysis + directive handed to the agent as its task; the routed delivery then carries the bare + findings, because the directive was already consumed upstream. Omit = the bare findings + JSON (and, for an agent analyst, a task of evidence only). ##### over? @@ -1273,6 +1294,36 @@ Semantic restart key, when the manager supplied one. Trusted candidate/campaign attribution attached by product authorization. +##### analyst? + +> `readonly` `optional` **analyst?**: `string` + +Present (as the analyst id) ONLY when this spawn is an analyst-AGENT run initiated by the + runtime's analyst-on-settle hook ([AnalyzeOnSettleRoute.agent](#agent)) — authored by the + runtime, never accepted from a driver's tool arguments. A node-pinning `makeWorkerAgent` + reads it to admit the analyst node it would refuse as a driver-authored spawn. + +*** + +### WorkerWatchOptions + +Online-detector wiring for spawned workers (`CoordinationToolsOptions.watchWorkers`). + +#### Properties + +##### detectors? + +> `readonly` `optional` **detectors?**: readonly `StreamingDetector`[] + +Detector panel; omit for the default stuck-loop + error-streak pair. + +##### maxFindingsPerWorker? + +> `readonly` `optional` **maxFindingsPerWorker?**: `number` + +Raise at most this many findings per worker, so one pathological worker cannot flood the + driver's inbox with the same signal every span. Default 3; `<= 0` = unlimited. + *** ### WorktreeCommandResult @@ -3562,7 +3613,7 @@ firewall is enforced afterwards by `createScopeAnalyst`, not by the analyst itse ##### analyst -> `readonly` **analyst**: [`Agent`](#agent-1)\<`unknown`, readonly `AnalystFinding`[]\> +> `readonly` **analyst**: [`Agent`](#agent-2)\<`unknown`, readonly `AnalystFinding`[]\> The analyst agent the combinator spawns over the trace. `harness` is the persona's choice (`null` for an inline router analyst, a `BackendType` for a sandboxed one). Its `act` returns @@ -3867,7 +3918,7 @@ The scope analyst (selector≠judge firewall) the combinator steers from. Absent ##### spawnChild() -> **spawnChild**(`name`, `spec`): [`Agent`](#agent-1)\<`unknown`, [`Outcome`](#outcome-2)\<`D`\>\> +> **spawnChild**(`name`, `spec`): [`Agent`](#agent-2)\<`unknown`, [`Outcome`](#outcome-2)\<`D`\>\> Wrap an `AgentSpec` into a leaf `Agent` carrying it as `executorSpec`, so the shape can `scope.spawn(spawnChild(spec), task, opts)`. `name` labels the child for traces. The @@ -3886,7 +3937,7 @@ spec drives the resolved `Executor`; `act` exists only to satisfy the `Agent` sh ###### Returns -[`Agent`](#agent-1)\<`unknown`, [`Outcome`](#outcome-2)\<`D`\>\> +[`Agent`](#agent-2)\<`unknown`, [`Outcome`](#outcome-2)\<`D`\>\> ##### childSpec() @@ -8445,7 +8496,7 @@ The cost vector, stamped by `runAgentic` from the Supervisor's conserved pool: r ##### driver() -> **driver**(`surface`, `task`, `opts`, `budget`): [`Agent`](#agent-1)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> +> **driver**(`surface`, `task`, `opts`, `budget`): [`Agent`](#agent-2)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> ###### Parameters @@ -8467,7 +8518,7 @@ The cost vector, stamped by `runAgentic` from the Supervisor's conserved pool: r ###### Returns -[`Agent`](#agent-1)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> +[`Agent`](#agent-2)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> *** @@ -9933,7 +9984,7 @@ Analyst kind ids run AUTOMATICALLY when a worker settles `done` — each result ##### watchWorkers? -> `readonly` `optional` **watchWorkers?**: [`WorkerWatchOptions`](mcp.md#workerwatchoptions) +> `readonly` `optional` **watchWorkers?**: [`WorkerWatchOptions`](#workerwatchoptions) Run the ONLINE detector panel over each worker's LIVE tool trace and raise a `finding` the moment it loops/error-storms — mid-run evidence to steer on, not a settle-time post-mortem. @@ -10457,7 +10508,7 @@ One unit of queued work: the agent to run, its task, and the spawn options (budg ##### agent -> `readonly` **agent**: [`Agent`](#agent-1)\<`unknown`, `Out`\> +> `readonly` **agent**: [`Agent`](#agent-2)\<`unknown`, `Out`\> ##### task @@ -10983,7 +11034,18 @@ Inject the driver brain directly (offline tests / advanced). > `readonly` `optional` **analysts?**: [`AnalystRegistry`](index.md#analystregistry) -The analyst lens registry `analyzes` edges resolve against. ENVIRONMENT, not nodes. +The analyst lens registry `analyzes` edges resolve against. ENVIRONMENT — needed only for + lens analysts; an analyzes edge naming a graph NODE as its analyst needs no registry. + +##### watchWorkers? + +> `readonly` `optional` **watchWorkers?**: [`WorkerWatchOptions`](#workerwatchoptions) + +Watch every worker's LIVE tool trace with the online detector panel and raise a `finding` + on the bus the moment one loops or error-storms — forwarded to `supervise()` verbatim (see + `SuperviseOptions.watchWorkers`). Online findings (`analyst: 'online:'`) are bus + events for the driver, not graph edges, so they are never ledgered as traversals. Omit = + off (no online watching, no extra events). ##### registry? @@ -13515,7 +13577,7 @@ Analyst kind ids run AUTOMATICALLY when a worker settles `done` — each re-ente ##### watchWorkers? -> `readonly` `optional` **watchWorkers?**: [`WorkerWatchOptions`](mcp.md#workerwatchoptions) +> `readonly` `optional` **watchWorkers?**: [`WorkerWatchOptions`](#workerwatchoptions) Watch every worker's LIVE tool trace with the online detector panel and raise a `finding` the moment one loops or error-storms — so the supervisor learns it mid-run (via `await_event`) @@ -14297,7 +14359,7 @@ Analyst kinds run on each worker-settle → a `finding` the driver composes its ##### watchWorkers? -> `readonly` `optional` **watchWorkers?**: [`WorkerWatchOptions`](mcp.md#workerwatchoptions) +> `readonly` `optional` **watchWorkers?**: [`WorkerWatchOptions`](#workerwatchoptions) Run the ONLINE detector panel over each worker's LIVE tool trace (both arms) so the driver learns a worker is looping mid-run instead of at settle. Omit = no online watching. @@ -18065,7 +18127,7 @@ Product decision over an exact continuation before it is durably recorded or del ### MakeWorkerAgent -> **MakeWorkerAgent** = (`profile`, `context?`) => [`Agent`](#agent-1)\<`unknown`, `unknown`\> +> **MakeWorkerAgent** = (`profile`, `context?`) => [`Agent`](#agent-2)\<`unknown`, `unknown`\> #### Parameters @@ -18079,7 +18141,7 @@ Product decision over an exact continuation before it is durably recorded or del #### Returns -[`Agent`](#agent-1)\<`unknown`, `unknown`\> +[`Agent`](#agent-2)\<`unknown`, `unknown`\> *** @@ -18252,7 +18314,7 @@ Builds a frozen `Persona`, failing loud on the executors-supplied invariant (nei ### LoopShape -> **LoopShape**\<`Task`, `D`\> = (`ctx`) => [`Agent`](#agent-1)\<`Task`, [`Outcome`](#outcome-2)\<`D`\>\> +> **LoopShape**\<`Task`, `D`\> = (`ctx`) => [`Agent`](#agent-2)\<`Task`, [`Outcome`](#outcome-2)\<`D`\>\> A reusable act-body factory. Given the persona's content + seams (`ShapeContext`), it returns the root `Agent>` whose `act` decomposes the task, fans out @@ -18278,7 +18340,7 @@ synthesizes the terminal `Outcome`. The shape is STRUCTURE; the persona is CO #### Returns -[`Agent`](#agent-1)\<`Task`, [`Outcome`](#outcome-2)\<`D`\>\> +[`Agent`](#agent-2)\<`Task`, [`Outcome`](#outcome-2)\<`D`\>\> *** @@ -19098,9 +19160,9 @@ Cyclic-graph backstop: traversals beyond this REFUSE (fail loud). Default \{ `kind`: `"analyzes"`; `analyst`: `string`; `over`: `ReadonlyArray`\<[`NodeId`](#nodeid-5)\>; `to`: [`NodeId`](#nodeid-5); `directive`: [`PromptHandle`](#prompthandle); `maxTraversals?`: `number`; \} -Findings flow anywhere: an analyst LENS (environment, never a node) over N nodes' settled - traces, delivered to ONE node wrapped in a directive telling the recipient what to do with - the analysis. +Findings flow anywhere: an analyst over N nodes' settled traces, delivered to ONE node. + With a LENS analyst the directive wraps the findings for the recipient; with a NODE analyst + the directive is the analyst agent's task and the findings are its settle output. ###### kind @@ -19110,7 +19172,12 @@ Findings flow anywhere: an analyst LENS (environment, never a node) over N nodes > `readonly` **analyst**: `string` -The analyst lens id, resolved against `RunGraphOptions.analysts`. NOT a node id. +The analyst REFERENCE, in one of two forms: a lens id resolved against + `RunGraphOptions.analysts` (environment), or the id of a graph NODE with no delegates + edge pointing at it — then each matching settle spawns that node's pinned profile as a + tool-equipped analyst WORKER (same spawn machinery, conserved budget, trace join) whose + task is this edge's directive plus the settled worker's trace evidence and whose settle + output is the findings. An id that is both a node and a registry lens is refused. ###### over @@ -23051,7 +23118,7 @@ Multi-generation strategy search: author candidates from tournament losses, play ### depthStrategy() -> **depthStrategy**(`surface`, `task`, `opts`, `cfg`): [`Agent`](#agent-1)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> +> **depthStrategy**(`surface`, `task`, `opts`, `cfg`): [`Agent`](#agent-2)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> DEPTH: one persistent artifact, carried across analyst-steered shots. @@ -23077,13 +23144,13 @@ DEPTH: one persistent artifact, carried across analyst-steered shots. #### Returns -[`Agent`](#agent-1)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> +[`Agent`](#agent-2)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> *** ### breadthStrategy() -> **breadthStrategy**(`_surface`, `task`, `opts`, `cfg`): [`Agent`](#agent-1)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> +> **breadthStrategy**(`_surface`, `task`, `opts`, `cfg`): [`Agent`](#agent-2)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> BREADTH: K independent rollouts (each own artifact), verifier picks the best. @@ -23109,7 +23176,7 @@ BREADTH: K independent rollouts (each own artifact), verifier picks the best. #### Returns -[`Agent`](#agent-1)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> +[`Agent`](#agent-2)\<`unknown`, [`Outcome`](#outcome-2)\<`unknown`\>\> *** @@ -23616,7 +23683,7 @@ The supervisor SKILL — the how-to the supervisor reads (its system prompt). TH ### authoredWorker() -> **authoredWorker**(`profile`, `opts`): [`Agent`](#agent-1)\<`unknown`, `unknown`\> +> **authoredWorker**(`profile`, `opts`): [`Agent`](#agent-2)\<`unknown`, `unknown`\> Build a router-only worker from an authored profile. This helper executes the prompt/model axes; use `workerFromBackend` for full materialization of tools, MCP, resources, hooks, and subagents. @@ -23647,7 +23714,7 @@ Build a router-only worker from an authored profile. This helper executes the pr #### Returns -[`Agent`](#agent-1)\<`unknown`, `unknown`\> +[`Agent`](#agent-2)\<`unknown`, `unknown`\> *** @@ -23793,7 +23860,7 @@ executor has produced its output. The inner `score` is preserved; only `valid` i ### driverAgent() -> **driverAgent**(`opts`): [`Agent`](#agent-1)\<`unknown`, `unknown`\> +> **driverAgent**(`opts`): [`Agent`](#agent-2)\<`unknown`, `unknown`\> Build the intelligent recursive driver. Its `act` is the LLM tool-loop; spawn it as a `driverChild` (`driver-executor.ts`) to run it inside a nested scope, recursively. @@ -23806,7 +23873,7 @@ Build the intelligent recursive driver. Its `act` is the LLM tool-loop; spawn it #### Returns -[`Agent`](#agent-1)\<`unknown`, `unknown`\> +[`Agent`](#agent-2)\<`unknown`, `unknown`\> *** @@ -23924,7 +23991,7 @@ Analyst kinds to auto-run when a worker settles `done` — findings flow up the ###### watchWorkers? -[`WorkerWatchOptions`](mcp.md#workerwatchoptions) +[`WorkerWatchOptions`](#workerwatchoptions) Run the ONLINE detector panel over each worker's live tool trace (raises `finding` events). @@ -25206,7 +25273,7 @@ explicit, recorded acknowledgment — never a silent bind. ### supervisorAgent() -> **supervisorAgent**(`profile`, `deps`): [`Agent`](#agent-1)\<`unknown`, `unknown`\> +> **supervisorAgent**(`profile`, `deps`): [`Agent`](#agent-2)\<`unknown`, `unknown`\> Build a supervisor `Agent` from its profile: the brain resolves from `profile.harness` (backend-as-data), the same resolution rule as every worker. @@ -25222,7 +25289,7 @@ Build a supervisor `Agent` from its profile: the brain resolves from `profile.ha #### Returns -[`Agent`](#agent-1)\<`unknown`, `unknown`\> +[`Agent`](#agent-2)\<`unknown`, `unknown`\> *** diff --git a/docs/canonical-api.md b/docs/canonical-api.md index 37fe92f3..457fe523 100644 --- a/docs/canonical-api.md +++ b/docs/canonical-api.md @@ -4,7 +4,7 @@ Generated signatures and the complete export list live in docs/api/. Run pnpm docs:freshness after editing this file. --> -> **Version 0.123.1.** +> **Version 0.124.0.** > [`docs/api/primitive-catalog.md`](./api/primitive-catalog.md) lists every export and import path. > `agent-eval` must satisfy `>=0.142.2 <0.143.0`. > `sandbox` must satisfy `>=0.17.2 <0.18.0`. diff --git a/examples/graphs/README.md b/examples/graphs/README.md index c1e5ab5d..457cbe6d 100644 --- a/examples/graphs/README.md +++ b/examples/graphs/README.md @@ -1,25 +1,27 @@ # graphs — agent topologies as plain data -Four runnable topologies for `runGraph` (the agent-graph layer over `supervise()`). +Five runnable topologies for `runGraph` (the agent-graph layer over `supervise()`). Each file's graph is a ≤25-line data literal — nodes are canonical `AgentProfile`s, edges are typed values carrying versioned registry directives — and each `main()` prints the EDGE LEDGER as the proof artifact: every traversal, its outcome (`delivered | stripped | empty | unpropagated`), its byte count, and the concrete worker it reached. -All four run offline at $0 (scripted driver brain + in-process leaf workers, in [`shared.ts`](./shared.ts) — the same seams the kernel's own graph tests use). +All five run offline at $0 (scripted driver brain + in-process leaf workers, in [`shared.ts`](./shared.ts) — the same seams the kernel's own graph tests use). ```bash pnpm tsx examples/graphs/collaborates-review-loop.ts pnpm tsx examples/graphs/best-of-n.ts pnpm tsx examples/graphs/watchdog-steer.ts pnpm tsx examples/graphs/shot-loop.ts +pnpm tsx examples/graphs/analyst-agent-review.ts ``` | Example | Topology | What the ledger proves | |---|---|---| | [`collaborates-review-loop.ts`](./collaborates-review-loop.ts) | root + implementer + reviewer; `analyzes` critique → reviewer, `analyzes` verdict → driver | Peer collaboration is MEDIATED: findings cross worker→worker only as a ledgered lens route (a direct worker-to-worker channel is not a first-class edge), and a route with no live target is `unpropagated`, never dropped. | | [`best-of-n.ts`](./best-of-n.ts) | root + two candidate coder nodes, one `delegates` edge each, `maxLiveWorkers: 2` | Breadth is two edges in the data: exactly two delivered spawn traversals, winner decided by the deliverable. | -| [`watchdog-steer.ts`](./watchdog-steer.ts) | root + one builder with a live trace; shipped online detector panel (`watchTrace`) | Mid-run intervention: the detector fires while the worker runs, and the corrective steer lands as the delegates edge's second delivered traversal BEFORE settle. | +| [`watchdog-steer.ts`](./watchdog-steer.ts) | root + one builder with a live trace; `watchWorkers` passthrough runs the shipped online detector panel | Mid-run intervention: the detector's finding reaches the driver over the bus, and the corrective steer lands as the delegates edge's second delivered traversal BEFORE settle. | | [`shot-loop.ts`](./shot-loop.ts) | reviewer(root) ↔ coder; `delegates maxTraversals: 3`, `analyzes` verify → reviewer | The multishot loop as data: each shot and each verify report is one ledgered traversal, the shot budget lives on the edge, and the deliverable gates on the verdict. | +| [`analyst-agent-review.ts`](./analyst-agent-review.ts) | root + implementer; `analyzes` whose analyst is the `reviewer` NODE (no delegates edge to it) | The analyst as a tool-equipped AGENT: the reviewer node is spawned on the implementer's settle with directive + trace evidence as its task, its settle output IS the finding, and its spend lands in the one conserved budget. | -The offline proof for all four (exact ledger counts, outcomes, destinations) lives in `tests/examples/graph-topologies.test.ts`. +The offline proof for all five (exact ledger counts, outcomes, destinations) lives in `tests/examples/graph-topologies.test.ts`. ## Two ledger semantics worth knowing diff --git a/examples/graphs/analyst-agent-review.ts b/examples/graphs/analyst-agent-review.ts new file mode 100644 index 00000000..6ddf9b98 --- /dev/null +++ b/examples/graphs/analyst-agent-review.ts @@ -0,0 +1,95 @@ +/** + * analyst-agent-review — the reviewer as a TOOL-EQUIPPED AGENT, not a registry lens. + * + * The analyzes edge names a graph NODE ('reviewer') as its analyst. When the implementer + * settles, `runGraph` spawns the reviewer node's pinned profile as a real WORKER through the + * same spawn machinery every worker uses: its task is the analyzes directive plus the + * implementer's tool-trace evidence, its spend reserves from the graph's one conserved budget, + * and its settle OUTPUT is the findings — published to the driver as a `finding` event and + * ledgered as the analyzes traversal, exactly like a registry-analyst finding. + * + * Oracle doctrine holds structurally: the reviewer has NO delegates edge pointing at it (a graph + * that adds one is refused), so the driver can never hand it work or smuggle capabilities into + * it — it runs only as the lens over the evidence, with whatever tools its own profile grants. + * + * Fully offline (scripted brain + leaf seam). Run: pnpm tsx examples/graphs/analyst-agent-review.ts + */ + +import type { AgentProfile } from '@tangle-network/agent-interface' +import { + type AgentGraph, + promptHandle, + type RunGraphOptions, + runGraph, +} from '@tangle-network/agent-runtime/kernel' +import { leafSeam, printLedger, scriptedBrain } from './shared' + +const brief = promptHandle('delegates/worker-brief/v1') +const report = promptHandle('analyzes/findings-report/v1') + +export function analystAgentReview(): { graph: AgentGraph; opts: RunGraphOptions } { + // ── The topology: plain data (the analyst is the 'reviewer' NODE, not a registry lens) ── + const graph: AgentGraph = { + nodes: [ + { id: 'driver', profile: { name: 'driver', prompt: { systemPrompt: 'Drive the build.' } } }, + { id: 'implementer', profile: { name: 'implementer', prompt: { systemPrompt: 'Build.' } } }, + { + id: 'reviewer', + profile: { name: 'reviewer', prompt: { systemPrompt: 'Review the trace evidence.' } }, + }, + ], + edges: [ + { kind: 'delegates', from: 'driver', to: 'implementer', directive: brief }, + { + kind: 'analyzes', + analyst: 'reviewer', + over: ['implementer'], + to: 'driver', + directive: report, + }, + ], + deliverable: { describe: 'the built artifact', check: (out) => out !== undefined }, + budget: { maxIterations: 20, maxTokens: 50_000 }, + } + + const received: AgentProfile[] = [] + const opts: RunGraphOptions = { + runId: 'rev', + makeWorkerAgent: leafSeam(received, { + implementer: { withTrace: true }, + // The reviewer node settles with its review — that OUTPUT is the findings the driver gets. + reviewer: { + shots: [{ out: { verdict: 'needs-changes', defects: ['no tests'] }, valid: true }], + }, + }), + brain: scriptedBrain([ + { + toolCalls: [ + { + name: 'spawn_agent', + arguments: { profile: { name: 'implementer' }, task: 'implement the feature' }, + }, + ], + }, + // implementer settles → the reviewer AGENT is spawned over its trace → its settle output + // arrives as the finding. Two bus events: settled, finding. + { toolCalls: [{ name: 'await_event', arguments: {} }] }, + { toolCalls: [{ name: 'await_event', arguments: {} }] }, + { content: 'done' }, + ]), + } + return { graph, opts } +} + +export async function main(): Promise { + const { graph, opts } = analystAgentReview() + const res = await runGraph(graph, opts) + printLedger('analyst-agent-review', res) +} + +if (import.meta.url === `file://${process.argv[1]}`) { + main().catch((err) => { + console.error(err) + process.exit(1) + }) +} diff --git a/examples/graphs/watchdog-steer.ts b/examples/graphs/watchdog-steer.ts index 7998bd89..9776d1e4 100644 --- a/examples/graphs/watchdog-steer.ts +++ b/examples/graphs/watchdog-steer.ts @@ -2,37 +2,51 @@ * watchdog-steer — the mid-run intervention loop: detect a stuck worker WHILE it runs, correct * it BEFORE it settles. * - * The builder node exposes a live tool-trace source. The shipped online detector panel - * (`watchTrace` + `defaultToolDetectors` — the same streaming stuck-loop/error-streak kernel - * agent-eval ships) watches that trace and fires the moment the builder starts hammering the - * same failing command. The driver waits on that signal, composes a corrective instruction FROM - * it, and steers the still-live builder; the steer is the mid-run leg of the delegates edge and - * lands in the ledger like every other traversal. - * - * Wiring note, stated plainly: `supervise()` accepts `watchWorkers` to run this exact panel and - * raise bus `finding`s itself, but `RunGraphOptions` does not forward it — so this example wires - * the SAME shipped panel directly over the worker's trace source at the leaf seam and hands the - * signal to the driver brain. The corrective steer still flows driver → worker over the - * delegates edge, authorized and ledgered. + * The builder node exposes a live tool-trace source. `RunGraphOptions.watchWorkers` forwards to + * `supervise()`'s online detector panel (`watchTrace` + `defaultToolDetectors` — the same + * streaming stuck-loop/error-streak kernel agent-eval ships), which watches every worker's live + * trace and raises a `finding` on the coordination bus the moment the builder starts hammering + * the same failing command. The driver pulls that finding from `await_event`, composes a + * corrective instruction FROM it, and steers the still-live builder; the steer is the mid-run + * leg of the delegates edge and lands in the ledger like every other traversal. * * Fully offline (reactive brain + leaf seam). Run: pnpm tsx examples/graphs/watchdog-steer.ts */ -import type { DetectorSignal } from '@tangle-network/agent-eval' import type { AgentProfile } from '@tangle-network/agent-interface' import { type AgentGraph, - defaultToolDetectors, promptHandle, type RunGraphOptions, runGraph, type ToolLoopChat, - watchTrace, } from '@tangle-network/agent-runtime/kernel' import { leafSeam, printLedger } from './shared' const brief = promptHandle('delegates/worker-brief/v1') +/** The online finding the watchdog raised, read back out of the driver's own transcript. */ +function onlineFinding( + messages: ReadonlyArray>, +): { detector: string; reason: string; streak: number } | undefined { + for (const message of messages) { + const content = typeof message.content === 'string' ? message.content : undefined + if (content === undefined || !content.includes('"analyst":"online:')) continue + try { + const event = JSON.parse(content) as { + findings?: { detector?: string; reason?: string; streak?: number } + } + const f = event.findings + if (f?.detector !== undefined) { + return { detector: f.detector, reason: f.reason ?? '', streak: f.streak ?? 0 } + } + } catch { + // a non-JSON tool message is simply not the finding + } + } + return undefined +} + export function watchdogSteer(): { graph: AgentGraph; opts: RunGraphOptions } { // ── The topology: plain data ── const graph: AgentGraph = { @@ -45,32 +59,17 @@ export function watchdogSteer(): { graph: AgentGraph; opts: RunGraphOptions } { budget: { maxIterations: 20, maxTokens: 50_000 }, } - // ── The watchdog: the online detector panel over the builder's LIVE trace ── - let fireSignal: (signal: DetectorSignal) => void - const firstSignal = new Promise((resolve) => { - fireSignal = resolve - }) + // The builder blocks until a steer arrives, and its trace replays a stuck loop: the same + // failing `pnpm test` five times — the storm the repeated-action/error-streak panel catches. const received: AgentProfile[] = [] - const seam = leafSeam( - received, - // The builder blocks until a steer arrives, and its trace replays a stuck loop: the same - // failing `pnpm test` five times — the storm the repeated-action/error-streak panel catches. - { builder: { awaitSteer: true, withTrace: true, storm: 5 } }, - { - onTraceSource: (_nodeId, source) => { - watchTrace(source, { - detectors: defaultToolDetectors(), - onSignal: (signal) => fireSignal(signal), - }) - }, - }, - ) + const seam = leafSeam(received, { builder: { awaitSteer: true, withTrace: true, storm: 5 } }) - // ── The driver: spawn, WAIT for the watchdog, steer with the evidence, settle ── - let turn = 0 - const brain: ToolLoopChat = async () => { - turn += 1 - if (turn === 1) { + // ── The driver: spawn, PULL the watchdog finding, steer with the evidence, settle ── + let spawned = false + let steered = false + const brain: ToolLoopChat = async (messages) => { + if (!spawned) { + spawned = true return { toolCalls: [ { @@ -81,8 +80,20 @@ export function watchdogSteer(): { graph: AgentGraph; opts: RunGraphOptions } { ], } } - if (turn === 2) { - const signal = await firstSignal + const finding = onlineFinding(messages) + if (finding === undefined) { + // Yield one macrotask so the freshly-spawned builder's executor starts and the detector's + // finding reaches the bus BEFORE this pull — otherwise the pull races the spawn by a few + // microtasks and the driver burns a full await fence learning nothing. + await new Promise((resolve) => setImmediate(resolve)) + return { + toolCalls: [ + { id: 'cw', name: 'await_event', arguments: JSON.stringify({ kinds: ['finding'] }) }, + ], + } + } + if (!steered) { + steered = true return { toolCalls: [ { @@ -91,20 +102,29 @@ export function watchdogSteer(): { graph: AgentGraph; opts: RunGraphOptions } { arguments: JSON.stringify({ workerId: 'wd:s0', instruction: - `Watchdog: ${signal.detector} fired (streak ${signal.streak}) — ${signal.reason}. ` + + `Watchdog: ${finding.detector} fired (streak ${finding.streak}) — ${finding.reason}. ` + 'Stop repeating the failing command and deliver what you have.', }), }, ], } } - if (turn === 3) { + const settled = messages.some( + (message) => + typeof message.content === 'string' && message.content.includes('"type":"settled"'), + ) + if (!settled) { return { toolCalls: [{ id: 'c3', name: 'await_event', arguments: JSON.stringify({}) }] } } return { content: 'done', toolCalls: [] } } - const opts: RunGraphOptions = { runId: 'wd', makeWorkerAgent: seam, brain } + const opts: RunGraphOptions = { + runId: 'wd', + makeWorkerAgent: seam, + brain, + watchWorkers: { maxFindingsPerWorker: 1 }, + } return { graph, opts } } diff --git a/package.json b/package.json index 7fba3565..40714eb8 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "@tangle-network/agent-runtime", - "version": "0.123.1", + "version": "0.124.0", "description": "Shared task-lifecycle skeleton for agents: a recursive loop kernel for chat turns, one-shot tasks, and multi-attempt loops, with trace capture and eval-gated self-improvement. Domain behavior lives in adapters; scoring and ship-gates in @tangle-network/agent-eval.", "homepage": "https://github.com/tangle-network/agent-runtime#readme", "repository": { diff --git a/src/mcp/tools/coordination.ts b/src/mcp/tools/coordination.ts index af6e9a1b..eb108797 100644 --- a/src/mcp/tools/coordination.ts +++ b/src/mcp/tools/coordination.ts @@ -35,7 +35,10 @@ import { type WatchTraceOptions, watchTrace } from '../../runtime/supervise/dete import { freeSlots } from '../../runtime/supervise/dispatch' import { type BusRecord, type BusStats, createEventBus } from '../../runtime/supervise/event-bus' import type { WorkerProgress } from '../../runtime/supervise/progress' -import { workerTraceAnalysisStore } from '../../runtime/supervise/trace-evidence' +import { + parseWorkerToolTraceArtifact, + workerTraceAnalysisStore, +} from '../../runtime/supervise/trace-evidence' import type { McpToolDescriptor } from '../server' // The floors a root must know BEFORE it authors a child budget, generated from the measured @@ -184,8 +187,24 @@ export function canonicalFindingEvent(finding: AnalystFindingEvent): AnalystFind * (the audit trail), routing adds delivery, never replaces the record. */ export interface AnalyzeOnSettleRoute { - /** The analyst lens id (resolved against the `analysts` registry). */ + /** The analyst id: a lens id resolved against the `analysts` registry, or — when `agent` is + * present — the AGENT analyst's stable identity carried on its finding/steer events (it need + * not exist in any registry). */ readonly kind: string + /** + * Make this analyst a tool-equipped AGENT instead of a registry lens: on each matching settle + * the runtime spawns this profile as a WORKER through the SAME spawn machinery a driver spawn + * uses (`Scope.spawn` + the run's `makeWorkerAgent` seam) — so its spend reserves from the + * conserved pool, its node is journaled and traced like any worker, and a node-pinning seam + * sees the spawn context marker (`WorkerSpawnContext.analyst`). Its task is `directive` plus + * the settled worker's tool-trace evidence; its settle OUTPUT is the findings, published as a + * `finding` event (same canonicalization) and delivered per `to` exactly like registry-analyst + * findings. A settle that failed publishes `{ analystRunFailed }`; a spawn the pool or fences + * refuse publishes `{ analystSpawnRefused }` — observable, never a silent drop. An agent + * analyst's settlement never enters the settled-worker ledger, never feeds the finalizer, and + * never re-fires the analyst-on-settle hook (no analyst-on-analyst cascade by construction). + */ + readonly agent?: AgentProfile /** Deliver the findings to this live worker, named by its PROFILE NAME (the stable node * identity a graph pins) or its spawn label. Omit = the driver (bus only). Delivery goes * through the same authorization + steer machinery a driver-authored steer uses and is @@ -193,7 +212,10 @@ export interface AnalyzeOnSettleRoute { * failed one (`delivered: false`) is a recorded outcome, never a silent drop. */ readonly to?: string /** Standing instruction wrapped around the findings on a routed delivery — what the recipient - * should DO with the analysis. Omit = the bare findings JSON. */ + * should DO with the analysis. For an AGENT analyst (`agent` set) it is instead the analysis + * directive handed to the agent as its task; the routed delivery then carries the bare + * findings, because the directive was already consumed upstream. Omit = the bare findings + * JSON (and, for an agent analyst, a task of evidence only). */ readonly directive?: string /** Restrict which settled workers feed this lens, by profile name or spawn label. Omit = * every settled `done` worker. */ @@ -311,6 +333,11 @@ export interface WorkerSpawnContext { readonly key?: string /** Trusted candidate/campaign attribution attached by product authorization. */ readonly execution?: AgentExecutionRef + /** Present (as the analyst id) ONLY when this spawn is an analyst-AGENT run initiated by the + * runtime's analyst-on-settle hook ({@link AnalyzeOnSettleRoute.agent}) — authored by the + * runtime, never accepted from a driver's tool arguments. A node-pinning `makeWorkerAgent` + * reads it to admit the analyst node it would refuse as a driver-authored spawn. */ + readonly analyst?: string } export type MakeWorkerAgent = ( @@ -348,8 +375,11 @@ export interface CoordinationToolsOptions { * {@link AnalyzeOnSettleRoute} generalizes the DESTINATION: findings can be delivered to a * named live WORKER (wrapped in the route's directive, through the same authorized steer * machinery a driver steer uses) instead of being hardwired to the spawning driver, and `over` - * restricts which settled workers feed the lens. Omit/empty = no auto-analysis (default; the - * driver can still run lenses on demand via `run_analyst`). Requires `analysts`. */ + * restricts which settled workers feed the lens. A route carrying `agent` generalizes the + * ANALYST itself: a tool-equipped agent spawned as a worker whose settle output is the + * findings (see {@link AnalyzeOnSettleRoute.agent}). Omit/empty = no auto-analysis (default; + * the driver can still run lenses on demand via `run_analyst`). Lens routes require + * `analysts`; agent routes do not. */ readonly analyzeOnSettle?: ReadonlyArray /** Hard cap on how many workers may be LIVE (spawned but not yet settled) at once. `spawn_agent` * counts the scope's non-terminal nodes and fails closed (`error: 'max-live-workers'`) BEFORE @@ -872,16 +902,115 @@ export function createCoordinationTools(opts: CoordinationToolsOptions): Coordin // `Scope.next()` is once-only, while an awaited observer may commit and lose its acknowledgement. // Retain the exact event until publication succeeds so the next await retries rather than losing - // the settlement between the spawn journal and the product transaction. + // the settlement between the spawn journal and the product transaction. `analystRun` marks a + // settlement that IS an analyst-agent's findings (see `analystRuns`), whose event is a `finding`. let pendingSettlement: | { readonly settled: Settled readonly worker: SettledWorker readonly event: CoordinationEvent readonly analyze: boolean + readonly analystRun?: AnalystRunInFlight } | undefined + // Analyst-AGENT runs in flight (`AnalyzeOnSettleRoute.agent`): worker id → the route that + // spawned it + the settled source worker whose evidence it analyzes. A member's settlement is a + // FINDING, not a worker settle: it never enters the settled ledger, never feeds the finalizer, + // and never re-fires the analyst-on-settle hook — so an analyst cannot cascade onto itself. + interface AnalystRunInFlight { + readonly route: AnalyzeOnSettleRoute + readonly sourceWorker: string + } + // PROCESS-LOCAL by design: a durable-run RESUME does not repopulate this map, so an + // analyst spawn from a PRIOR process settles as an ordinary worker on resume (it re-enters + // the settled ledger instead of being intercepted as a finding). Journal-level marking of + // analyst assignments is the fix; until then resume + analyst-node graphs do not compose. + const analystRuns = new Map() + let analystRunOrdinal = 0 + + /** The `finding` event an analyst-agent settlement becomes: its settle OUTPUT is the findings + * (a failed run publishes the failure as findings — degraded beats vanished). */ + const analystRunFinding = ( + run: AnalystRunInFlight, + settled: Settled, + ): CoordinationEvent => + deepFreezeDetached({ + type: 'finding', + finding: canonicalFindingEvent({ + fromWorker: run.sourceWorker, + analyst: run.route.kind, + findings: settled.kind === 'done' ? settled.out : { analystRunFailed: settled.reason }, + }), + }) + + /** + * Spawn one analyst-AGENT run over a settled worker's evidence, through the SAME spawn + * machinery a driver spawn uses (`scope.spawn` + `makeWorkerAgent`): the analyst's spend + * reserves from the conserved pool, its node is journaled/traced like any worker, and a + * node-pinning seam sees `context.analyst`. Its task is the route directive plus the settled + * worker's persisted tool-trace spans. A refused spawn publishes a finding RECORDING the + * refusal — observable, never silent — and must never take down the settlement path. + */ + const spawnAnalystRun = async ( + route: AnalyzeOnSettleRoute, + worker: SettledWorker, + ): Promise => { + let spansText = '' + let spanCount = 0 + if (worker.trace.status === 'available') { + try { + const raw = await opts.blobs.get(worker.trace.traceRef) + const artifact = parseWorkerToolTraceArtifact(raw, worker.trace.traceRef) + spanCount = artifact.spans.length + spansText = safeJsonText(artifact.spans) + } catch { + spansText = '' // degraded evidence is stated below, never a thrown settle path + } + } + const task = [ + ...(route.directive === undefined || route.directive.length === 0 ? [] : [route.directive]), + `Evidence — settled worker '${worker.id}' tool trace (${spanCount} spans):`, + spansText.length === 0 ? '(no tool spans available)' : spansText, + ].join('\n\n') + const assignmentId = `analyst:${route.kind}:o${analystRunOrdinal++}` + const label = `analyst:${route.kind}` + const context: WorkerSpawnContext = Object.freeze({ + assignmentId, + parentNodeId: opts.scope.view.root, + budget: opts.perWorker, + task, + label, + analyst: route.kind, + }) + let refusal: string | undefined + let spawnedId: string | undefined + try { + const res = opts.scope.spawn( + () => opts.makeWorkerAgent(route.agent as AgentProfile, context), + task, + { budget: opts.perWorker, label, assignmentId }, + ) + if (res.ok) spawnedId = res.handle.id + else refusal = String(res.reason) + } catch (cause) { + refusal = cause instanceof Error ? cause.message : String(cause) + } + if (spawnedId === undefined) { + await bus.publish({ + type: 'finding', + finding: canonicalFindingEvent({ + fromWorker: worker.id, + analyst: route.kind, + findings: { analystSpawnRefused: refusal ?? 'unknown' }, + }), + }) + return + } + analystRuns.set(spawnedId, { route, sourceWorker: worker.id }) + watchWorker(spawnedId) + } + // The names an analyst route may know a worker by: its authored profile name (the stable node // identity) and its spawn label (the driver's free-text choice). const workerRouteNames = (workerId: string): Set => { @@ -989,13 +1118,29 @@ export function createCoordinationTools(opts: CoordinationToolsOptions): Coordin const pending = pendingSettlement if (!pending) return false await bus.publish(pending.event) + // An analyst-AGENT settlement IS its finding (just published): it never enters the settled + // ledger or the finalizer, and it re-fires no analyst hook. Its routed delivery goes through + // the exact machinery a registry-analyst finding uses — with no wrap directive, because the + // route's directive was already consumed as the analyst's task. + if (pending.analystRun) { + analystRuns.delete(pending.worker.id) + unwatchWorker(pending.worker.id) + pendingSettlement = undefined + const { route } = pending.analystRun + if (route.to !== undefined && pending.event.type === 'finding') { + await deliverRoutedFinding( + { kind: route.kind, to: route.to }, + pending.event.finding.findings, + ) + } + return true + } commitSettled(pending.settled, pending.worker) pendingSettlement = undefined if ( pending.analyze && pending.worker.status === 'done' && pending.worker.trace.status === 'available' && - opts.analysts && opts.analyzeOnSettle?.length ) { const routes = opts.analyzeOnSettle.map(normalizeAnalyzeOnSettle) @@ -1003,9 +1148,12 @@ export function createCoordinationTools(opts: CoordinationToolsOptions): Coordin const applicable = routes.filter( (route) => route.over === undefined || route.over.some((name) => sourceNames.has(name)), ) - if (applicable.length > 0) { + // Registry lenses run in-process over the trace store; agent analysts spawn as workers. + const lensRoutes = applicable.filter((route) => route.agent === undefined) + const agentRoutes = applicable.filter((route) => route.agent !== undefined) + if (lensRoutes.length > 0 && opts.analysts) { const trace = await workerTraceAnalysisStore(pending.worker.trace, opts.blobs) - for (const route of applicable) { + for (const route of lensRoutes) { const findings = await opts.analysts.run(route.kind, trace) // The finding ALWAYS lands on the bus — the audit trail and the driver's pullable copy — // whether or not the route also delivers it to a named worker. @@ -1020,6 +1168,7 @@ export function createCoordinationTools(opts: CoordinationToolsOptions): Coordin if (route.to !== undefined) await deliverRoutedFinding(route, findings) } } + for (const route of agentRoutes) await spawnAnalystRun(route, pending.worker) } return true } @@ -1033,19 +1182,30 @@ export function createCoordinationTools(opts: CoordinationToolsOptions): Coordin const settled = await opts.scope.next() if (!settled) return false const worker = projectSettled(settled) - pendingSettlement = { - settled, - worker, - event: deepFreezeDetached({ type: 'settled', worker }), - analyze: true, - } + const run = analystRuns.get(settled.handle.id) + pendingSettlement = run + ? { + settled, + worker, + event: analystRunFinding(run, settled), + analyze: false, + analystRun: run, + } + : { + settled, + worker, + event: deepFreezeDetached({ type: 'settled', worker }), + analyze: true, + } } return flushPendingSettlement() } // Post-loop drain: every ALREADY-settled, unpulled child enters the ledger + audit trail. No // analyst-on-settle here — the driver has stopped, so a finding has no reader and an analyst - // spawn would spend real compute for nothing. + // spawn would spend real compute for nothing. An analyst-agent run ALREADY in flight is not a + // new spawn: its settlement still becomes its finding (the audit trail must not lose paid-for + // findings), though a routed delivery will usually record `delivered:false` this late. const drainResolved = async (): Promise => { let drained = 0 for (;;) { @@ -1053,12 +1213,21 @@ export function createCoordinationTools(opts: CoordinationToolsOptions): Coordin const settled = await opts.scope.nextResolved() if (!settled) return drained const worker = projectSettled(settled) - pendingSettlement = { - settled, - worker, - event: deepFreezeDetached({ type: 'settled', worker }), - analyze: false, - } + const run = analystRuns.get(settled.handle.id) + pendingSettlement = run + ? { + settled, + worker, + event: analystRunFinding(run, settled), + analyze: false, + analystRun: run, + } + : { + settled, + worker, + event: deepFreezeDetached({ type: 'settled', worker }), + analyze: false, + } } await flushPendingSettlement() drained += 1 diff --git a/src/runtime/index.ts b/src/runtime/index.ts index 7cc438b5..a6ac1b4a 100644 --- a/src/runtime/index.ts +++ b/src/runtime/index.ts @@ -72,6 +72,7 @@ export type { DownMessageEvent, MakeWorkerAgent, WorkerSpawnContext, + WorkerWatchOptions, } from './../mcp/tools/coordination' export { canonicalFindingEvent, diff --git a/src/runtime/supervise/coordination-driver.ts b/src/runtime/supervise/coordination-driver.ts index 27438953..579d0ea5 100644 --- a/src/runtime/supervise/coordination-driver.ts +++ b/src/runtime/supervise/coordination-driver.ts @@ -36,6 +36,7 @@ import { coordinationVerbNames, createCoordinationTools, type MakeWorkerAgent, + normalizeAnalyzeOnSettle, type SettledWorker, type WorkerWatchOptions, } from '../../mcp/tools/coordination' @@ -293,7 +294,13 @@ export function driverAgent(opts: DriverAgentOptions): Agent { } // Fail loud on a half-wired analyst seam (matches the extraTools pattern): analyze-on-settle with no // lens registry is a silent no-op the house rules forbid — the driver would get no findings, no error. - if ((opts.analyzeOnSettle?.length ?? 0) > 0 && !opts.analysts) { + // Only LENS routes resolve against the registry; an agent route carries its own analyst profile. + if ( + (opts.analyzeOnSettle ?? []) + .map(normalizeAnalyzeOnSettle) + .some((route) => route.agent === undefined) && + !opts.analysts + ) { throw new ValidationError( 'driverAgent: analyzeOnSettle requires analysts (the lens registry the kinds resolve against)', ) diff --git a/src/runtime/supervise/graph.ts b/src/runtime/supervise/graph.ts index a546877f..8a8f67d5 100644 --- a/src/runtime/supervise/graph.ts +++ b/src/runtime/supervise/graph.ts @@ -28,10 +28,13 @@ * 4. **Per-edge traversal caps** — the cyclic-graph backstop. A delegates edge whose cap is * exhausted REFUSES further traversals (fail loud), so a cycle cannot spin the pool dry. * - * ORACLES ARE ENVIRONMENT, NEVER NODES. Graders/verifiers must not be addressable in the graph — - * an edge to them leaks the rubric. Analysts (`analyzes` edges) are LENSES from the environment's - * registry reading trace evidence; they are not nodes either, and a graph that names a node id as - * its analyst is refused. + * ORACLES ARE ENVIRONMENT, NEVER WORKERS. Graders/verifiers must not be spawnable in the graph — + * a delegates edge to them leaks the rubric. An `analyzes` edge names its analyst in one of two + * forms: a LENS id from the environment's registry (a pure function over trace evidence), or the + * id of a graph NODE — a tool-equipped analyst AGENT spawned on each matching settle with the + * node's pinned profile, whose settle output IS the findings. Either way the oracle doctrine + * holds: an analyst node can never be a delegates target (refused loudly), so no driver can hand + * it work, and an id living in both the registry and the nodes is refused as ambiguous. * * @experimental */ @@ -48,6 +51,7 @@ import type { AnalyzeOnSettleRoute, CoordinationEvent, MakeWorkerAgent, + WorkerWatchOptions, } from '../../mcp/tools/coordination' import { composeRuntimeHooks, type RuntimeHooks } from '../../runtime-hooks' import type { RouterConfig } from '../router-client' @@ -86,12 +90,17 @@ export type GraphEdge = * {@link defaultEdgeTraversalCap}. */ readonly maxTraversals?: number } - /** Findings flow anywhere: an analyst LENS (environment, never a node) over N nodes' settled - * traces, delivered to ONE node wrapped in a directive telling the recipient what to do with - * the analysis. */ + /** Findings flow anywhere: an analyst over N nodes' settled traces, delivered to ONE node. + * With a LENS analyst the directive wraps the findings for the recipient; with a NODE analyst + * the directive is the analyst agent's task and the findings are its settle output. */ | { readonly kind: 'analyzes' - /** The analyst lens id, resolved against `RunGraphOptions.analysts`. NOT a node id. */ + /** The analyst REFERENCE, in one of two forms: a lens id resolved against + * `RunGraphOptions.analysts` (environment), or the id of a graph NODE with no delegates + * edge pointing at it — then each matching settle spawns that node's pinned profile as a + * tool-equipped analyst WORKER (same spawn machinery, conserved budget, trace join) whose + * task is this edge's directive plus the settled worker's trace evidence and whose settle + * output is the findings. An id that is both a node and a registry lens is refused. */ readonly analyst: string readonly over: ReadonlyArray readonly to: NodeId @@ -176,8 +185,15 @@ export interface RunGraphOptions { readonly hooks?: RuntimeHooks /** Inject the driver brain directly (offline tests / advanced). */ readonly brain?: ToolLoopChat - /** The analyst lens registry `analyzes` edges resolve against. ENVIRONMENT, not nodes. */ + /** The analyst lens registry `analyzes` edges resolve against. ENVIRONMENT — needed only for + * lens analysts; an analyzes edge naming a graph NODE as its analyst needs no registry. */ readonly analysts?: AnalystRegistry + /** Watch every worker's LIVE tool trace with the online detector panel and raise a `finding` + * on the bus the moment one loops or error-storms — forwarded to `supervise()` verbatim (see + * `SuperviseOptions.watchWorkers`). Online findings (`analyst: 'online:'`) are bus + * events for the driver, not graph edges, so they are never ledgered as traversals. Omit = + * off (no online watching, no extra events). */ + readonly watchWorkers?: WorkerWatchOptions /** Directive registry. Default: the seeded kernel registry (`kernelPromptRegistry()`). */ readonly registry?: PromptRegistry /** The run journal the edge ledger and every spawn/settle ride. Default: in-memory. */ @@ -219,6 +235,9 @@ interface ValidatedGraph { readonly workers: ReadonlyMap readonly delegatesByWorker: ReadonlyMap> readonly analyzes: ReadonlyArray> + /** Nodes referenced as an analyzes edge's ANALYST (the analyst-agent form): reachable through + * their analyzes edge (spawned on settle), never through a delegates edge. */ + readonly analystNodes: ReadonlyMap } function edgeId(edge: GraphEdge): string { @@ -319,6 +338,7 @@ function validateGraph( } } const analystIds = new Set() + const analystNodes = new Map() for (const edge of analyzes) { // The runner's traversal ledger resolves a finding/steer back to its edge BY ANALYST ID // alone, so a second edge sharing an analyst would silently absorb the first edge's @@ -332,21 +352,46 @@ function validateGraph( ) } analystIds.add(edge.analyst) - if (byId.has(edge.analyst)) { + // The analyst REFERENCE has two forms — a registry lens id or a graph node id — and the id + // itself is what distinguishes them, so an id living in both is refused as ambiguous rather + // than silently resolved by precedence. + const analystNode = byId.get(edge.analyst) + const inRegistry = analysts?.kinds.some((kind) => kind.id === edge.analyst) === true + if (analystNode !== undefined && inRegistry) { throw new ValidationError( - `runGraph: ${edgeId(edge)} names node '${edge.analyst}' as its analyst — oracles and ` + - 'analysts are ENVIRONMENT, never nodes; pass a lens id from RunGraphOptions.analysts', + `runGraph: ${edgeId(edge)} analyst '${edge.analyst}' is BOTH a graph node and a lens in ` + + 'the analysts registry — the id alone distinguishes the two analyst forms, so this is ' + + 'ambiguous; rename the node or register the lens under another id', ) } - if (!analysts) { + if (analystNode !== undefined) { + // The analyst-AGENT form. Oracle doctrine holds structurally: the analyst node can never + // receive work — not from the root (a delegates edge to it is refused) and not by being + // the root (the root delegates by definition). + if (analystNode.id === root.id) { + throw new ValidationError( + `runGraph: ${edgeId(edge)} names the ROOT as its analyst — the root is the driver; ` + + 'give the analyst its own node with no delegates edge pointing at it', + ) + } + if (delegatedTo.has(analystNode.id)) { + throw new ValidationError( + `runGraph: ${edgeId(edge)} names node '${edge.analyst}' as its analyst, but that node ` + + 'is a delegates target — oracle doctrine: an analyst is never delegated to. An ' + + 'analyst NODE is legal only with NO delegates edge pointing at it; give the analyst ' + + 'its own delegates-free node or pass a lens id from RunGraphOptions.analysts.', + ) + } + analystNodes.set(analystNode.id, analystNode) + } else if (!analysts) { throw new ValidationError( - `runGraph: ${edgeId(edge)} needs RunGraphOptions.analysts (the lens registry its analyst resolves against)`, + `runGraph: ${edgeId(edge)} analyst '${edge.analyst}' is not a graph node, and no ` + + 'RunGraphOptions.analysts registry was provided to resolve it as a lens', ) - } - if (!analysts.kinds.some((kind) => kind.id === edge.analyst)) { + } else if (!inRegistry) { throw new ValidationError( - `runGraph: ${edgeId(edge)} analyst '${edge.analyst}' is not in the analysts registry ` + - `(known: ${analysts.kinds.map((kind) => kind.id).join(', ') || 'none'})`, + `runGraph: ${edgeId(edge)} analyst '${edge.analyst}' is neither a graph node nor in the ` + + `analysts registry (known lenses: ${analysts.kinds.map((kind) => kind.id).join(', ') || 'none'})`, ) } if (edge.over.length === 0) { @@ -355,6 +400,20 @@ function validateGraph( for (const over of edge.over) requireNode(over, edgeId(edge)) requireNode(edge.to, edgeId(edge)) } + // Second pass, once every analyst NODE is known: an analyst run's settlement is a FINDING, + // never a worker settle, so an analyzes edge OVER an analyst node would silently never fire — + // refuse it rather than let it rot unobserved. + for (const edge of analyzes) { + for (const over of edge.over) { + if (analystNodes.has(over)) { + throw new ValidationError( + `runGraph: ${edgeId(edge)} analyzes '${over}', which is an analyst node — an analyst ` + + 'run settles as a finding, never as a worker, so this edge would silently never ' + + 'fire; analyst nodes are not analyzable', + ) + } + } + } const workers = new Map() const delegatesByWorker = new Map>() for (const edge of delegates) { @@ -367,15 +426,16 @@ function validateGraph( delegatesByWorker.set(edge.to, edge) workers.set(edge.to, requireNode(edge.to, edgeId(edge))) } - // Every non-root node must be reachable by a delegates edge, or it can never run. + // Every non-root node must be reachable by SOME edge, or it can never run. A worker node is + // reached by its delegates edge; an analyst node by its analyzes edge (spawned on settle). for (const node of graph.nodes) { - if (node.id !== root.id && !workers.has(node.id)) { + if (node.id !== root.id && !workers.has(node.id) && !analystNodes.has(node.id)) { throw new ValidationError( `runGraph: node '${node.id}' has no delegates edge to it — an unreachable node never runs`, ) } } - return { root, workers, delegatesByWorker, analyzes } + return { root, workers, delegatesByWorker, analyzes, analystNodes } } // ── The runner ───────────────────────────────────────────────────────────────── @@ -401,7 +461,7 @@ function stringifyPayload(payload: unknown): string { */ export function runGraph(graph: AgentGraph, opts: RunGraphOptions): Promise { const registry = opts.registry ?? kernelPromptRegistry() - const { root, workers, delegatesByWorker, analyzes } = validateGraph( + const { root, workers, delegatesByWorker, analyzes, analystNodes } = validateGraph( graph, registry, opts.analysts, @@ -464,6 +524,22 @@ export function runGraph(graph: AgentGraph, opts: RunGraphOptions): Promise = analyzes.map((edge) => - edge.to === root.id + // A LENS edge's directive wraps the findings for the recipient (so a driver-destined lens + // route carries no directive — it becomes the driver brief below). A NODE edge's directive is + // the analyst AGENT's task, so it always rides the route, wherever the findings go. + const routes: Array = analyzes.map((edge) => { + const analystNode = analystNodes.get(edge.analyst) + if (analystNode) { + return { + kind: edge.analyst, + over: edge.over, + agent: analystNode.profile, + directive: registry.resolve(edge.directive).text, + ...(edge.to === root.id ? {} : { to: edge.to }), + } + } + return edge.to === root.id ? { kind: edge.analyst, over: edge.over } : { kind: edge.analyst, over: edge.over, to: edge.to, directive: registry.resolve(edge.directive).text, - }, - ) - // Driver-destined analyzes directives are standing instructions for the ROOT: the findings - // arrive as bus events; the directive tells the driver what to do with them. + } + }) + // Driver-destined analyzes findings are standing knowledge for the ROOT: the findings arrive + // as bus events. For a lens edge the directive tells the driver what to do with them; for a + // node edge the directive already went to the analyst agent as its task. const driverAnalyzesBriefs = analyzes .filter((edge) => edge.to === root.id) - .map( - (edge) => - `Findings from analyst '${edge.analyst}' (over: ${edge.over.join(', ')}) will arrive as ` + - `finding events.\n${registry.resolve(edge.directive).text}`, + .map((edge) => + analystNodes.has(edge.analyst) + ? `Findings from analyst '${edge.analyst}' (a tool-equipped analyst agent node, over: ` + + `${edge.over.join(', ')}) will arrive as finding events.` + : `Findings from analyst '${edge.analyst}' (over: ${edge.over.join(', ')}) will arrive as ` + + `finding events.\n${registry.resolve(edge.directive).text}`, ) // ── The driver graph brief: which nodes it may spawn, by exact name ── @@ -753,9 +845,12 @@ export function runGraph(graph: AgentGraph, opts: RunGraphOptions): Promise 0 && opts.analysts - ? { analysts: opts.analysts, analyzeOnSettle: routes } + // Lens routes resolve against the registry; agent routes carry their own analyst profile, + // so a graph whose only analysts are nodes needs no registry at all. + ...(routes.length > 0 + ? { analyzeOnSettle: routes, ...(opts.analysts ? { analysts: opts.analysts } : {}) } : {}), + ...(opts.watchWorkers ? { watchWorkers: opts.watchWorkers } : {}), ...(opts.router ? { router: opts.router } : {}), ...(opts.brain ? { brain: opts.brain } : {}), ...(authorizeMessage ? { authorizeMessage } : {}), diff --git a/src/testing/fixtures/agent-improvement-proposal.json b/src/testing/fixtures/agent-improvement-proposal.json index ed72b64c..0e39fdda 100644 --- a/src/testing/fixtures/agent-improvement-proposal.json +++ b/src/testing/fixtures/agent-improvement-proposal.json @@ -1,6 +1,6 @@ { "changedSurfaces": ["prompt"], - "digest": "sha256:fb18b750e1c943753d05266412a3240b548e47b3a7b049c6b49baf4387d14578", + "digest": "sha256:2e3f72e29ef32b74e83428b297609ad49742c70e78ee18d3a30ef93f1270889f", "evaluation": { "decision": { "contributingChecks": [ @@ -4810,7 +4810,7 @@ ], "metadata": { "fixture": "agent-improvement-proposal", - "runtimeVersion": "0.123.1" + "runtimeVersion": "0.124.0" }, "objectives": [ { @@ -4921,8 +4921,8 @@ "baselineContentHash": "sha256:5c21ee53e513fc604cb09754e21c392b24a424da0ef37dbf8f1ee4a8a0b08f09", "candidateContentHash": "sha256:60fcbb1c728194bd51d7d19cb732d1c3f1881dce7e0a6266b41c8b98cfd65693", "kind": "agent-eval-loop", - "recordDigest": "sha256:9ff787d43525daee437bd6dd1518ef0d119ff0e3d91c5907068b29bac1a31d0a", - "runId": "agent-runtime-0.123.1-proposal-fixture", + "recordDigest": "sha256:c0025edf0ab73091b6d740c534ed4c3b4b660f90e82c4fadf0dd65fcd3a50d76", + "runId": "agent-runtime-0.124.0-proposal-fixture", "schema": "agent-candidate-experiment" } }, @@ -4949,5 +4949,5 @@ ], "kind": "agent-improvement-proposal", "proposedAt": "2026-07-10T01:00:00.000Z", - "runId": "agent-runtime-0.123.1-proposal-fixture" + "runId": "agent-runtime-0.124.0-proposal-fixture" } diff --git a/src/testing/fixtures/agent-profile-improvement-proposal.json b/src/testing/fixtures/agent-profile-improvement-proposal.json index bab9660b..d9d0b85c 100644 --- a/src/testing/fixtures/agent-profile-improvement-proposal.json +++ b/src/testing/fixtures/agent-profile-improvement-proposal.json @@ -1,6 +1,6 @@ { "changedSurfaces": ["prompt", "skills"], - "digest": "sha256:af06351a86081b1db9bdd82aa7c01aade535163410bb5d769e0ce5fcf67fabd1", + "digest": "sha256:cf9cab31f85543da571017e87c5cca7aa07656acd704d515d7832754f3ca4aec", "evaluation": { "decision": { "contributingChecks": [ @@ -1715,7 +1715,7 @@ ], "metadata": { "fixture": "agent-profile-improvement-proposal", - "runtimeVersion": "0.123.1" + "runtimeVersion": "0.124.0" }, "objectives": [ { @@ -1826,7 +1826,7 @@ "baselineContentHash": "sha256:21c495a37c418c10bde64fbaa188beddeed31f1f051ea60a6a6582a9ee0db704", "candidateContentHash": "sha256:103f77bc8481601eef1ad5fe6ba84a40dffabc3a44f421f8c8559121edab84e9", "kind": "agent-eval-loop", - "recordDigest": "sha256:b2cdacb049f574cded93ebeddfe3f90dbce5f8b9c16abd59b6e310d74eb07e08", + "recordDigest": "sha256:5d84308cbe6eb3596d9d3e55604d8830946622e922c5be16f81442f6b7cdf824", "runId": "profile-improvement-1", "schema": "agent-profile-improvement-experiment" } diff --git a/tests/examples/graph-topologies.test.ts b/tests/examples/graph-topologies.test.ts index ffa6dc10..e6b8e613 100644 --- a/tests/examples/graph-topologies.test.ts +++ b/tests/examples/graph-topologies.test.ts @@ -1,5 +1,5 @@ /** - * The four example graph topologies (`examples/graphs/`) run offline end-to-end, and each one's + * The five example graph topologies (`examples/graphs/`) run offline end-to-end, and each one's * DECISIVE ledger facts hold — the counts, outcomes, and destinations that make the example's * claim true, not just "it ran": * @@ -10,21 +10,26 @@ * silently dropped. * 2. best-of-n — exactly two delivered spawn traversals, one per candidate edge, and the winner * is the candidate whose settle passed the deliverable. - * 3. watchdog-steer — the corrective steer lands as the delegates edge's SECOND delivered - * traversal on the SAME live worker, before settle, carrying the detector's evidence. + * 3. watchdog-steer — the online finding arrives over the `watchWorkers` passthrough and the + * corrective steer lands as the delegates edge's SECOND delivered traversal on the SAME + * live worker, before settle, carrying the detector's evidence. * 4. shot-loop — two delivered shots under a 3-traversal cap (no exhaustion), each shot * followed by a delivered verify traversal to the reviewer root; the winner is the shot * whose tests pass. + * 5. analyst-agent-review — the analyzes analyst is a NODE: the reviewer agent is spawned on + * the implementer's settle and its settle output arrives as the driver's finding, one + * delivered analyzes traversal from the implementer's worker id. */ import { runGraph } from '@tangle-network/agent-runtime/kernel' import { describe, expect, it } from 'vitest' +import { analystAgentReview } from '../../examples/graphs/analyst-agent-review' import { bestOfN } from '../../examples/graphs/best-of-n' import { collaboratesReviewLoop } from '../../examples/graphs/collaborates-review-loop' import { shotLoop } from '../../examples/graphs/shot-loop' import { watchdogSteer } from '../../examples/graphs/watchdog-steer' -describe('examples/graphs — the four topologies run offline with truthful ledgers', () => { +describe('examples/graphs — the five topologies run offline with truthful ledgers', () => { it('collaborates-review-loop: every peer hop is mediated, ledgered, and addressed', async () => { const { graph, opts } = collaboratesReviewLoop() const res = await runGraph(graph, opts) @@ -106,4 +111,26 @@ describe('examples/graphs — the four topologies run offline with truthful ledg // The shot budget lives on the edge: 2 of 3 traversals used, nothing exhausted. expect(res.exhaustedEdges).toEqual([]) }) + + it('analyst-agent-review: the reviewer NODE runs as the analyst and its output is the finding', async () => { + const { graph, opts } = analystAgentReview() + const res = await runGraph(graph, opts) + + expect(res.result.kind).toBe('winner') + if (res.result.kind === 'winner') { + expect(res.result.out).toEqual({ built: 'implementer', attempt: 1 }) + } + + // One delivered spawn traversal (the implementer), one delivered analyzes traversal whose + // workerId is the SOURCE worker (driver-destined finding rows carry the analyzed worker). + expect(res.ledger.map((row) => [row.edge, row.outcome, row.workerId])).toEqual([ + ['delegates:driver->implementer', 'delivered', 'rev:s0'], + ['analyzes:reviewer:implementer->driver', 'delivered', 'rev:s0'], + ]) + // The finding carried the reviewer AGENT's settle output — real findings bytes, not a ping. + expect(res.ledger[1]!.bytes).toBeGreaterThan( + JSON.stringify({ verdict: 'needs-changes', defects: ['no tests'] }).length, + ) + expect(res.exhaustedEdges).toEqual([]) + }) }) diff --git a/tests/kernel/graph.test.ts b/tests/kernel/graph.test.ts index 24a9efb3..119d6dcc 100644 --- a/tests/kernel/graph.test.ts +++ b/tests/kernel/graph.test.ts @@ -14,12 +14,20 @@ * it, and a cap-killed no-winner run throws `GraphEdgeCapError` with the evidence attached. * 5. Analyzes edges route findings to a real DESTINATION (driver via the bus, a live worker via * an authorized steer carrying `analyst`), each traversal ledgered. - * 6. Oracles/analysts are ENVIRONMENT: a graph naming a node as its analyst is refused. + * 6. Oracle doctrine: an analyst that is a delegates TARGET is refused — a node that receives + * work can never also be the lens over it. * 7. The ledger never lies by omission or mislabel — the three truthfulness probes: * an undefined-findings analyst is ledgered `empty` (the event publishes; it cannot vanish * in the digest), an exhausted ANALYZES cap is observable but never GraphEdgeCapError (only * delegates caps refuse), and a spawn refused AFTER the factory ran is rewritten * `unpropagated`, never left `delivered`. + * 8. Analyst NODES: an analyzes edge naming a graph node spawns that node's pinned profile as + * a tool-equipped analyst WORKER on each matching settle — directive+evidence as its task, + * settle output as the findings, spend in the one conserved pool, same ledger rows — and + * validateGraph refuses the delegates-target, unknown, ambiguous, and analyzed-analyst forms. + * 9. watchWorkers passthrough: RunGraphOptions forwards the online detector panel to + * supervise(), so a live worker's stuck-loop finding reaches the driver on the bus with no + * leaf-seam wiring; omitted = off. */ import type { ToolSpan } from '@tangle-network/agent-eval' @@ -27,7 +35,7 @@ import type { AgentProfile } from '@tangle-network/agent-interface' import { describe, expect, it } from 'vitest' import { InMemorySpawnJournal } from '../../src/durable/spawn-journal' import { ValidationError } from '../../src/errors' -import type { MakeWorkerAgent } from '../../src/mcp/tools/coordination' +import type { MakeWorkerAgent, WorkerSpawnContext } from '../../src/mcp/tools/coordination' import { type AgentGraph, GraphEdgeCapError, runGraph } from '../../src/runtime/supervise/graph' import { analyzesFindingsReportPrompt, @@ -37,6 +45,7 @@ import { kernelPromptRegistry, promptHandle, } from '../../src/runtime/supervise/prompt-registry' +import { createPushTraceSource } from '../../src/runtime/supervise/trace-source' import type { Agent, AgentSpec, @@ -66,6 +75,9 @@ interface LeafOptions { awaitSteer?: boolean /** Expose a tool-trace source so settle-time analysts have evidence to read. */ withTrace?: boolean + /** Record this many IDENTICAL failing tool calls into a LIVE (push) trace at execute start — + * the stuck-loop storm the online detector panel fires on. Implies a live trace source. */ + storm?: number /** Settle by throwing instead of delivering. */ fail?: boolean /** Settle `done` with an INVALID verdict — a completed worker that delivered nothing usable. */ @@ -73,20 +85,24 @@ interface LeafOptions { } /** A leaf agent whose PROFILE (what the graph pinned + the directive) is captured for assertion. - * `opts` may be one option set for every node, or per-node-name option sets. */ + * `opts` may be one option set for every node, or per-node-name option sets. `contexts` captures + * each spawn's `WorkerSpawnContext` (the analyst marker + composed task assertions read it). */ function leafSeam( received: AgentProfile[], optsByNode: LeafOptions | Record = {}, + contexts?: Array, ): MakeWorkerAgent { const optionsFor = (name: string): LeafOptions => 'awaitSteer' in optsByNode || 'withTrace' in optsByNode || + 'storm' in optsByNode || 'fail' in optsByNode || 'invalid' in optsByNode ? (optsByNode as LeafOptions) : ((optsByNode as Record)[name] ?? {}) - return (profile) => { + return (profile, context) => { received.push(profile) + contexts?.push(context) const name = profile.name ?? 'leaf' const opts = optionsFor(name) let release: (() => void) | undefined @@ -95,6 +111,8 @@ function leafSeam( release = resolve }) : undefined + const pushed = + opts.storm !== undefined ? createPushTraceSource({ runId: `leaf-${name}` }) : undefined let artifact: ExecutorResult | undefined const ex: Executor = { runtime: 'router', @@ -106,15 +124,25 @@ function leafSeam( }, } : {}), - ...(opts.withTrace - ? { - traceSource: () => ({ - onSpan: () => () => {}, - collect: async () => [toolSpan(`leaf-${name}`)], - }), - } - : {}), + ...(pushed + ? { traceSource: () => pushed.source } + : opts.withTrace + ? { + traceSource: () => ({ + onSpan: () => () => {}, + collect: async () => [toolSpan(`leaf-${name}`)], + }), + } + : {}), async execute() { + for (let i = 0; i < (opts.storm ?? 0); i += 1) { + pushed?.record({ + toolName: 'bash', + args: { cmd: 'pnpm test' }, + status: 'error', + error: '1 failing', + }) + } if (gate) await gate if (opts.fail) throw new Error(`${name}: deliberate failure`) artifact = { @@ -741,7 +769,9 @@ describe('runGraph — analyzes edges (analysts are environment, findings get a expect(analyzed[0]!.reason).toContain('traversal-cap-exhausted (max 0)') }) - it('refuses a graph whose analyst is a NODE — oracles/analysts are environment', () => { + it('refuses an analyst node that is a delegates TARGET — oracle doctrine holds structurally', () => { + // 'worker' receives work from the driver, so it can never also be the lens over that work — + // an analyst NODE is legal only with no delegates edge pointing at it. const graph = twoNodeGraph({ edges: [ { @@ -761,7 +791,350 @@ describe('runGraph — analyzes edges (analysts are environment, findings get a }) expect(() => runGraph(graph, { analysts, makeWorkerAgent: leafSeam([]), brain: scriptedBrain([]) }), - ).toThrow(/analysts are ENVIRONMENT, never nodes/) + ).toThrow(/oracle doctrine: an analyst is never delegated to/) + }) +}) + +describe('runGraph — analyst NODES (the analyzes lens as a tool-equipped agent)', () => { + /** driver → worker, with an 'inspector' NODE as the analyzes analyst. The inspector has no + * delegates edge — it is spawned by the settle hook, with its own pinned profile. */ + const inspectorGraph = (to: 'driver' | 'fixer'): AgentGraph => ({ + nodes: [ + { id: 'driver', profile: { name: 'driver', prompt: { systemPrompt: 'Drive.' } } }, + { id: 'worker', profile: { name: 'worker', prompt: { systemPrompt: 'Build.' } } }, + ...(to === 'fixer' + ? [{ id: 'fixer', profile: { name: 'fixer', prompt: { systemPrompt: 'Fix.' } } }] + : []), + { id: 'inspector', profile: { name: 'inspector', prompt: { systemPrompt: 'Inspect.' } } }, + ], + edges: [ + { + kind: 'delegates', + from: 'driver', + to: 'worker', + directive: promptHandle('delegates/worker-brief/v1'), + }, + ...(to === 'fixer' + ? [ + { + kind: 'delegates', + from: 'driver', + to: 'fixer', + directive: promptHandle('delegates/worker-brief/v1'), + } as const, + ] + : []), + { + kind: 'analyzes', + analyst: 'inspector', + over: ['worker'], + to, + directive: promptHandle('analyzes/findings-report/v1'), + }, + ], + deliverable: { describe: 'the built artifact', check: (out) => out !== undefined }, + budget: { maxIterations: 20, maxTokens: 50_000 }, + }) + + it('spawns the analyst node on settle: pinned profile, directive+evidence task, settle output as findings, spend conserved', async () => { + const received: AgentProfile[] = [] + const contexts: Array = [] + const journal = new InMemorySpawnJournal() + const res = await runGraph(inspectorGraph('driver'), { + runId: 'gan', + journal, + makeWorkerAgent: leafSeam(received, { worker: { withTrace: true }, inspector: {} }, contexts), + brain: scriptedBrain([ + { + toolCalls: [ + { name: 'spawn_agent', arguments: { profile: { name: 'worker' }, task: 'build it' } }, + ], + }, + { toolCalls: [{ name: 'await_event', arguments: {} }] }, // settled(worker) + { toolCalls: [{ name: 'await_event', arguments: {} }] }, // finding(inspector output) + { content: 'done' }, + ]), + }) + expect(res.result.kind).toBe('winner') + + // Node pinning holds for the analyst too: the inspector ran under ITS canonical profile, + // spawned by the settle hook (context.analyst), never by a driver tool call. + expect(received.map((p) => p.name)).toEqual(['worker', 'inspector']) + expect(received[1]!.prompt?.systemPrompt).toBe('Inspect.') + const inspectorContext = contexts[1] + expect(inspectorContext?.analyst).toBe('inspector') + expect(inspectorContext?.label).toBe('analyst:inspector') + // The task is the analyzes directive PLUS the settled worker's trace evidence. + const task = String(inspectorContext?.task) + expect(task).toContain(analyzesFindingsReportPrompt.text) + expect(task).toContain('write_file') // the worker's recorded tool span crossed as evidence + expect(task).toContain("settled worker 'gan:s0'") + + // The analyzes traversal is ledgered exactly like a registry analyst's: driver-destined + // finding, source worker as the workerId, directive + findings bytes. + const analyzed = res.ledger.filter((row) => row.kind === 'analyzes') + expect(analyzed).toHaveLength(1) + expect(analyzed[0]!.edge).toBe('analyzes:inspector:worker->driver') + expect(analyzed[0]!.outcome).toBe('delivered') + expect(analyzed[0]!.workerId).toBe('gan:s0') + expect(analyzed[0]!.bytes).toBe( + Buffer.byteLength(analyzesFindingsReportPrompt.text, 'utf8') + + Buffer.byteLength(JSON.stringify({ built: 'inspector' }), 'utf8'), + ) + // The journal twin exists — same observable-edge contract as every traversal. + const events = (await journal.loadTree('gan')) ?? [] + const edgeEvents = events.filter( + (ev): ev is Extract => + ev.kind === 'edge' && ev.edge.kind === 'analyzes', + ) + expect(edgeEvents).toHaveLength(1) + expect(edgeEvents[0]).toMatchObject({ outcome: 'delivered', traversal: 1 }) + + // Budget accounting: the analyst run's spend lands in the graph's ONE conserved Spend — + // worker (5/5) + inspector (5/5). + if (res.result.kind === 'winner') { + expect(res.result.spentTotal.tokens.input).toBe(10) + expect(res.result.spentTotal.tokens.output).toBe(10) + } + }) + + it("routes an analyst node's findings to a live WORKER through the same authorized steer machinery", async () => { + const received: AgentProfile[] = [] + const res = await runGraph(inspectorGraph('fixer'), { + runId: 'gar', + makeWorkerAgent: leafSeam(received, { + worker: { withTrace: true }, + fixer: { awaitSteer: true }, + inspector: {}, + }), + brain: scriptedBrain([ + { + toolCalls: [ + { name: 'spawn_agent', arguments: { profile: { name: 'worker' }, task: 'build' } }, + ], + }, + { + toolCalls: [ + { name: 'spawn_agent', arguments: { profile: { name: 'fixer' }, task: 'stand by' } }, + ], + }, + { toolCalls: [{ name: 'await_event', arguments: {} }] }, // settled(worker) + { toolCalls: [{ name: 'await_event', arguments: {} }] }, // finding (audit copy) + { toolCalls: [{ name: 'await_event', arguments: {} }] }, // settled(fixer, released by the steer) + { content: 'done' }, + ]), + }) + expect(res.result.kind).toBe('winner') + expect(received.map((p) => p.name)).toEqual(['worker', 'fixer', 'inspector']) + const analyzed = res.ledger.filter((row) => row.kind === 'analyzes') + expect(analyzed).toHaveLength(1) + expect(analyzed[0]!.edge).toBe('analyzes:inspector:worker->fixer') + expect(analyzed[0]!.outcome).toBe('delivered') + // The routed delivery reached the DESTINATION worker (the steer leg), releasing it. + expect(analyzed[0]!.workerId).toBe('gar:s1') + // The steered instruction is the BARE findings — the directive was the analyst's task. + expect(analyzed[0]!.bytes).toBe( + Buffer.byteLength(JSON.stringify({ built: 'inspector' }), 'utf8'), + ) + }) + + it('refuses an UNKNOWN analyst reference — neither node nor registry lens', () => { + const withRegistry = twoNodeGraph({ + edges: [ + { + kind: 'delegates', + from: 'driver', + to: 'worker', + directive: promptHandle('delegates/worker-brief/v1'), + }, + { + kind: 'analyzes', + analyst: 'ghost', + over: ['worker'], + to: 'driver', + directive: promptHandle('analyzes/findings-report/v1'), + }, + ], + }) + expect(() => + runGraph(withRegistry, { + analysts: { + kinds: [{ id: 'convergence', description: 'x', area: 'progress' }], + run: async () => [], + }, + makeWorkerAgent: leafSeam([]), + brain: scriptedBrain([]), + }), + ).toThrow(/analyst 'ghost' is neither a graph node nor in the analysts registry/) + // Without a registry the refusal names the missing registry, not a phantom lens. + expect(() => + runGraph(withRegistry, { makeWorkerAgent: leafSeam([]), brain: scriptedBrain([]) }), + ).toThrow(/analyst 'ghost' is not a graph node, and no RunGraphOptions.analysts registry/) + }) + + it('refuses an AMBIGUOUS analyst id — both a graph node and a registry lens', () => { + expect(() => + runGraph(inspectorGraph('driver'), { + analysts: { + kinds: [{ id: 'inspector', description: 'the same id as the node', area: 'review' }], + run: async () => [], + }, + makeWorkerAgent: leafSeam([]), + brain: scriptedBrain([]), + }), + ).toThrow(/'inspector' is BOTH a graph node and a lens/) + }) + + it('refuses the ROOT as an analyst — the root is the driver, not a lens', () => { + const graph = twoNodeGraph({ + edges: [ + { + kind: 'delegates', + from: 'driver', + to: 'worker', + directive: promptHandle('delegates/worker-brief/v1'), + }, + { + kind: 'analyzes', + analyst: 'driver', + over: ['worker'], + to: 'driver', + directive: promptHandle('analyzes/findings-report/v1'), + }, + ], + }) + expect(() => + runGraph(graph, { makeWorkerAgent: leafSeam([]), brain: scriptedBrain([]) }), + ).toThrow(/names the ROOT as its analyst/) + }) + + it('refuses an analyzes edge OVER an analyst node — it would silently never fire', () => { + const graph = inspectorGraph('driver') + const overAnalyst: AgentGraph = { + ...graph, + edges: [ + ...graph.edges, + { + kind: 'analyzes', + analyst: 'convergence', + over: ['inspector'], + to: 'driver', + directive: promptHandle('analyzes/findings-report/v1'), + }, + ], + } + expect(() => + runGraph(overAnalyst, { + analysts: { + kinds: [{ id: 'convergence', description: 'x', area: 'progress' }], + run: async () => [], + }, + makeWorkerAgent: leafSeam([]), + brain: scriptedBrain([]), + }), + ).toThrow(/analyst nodes are not analyzable/) + }) +}) + +describe('runGraph — watchWorkers passthrough (the online detector panel over live workers)', () => { + it('forwards watchWorkers to supervise(): a detector finding reaches the driver on the bus, no leaf-seam wiring', async () => { + // The builder blocks until a steer arrives while its live trace replays the same failing + // command — the storm the shipped repeated-action/error-streak panel fires on. NOTHING here + // wires watchTrace at the leaf seam: the passthrough is the whole test. + let sawOnlineFinding = false + let spawned = false + let steered = false + const brain: ToolLoopChat = async (messages) => { + if ( + messages.some( + (m) => typeof m.content === 'string' && m.content.includes('"analyst":"online:'), + ) + ) { + sawOnlineFinding = true + } + if (!spawned) { + spawned = true + return { + toolCalls: [ + { + id: 'c1', + name: 'spawn_agent', + arguments: JSON.stringify({ profile: { name: 'worker' }, task: 'build it' }), + }, + ], + } + } + if (!sawOnlineFinding) { + // Yield one macrotask so the spawned executor starts and the detector publishes before + // this pull — the deterministic offline ordering, not a sleep. + await new Promise((resolve) => setImmediate(resolve)) + return { + toolCalls: [ + { id: 'cw', name: 'await_event', arguments: JSON.stringify({ kinds: ['finding'] }) }, + ], + } + } + if (!steered) { + steered = true + return { + toolCalls: [ + { + id: 'c2', + name: 'steer_agent', + arguments: JSON.stringify({ + workerId: 'gw:s0', + instruction: 'Watchdog fired: stop repeating the failing command and deliver.', + }), + }, + ], + } + } + const settled = messages.some( + (m) => typeof m.content === 'string' && m.content.includes('"type":"settled"'), + ) + if (!settled) { + return { toolCalls: [{ id: 'c3', name: 'await_event', arguments: JSON.stringify({}) }] } + } + return { content: 'done', toolCalls: [] } + } + const res = await runGraph(twoNodeGraph(), { + runId: 'gw', + makeWorkerAgent: leafSeam([], { worker: { awaitSteer: true, storm: 5 } }), + watchWorkers: { maxFindingsPerWorker: 1 }, + brain, + }) + expect(res.result.kind).toBe('winner') + // The detector finding crossed the bus TO THE DRIVER under runGraph — the passthrough works. + expect(sawOnlineFinding).toBe(true) + // The corrective steer is the mid-run leg of the delegates edge, on the same live worker. + expect(res.ledger.map((row) => [row.traversal, row.outcome, row.workerId])).toEqual([ + [1, 'delivered', 'gw:s0'], + [2, 'delivered', 'gw:s0'], + ]) + }) + + it('stays OFF when omitted — no online findings without watchWorkers', async () => { + const seen: Array>> = [] + const res = await runGraph(twoNodeGraph(), { + runId: 'gw0', + makeWorkerAgent: leafSeam([], { worker: { storm: 5 } }), + brain: scriptedBrain( + [ + { + toolCalls: [ + { name: 'spawn_agent', arguments: { profile: { name: 'worker' }, task: 'build' } }, + ], + }, + { toolCalls: [{ name: 'await_event', arguments: {} }] }, + { content: 'done' }, + ], + seen, + ), + }) + expect(res.result.kind).toBe('winner') + // No turn's transcript ever carried an online finding: the panel is opt-in, off by default. + const transcript = JSON.stringify(seen) + expect(transcript).not.toContain('online:') }) })