Skip to content

Commit c9dfb98

Browse files
An agent node on the Claude CLI now delegates instead of refusing (#58)
`AgentNode` used to raise `AgentConfigError` when handed the Claude CLI: the CLI has no tool-calling wire format, so GraphARC cannot run its own gated loop over it. That was accurate and useless — the CLI *is* a complete agent, and refusing to use it meant the subscription backend could do no agentic work inside a graph at all. It now hands the whole loop to Claude Code's headless agent, reusing the same machinery `grapharc agent --executor claude-cli` already used. That path's subprocess-and-parse core is extracted as `delegate_task()`, so the CLI command and the node cannot drift on what actually gets spawned. This is a real widening of the trust boundary and is treated as one: - `DelegatedToolUseWarning` at construction, naming each thing given up — every tool Claude Code has, not checked by this graph's policy, not confined by the sandbox, token figure self-reported. Its own category, so `-W error::grapharc.harness.agent.DelegatedToolUseWarning` restores the old refusal for anyone who wants it. - **Every trace event from a delegated node carries `executor=delegated`**, plus the permission mode and `governed_by`. A warning is gone by the time someone reads the run back; without this, a JSONL reader six months later sees an agent node that completed and cannot tell that its tool calls never reached the permission engine. That is the project's own claim about its traces, so the marking is the part that keeps it true. One thing found by running it rather than reasoning about it: omitting `--allowedTools` does **not** mean "every tool". It leaves Claude Code's own gating in place, and headless there is nobody to approve a Write — the first delegated node came back reporting it could not create the file. Only `--permission-mode bypassPermissions` means what "everything Claude Code has" was meant to mean, so that is what a delegated node runs under, named in the warning and recorded in the trace. `delegate_task` takes the two as separate arguments because conflating them is the trap. Detection is on `_llm_type == "grapharc-claude-cli"`, not on "does this model lack `bind_tools`" — `ScriptedChatModel` lacks it too, and matching that way would have silently delegated every mocked agent in the suite to a real subprocess. A test pins that. Verified live end to end: a node given `get_model("claude-cli")` created a file in its workspace using Claude Code's own tools (2 turns, target_met), and the trace shows `executor=delegated` with `permission_mode=bypassPermissions` on every event. Docs updated in README's limits and `stdlib`'s module docstring, both of which asserted the old refusal. Full suite green on 3.12; ruff clean. Co-authored-by: Claude <noreply@anthropic.com>
1 parent 6f9463e commit c9dfb98

5 files changed

Lines changed: 414 additions & 6 deletions

File tree

README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -482,7 +482,7 @@ Re-derived on 2026-07-28 by running each item, not by reading the commit log.
482482
- **Cost is recorded when a backend reports one, estimated when it does not.** Both gateways publish the provider's `cost_usd` through the same `llm_output` envelope, the runtime's usage callback writes it onto the node's `end` event, and an agent's `model` events carry the per-call breakdown. A backend that reports no price still falls back to a `RateCard` estimate, and the two figures stay apart — `recorded_cost_usd` is never a guess. Still missing: no tenant on a trace event, so per-tenant attribution is not offered.
483483
- **A node's tokens are its own, not the run's movement while it ran.** Worth stating because it was the other way round: an `end` event carried the difference between two readings of the run's *shared* meter, so under fan-out the workers' windows overlapped and each was credited with its siblings' concurrent spend. Three workers costing 8 tokens each traced as 24/16/8, and `metrics` and `cost` agreed on 48 for 24 tokens of real work — doubling the estimated bill purely because the work ran in parallel. Attribution now comes from a per-node scope on the meter, so the same work costs the same serially and in parallel; a hand charge the usage callback never saw still lands on the node that made it.
484484
- **A planning round is an envelope, not a measurement.** A `round` event used to carry the planner's `tokens` and the round's `duration_ms`, both of which `metrics`, `cost` and `replay` add on top of node totals — and the planner's spend was already reported by its own `plan` event, so it was counted twice, and a round's duration encloses the plan plus every node it ran. Neither is on the event now; both are on its `state_delta` as `round_tokens` / `round_iterations` / `round_duration_ms`, where no reader sums them. `RoundRecord.iterations` also holds a figure now rather than always `0`.
485-
- **The Claude CLI backend is completion-only.** Tool calling and structured output need one of the OpenAI-wire backends: `openrouter`, `openai`, or a local `ollama`.
485+
- **The Claude CLI backend is completion-only, and an agent node on it is *delegated* rather than governed.** The CLI has no tool-calling wire format, so GraphARC cannot run its own gated loop over it. Rather than refuse, `AgentNode` hands the whole loop to Claude Code's headless agent — which means every tool Claude Code has, under its `bypassPermissions` mode: those calls are not checked by this graph's permission policy, not confined by the sandbox executor, and the token figure is the sub-agent's own rather than one GraphARC metered call by call. The workspace boundary and the wall-clock ceiling still hold. It warns on `DelegatedToolUseWarning` at construction and marks every trace event `executor=delegated`, so a run stays auditable as delegated; filter that warning to an error to get the old refusal back. Structured output still needs an OpenAI-wire backend: `openrouter`, `openai`, or a local `ollama`.
486486
- **A session turn is synchronous**, and a runner claim is a claim rather than a lease — nothing reclaims a session whose runner died holding it.
487487
- **`.env` is found by walking up parent directories; `grapharc.toml` is not.** The config layer refuses an upward search on purpose — a run must not be governed by a file you did not know about. The credential loader predates that decision and still searches upward, so the thing that *spends money* is discovered more eagerly than the thing that *constrains* it.
488488
- **`grapharc run` has no budget unless you give it one.** Set any of `--max-tokens`, `--max-iterations`, `--max-seconds`, or `--max-concurrency`; without them each dimension is unlimited and the gate admits a topology of any worst-case cost.

grapharc/cli/delegate.py

Lines changed: 132 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,7 @@
2525
import shutil
2626
import subprocess
2727
import uuid
28+
from dataclasses import dataclass
2829
from pathlib import Path
2930

3031
from grapharc.cli import style
@@ -35,6 +36,130 @@
3536
DEFAULT_DELEGATED_TOOLS = ("Read", "Glob", "Grep", "LS", "Edit", "Write", "Bash")
3637

3738

39+
40+
# ---------------------------------------------------------------------------
41+
# The reusable core. `run_delegated` below is the CLI's presentation of it, and
42+
# `grapharc.harness.agent.AgentNode` calls it directly when its backend is the
43+
# Claude CLI — one implementation, so the two paths cannot drift on what
44+
# actually gets spawned.
45+
46+
47+
class DelegationError(Exception):
48+
"""A delegated run could not be started, or came back unreadable.
49+
50+
`reason` is the machine-readable form, written to the trace so a run that
51+
failed here is distinguishable afterwards from one that ran and refused.
52+
"""
53+
54+
def __init__(self, message: str, *, reason: str) -> None:
55+
super().__init__(message)
56+
self.reason = reason
57+
58+
59+
@dataclass(frozen=True)
60+
class DelegatedRun:
61+
"""What Claude Code reported back. Every number here is *its* figure.
62+
63+
`tokens_reported` is named for what it is: the sub-agent's own count, not
64+
something GraphARC metered call by call. Nothing in this object was
65+
observed by the permission engine.
66+
"""
67+
68+
ok: bool
69+
answer: str
70+
reason: str
71+
turns: int
72+
tokens_reported: int
73+
cost_usd: float | None
74+
session_id: str | None
75+
allowed: list[str] | None # None == no --allowedTools restriction at all
76+
denied: list[str]
77+
78+
79+
def delegate_task(
80+
task: str,
81+
*,
82+
workspace: Path,
83+
model: str | None = None,
84+
allow: list[str] | None = None,
85+
deny: list[str] | None = None,
86+
max_turns: int = 20,
87+
max_seconds: float | None = None,
88+
system_prompt: str | None = None,
89+
permission_mode: str | None = None,
90+
) -> DelegatedRun:
91+
"""Run one headless `claude -p` agent loop in `workspace`.
92+
93+
Two separate axes, and conflating them is a trap worth naming. `allow`
94+
controls *which* tools exist; `permission_mode` controls whether the ones
95+
that mutate anything are allowed to run without a human answering a prompt.
96+
Omitting `--allowedTools` does **not** mean "every tool": it means Claude
97+
Code's own default gating, and headless there is nobody to approve a Write,
98+
so the sub-agent reports back that it could not create the file. Measured,
99+
not assumed. `permission_mode="bypassPermissions"` is what actually means
100+
"everything", and it means it literally — no checks at all.
101+
102+
Either way the caller is responsible for having said so out loud;
103+
`AgentNode` warns at construction and marks every trace event.
104+
"""
105+
binary = shutil.which("claude")
106+
if binary is None:
107+
raise DelegationError(
108+
"the delegated executor shells out to `claude`, which is not on PATH; "
109+
"install Claude Code or use a tool-calling backend",
110+
reason="claude_not_found",
111+
)
112+
113+
workspace = Path(workspace).expanduser().resolve()
114+
workspace.mkdir(parents=True, exist_ok=True)
115+
116+
argv = [binary, "-p", task, "--output-format", "json", "--max-turns", str(max_turns)]
117+
if allow is not None:
118+
argv += ["--allowedTools", ",".join(allow)]
119+
if deny:
120+
argv += ["--disallowedTools", ",".join(deny)]
121+
if permission_mode:
122+
argv += ["--permission-mode", permission_mode]
123+
if model:
124+
argv += ["--model", model]
125+
if system_prompt:
126+
argv += ["--append-system-prompt", system_prompt]
127+
128+
try:
129+
completed = subprocess.run(
130+
argv, cwd=workspace, capture_output=True, text=True, timeout=max_seconds
131+
)
132+
except subprocess.TimeoutExpired as exc:
133+
raise DelegationError(
134+
f"max_seconds ({max_seconds}) reached; the delegated run was stopped",
135+
reason="deadline_exceeded",
136+
) from exc
137+
138+
try:
139+
report = json.loads(completed.stdout)
140+
except (json.JSONDecodeError, ValueError) as exc:
141+
detail = (completed.stderr or completed.stdout or "").strip()[-500:]
142+
raise DelegationError(
143+
f"claude exited {completed.returncode} without a readable JSON report: {detail}",
144+
reason="unreadable_report",
145+
) from exc
146+
147+
usage = report.get("usage") or {}
148+
met = report.get("subtype") == "success" and not report.get("is_error", False)
149+
return DelegatedRun(
150+
ok=met,
151+
answer=str(report.get("result") or "").strip(),
152+
reason="target_met" if met else str(report.get("subtype") or "error"),
153+
turns=int(report.get("num_turns") or 0),
154+
tokens_reported=int(usage.get("input_tokens") or 0)
155+
+ int(usage.get("output_tokens") or 0),
156+
cost_usd=report.get("total_cost_usd"),
157+
session_id=report.get("session_id"),
158+
allowed=list(allow) if allow is not None else None,
159+
denied=list(deny or []),
160+
)
161+
162+
38163
def run_delegated(
39164
task: str,
40165
*,
@@ -217,4 +342,10 @@ def run_delegated(
217342
return EXIT_OK if met else EXIT_FAILED
218343

219344

220-
__all__ = ["DEFAULT_DELEGATED_TOOLS", "run_delegated"]
345+
__all__ = [
346+
"DEFAULT_DELEGATED_TOOLS",
347+
"DelegatedRun",
348+
"DelegationError",
349+
"delegate_task",
350+
"run_delegated",
351+
]

grapharc/harness/agent.py

Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -53,8 +53,10 @@
5353
import types
5454
import typing
5555
import uuid
56+
import warnings
5657
from collections.abc import Callable, Mapping
5758
from enum import StrEnum
59+
from pathlib import Path
5860
from typing import Any
5961

6062
from langchain_core.language_models.chat_models import BaseChatModel
@@ -297,6 +299,48 @@ def _coerce_args(raw: Any) -> dict[str, Any]:
297299
raise TypeError(f"tool arguments must be a JSON object, got {type(raw).__name__}")
298300

299301

302+
303+
class DelegatedToolUseWarning(UserWarning):
304+
"""An `AgentNode` is running Claude Code's tool loop instead of GraphARC's.
305+
306+
Its own category so it can be filtered, asserted on in tests, or turned
307+
into an error with `-W error::grapharc.harness.agent.DelegatedToolUseWarning`
308+
by anyone who wants the old refusal back.
309+
"""
310+
311+
312+
#: What the delegated loop runs under. `bypassPermissions` is Claude Code's
313+
#: "no checks at all" mode, and it is deliberate: omitting `--allowedTools`
314+
#: leaves its default gating in place, and headless there is no one to approve
315+
#: a Write — the sub-agent simply reports that it could not create the file.
316+
#: "Every tool Claude Code has" only means that with this set.
317+
DELEGATED_PERMISSION_MODE = "bypassPermissions"
318+
319+
_DELEGATION_WARNING = (
320+
"agent node {name!r} is backed by the Claude CLI, which has no tool-calling "
321+
"wire format, so GraphARC cannot run its own tool loop over it. The whole "
322+
"loop is delegated to Claude Code's headless agent, which means: it uses "
323+
"EVERY tool Claude Code has (Bash, Write, WebFetch, Task, ...) under its "
324+
"bypassPermissions mode, so those calls are NOT checked by this graph's "
325+
"permission policy, NOT confined by the sandbox executor, and NOT gated by "
326+
"Claude Code's own prompts either. The token figure is what the sub-agent "
327+
"reports rather than what GraphARC metered. The workspace boundary and the wall-clock "
328+
"ceiling still apply. Every trace event from this node is marked "
329+
"executor=delegated so the run stays auditable; use a tool-calling backend "
330+
"(openrouter/*, openai/*, ollama/*) for a governed loop."
331+
)
332+
333+
334+
def _is_claude_cli(model: Any) -> bool:
335+
"""Is this the Claude CLI backend?
336+
337+
Matched on `_llm_type` rather than `isinstance`, so this module does not
338+
import the gateway, and rather than "does it lack bind_tools" — which is
339+
also true of `ScriptedChatModel` and would silently delegate every mock.
340+
"""
341+
return getattr(model, "_llm_type", None) == "grapharc-claude-cli"
342+
343+
300344
class AgentNode:
301345
"""A tool-using agent loop, shaped as a GraphARC node.
302346
@@ -341,6 +385,16 @@ def __init__(
341385
self.prompt_fn = prompt_fn
342386
self.trace = trace
343387
self.max_tool_result_chars = max_tool_result_chars
388+
#: True when the backend is the Claude CLI, which has no tool-calling
389+
#: wire format and therefore cannot be driven as a raw model. The loop
390+
#: is handed to Claude Code instead — see `_run_delegated`.
391+
self.delegated = _is_claude_cli(model)
392+
if self.delegated:
393+
warnings.warn(
394+
_DELEGATION_WARNING.format(name=name),
395+
DelegatedToolUseWarning,
396+
stacklevel=2,
397+
)
344398

345399
@property
346400
def writes(self) -> set[str]:
@@ -385,6 +439,9 @@ def run(self, prompt: str, ctx: RunContext | None = None) -> AgentResult:
385439
run_id=uuid.uuid4().hex[:12], graph=self.name, meter=BudgetMeter(Budget())
386440
)
387441

442+
if self.delegated:
443+
return self._run_delegated(prompt, ctx)
444+
388445
model = self._bind_tools()
389446
messages: list[BaseMessage] = [
390447
SystemMessage(content=self.system_prompt),
@@ -524,6 +581,89 @@ def run(self, prompt: str, ctx: RunContext | None = None) -> AgentResult:
524581

525582
# -- internals ------------------------------------------------------------
526583

584+
def _run_delegated(self, prompt: str, ctx: RunContext) -> AgentResult:
585+
"""Hand the whole task to Claude Code's headless agent.
586+
587+
The trade is stated in `_DELEGATION_WARNING` and repeated on every trace
588+
event this writes, because a warning at construction is gone by the time
589+
anyone reads the run back. `executor="delegated"` on the events is what
590+
stops a reader six months later from assuming this graph's permission
591+
policy saw these tool calls. It did not.
592+
593+
The workspace boundary and the wall-clock ceiling still hold: the CLI is
594+
spawned with `cwd` set to the harness workspace, and `max_seconds` is
595+
enforced from outside by the subprocess timeout. Everything finer than
596+
that is Claude Code's.
597+
"""
598+
from grapharc.cli.delegate import DelegationError, delegate_task
599+
600+
remaining = ctx.meter.remaining_seconds() if ctx.meter else None
601+
step = 1
602+
if self.trace is not None:
603+
self.trace.event(
604+
run_id=ctx.run_id, graph=ctx.graph, node=self.name, phase="model",
605+
step=step, thread_id=ctx.thread_id, attempt=ctx.attempt,
606+
state_delta={"executor": "delegated", "tools": "all of Claude Code's",
607+
"permission_mode": DELEGATED_PERMISSION_MODE,
608+
"governed_by": "Claude Code, not this graph's policy"},
609+
)
610+
try:
611+
workspace = getattr(self.harness.executor, "workspace", None)
612+
if workspace is None:
613+
raise DelegationError(
614+
"the delegated executor needs a workspace directory, and this "
615+
f"harness's executor ({type(self.harness.executor).__name__}) "
616+
"does not expose one",
617+
reason="no_workspace",
618+
)
619+
run = delegate_task(
620+
prompt,
621+
workspace=Path(workspace),
622+
max_turns=self.max_iterations,
623+
max_seconds=remaining,
624+
system_prompt=self.system_prompt,
625+
permission_mode=DELEGATED_PERMISSION_MODE,
626+
)
627+
except DelegationError as exc:
628+
if self.trace is not None:
629+
self.trace.event(
630+
run_id=ctx.run_id, graph=ctx.graph, node=self.name, phase="stop",
631+
step=step, thread_id=ctx.thread_id, attempt=ctx.attempt,
632+
state_delta={"executor": "delegated", "termination_reason": exc.reason},
633+
error=str(exc),
634+
)
635+
return AgentResult(
636+
termination_reason=StopReason.ERROR, iterations=0, note=str(exc)
637+
)
638+
639+
# The sub-agent's own count, charged so a budget is not simply blind to
640+
# a delegated node — but named `tokens_reported` everywhere it surfaces,
641+
# because GraphARC did not meter these call by call.
642+
if ctx.meter and run.tokens_reported:
643+
ctx.meter.charge_tokens(run.tokens_reported)
644+
reason = StopReason.TARGET_MET if run.ok else StopReason.ERROR
645+
if self.trace is not None:
646+
self.trace.event(
647+
run_id=ctx.run_id, graph=ctx.graph, node=self.name, phase="stop",
648+
step=step, thread_id=ctx.thread_id, attempt=ctx.attempt,
649+
tokens=run.tokens_reported or None,
650+
cost_usd=run.cost_usd,
651+
state_delta={"executor": "delegated", "termination_reason": reason.value,
652+
"turns": run.turns, "tokens_reported": run.tokens_reported,
653+
"session_id": run.session_id},
654+
)
655+
return AgentResult(
656+
output=run.answer if run.ok else "",
657+
partial_output="" if run.ok else run.answer,
658+
termination_reason=reason,
659+
iterations=run.turns,
660+
note=(
661+
f"delegated to Claude Code: {run.turns} turn(s), "
662+
f"{run.tokens_reported:,} tokens reported by the sub-agent, "
663+
"tool calls not checked by this graph's policy"
664+
),
665+
)
666+
527667
def _bind_tools(self) -> Any:
528668
"""Bind the policy-filtered tool set. Denied tools are never described."""
529669
schemas = tool_schemas(self.harness)

grapharc/stdlib.py

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,10 +23,14 @@
2323
not chosen by the model at run time. `apply_change` is the only one that can
2424
write, which is what makes it the one worth denying by default.
2525
26-
All but one of the agent kinds need a **tool-calling** backend: `AgentNode`
27-
refuses a model with no `bind_tools` rather than degrading, so the Claude CLI
28-
subscription cannot drive them. `summarize` is the exception — it is toolless by
29-
design, so it binds nothing and runs anywhere.
26+
All but one of the agent kinds want a **tool-calling** backend, because that is
27+
the only way GraphARC can run the loop itself and gate each call. Given the
28+
Claude CLI — which has no tool-calling wire format — `AgentNode` delegates the
29+
whole loop to Claude Code instead, warning at construction and marking the
30+
trace: the fixed allowlists described above do not apply to a delegated run,
31+
because the tools are Claude Code's rather than this registry's. `summarize` is
32+
the exception either way — it is toolless by design, so it binds nothing and
33+
runs anywhere.
3034
3135
Registered but denied is the interesting state: **given a model**, `apply_change`
3236
is in the registry because changing files is a real capability, and the default

0 commit comments

Comments
 (0)