Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -482,7 +482,7 @@ Re-derived on 2026-07-28 by running each item, not by reading the commit log.
- **Cost is recorded when a backend reports one, estimated when it does not.** Both gateways publish the provider's `cost_usd` through the same `llm_output` envelope, the runtime's usage callback writes it onto the node's `end` event, and an agent's `model` events carry the per-call breakdown. A backend that reports no price still falls back to a `RateCard` estimate, and the two figures stay apart — `recorded_cost_usd` is never a guess. Still missing: no tenant on a trace event, so per-tenant attribution is not offered.
- **A node's tokens are its own, not the run's movement while it ran.** Worth stating because it was the other way round: an `end` event carried the difference between two readings of the run's *shared* meter, so under fan-out the workers' windows overlapped and each was credited with its siblings' concurrent spend. Three workers costing 8 tokens each traced as 24/16/8, and `metrics` and `cost` agreed on 48 for 24 tokens of real work — doubling the estimated bill purely because the work ran in parallel. Attribution now comes from a per-node scope on the meter, so the same work costs the same serially and in parallel; a hand charge the usage callback never saw still lands on the node that made it.
- **A planning round is an envelope, not a measurement.** A `round` event used to carry the planner's `tokens` and the round's `duration_ms`, both of which `metrics`, `cost` and `replay` add on top of node totals — and the planner's spend was already reported by its own `plan` event, so it was counted twice, and a round's duration encloses the plan plus every node it ran. Neither is on the event now; both are on its `state_delta` as `round_tokens` / `round_iterations` / `round_duration_ms`, where no reader sums them. `RoundRecord.iterations` also holds a figure now rather than always `0`.
- **The Claude CLI backend is completion-only.** Tool calling and structured output need one of the OpenAI-wire backends: `openrouter`, `openai`, or a local `ollama`.
- **The Claude CLI backend is completion-only, and an agent node on it is *delegated* rather than governed.** The CLI has no tool-calling wire format, so GraphARC cannot run its own gated loop over it. Rather than refuse, `AgentNode` hands the whole loop to Claude Code's headless agent — which means every tool Claude Code has, under its `bypassPermissions` mode: those calls are not checked by this graph's permission policy, not confined by the sandbox executor, and the token figure is the sub-agent's own rather than one GraphARC metered call by call. The workspace boundary and the wall-clock ceiling still hold. It warns on `DelegatedToolUseWarning` at construction and marks every trace event `executor=delegated`, so a run stays auditable as delegated; filter that warning to an error to get the old refusal back. Structured output still needs an OpenAI-wire backend: `openrouter`, `openai`, or a local `ollama`.
- **A session turn is synchronous**, and a runner claim is a claim rather than a lease — nothing reclaims a session whose runner died holding it.
- **`.env` is found by walking up parent directories; `grapharc.toml` is not.** The config layer refuses an upward search on purpose — a run must not be governed by a file you did not know about. The credential loader predates that decision and still searches upward, so the thing that *spends money* is discovered more eagerly than the thing that *constrains* it.
- **`grapharc run` has no budget unless you give it one.** Set any of `--max-tokens`, `--max-iterations`, `--max-seconds`, or `--max-concurrency`; without them each dimension is unlimited and the gate admits a topology of any worst-case cost.
Expand Down
133 changes: 132 additions & 1 deletion grapharc/cli/delegate.py
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,7 @@
import shutil
import subprocess
import uuid
from dataclasses import dataclass
from pathlib import Path

from grapharc.cli import style
Expand All @@ -35,6 +36,130 @@
DEFAULT_DELEGATED_TOOLS = ("Read", "Glob", "Grep", "LS", "Edit", "Write", "Bash")



# ---------------------------------------------------------------------------
# The reusable core. `run_delegated` below is the CLI's presentation of it, and
# `grapharc.harness.agent.AgentNode` calls it directly when its backend is the
# Claude CLI — one implementation, so the two paths cannot drift on what
# actually gets spawned.


class DelegationError(Exception):
"""A delegated run could not be started, or came back unreadable.

`reason` is the machine-readable form, written to the trace so a run that
failed here is distinguishable afterwards from one that ran and refused.
"""

def __init__(self, message: str, *, reason: str) -> None:
super().__init__(message)
self.reason = reason


@dataclass(frozen=True)
class DelegatedRun:
"""What Claude Code reported back. Every number here is *its* figure.

`tokens_reported` is named for what it is: the sub-agent's own count, not
something GraphARC metered call by call. Nothing in this object was
observed by the permission engine.
"""

ok: bool
answer: str
reason: str
turns: int
tokens_reported: int
cost_usd: float | None
session_id: str | None
allowed: list[str] | None # None == no --allowedTools restriction at all
denied: list[str]


def delegate_task(
task: str,
*,
workspace: Path,
model: str | None = None,
allow: list[str] | None = None,
deny: list[str] | None = None,
max_turns: int = 20,
max_seconds: float | None = None,
system_prompt: str | None = None,
permission_mode: str | None = None,
) -> DelegatedRun:
"""Run one headless `claude -p` agent loop in `workspace`.

Two separate axes, and conflating them is a trap worth naming. `allow`
controls *which* tools exist; `permission_mode` controls whether the ones
that mutate anything are allowed to run without a human answering a prompt.
Omitting `--allowedTools` does **not** mean "every tool": it means Claude
Code's own default gating, and headless there is nobody to approve a Write,
so the sub-agent reports back that it could not create the file. Measured,
not assumed. `permission_mode="bypassPermissions"` is what actually means
"everything", and it means it literally — no checks at all.

Either way the caller is responsible for having said so out loud;
`AgentNode` warns at construction and marks every trace event.
"""
binary = shutil.which("claude")
if binary is None:
raise DelegationError(
"the delegated executor shells out to `claude`, which is not on PATH; "
"install Claude Code or use a tool-calling backend",
reason="claude_not_found",
)

workspace = Path(workspace).expanduser().resolve()
workspace.mkdir(parents=True, exist_ok=True)

argv = [binary, "-p", task, "--output-format", "json", "--max-turns", str(max_turns)]
if allow is not None:
argv += ["--allowedTools", ",".join(allow)]
if deny:
argv += ["--disallowedTools", ",".join(deny)]
if permission_mode:
argv += ["--permission-mode", permission_mode]
if model:
argv += ["--model", model]
if system_prompt:
argv += ["--append-system-prompt", system_prompt]

try:
completed = subprocess.run(
argv, cwd=workspace, capture_output=True, text=True, timeout=max_seconds
)
except subprocess.TimeoutExpired as exc:
raise DelegationError(
f"max_seconds ({max_seconds}) reached; the delegated run was stopped",
reason="deadline_exceeded",
) from exc

try:
report = json.loads(completed.stdout)
except (json.JSONDecodeError, ValueError) as exc:
detail = (completed.stderr or completed.stdout or "").strip()[-500:]
raise DelegationError(
f"claude exited {completed.returncode} without a readable JSON report: {detail}",
reason="unreadable_report",
) from exc

usage = report.get("usage") or {}
met = report.get("subtype") == "success" and not report.get("is_error", False)
return DelegatedRun(
ok=met,
answer=str(report.get("result") or "").strip(),
reason="target_met" if met else str(report.get("subtype") or "error"),
turns=int(report.get("num_turns") or 0),
tokens_reported=int(usage.get("input_tokens") or 0)
+ int(usage.get("output_tokens") or 0),
cost_usd=report.get("total_cost_usd"),
session_id=report.get("session_id"),
allowed=list(allow) if allow is not None else None,
denied=list(deny or []),
)


def run_delegated(
task: str,
*,
Expand Down Expand Up @@ -217,4 +342,10 @@ def run_delegated(
return EXIT_OK if met else EXIT_FAILED


__all__ = ["DEFAULT_DELEGATED_TOOLS", "run_delegated"]
__all__ = [
"DEFAULT_DELEGATED_TOOLS",
"DelegatedRun",
"DelegationError",
"delegate_task",
"run_delegated",
]
140 changes: 140 additions & 0 deletions grapharc/harness/agent.py
Original file line number Diff line number Diff line change
Expand Up @@ -53,8 +53,10 @@
import types
import typing
import uuid
import warnings
from collections.abc import Callable, Mapping
from enum import StrEnum
from pathlib import Path
from typing import Any

from langchain_core.language_models.chat_models import BaseChatModel
Expand Down Expand Up @@ -297,6 +299,48 @@ def _coerce_args(raw: Any) -> dict[str, Any]:
raise TypeError(f"tool arguments must be a JSON object, got {type(raw).__name__}")



class DelegatedToolUseWarning(UserWarning):
"""An `AgentNode` is running Claude Code's tool loop instead of GraphARC's.

Its own category so it can be filtered, asserted on in tests, or turned
into an error with `-W error::grapharc.harness.agent.DelegatedToolUseWarning`
by anyone who wants the old refusal back.
"""


#: What the delegated loop runs under. `bypassPermissions` is Claude Code's
#: "no checks at all" mode, and it is deliberate: omitting `--allowedTools`
#: leaves its default gating in place, and headless there is no one to approve
#: a Write — the sub-agent simply reports that it could not create the file.
#: "Every tool Claude Code has" only means that with this set.
DELEGATED_PERMISSION_MODE = "bypassPermissions"

_DELEGATION_WARNING = (
"agent node {name!r} is backed by the Claude CLI, which has no tool-calling "
"wire format, so GraphARC cannot run its own tool loop over it. The whole "
"loop is delegated to Claude Code's headless agent, which means: it uses "
"EVERY tool Claude Code has (Bash, Write, WebFetch, Task, ...) under its "
"bypassPermissions mode, so those calls are NOT checked by this graph's "
"permission policy, NOT confined by the sandbox executor, and NOT gated by "
"Claude Code's own prompts either. The token figure is what the sub-agent "
"reports rather than what GraphARC metered. The workspace boundary and the wall-clock "
"ceiling still apply. Every trace event from this node is marked "
"executor=delegated so the run stays auditable; use a tool-calling backend "
"(openrouter/*, openai/*, ollama/*) for a governed loop."
)


def _is_claude_cli(model: Any) -> bool:
"""Is this the Claude CLI backend?

Matched on `_llm_type` rather than `isinstance`, so this module does not
import the gateway, and rather than "does it lack bind_tools" — which is
also true of `ScriptedChatModel` and would silently delegate every mock.
"""
return getattr(model, "_llm_type", None) == "grapharc-claude-cli"


class AgentNode:
"""A tool-using agent loop, shaped as a GraphARC node.

Expand Down Expand Up @@ -341,6 +385,16 @@ def __init__(
self.prompt_fn = prompt_fn
self.trace = trace
self.max_tool_result_chars = max_tool_result_chars
#: True when the backend is the Claude CLI, which has no tool-calling
#: wire format and therefore cannot be driven as a raw model. The loop
#: is handed to Claude Code instead — see `_run_delegated`.
self.delegated = _is_claude_cli(model)
if self.delegated:
warnings.warn(
_DELEGATION_WARNING.format(name=name),
DelegatedToolUseWarning,
stacklevel=2,
)

@property
def writes(self) -> set[str]:
Expand Down Expand Up @@ -385,6 +439,9 @@ def run(self, prompt: str, ctx: RunContext | None = None) -> AgentResult:
run_id=uuid.uuid4().hex[:12], graph=self.name, meter=BudgetMeter(Budget())
)

if self.delegated:
return self._run_delegated(prompt, ctx)

model = self._bind_tools()
messages: list[BaseMessage] = [
SystemMessage(content=self.system_prompt),
Expand Down Expand Up @@ -524,6 +581,89 @@ def run(self, prompt: str, ctx: RunContext | None = None) -> AgentResult:

# -- internals ------------------------------------------------------------

def _run_delegated(self, prompt: str, ctx: RunContext) -> AgentResult:
"""Hand the whole task to Claude Code's headless agent.

The trade is stated in `_DELEGATION_WARNING` and repeated on every trace
event this writes, because a warning at construction is gone by the time
anyone reads the run back. `executor="delegated"` on the events is what
stops a reader six months later from assuming this graph's permission
policy saw these tool calls. It did not.

The workspace boundary and the wall-clock ceiling still hold: the CLI is
spawned with `cwd` set to the harness workspace, and `max_seconds` is
enforced from outside by the subprocess timeout. Everything finer than
that is Claude Code's.
"""
from grapharc.cli.delegate import DelegationError, delegate_task

remaining = ctx.meter.remaining_seconds() if ctx.meter else None
step = 1
if self.trace is not None:
self.trace.event(
run_id=ctx.run_id, graph=ctx.graph, node=self.name, phase="model",
step=step, thread_id=ctx.thread_id, attempt=ctx.attempt,
state_delta={"executor": "delegated", "tools": "all of Claude Code's",
"permission_mode": DELEGATED_PERMISSION_MODE,
"governed_by": "Claude Code, not this graph's policy"},
)
try:
workspace = getattr(self.harness.executor, "workspace", None)
if workspace is None:
raise DelegationError(
"the delegated executor needs a workspace directory, and this "
f"harness's executor ({type(self.harness.executor).__name__}) "
"does not expose one",
reason="no_workspace",
)
run = delegate_task(
prompt,
workspace=Path(workspace),
max_turns=self.max_iterations,
max_seconds=remaining,
system_prompt=self.system_prompt,
permission_mode=DELEGATED_PERMISSION_MODE,
)
except DelegationError as exc:
if self.trace is not None:
self.trace.event(
run_id=ctx.run_id, graph=ctx.graph, node=self.name, phase="stop",
step=step, thread_id=ctx.thread_id, attempt=ctx.attempt,
state_delta={"executor": "delegated", "termination_reason": exc.reason},
error=str(exc),
)
return AgentResult(
termination_reason=StopReason.ERROR, iterations=0, note=str(exc)
)

# The sub-agent's own count, charged so a budget is not simply blind to
# a delegated node — but named `tokens_reported` everywhere it surfaces,
# because GraphARC did not meter these call by call.
if ctx.meter and run.tokens_reported:
ctx.meter.charge_tokens(run.tokens_reported)
reason = StopReason.TARGET_MET if run.ok else StopReason.ERROR
if self.trace is not None:
self.trace.event(
run_id=ctx.run_id, graph=ctx.graph, node=self.name, phase="stop",
step=step, thread_id=ctx.thread_id, attempt=ctx.attempt,
tokens=run.tokens_reported or None,
cost_usd=run.cost_usd,
state_delta={"executor": "delegated", "termination_reason": reason.value,
"turns": run.turns, "tokens_reported": run.tokens_reported,
"session_id": run.session_id},
)
return AgentResult(
output=run.answer if run.ok else "",
partial_output="" if run.ok else run.answer,
termination_reason=reason,
iterations=run.turns,
note=(
f"delegated to Claude Code: {run.turns} turn(s), "
f"{run.tokens_reported:,} tokens reported by the sub-agent, "
"tool calls not checked by this graph's policy"
),
)

def _bind_tools(self) -> Any:
"""Bind the policy-filtered tool set. Denied tools are never described."""
schemas = tool_schemas(self.harness)
Expand Down
12 changes: 8 additions & 4 deletions grapharc/stdlib.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,14 @@
not chosen by the model at run time. `apply_change` is the only one that can
write, which is what makes it the one worth denying by default.

All but one of the agent kinds need a **tool-calling** backend: `AgentNode`
refuses a model with no `bind_tools` rather than degrading, so the Claude CLI
subscription cannot drive them. `summarize` is the exception — it is toolless by
design, so it binds nothing and runs anywhere.
All but one of the agent kinds want a **tool-calling** backend, because that is
the only way GraphARC can run the loop itself and gate each call. Given the
Claude CLI — which has no tool-calling wire format — `AgentNode` delegates the
whole loop to Claude Code instead, warning at construction and marking the
trace: the fixed allowlists described above do not apply to a delegated run,
because the tools are Claude Code's rather than this registry's. `summarize` is
the exception either way — it is toolless by design, so it binds nothing and
runs anywhere.

Registered but denied is the interesting state: **given a model**, `apply_change`
is in the registry because changing files is a real capability, and the default
Expand Down
Loading
Loading