From 68df509dd45e289b6f710c823cb251697d41a780 Mon Sep 17 00:00:00 2001 From: Claude Date: Mon, 3 Aug 2026 17:22:17 +0000 Subject: [PATCH] An agent node on the Claude CLI now delegates instead of refusing MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `AgentNode` used to raise `AgentConfigError` when handed the Claude CLI: the CLI has no tool-calling wire format, so GraphARC cannot run its own gated loop over it. That was accurate and useless — the CLI *is* a complete agent, and refusing to use it meant the subscription backend could do no agentic work inside a graph at all. It now hands the whole loop to Claude Code's headless agent, reusing the same machinery `grapharc agent --executor claude-cli` already used. That path's subprocess-and-parse core is extracted as `delegate_task()`, so the CLI command and the node cannot drift on what actually gets spawned. This is a real widening of the trust boundary and is treated as one: - `DelegatedToolUseWarning` at construction, naming each thing given up — every tool Claude Code has, not checked by this graph's policy, not confined by the sandbox, token figure self-reported. Its own category, so `-W error::grapharc.harness.agent.DelegatedToolUseWarning` restores the old refusal for anyone who wants it. - **Every trace event from a delegated node carries `executor=delegated`**, plus the permission mode and `governed_by`. A warning is gone by the time someone reads the run back; without this, a JSONL reader six months later sees an agent node that completed and cannot tell that its tool calls never reached the permission engine. That is the project's own claim about its traces, so the marking is the part that keeps it true. One thing found by running it rather than reasoning about it: omitting `--allowedTools` does **not** mean "every tool". It leaves Claude Code's own gating in place, and headless there is nobody to approve a Write — the first delegated node came back reporting it could not create the file. Only `--permission-mode bypassPermissions` means what "everything Claude Code has" was meant to mean, so that is what a delegated node runs under, named in the warning and recorded in the trace. `delegate_task` takes the two as separate arguments because conflating them is the trap. Detection is on `_llm_type == "grapharc-claude-cli"`, not on "does this model lack `bind_tools`" — `ScriptedChatModel` lacks it too, and matching that way would have silently delegated every mocked agent in the suite to a real subprocess. A test pins that. Verified live end to end: a node given `get_model("claude-cli")` created a file in its workspace using Claude Code's own tools (2 turns, target_met), and the trace shows `executor=delegated` with `permission_mode=bypassPermissions` on every event. Docs updated in README's limits and `stdlib`'s module docstring, both of which asserted the old refusal. Full suite green on 3.12; ruff clean. Co-Authored-By: Claude Opus 5 --- README.md | 2 +- grapharc/cli/delegate.py | 133 ++++++++++++++++++++++++++++++++- grapharc/harness/agent.py | 140 +++++++++++++++++++++++++++++++++++ grapharc/stdlib.py | 12 ++- tests/test_agent_delegate.py | 133 +++++++++++++++++++++++++++++++++ 5 files changed, 414 insertions(+), 6 deletions(-) diff --git a/README.md b/README.md index 8e50429..4153ea4 100644 --- a/README.md +++ b/README.md @@ -482,7 +482,7 @@ Re-derived on 2026-07-28 by running each item, not by reading the commit log. - **Cost is recorded when a backend reports one, estimated when it does not.** Both gateways publish the provider's `cost_usd` through the same `llm_output` envelope, the runtime's usage callback writes it onto the node's `end` event, and an agent's `model` events carry the per-call breakdown. A backend that reports no price still falls back to a `RateCard` estimate, and the two figures stay apart — `recorded_cost_usd` is never a guess. Still missing: no tenant on a trace event, so per-tenant attribution is not offered. - **A node's tokens are its own, not the run's movement while it ran.** Worth stating because it was the other way round: an `end` event carried the difference between two readings of the run's *shared* meter, so under fan-out the workers' windows overlapped and each was credited with its siblings' concurrent spend. Three workers costing 8 tokens each traced as 24/16/8, and `metrics` and `cost` agreed on 48 for 24 tokens of real work — doubling the estimated bill purely because the work ran in parallel. Attribution now comes from a per-node scope on the meter, so the same work costs the same serially and in parallel; a hand charge the usage callback never saw still lands on the node that made it. - **A planning round is an envelope, not a measurement.** A `round` event used to carry the planner's `tokens` and the round's `duration_ms`, both of which `metrics`, `cost` and `replay` add on top of node totals — and the planner's spend was already reported by its own `plan` event, so it was counted twice, and a round's duration encloses the plan plus every node it ran. Neither is on the event now; both are on its `state_delta` as `round_tokens` / `round_iterations` / `round_duration_ms`, where no reader sums them. `RoundRecord.iterations` also holds a figure now rather than always `0`. -- **The Claude CLI backend is completion-only.** Tool calling and structured output need one of the OpenAI-wire backends: `openrouter`, `openai`, or a local `ollama`. +- **The Claude CLI backend is completion-only, and an agent node on it is *delegated* rather than governed.** The CLI has no tool-calling wire format, so GraphARC cannot run its own gated loop over it. Rather than refuse, `AgentNode` hands the whole loop to Claude Code's headless agent — which means every tool Claude Code has, under its `bypassPermissions` mode: those calls are not checked by this graph's permission policy, not confined by the sandbox executor, and the token figure is the sub-agent's own rather than one GraphARC metered call by call. The workspace boundary and the wall-clock ceiling still hold. It warns on `DelegatedToolUseWarning` at construction and marks every trace event `executor=delegated`, so a run stays auditable as delegated; filter that warning to an error to get the old refusal back. Structured output still needs an OpenAI-wire backend: `openrouter`, `openai`, or a local `ollama`. - **A session turn is synchronous**, and a runner claim is a claim rather than a lease — nothing reclaims a session whose runner died holding it. - **`.env` is found by walking up parent directories; `grapharc.toml` is not.** The config layer refuses an upward search on purpose — a run must not be governed by a file you did not know about. The credential loader predates that decision and still searches upward, so the thing that *spends money* is discovered more eagerly than the thing that *constrains* it. - **`grapharc run` has no budget unless you give it one.** Set any of `--max-tokens`, `--max-iterations`, `--max-seconds`, or `--max-concurrency`; without them each dimension is unlimited and the gate admits a topology of any worst-case cost. diff --git a/grapharc/cli/delegate.py b/grapharc/cli/delegate.py index 8d70e89..472b459 100644 --- a/grapharc/cli/delegate.py +++ b/grapharc/cli/delegate.py @@ -25,6 +25,7 @@ import shutil import subprocess import uuid +from dataclasses import dataclass from pathlib import Path from grapharc.cli import style @@ -35,6 +36,130 @@ DEFAULT_DELEGATED_TOOLS = ("Read", "Glob", "Grep", "LS", "Edit", "Write", "Bash") + +# --------------------------------------------------------------------------- +# The reusable core. `run_delegated` below is the CLI's presentation of it, and +# `grapharc.harness.agent.AgentNode` calls it directly when its backend is the +# Claude CLI — one implementation, so the two paths cannot drift on what +# actually gets spawned. + + +class DelegationError(Exception): + """A delegated run could not be started, or came back unreadable. + + `reason` is the machine-readable form, written to the trace so a run that + failed here is distinguishable afterwards from one that ran and refused. + """ + + def __init__(self, message: str, *, reason: str) -> None: + super().__init__(message) + self.reason = reason + + +@dataclass(frozen=True) +class DelegatedRun: + """What Claude Code reported back. Every number here is *its* figure. + + `tokens_reported` is named for what it is: the sub-agent's own count, not + something GraphARC metered call by call. Nothing in this object was + observed by the permission engine. + """ + + ok: bool + answer: str + reason: str + turns: int + tokens_reported: int + cost_usd: float | None + session_id: str | None + allowed: list[str] | None # None == no --allowedTools restriction at all + denied: list[str] + + +def delegate_task( + task: str, + *, + workspace: Path, + model: str | None = None, + allow: list[str] | None = None, + deny: list[str] | None = None, + max_turns: int = 20, + max_seconds: float | None = None, + system_prompt: str | None = None, + permission_mode: str | None = None, +) -> DelegatedRun: + """Run one headless `claude -p` agent loop in `workspace`. + + Two separate axes, and conflating them is a trap worth naming. `allow` + controls *which* tools exist; `permission_mode` controls whether the ones + that mutate anything are allowed to run without a human answering a prompt. + Omitting `--allowedTools` does **not** mean "every tool": it means Claude + Code's own default gating, and headless there is nobody to approve a Write, + so the sub-agent reports back that it could not create the file. Measured, + not assumed. `permission_mode="bypassPermissions"` is what actually means + "everything", and it means it literally — no checks at all. + + Either way the caller is responsible for having said so out loud; + `AgentNode` warns at construction and marks every trace event. + """ + binary = shutil.which("claude") + if binary is None: + raise DelegationError( + "the delegated executor shells out to `claude`, which is not on PATH; " + "install Claude Code or use a tool-calling backend", + reason="claude_not_found", + ) + + workspace = Path(workspace).expanduser().resolve() + workspace.mkdir(parents=True, exist_ok=True) + + argv = [binary, "-p", task, "--output-format", "json", "--max-turns", str(max_turns)] + if allow is not None: + argv += ["--allowedTools", ",".join(allow)] + if deny: + argv += ["--disallowedTools", ",".join(deny)] + if permission_mode: + argv += ["--permission-mode", permission_mode] + if model: + argv += ["--model", model] + if system_prompt: + argv += ["--append-system-prompt", system_prompt] + + try: + completed = subprocess.run( + argv, cwd=workspace, capture_output=True, text=True, timeout=max_seconds + ) + except subprocess.TimeoutExpired as exc: + raise DelegationError( + f"max_seconds ({max_seconds}) reached; the delegated run was stopped", + reason="deadline_exceeded", + ) from exc + + try: + report = json.loads(completed.stdout) + except (json.JSONDecodeError, ValueError) as exc: + detail = (completed.stderr or completed.stdout or "").strip()[-500:] + raise DelegationError( + f"claude exited {completed.returncode} without a readable JSON report: {detail}", + reason="unreadable_report", + ) from exc + + usage = report.get("usage") or {} + met = report.get("subtype") == "success" and not report.get("is_error", False) + return DelegatedRun( + ok=met, + answer=str(report.get("result") or "").strip(), + reason="target_met" if met else str(report.get("subtype") or "error"), + turns=int(report.get("num_turns") or 0), + tokens_reported=int(usage.get("input_tokens") or 0) + + int(usage.get("output_tokens") or 0), + cost_usd=report.get("total_cost_usd"), + session_id=report.get("session_id"), + allowed=list(allow) if allow is not None else None, + denied=list(deny or []), + ) + + def run_delegated( task: str, *, @@ -217,4 +342,10 @@ def run_delegated( return EXIT_OK if met else EXIT_FAILED -__all__ = ["DEFAULT_DELEGATED_TOOLS", "run_delegated"] +__all__ = [ + "DEFAULT_DELEGATED_TOOLS", + "DelegatedRun", + "DelegationError", + "delegate_task", + "run_delegated", +] diff --git a/grapharc/harness/agent.py b/grapharc/harness/agent.py index 882f7ea..3afd123 100644 --- a/grapharc/harness/agent.py +++ b/grapharc/harness/agent.py @@ -53,8 +53,10 @@ import types import typing import uuid +import warnings from collections.abc import Callable, Mapping from enum import StrEnum +from pathlib import Path from typing import Any from langchain_core.language_models.chat_models import BaseChatModel @@ -297,6 +299,48 @@ def _coerce_args(raw: Any) -> dict[str, Any]: raise TypeError(f"tool arguments must be a JSON object, got {type(raw).__name__}") + +class DelegatedToolUseWarning(UserWarning): + """An `AgentNode` is running Claude Code's tool loop instead of GraphARC's. + + Its own category so it can be filtered, asserted on in tests, or turned + into an error with `-W error::grapharc.harness.agent.DelegatedToolUseWarning` + by anyone who wants the old refusal back. + """ + + +#: What the delegated loop runs under. `bypassPermissions` is Claude Code's +#: "no checks at all" mode, and it is deliberate: omitting `--allowedTools` +#: leaves its default gating in place, and headless there is no one to approve +#: a Write — the sub-agent simply reports that it could not create the file. +#: "Every tool Claude Code has" only means that with this set. +DELEGATED_PERMISSION_MODE = "bypassPermissions" + +_DELEGATION_WARNING = ( + "agent node {name!r} is backed by the Claude CLI, which has no tool-calling " + "wire format, so GraphARC cannot run its own tool loop over it. The whole " + "loop is delegated to Claude Code's headless agent, which means: it uses " + "EVERY tool Claude Code has (Bash, Write, WebFetch, Task, ...) under its " + "bypassPermissions mode, so those calls are NOT checked by this graph's " + "permission policy, NOT confined by the sandbox executor, and NOT gated by " + "Claude Code's own prompts either. The token figure is what the sub-agent " + "reports rather than what GraphARC metered. The workspace boundary and the wall-clock " + "ceiling still apply. Every trace event from this node is marked " + "executor=delegated so the run stays auditable; use a tool-calling backend " + "(openrouter/*, openai/*, ollama/*) for a governed loop." +) + + +def _is_claude_cli(model: Any) -> bool: + """Is this the Claude CLI backend? + + Matched on `_llm_type` rather than `isinstance`, so this module does not + import the gateway, and rather than "does it lack bind_tools" — which is + also true of `ScriptedChatModel` and would silently delegate every mock. + """ + return getattr(model, "_llm_type", None) == "grapharc-claude-cli" + + class AgentNode: """A tool-using agent loop, shaped as a GraphARC node. @@ -341,6 +385,16 @@ def __init__( self.prompt_fn = prompt_fn self.trace = trace self.max_tool_result_chars = max_tool_result_chars + #: True when the backend is the Claude CLI, which has no tool-calling + #: wire format and therefore cannot be driven as a raw model. The loop + #: is handed to Claude Code instead — see `_run_delegated`. + self.delegated = _is_claude_cli(model) + if self.delegated: + warnings.warn( + _DELEGATION_WARNING.format(name=name), + DelegatedToolUseWarning, + stacklevel=2, + ) @property def writes(self) -> set[str]: @@ -385,6 +439,9 @@ def run(self, prompt: str, ctx: RunContext | None = None) -> AgentResult: run_id=uuid.uuid4().hex[:12], graph=self.name, meter=BudgetMeter(Budget()) ) + if self.delegated: + return self._run_delegated(prompt, ctx) + model = self._bind_tools() messages: list[BaseMessage] = [ SystemMessage(content=self.system_prompt), @@ -524,6 +581,89 @@ def run(self, prompt: str, ctx: RunContext | None = None) -> AgentResult: # -- internals ------------------------------------------------------------ + def _run_delegated(self, prompt: str, ctx: RunContext) -> AgentResult: + """Hand the whole task to Claude Code's headless agent. + + The trade is stated in `_DELEGATION_WARNING` and repeated on every trace + event this writes, because a warning at construction is gone by the time + anyone reads the run back. `executor="delegated"` on the events is what + stops a reader six months later from assuming this graph's permission + policy saw these tool calls. It did not. + + The workspace boundary and the wall-clock ceiling still hold: the CLI is + spawned with `cwd` set to the harness workspace, and `max_seconds` is + enforced from outside by the subprocess timeout. Everything finer than + that is Claude Code's. + """ + from grapharc.cli.delegate import DelegationError, delegate_task + + remaining = ctx.meter.remaining_seconds() if ctx.meter else None + step = 1 + if self.trace is not None: + self.trace.event( + run_id=ctx.run_id, graph=ctx.graph, node=self.name, phase="model", + step=step, thread_id=ctx.thread_id, attempt=ctx.attempt, + state_delta={"executor": "delegated", "tools": "all of Claude Code's", + "permission_mode": DELEGATED_PERMISSION_MODE, + "governed_by": "Claude Code, not this graph's policy"}, + ) + try: + workspace = getattr(self.harness.executor, "workspace", None) + if workspace is None: + raise DelegationError( + "the delegated executor needs a workspace directory, and this " + f"harness's executor ({type(self.harness.executor).__name__}) " + "does not expose one", + reason="no_workspace", + ) + run = delegate_task( + prompt, + workspace=Path(workspace), + max_turns=self.max_iterations, + max_seconds=remaining, + system_prompt=self.system_prompt, + permission_mode=DELEGATED_PERMISSION_MODE, + ) + except DelegationError as exc: + if self.trace is not None: + self.trace.event( + run_id=ctx.run_id, graph=ctx.graph, node=self.name, phase="stop", + step=step, thread_id=ctx.thread_id, attempt=ctx.attempt, + state_delta={"executor": "delegated", "termination_reason": exc.reason}, + error=str(exc), + ) + return AgentResult( + termination_reason=StopReason.ERROR, iterations=0, note=str(exc) + ) + + # The sub-agent's own count, charged so a budget is not simply blind to + # a delegated node — but named `tokens_reported` everywhere it surfaces, + # because GraphARC did not meter these call by call. + if ctx.meter and run.tokens_reported: + ctx.meter.charge_tokens(run.tokens_reported) + reason = StopReason.TARGET_MET if run.ok else StopReason.ERROR + if self.trace is not None: + self.trace.event( + run_id=ctx.run_id, graph=ctx.graph, node=self.name, phase="stop", + step=step, thread_id=ctx.thread_id, attempt=ctx.attempt, + tokens=run.tokens_reported or None, + cost_usd=run.cost_usd, + state_delta={"executor": "delegated", "termination_reason": reason.value, + "turns": run.turns, "tokens_reported": run.tokens_reported, + "session_id": run.session_id}, + ) + return AgentResult( + output=run.answer if run.ok else "", + partial_output="" if run.ok else run.answer, + termination_reason=reason, + iterations=run.turns, + note=( + f"delegated to Claude Code: {run.turns} turn(s), " + f"{run.tokens_reported:,} tokens reported by the sub-agent, " + "tool calls not checked by this graph's policy" + ), + ) + def _bind_tools(self) -> Any: """Bind the policy-filtered tool set. Denied tools are never described.""" schemas = tool_schemas(self.harness) diff --git a/grapharc/stdlib.py b/grapharc/stdlib.py index 414bf7e..dbd1abc 100644 --- a/grapharc/stdlib.py +++ b/grapharc/stdlib.py @@ -23,10 +23,14 @@ not chosen by the model at run time. `apply_change` is the only one that can write, which is what makes it the one worth denying by default. -All but one of the agent kinds need a **tool-calling** backend: `AgentNode` -refuses a model with no `bind_tools` rather than degrading, so the Claude CLI -subscription cannot drive them. `summarize` is the exception — it is toolless by -design, so it binds nothing and runs anywhere. +All but one of the agent kinds want a **tool-calling** backend, because that is +the only way GraphARC can run the loop itself and gate each call. Given the +Claude CLI — which has no tool-calling wire format — `AgentNode` delegates the +whole loop to Claude Code instead, warning at construction and marking the +trace: the fixed allowlists described above do not apply to a delegated run, +because the tools are Claude Code's rather than this registry's. `summarize` is +the exception either way — it is toolless by design, so it binds nothing and +runs anywhere. Registered but denied is the interesting state: **given a model**, `apply_change` is in the registry because changing files is a real capability, and the default diff --git a/tests/test_agent_delegate.py b/tests/test_agent_delegate.py index 0e3be9a..7437721 100644 --- a/tests/test_agent_delegate.py +++ b/tests/test_agent_delegate.py @@ -117,3 +117,136 @@ def test_a_missing_binary_is_exit_2_with_the_reason(tmp_path, monkeypatch, capsy code = _run(tmp_path) assert code == 2 assert "not on PATH" in capsys.readouterr().err + + +# ---- the same delegation, reached from an AgentNode --------------------------- +# +# `AgentNode` used to refuse the Claude CLI outright: it has no tool-calling wire +# format, so GraphARC cannot run its own gated loop over it. It now delegates the +# whole loop to Claude Code instead, which is a genuine widening of the trust +# boundary — so what these gates pin is that the widening is *visible*, at +# construction and afterwards in the trace. + + +def _node(workspace, trace=None, name="worker"): + from grapharc.gateway import get_model + from grapharc.harness import Harness, PermissionPolicy, PermissionRule, ToolRegistry + from grapharc.harness.agent import AgentNode + + harness = Harness( + ToolRegistry(), + PermissionPolicy(rules=[PermissionRule(action="allow", pattern="*")]), + workspace=str(workspace), + ) + with pytest.warns(Warning): + return AgentNode(get_model("claude-cli"), harness, name=name, trace=trace) + + +def test_a_claude_cli_agent_node_warns_loudly_at_construction(tmp_path, fake_claude): + """A silent switch from "refuses" to "runs with every tool and no checks" + is the one thing this must not be. The warning names each thing given up. + """ + from grapharc.gateway import get_model + from grapharc.harness import Harness, PermissionPolicy, PermissionRule, ToolRegistry + from grapharc.harness.agent import AgentNode, DelegatedToolUseWarning + + harness = Harness( + ToolRegistry(), + PermissionPolicy(rules=[PermissionRule(action="allow", pattern="*")]), + workspace=str(tmp_path), + ) + with pytest.warns(DelegatedToolUseWarning) as caught: + node = AgentNode(get_model("claude-cli"), harness, name="worker") + + assert node.delegated is True + text = str(caught[0].message) + for claim in ("EVERY tool", "NOT checked", "NOT confined", "bypassPermissions"): + assert claim in text, f"the warning does not mention {claim!r}: {text}" + + +def test_a_tool_calling_backend_is_not_delegated_and_does_not_warn(tmp_path): + """The mock double must keep running GraphARC's own loop. + + Detection is on `_llm_type`, not on "does this model lack bind_tools" — + `ScriptedChatModel` lacks it too, and matching that way would have silently + delegated every mocked agent in the suite to a real subprocess. + """ + import warnings as _warnings + + from grapharc.gateway import get_model + from grapharc.harness import Harness, PermissionPolicy, PermissionRule, ToolRegistry + from grapharc.harness.agent import AgentNode + + harness = Harness( + ToolRegistry(), + PermissionPolicy(rules=[PermissionRule(action="allow", pattern="*")]), + workspace=str(tmp_path), + ) + with _warnings.catch_warnings(): + _warnings.simplefilter("error") # any warning at all fails this + node = AgentNode(get_model("mock/x", responses=["hi"]), harness, name="m") + assert node.delegated is False + + +def test_the_delegated_node_asks_for_every_tool_and_bypasses_the_prompt( + tmp_path, fake_claude +): + """Two axes, and conflating them was a real bug found by running it. + + Omitting `--allowedTools` does not mean "every tool" — it leaves Claude + Code's own gating on, and headless there is nobody to approve a Write, so + the sub-agent came back reporting it could not create the file. Only + `--permission-mode bypassPermissions` means what "everything Claude Code + has" was chosen to mean. + """ + workspace = tmp_path / "ws" + workspace.mkdir() + _node(workspace).run("do a thing") + + argv = json.loads(fake_claude.read_text()) + assert "--allowedTools" not in argv, "an allowlist would narrow the tool set" + assert "--permission-mode" in argv + assert argv[argv.index("--permission-mode") + 1] == "bypassPermissions" + + +def test_every_delegated_trace_event_says_it_was_delegated(tmp_path, fake_claude): + """The construction warning is gone by the time anyone reads the run back. + + Without this marking, a JSONL reader six months later sees an agent node + that completed and has no way to know its tool calls never reached this + graph's permission policy — which is exactly the claim the project makes + about its traces. + """ + from grapharc.observe.trace import TraceRecorder + + workspace = tmp_path / "ws" + workspace.mkdir() + trace_path = tmp_path / "t.jsonl" + _node(workspace, trace=TraceRecorder(trace_path)).run("do a thing") + + events = [json.loads(line) for line in trace_path.read_text().splitlines() if line.strip()] + assert events, "the delegated run recorded nothing" + for event in events: + delta = event.get("state_delta") or {} + assert delta.get("executor") == "delegated", event + + opening = events[0]["state_delta"] + assert opening["permission_mode"] == "bypassPermissions" + assert "not this graph's policy" in opening["governed_by"] + + +def test_a_delegated_run_charges_the_meter_what_the_sub_agent_reported(tmp_path, fake_claude): + """A budget must not be simply blind to a delegated node — but the figure is + the sub-agent's own, and every name it surfaces under says so. + """ + from grapharc.runtime.budget import Budget, BudgetMeter + from grapharc.runtime.graph import RunContext + + workspace = tmp_path / "ws" + workspace.mkdir() + ctx = RunContext(run_id="r", graph="g", meter=BudgetMeter(Budget())) + result = _node(workspace).run("do a thing", ctx) + + assert ctx.meter.tokens == REPORT["usage"]["input_tokens"] + REPORT["usage"]["output_tokens"] + assert "tokens reported by the sub-agent" in result.note + assert "not checked by this graph's policy" in result.note