Batch API: tool_use input wrapped in $PARAMETER_NAME placeholder key (claude-opus-4-7)

[ffishman]

Summary

When using the Messages Batch API with forced tool_use (tool_choice={"type":"tool","name":"..."}) on claude-opus-4-7, the returned tool input is frequently wrapped in a single key $PARAMETER_NAME instead of containing the schema's actual property names at the top level. The same prompt and schema, sent via client.messages.create (sync), returns the correct top-level keys 100% of the time.

Environment

• SDK: anthropic (Python), latest stable
• Model: claude-opus-4-7
• Endpoint: client.messages.batches.create / .retrieve / .results
• Tool config: tool_choice={"type":"tool","name":"save_topic_synthesis"} (forced)

Expected vs Observed

Expected:

tool_use.input == {
  "tldr": "...",
  "mechanism_summary": "...",
  "evidence_strength": "emerging",
  ...
}

Observed (most items):

tool_use.input == {
  "$PARAMETER_NAME": {
    "tldr": "...",
    "mechanism_summary": "...",
    ...
  }
}

The literal string $PARAMETER_NAME does not appear anywhere in our prompt, tool schema, or system message.

Distribution from a single batch (10 messages, all stop_reason=tool_use)

Request	Top-level keys	Output tokens
1	$PARAMETER_NAME (full inner)	4573
2	$PARAMETER_NAME (full inner)	5668
3	$PARAMETER_NAME (full inner)	5221
4	$PARAMETER_NAME (full inner)	5513
5	$PARAMETER_NAME (full inner)	4853
6	mixed: $PARAMETER_NAME + top-level tldr/mechanism_summary	5525
7	topic (different wrap key, single-key dict)	4929
8	$PARAMETER_NAME (empty inner dict)	49
9	top-level (correct)	5353
10	top-level (correct)	5915

8/10 had non-conforming top-level shape; 2/10 conformed. Pattern repeated in a second independent batch with different inputs (same 10 messages re-submitted on a different day after schema/prompt changes): again 1 item came back stuck ($PARAMETER_NAME: {}, ~50 output tokens, smallest input).

Possibly related: tool_use that stops early with empty wrapper

Item 8 above: stop_reason="tool_use" but only 49 output tokens emitted, and the $PARAMETER_NAME wrapper was an empty dict. Recurring across our two batches: the failing item is consistently the one with the smallest prompt input (~10K tokens vs. ~17K average). Both stop_reason and tool_use block presence suggest the model thinks it completed successfully.

Reproducer (sketch)

import anthropic
client = anthropic.Anthropic()

schema = {
    "type": "object",
    "properties": {
        "tldr": {"type": "string", "description": "..."},
        "mechanism_summary": {"type": "string"},
        "evidence_strength": {"type": "string", "enum": ["consolidated","emerging","speculative"]},
        # ...several more required string/array fields...
    },
    "required": ["tldr", "mechanism_summary", "evidence_strength", ...],
}

batch = client.messages.batches.create(requests=[
    {
        "custom_id": f"item-{i}",
        "params": {
            "model": "claude-opus-4-7",
            "max_tokens": 8000,
            "system": [{"type": "text", "text": "<system prompt>", "cache_control": {"type": "ephemeral"}}],
            "tools": [{
                "name": "save_topic_synthesis",
                "description": "Save structured composition.",
                "input_schema": schema,
            }],
            "tool_choice": {"type": "tool", "name": "save_topic_synthesis"},
            "messages": [{"role": "user", "content": "<long prompt with ~15-20K tokens of structured input>"}],
        },
    }
    for i in range(10)
])
# wait for batch.processing_status == "ended", then for line in client.messages.batches.results(batch.id):
#   inspect line.result.message.content[0].input
# Expect: most items wrap content in {"$PARAMETER_NAME": {...}}

Impact

Without client-side defensive unwrapping, the majority of batch responses fail strict downstream validation (Pydantic, JSON Schema validators) because expected fields are nested one level too deep. Forces every batch consumer to add a workaround like:

if len(input) == 1:
    sole = next(iter(input.values()))
    if isinstance(sole, dict) and "<sentinel_field>" in sole:
        input = sole

Compounded with the stuck-early issue, the practical failure rate on a fresh batch is ~10% even with that workaround.

Workaround in place on our side

Detect len(input)==1 and unwrap; on <sentinel>=None (stuck output), re-issue the same request as a synchronous messages.create and accept the result. This pattern is described and committed at our repo if useful as reference.

Ask

Could the SDK / Batch API surface either (a) normalize the response shape to match sync mode, or (b) document the wrap behavior so consumers know to expect it? Currently the SDK type hints suggest top-level fields per the declared schema, which is misleading.

---

Update (2026-05-29): retested on claude-opus-4-8 + minimal reproducer

Re-tested after the 4.8 release. The $PARAMETER_NAME single-key wrapper no longer reproduces on claude-opus-4-8 (0/10 on the original workload that previously showed 8/10). However, the root issue — tool-input serialization leaking into tool_use.input — persists in a new form, and a minimal reproducer narrows the trigger.

Trigger isolated: a nested array<object> sub-schema. A flattened schema (scalars / arrays-of-scalars only) was clean in 28/28 runs across both models and both transports. The nested schema breaks on both.

Failure mode by version:

• opus-4-7: top-level wrapped in a single non-schema key — $PARAMETER_NAME, topic, or input (the last not in the original report).
• opus-4-8: internal tool-call markup (<parameter name="..."> / </parameter>) leaks into a string field value, absorbing the following sibling field — so a required field silently disappears while the object still looks well-formed.

Corrections to the original report:

1. Not batch-only. It also reproduces via client.messages.create (sync), at a lower rate than batch. The original "sync 100% correct" was likely a small-sample / schema-specific effect.
2. Wrapper-key set on 4.7 includes input in addition to $PARAMETER_NAME and topic.

Environment: SDK anthropic 0.104.1, anthropic-version: 2023-06-01. Reproducer script and per-request IDs available on request.

[santihdzs]

Would an SDK-side normalization PR be appropriate here, or is this expected to be fixed server-side? Happy to work on either the normalization or the documentation angle depending on the preferred direction.

[ffishman]

Thanks for offering to help! Before picking a direction I re-tested on the newly released claude-opus-4-8 and built a minimal self-contained reproducer. The picture is more nuanced than the original report — sharing in case it changes the fix direction.

TL;DR: the exact $PARAMETER_NAME single-key wrapper is gone on claude-opus-4-8, but the underlying bug (tool-input serialization leaking into the output) isn't fixed — it changed shape. On 4.8 the internal tool-call markup (<parameter name="...">) leaks into string field values, swallowing sibling fields (including required ones). It's lower-rate but more insidious, since the result looks structurally valid and passes shallow checks.

The trigger is isolable: a nested array<object> sub-schema. A flat schema (only scalars / arrays-of-scalars) never reproduced it — 28/28 clean across both models and both transports.

Minimal repro: forced tool_choice, ~18K-token input, two schemas (one with effects: array<{effect, magnitude, ref_ids}>, one flattened), same SDK 0.104.1, anthropic-version: 2023-06-01, same day:

model	schema	transport	conforming	broken
opus-4-7	flat	batch	8/8	—
opus-4-7	nested	batch	2/8	6 × key-wrap ($PARAMETER_NAME/input/topic)
opus-4-7	nested	sync	3/4	1 × key-wrap
opus-4-8	flat	batch+sync	12/12	—
opus-4-8	nested	batch	6/8	2 × XML-markup contamination
opus-4-8	nested	sync	3/4	1 × XML-markup contamination

Two refinements to the original report: (1) it's not batch-only — it reproduces in messages.create too, just at a much lower rate than batch; (2) on 4.7 I also saw input as a wrapper key, not just $PARAMETER_NAME/topic. (Sync cells are n=4, so treat those rates as indicative, not precise.)

Re: SDK-side normalization vs server-side — I think the 4.8 variant argues for server-side. The 4.7 single-key wrap was cleanly unwrappable client-side, but the 4.8 contamination injects <parameter name="..."> inside a string value and drops the swallowed field entirely — there's nothing for the SDK to losslessly recover. A defensive SDK check could detect it (reject/retry on a <parameter name= substring or a missing required field) but not repair it. So: server-side fix for the leak; documentation still worthwhile in the meantime. Happy to share the full reproducer script + request-ids if useful.

[ffishman]

Minimal self-contained reproducer (no external deps beyond the SDK), in case it's useful for triage. Forced tool_choice, ~15K-token input, two schemas (nested array<object> vs flattened). Run submit_batch then report_batch <id>, or sync.

repro_minimal.py (~180 lines)

"""Minimal, self-contained reproducer for anthropic-sdk-python#1607.

Tool-input serialization leaks into `tool_use.input` under forced
`tool_choice`, triggered by a nested `array<object>` sub-schema + large
input. A flattened schema (scalars / arrays-of-scalars) never reproduces it.

Failure mode differs by model:
  - claude-opus-4-7: top-level wrapped in a single non-schema key
    ($PARAMETER_NAME / input / topic).
  - claude-opus-4-8: internal tool-call markup (<parameter name="...">)
    leaks INTO a string field value, swallowing the following sibling
    (a required field disappears while the object still looks well-formed).

Reproduces in BOTH messages.batches (high rate) and messages.create
(lower rate). SDK 0.104.1, anthropic-version 2023-06-01.

Usage:
    export ANTHROPIC_API_KEY=...
    python3 repro_minimal_public.py submit_batch
    python3 repro_minimal_public.py report_batch <batch_id>
    python3 repro_minimal_public.py sync
"""
import json
import os
import sys
from collections import Counter

import anthropic

MODELS = ["claude-opus-4-7", "claude-opus-4-8"]
N_PER_CELL = 8
N_SYNC = 4
MAX_TOKENS = 6000
TOOL_NAME = "save_report"

# --- The suspected trigger: a nested array<object> sub-schema ---
NESTED_SCHEMA = {
    "type": "object",
    "properties": {
        "summary": {"type": "string", "description": "2-3 sentence summary."},
        "strength": {"type": "string", "enum": ["strong", "moderate", "weak"]},
        "observations": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "observation": {"type": "string"},
                    "magnitude": {"type": ["string", "null"]},
                    "source_ids": {"type": "array", "items": {"type": "integer"}},
                },
                "required": ["observation"],
            },
        },
        "open_questions": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["summary", "strength", "observations", "open_questions"],
}

# --- Control: same information, NO nested array<object> ---
FLAT_SCHEMA = {
    "type": "object",
    "properties": {
        "summary": {"type": "string", "description": "2-3 sentence summary."},
        "strength": {"type": "string", "enum": ["strong", "moderate", "weak"]},
        "main_observation": {"type": "string"},
        "magnitude": {"type": ["string", "null"]},
        "source_ids": {"type": "array", "items": {"type": "integer"}},
        "open_questions": {"type": "array", "items": {"type": "string"}},
    },
    "required": ["summary", "strength", "main_observation", "source_ids", "open_questions"],
}

SCHEMAS = {"nested": NESTED_SCHEMA, "flat": FLAT_SCHEMA}

# Neutral filler to reach ~18K tokens (the regime where the bug appears).
_PARA = (
    "Report {i}: bench test of a mid-range smart thermostat firmware build under "
    "sustained load. Observed reduced standby power draw, faster sensor polling, and "
    "a regression in Wi-Fi reconnect time after router reboots. Telemetry captured: "
    "setpoint drift, relay actuation latency, OTA update duration, and crash counts "
    "across a 72-hour soak. The authors note effect size depends on ambient "
    "temperature and the scheduling profile. Internal source_id={i}. "
)


def _big_input(seed: int) -> str:
    parts = ["Synthesize the findings below by calling save_report exactly once.\n\n"]
    for k in range(130):
        parts.append(_PARA.format(i=(seed * 1000 + k)))
    parts.append("\n\nCompose: summary, strength, the main observations (each with "
                 "observation, magnitude and source_ids), and open_questions.")
    return "".join(parts)


SYSTEM = ("You are an evidence synthesizer. ALWAYS call save_report. "
          "Populate every field per the declared schema.")


def _client():
    return anthropic.Anthropic()


def _params(model, schema_key, seed):
    return {
        "model": model,
        "max_tokens": MAX_TOKENS,
        "system": [{"type": "text", "text": SYSTEM, "cache_control": {"type": "ephemeral"}}],
        "tools": [{"name": TOOL_NAME, "description": "Save structured report. Call exactly once.",
                   "input_schema": SCHEMAS[schema_key]}],
        "tool_choice": {"type": "tool", "name": TOOL_NAME},
        "messages": [{"role": "user", "content": _big_input(seed)}],
    }


def _classify(inp, schema_key):
    blob = json.dumps(inp, ensure_ascii=False)
    xml_leak = "<parameter name" in blob or "</parameter>" in blob or "$PARAMETER_NAME" in blob
    base = {"xml_leak": xml_leak}
    if not isinstance(inp, dict):
        return {"shape": "non_dict", **base}
    keys = list(inp.keys())
    schema = SCHEMAS[schema_key]
    real = set(schema["properties"].keys())
    missing = sorted(set(schema.get("required", [])) - set(keys))
    base.update(keys=keys, missing_required=missing)
    if len(keys) == 1 and keys[0] not in real:
        return {"shape": "wrapped", "wrap_key": keys[0], **base}
    if xml_leak:
        return {"shape": "contaminated", **base}
    if schema_key == "nested":
        obs = inp.get("observations")
        obs_ok = isinstance(obs, list) and (not obs or isinstance(obs[0], dict))
        if (isinstance(obs, str) or "magnitude" in inp or "source_ids" in inp) and not obs_ok:
            return {"shape": "flattened", **base}
        if not obs_ok:
            return {"shape": "other", **base}
    else:
        if not isinstance(inp.get("main_observation"), str) or "observations" in inp:
            return {"shape": "other", **base}
    return {"shape": "missing_required" if missing else "conforms", **base}


def submit_batch():
    reqs = [{"custom_id": f"{m}__{s}__{j}", "params": _params(m, s, j)}
            for m in MODELS for s in SCHEMAS for j in range(N_PER_CELL)]
    b = _client().messages.batches.create(requests=reqs)
    print(f"batch_id={b.id}  n={len(reqs)}")


def report_batch(batch_id):
    c = _client()
    rows = []
    for line in c.messages.batches.results(batch_id):
        m, s, j = line.custom_id.split("__")
        if line.result.type != "succeeded":
            rows.append({"model": m, "schema": s, "result": line.result.type}); continue
        msg = line.result.message
        blk = next((x for x in msg.content if x.type == "tool_use"), None)
        rows.append({"model": m, "schema": s, **_classify(blk.input if blk else None, s)})
    _summary(rows, "batch")


def sync():
    c = anthropic.Anthropic()
    rows = []
    for m, s in [("claude-opus-4-8", "nested"), ("claude-opus-4-8", "flat"),
                 ("claude-opus-4-7", "nested")]:
        for j in range(N_SYNC):
            r = c.messages.with_raw_response.create(**_params(m, s, j))
            msg = r.parse()
            blk = next((x for x in msg.content if x.type == "tool_use"), None)
            cl = _classify(blk.input if blk else None, s)
            rows.append({"model": m, "schema": s, "request_id": r.headers.get("request-id"), **cl})
            print(f"  {m}/{s}#{j} -> {cl['shape']}{' [LEAK]' if cl.get('xml_leak') else ''}")
    _summary(rows, "sync")


def _summary(rows, transport):
    cells = {}
    for r in rows:
        cells.setdefault((r["model"], r["schema"]), Counter())[r.get("shape", r.get("result"))] += 1
    print(f"\n{'model':18} {'schema':7} {'transport':9} shapes")
    for (m, s), cnt in sorted(cells.items()):
        leak = sum(1 for r in rows if r["model"] == m and r["schema"] == s and r.get("xml_leak"))
        print(f"  {m:18} {s:7} {transport:9} {dict(cnt)}   xml_leak={leak}")


if __name__ == "__main__":
    cmd = sys.argv[1] if len(sys.argv) > 1 else "submit_batch"
    {"submit_batch": submit_batch,
     "report_batch": lambda: report_batch(sys.argv[2]),
     "sync": sync}.get(cmd, lambda: print(__doc__))()