Skip to content

fix(bedrock): fall back to toolChoice.auto in structured_output for unsupported models#1887

Open
giulio-leone wants to merge 1 commit intostrands-agents:mainfrom
giulio-leone:fix/bedrock-structured-output-tool-choice
Open

fix(bedrock): fall back to toolChoice.auto in structured_output for unsupported models#1887
giulio-leone wants to merge 1 commit intostrands-agents:mainfrom
giulio-leone:fix/bedrock-structured-output-tool-choice

Conversation

@giulio-leone
Copy link
Contributor

Problem

Some Bedrock models (e.g., Meta Llama) do not support the toolConfig.toolChoice.any field. When structured_output() forces tool_choice={"any": {}}, these models fail with:

ValidationException: This model doesn't support the toolConfig.toolChoice.any field.
Remove toolConfig.toolChoice.any and try again

Root Cause

BedrockModel.structured_output() unconditionally passes tool_choice={"any": {}} (line 1082) to guarantee the model invokes the schema tool. However, not all Bedrock models support the any tool choice constraint — Llama models, for instance, only support auto.

Solution

Catch ClientError mentioning toolChoice in the error message and transparently retry the stream with tool_choice={"auto": {}}. This allows structured output to work with models that only support the auto tool choice, while preserving the stricter any constraint for models that support it (e.g., Claude).

Changes

  • src/strands/models/bedrock.py: Wrapped the stream call in a try/except that catches ClientError about toolChoice and retries with auto
  • tests/strands/models/test_bedrock.py: Added test verifying the fallback behavior (first call uses any, second call uses auto)

Testing

All 126 Bedrock unit tests pass (125 existing + 1 new).

Closes #1241

…odels that don't support toolChoice.any

Some Bedrock models (e.g., Llama) reject the toolConfig.toolChoice.any
field with a ValidationException. When structured_output() forces
tool_choice={'any': {}} to guarantee tool invocation, models that don't
support this field fail with:

  ValidationException: This model doesn't support the
  toolConfig.toolChoice.any field.

Fix: catch ClientError mentioning 'toolChoice' and retry the stream
with tool_choice={'auto': {}}, allowing the model to return a tool
call without the unsupported constraint.

Closes strands-agents#1241
@giulio-leone
Copy link
Contributor Author

Friendly ping — adds a fallback to toolChoice.auto in structured output mode for Bedrock models that don't support toolChoice.tool, preventing API errors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

1 participant