Add Anthropic structured output support#608
Conversation
Wire output_config into the Anthropic chat payload when a schema is
provided, using the GA format ({ format: { type: 'json_schema', schema } })
with no beta headers required. The build_output_config method deep-dups the
schema and strips :strict keys that Anthropic rejects.
Fix model detection regexes in capabilities to match Claude 4+ model IDs
(supports_functions?, supports_json_mode?, capabilities_for) and add
supports_structured_output? for Claude 4.5+ models.
Re-record Anthropic VCR cassettes.
llenodo
left a comment
There was a problem hiding this comment.
Nice PR I almost implemented the same thing before finding this.
One suggestion to make it more future proof which assumes future versions of claude (5+) will support structured outputs so we wont need to come back to this later
|
@hiasinho Any chance this PR could also include structured output for anthropic models through AWS Bedrock? The AWS bedrock integration was recently rewritten to use converse API, which support structured output: https://docs.aws.amazon.com/bedrock/latest/userguide/structured-output.html |
Done! |
Would love to do it, but I don't use AWS. So no creds to create VCRs. Tests will fail. Feel free to build on it. |
Extend the Bedrock Converse API provider to support structured output via outputConfig.textFormat, building on crmne#608's Anthropic implementation. Key differences from the direct Anthropic API: schema must be a JSON string (not a Hash), nested under structure.jsonSchema with a required name field, and uses camelCase Converse API naming. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Extend the Bedrock Converse API provider to support structured output via outputConfig.textFormat, building on crmne#608's Anthropic implementation. Key differences from the direct Anthropic API: schema must be a JSON string (not a Hash), nested under structure.jsonSchema with a required name field, and uses camelCase Converse API naming. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
spec/ruby_llm/chat_schema_spec.rb
Outdated
| end | ||
|
|
||
| # Test Anthropic provider | ||
| CHAT_MODELS.select { |model_info| model_info[:provider] == :anthropic }.each do |model_info| |
There was a problem hiding this comment.
This shouldn't be another test but use the test above.
…uctured-output # Conflicts: # spec/fixtures/vcr_cassettes/activerecord_actsas_extended_thinking_persistence_anthropic_claude-haiku-4-5_persists_thinking_data_and_replays_it_across_turns.yml # spec/fixtures/vcr_cassettes/chat_basic_chat_functionality_anthropic_claude-haiku-4-5_can_handle_multi-turn_conversations.yml # spec/fixtures/vcr_cassettes/chat_basic_chat_functionality_anthropic_claude-haiku-4-5_can_have_a_basic_conversation.yml # spec/fixtures/vcr_cassettes/chat_basic_chat_functionality_anthropic_claude-haiku-4-5_replaces_previous_system_messages_when_replace_true.yml # spec/fixtures/vcr_cassettes/chat_basic_chat_functionality_anthropic_claude-haiku-4-5_returns_raw_responses.yml # spec/fixtures/vcr_cassettes/chat_basic_chat_functionality_anthropic_claude-haiku-4-5_successfully_uses_the_system_prompt.yml # spec/fixtures/vcr_cassettes/chat_content_object_support_anthropic_claude-haiku-4-5_preserves_content_objects_returned_from_tools.yml # spec/fixtures/vcr_cassettes/chat_error_handling_with_anthropic_claude-haiku-4-5_raises_appropriate_auth_error.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_can_handle_multiple_tool_calls_in_a_single_response.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_can_use_parallel_tool_calls.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_can_use_tools.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_can_use_tools_in_multi-turn_conversations.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_can_use_tools_with_multi-turn_streaming_conversations.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_can_use_tools_without_parameters.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_can_use_tools_without_parameters_in_multi-turn_streaming_conversations.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_handles_anyof_params.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_handles_array_params.yml # spec/fixtures/vcr_cassettes/chat_function_calling_anthropic_claude-haiku-4-5_handles_object_params.yml # spec/fixtures/vcr_cassettes/chat_pdf_models_anthropic_claude-haiku-4-5_can_handle_array_of_mixed_files_with_auto-detection.yml # spec/fixtures/vcr_cassettes/chat_pdf_models_anthropic_claude-haiku-4-5_handles_multiple_pdfs.yml # spec/fixtures/vcr_cassettes/chat_pdf_models_anthropic_claude-haiku-4-5_understands_pdfs.yml # spec/fixtures/vcr_cassettes/chat_real_error_scenarios_anthropic_claude-haiku-4-5_handles_context_length_exceeded_errors.yml # spec/fixtures/vcr_cassettes/chat_streaming_responses_anthropic_claude-haiku-4-5_reports_consistent_token_counts_compared_to_non-streaming.yml # spec/fixtures/vcr_cassettes/chat_streaming_responses_anthropic_claude-haiku-4-5_supports_streaming_responses.yml # spec/fixtures/vcr_cassettes/chat_text_models_anthropic_claude-haiku-4-5_can_understand_remote_text.yml # spec/fixtures/vcr_cassettes/chat_text_models_anthropic_claude-haiku-4-5_can_understand_text.yml # spec/fixtures/vcr_cassettes/chat_vision_models_anthropic_claude-haiku-4-5_can_understand_local_images.yml # spec/fixtures/vcr_cassettes/chat_vision_models_anthropic_claude-haiku-4-5_can_understand_remote_images_without_extension.yml # spec/fixtures/vcr_cassettes/chat_with_extended_thinking_anthropic_claude-haiku-4-5_preserves_thinking_signatures_between_turns_when_provided.yml # spec/fixtures/vcr_cassettes/chat_with_extended_thinking_anthropic_claude-haiku-4-5_returns_thinking_when_available.yml # spec/fixtures/vcr_cassettes/chat_with_extended_thinking_anthropic_claude-haiku-4-5_streams_thinking_content_when_available.yml # spec/fixtures/vcr_cassettes/chat_with_params_anthropic_claude-haiku-4-5_supports_service_tier_param.yml
|
Fixed them myself to keep momentum. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #608 +/- ##
==========================================
- Coverage 80.89% 80.82% -0.07%
==========================================
Files 113 113
Lines 5099 5112 +13
Branches 1310 1313 +3
==========================================
+ Hits 4125 4132 +7
- Misses 974 980 +6 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
What this does
Adds native structured output support for Anthropic Claude 4.5+ models using the GA output_config API. When with_schema is called on a chat using a supported Anthropic model, the schema is sent via output_config.format with type json_schema, and the response is automatically parsed as JSON.
Because I can't contribute to PR #528, I opened this one.
Type of change
Scope check
Required for new features
PRs for new features or enhancements without a prior approved issue will be closed.
Quality check
overcommit --installand all hooks passbundle exec rake vcr:record[provider_name]bundle exec rspecmodels.json,aliases.json)AI-generated code
API changes