Skip to content

examples: validate Responses function tool output - #505

Merged
jbeckwith-oai merged 2 commits into
mainfrom
codex/validate-responses-function-tool-output
Aug 21, 2026
Merged

examples: validate Responses function tool output#505
jbeckwith-oai merged 2 commits into
mainfrom
codex/validate-responses-function-tool-output

Conversation

@jbeckwith-oai

Copy link
Copy Markdown
Contributor

Summary

  • inspect mixed Responses output items before reading parsed function arguments
  • print only parsed GetWeather function calls
  • fail clearly when the response has no valid parsed tool call
  • add offline regression coverage for mixed, empty, and invalid parsed outputs

Scope and compatibility

The change is limited to a handwritten example and its test. It does not modify generated code, runtime SDK behavior, public APIs, type signatures, transport, authentication, serialization, dependencies, or supported Ruby versions.

Generator ownership: examples/ and this regression test are handwritten repository artifacts, so no regeneration is required.

Validation

  • focused example regression: 3 runs, 17 assertions, 0 failures
  • Responses model tests: 8 runs, 8 assertions, 0 failures
  • example E2E harness tests: 15 runs, 187 assertions, 0 failures
  • offline example inventory: 30 examples classified
  • repository lint, rubyfmt, RuboCop, Sorbet, RBS, and directive validation: passed
  • non-Bedrock suite excluding one sandbox-incompatible realtime proxy socket test: 1089 runs, 9805 assertions, 0 failures
  • complete non-Bedrock attempt: 1090 runs, 9805 assertions, 0 failures, 1 unrelated local proxy socket error
  • thermo-nuclear code-quality review: no findings
  • git diff --check: passed

@jbeckwith-oai jbeckwith-oai added the codex-maintenance Low-risk maintenance changes created by Codex label Aug 21, 2026
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Castiron custom code

✅ No new custom-code files detected.

48 mixed files remain; 0 existing customizations changed.

Compared 851f53438f5ac2c1e7b9bf91. Generated baselines verified.

48 existing customizations unchanged
  • lib/openai.rb
  • lib/openai/client.rb
  • lib/openai/models/chat/chat_completion_message.rb
  • lib/openai/models/chat/chat_completion_message_function_tool_call.rb
  • lib/openai/models/chat/completion_create_params.rb
  • lib/openai/models/response_format_json_schema.rb
  • lib/openai/models/responses/function_tool.rb
  • lib/openai/models/responses/response.rb
  • lib/openai/models/responses/response_create_params.rb
  • lib/openai/models/responses/response_format_text_config.rb
  • lib/openai/models/responses/response_format_text_json_schema_config.rb
  • lib/openai/models/responses/response_function_tool_call.rb
  • lib/openai/models/responses/response_function_web_search.rb
  • lib/openai/models/responses/response_output_text.rb
  • lib/openai/models/responses/tool.rb
  • lib/openai/resources/beta/responses.rb
  • lib/openai/resources/beta/threads.rb
  • lib/openai/resources/chat/completions.rb
  • lib/openai/resources/files.rb
  • lib/openai/resources/responses.rb
  • lib/openai/resources/vector_stores/file_batches.rb
  • lib/openai/resources/vector_stores/files.rb
  • lib/openai/resources/webhooks.rb
  • rbi/openai/client.rbi
  • rbi/openai/models/chat/chat_completion_message.rbi
  • rbi/openai/models/chat/chat_completion_message_function_tool_call.rbi
  • rbi/openai/models/chat/completion_create_params.rbi
  • rbi/openai/models/response_format_json_schema.rbi
  • rbi/openai/models/responses/response.rbi
  • rbi/openai/models/responses/response_create_params.rbi
  • rbi/openai/models/responses/response_function_tool_call.rbi
  • rbi/openai/models/responses/response_function_web_search.rbi
  • rbi/openai/models/responses/response_output_text.rbi
  • rbi/openai/resources/chat/completions.rbi
  • rbi/openai/resources/files.rbi
  • rbi/openai/resources/responses.rbi
  • rbi/openai/resources/vector_stores/file_batches.rbi
  • rbi/openai/resources/vector_stores/files.rbi
  • scripts/castiron/README.md
  • scripts/castiron/custom_code_report.py

8 more in the full report.

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 32446845547 --repo openai/openai-ruby \
  --name castiron-custom-code-32446845547-1 --dir /tmp/castiron-custom-code-32446845547-1
git apply --stat /tmp/castiron-custom-code-32446845547-1/custom-code.patch
cat /tmp/castiron-custom-code-32446845547-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 851f53438f5afc7f32e3ccbe866fd03bbade576a c2c1e7b9bf91ff7ae5ad63d6f236ad551922fa49
python3 scripts/castiron/custom_code_report.py report \
  --base 851f53438f5afc7f32e3ccbe866fd03bbade576a \
  --head c2c1e7b9bf91ff7ae5ad63d6f236ad551922fa49 --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-c2c1e7b9bf91
cat /tmp/castiron-custom-code-c2c1e7b9bf91/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@jbeckwith-oai
jbeckwith-oai marked this pull request as ready for review August 21, 2026 04:24
@jbeckwith-oai
jbeckwith-oai requested a review from a team as a code owner August 21, 2026 04:24
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-21T04:25:35.010544Z c2c1e7b Draft marked ready
🔒 Security Review Completed 2026-08-21T04:25:49.874349Z c2c1e7b Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@jbeckwith-oai jbeckwith-oai changed the title fix(examples): validate Responses function tool output examples: validate Responses function tool output Aug 21, 2026
@openai-sdks

openai-sdks Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

OkTest Summary

237/237 SDK tests passed in 9.22s for Ruby SDK PR #505.

Test results — 42 files
Test Result Time
tests/chat-completions-complex-body.test.ts ✅ Passed 147ms
tests/chat-completions-create.test.ts ✅ Passed 188ms
tests/chat-completions-stream.test.ts ✅ Passed 116ms
tests/files-content-binary.test.ts ✅ Passed 172ms
tests/files-create-multipart.test.ts ✅ Passed 275ms
tests/files-list-pagination.test.ts ✅ Passed 128ms
tests/initialize-config.test.ts ✅ Passed 244ms
tests/instance-isolation.test.ts ✅ Passed 162ms
tests/models-list.test.ts ✅ Passed 266ms
tests/responses-background-lifecycle.test.ts ✅ Passed 135ms
tests/responses-body-method-errors.test.ts ✅ Passed 376ms
tests/responses-cancel-timeout.test.ts ✅ Passed 175ms
tests/responses-cancel.test.ts ✅ Passed 186ms
tests/responses-compact-retries.test.ts ✅ Passed 219ms
tests/responses-compact.test.ts ✅ Passed 237ms
tests/responses-create-advanced-stream.test.ts ✅ Passed 140ms
tests/responses-create-advanced.test.ts ✅ Passed 141ms
tests/responses-create-disconnect.test.ts ✅ Passed 1.063s
tests/responses-create-errors.test.ts ✅ Passed 385ms
tests/responses-create-malformed-api-responses.test.ts ✅ Passed 149ms
tests/responses-create-retries.test.ts ✅ Passed 244ms
tests/responses-create-stream-failures.test.ts ✅ Passed 125ms
tests/responses-create-stream-timeout.test.ts ✅ Passed 193ms
tests/responses-create-stream-wire.test.ts ✅ Passed 2.359s
tests/responses-create-stream.test.ts ✅ Passed 73ms
tests/responses-create-terminal-states.test.ts ✅ Passed 295ms
tests/responses-create-timeout.test.ts ✅ Passed 174ms
tests/responses-create.test.ts ✅ Passed 220ms
tests/responses-delete.test.ts ✅ Passed 245ms
tests/responses-input-items-errors.test.ts ✅ Passed 144ms
tests/responses-input-items-list.test.ts ✅ Passed 151ms
tests/responses-input-items-options.test.ts ✅ Passed 147ms
tests/responses-input-tokens-count-timeout.test.ts ✅ Passed 213ms
tests/responses-input-tokens-count.test.ts ✅ Passed 201ms
tests/responses-malformed-inputs.test.ts ✅ Passed 2.027s
tests/responses-not-found-errors.test.ts ✅ Passed 275ms
tests/responses-parse.test.ts ✅ Passed 164ms
tests/responses-retrieve-retries.test.ts ✅ Passed 273ms
tests/responses-retrieve.test.ts ✅ Passed 177ms
tests/responses-stored-method-errors.test.ts ✅ Passed 607ms
tests/retry-behavior.test.ts ✅ Passed 3.027s
tests/sdk-error-shape.test.ts ✅ Passed 366ms

View OkTest run #32446847158

SDK merge (7fe8b4b34904) · head (c2c1e7b9bf91) · base (851f53438f5a) · OkTest (2b1bdfd25e98)

@apcha-oai apcha-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[sdk codex review] Approved after two independent reviews of c2c1e7b. Verified mixed-output discrimination, typed GetWeather parsing, multiple valid calls, clear empty/invalid-output failure, and serial-test isolation without SDK API changes. Independent checks passed: 13 focused/helper tests with 53 assertions, five separate real-parser/example scenarios, syntax, and offline inventory. The local test run used a fallback mock dependency rather than the exact locked bundle; no live API tests were run. All current required checks pass. Latest main's tooling-only changes were reviewed, and head/base/author/draft/mergeability and feedback were rechecked before submission.

@jbeckwith-oai
jbeckwith-oai added this pull request to the merge queue Aug 21, 2026
Merged via the queue into main with commit 17d43f3 Aug 21, 2026
16 checks passed
@jbeckwith-oai
jbeckwith-oai deleted the codex/validate-responses-function-tool-output branch August 21, 2026 19:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-maintenance Low-risk maintenance changes created by Codex

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants