Skip to content

examples: validate E2E manifest mappings - #503

Merged
jbeckwith-oai merged 1 commit into
mainfrom
codex/validate-examples-e2e-manifest-mappings
Aug 21, 2026
Merged

examples: validate E2E manifest mappings#503
jbeckwith-oai merged 1 commit into
mainfrom
codex/validate-examples-e2e-manifest-mappings

Conversation

@jbeckwith-oai

Copy link
Copy Markdown
Contributor

Summary

  • validate that the examples E2E YAML root is a mapping before reading manifest fields
  • validate that every per-example configuration is a mapping before constructing inventory entries
  • include manifest and example paths in configuration errors

Evidence

Malformed YAML shapes currently reach hash access and fail with incidental exceptions. The focused tests cover nil, array, and string roots and per-example configurations, and assert stable ConfigurationError messages.

Blast radius and compatibility

This is limited to the handwritten examples E2E inventory script and its dedicated tests. It does not change the public SDK API, generated code, request transport, wire behavior, dependencies, RBI/RBS contracts, or supported Ruby versions. Valid manifests retain existing behavior.

Generator ownership

Both changed files are handwritten repository tooling and tests; no generator-owned source or output is modified.

Validation

  • bundle exec rake test TEST=test/scripts/examples_e2e_test.rb — 17 runs, 199 assertions, 0 failures, 0 errors
  • offline examples E2E inventory validation — passes for all 30 classified examples
  • bundle exec rake lint — 2,728 Ruby files inspected with no offenses; 1,239 RBS files validated; Sorbet, rubyfmt, and directive validation pass
  • thermo-nuclear code-quality review — no findings
  • git diff --check — clean
  • full non-Bedrock suite — 1,089 runs, 9,800 assertions, with one unrelated realtime proxy socket error that reproduces in isolation before any assertion

Because this touches manifest deserialization validation, please include @openai/sdks-team in review routing.

@jbeckwith-oai jbeckwith-oai added the codex-maintenance Low-risk maintenance changes created by Codex label Aug 21, 2026
@github-actions

github-actions Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

Castiron custom code

✅ No new custom-code files detected.

48 mixed files remain; 0 existing customizations changed.

Compared 851f53438f5ab518c5c1b3ac. Generated baselines verified.

48 existing customizations unchanged
  • lib/openai.rb
  • lib/openai/client.rb
  • lib/openai/models/chat/chat_completion_message.rb
  • lib/openai/models/chat/chat_completion_message_function_tool_call.rb
  • lib/openai/models/chat/completion_create_params.rb
  • lib/openai/models/response_format_json_schema.rb
  • lib/openai/models/responses/function_tool.rb
  • lib/openai/models/responses/response.rb
  • lib/openai/models/responses/response_create_params.rb
  • lib/openai/models/responses/response_format_text_config.rb
  • lib/openai/models/responses/response_format_text_json_schema_config.rb
  • lib/openai/models/responses/response_function_tool_call.rb
  • lib/openai/models/responses/response_function_web_search.rb
  • lib/openai/models/responses/response_output_text.rb
  • lib/openai/models/responses/tool.rb
  • lib/openai/resources/beta/responses.rb
  • lib/openai/resources/beta/threads.rb
  • lib/openai/resources/chat/completions.rb
  • lib/openai/resources/files.rb
  • lib/openai/resources/responses.rb
  • lib/openai/resources/vector_stores/file_batches.rb
  • lib/openai/resources/vector_stores/files.rb
  • lib/openai/resources/webhooks.rb
  • rbi/openai/client.rbi
  • rbi/openai/models/chat/chat_completion_message.rbi
  • rbi/openai/models/chat/chat_completion_message_function_tool_call.rbi
  • rbi/openai/models/chat/completion_create_params.rbi
  • rbi/openai/models/response_format_json_schema.rbi
  • rbi/openai/models/responses/response.rbi
  • rbi/openai/models/responses/response_create_params.rbi
  • rbi/openai/models/responses/response_function_tool_call.rbi
  • rbi/openai/models/responses/response_function_web_search.rbi
  • rbi/openai/models/responses/response_output_text.rbi
  • rbi/openai/resources/chat/completions.rbi
  • rbi/openai/resources/files.rbi
  • rbi/openai/resources/responses.rbi
  • rbi/openai/resources/vector_stores/file_batches.rbi
  • rbi/openai/resources/vector_stores/files.rbi
  • scripts/castiron/README.md
  • scripts/castiron/custom_code_report.py

8 more in the full report.

A changed generated baseline means this report cannot reliably identify which handwritten lines changed.

Inspect the custom-code diff

Download the exact patch produced by this run (requires repository access):

gh run download 32446588835 --repo openai/openai-ruby \
  --name castiron-custom-code-32446588835-1 --dir /tmp/castiron-custom-code-32446588835-1
git apply --stat /tmp/castiron-custom-code-32446588835-1/custom-code.patch
cat /tmp/castiron-custom-code-32446588835-1/custom-code.patch

Or reproduce it from an SDK checkout containing the vendored reporter:

git fetch --no-tags origin 851f53438f5afc7f32e3ccbe866fd03bbade576a b518c5c1b3ac5defbee9e0b6ead0f51c85dfeb05
python3 scripts/castiron/custom_code_report.py report \
  --base 851f53438f5afc7f32e3ccbe866fd03bbade576a \
  --head b518c5c1b3ac5defbee9e0b6ead0f51c85dfeb05 --fetch --require-head-hash --public \
  --out /tmp/castiron-custom-code-b518c5c1b3ac
cat /tmp/castiron-custom-code-b518c5c1b3ac/custom-code.patch

This is the current full custom patch for mixed files, not an attribution of only the handwritten lines changed by this PR.

Full report and patch

@jbeckwith-oai
jbeckwith-oai requested a review from a team August 21, 2026 04:16
@jbeckwith-oai
jbeckwith-oai marked this pull request as ready for review August 21, 2026 04:20
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Aug 21, 2026

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review Completed 2026-08-21T04:21:21.527344Z b518c5c Draft marked ready
🔒 Security Review Completed 2026-08-21T04:22:01.029604Z b518c5c Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@openai-sdks

openai-sdks Bot commented Aug 21, 2026

Copy link
Copy Markdown
Contributor

OkTest Summary

237/237 SDK tests passed in 8.915s for Ruby SDK PR #503.

Test results — 42 files
Test Result Time
tests/chat-completions-complex-body.test.ts ✅ Passed 133ms
tests/chat-completions-create.test.ts ✅ Passed 191ms
tests/chat-completions-stream.test.ts ✅ Passed 101ms
tests/files-content-binary.test.ts ✅ Passed 118ms
tests/files-create-multipart.test.ts ✅ Passed 181ms
tests/files-list-pagination.test.ts ✅ Passed 120ms
tests/initialize-config.test.ts ✅ Passed 141ms
tests/instance-isolation.test.ts ✅ Passed 181ms
tests/models-list.test.ts ✅ Passed 133ms
tests/responses-background-lifecycle.test.ts ✅ Passed 160ms
tests/responses-body-method-errors.test.ts ✅ Passed 362ms
tests/responses-cancel-timeout.test.ts ✅ Passed 307ms
tests/responses-cancel.test.ts ✅ Passed 218ms
tests/responses-compact-retries.test.ts ✅ Passed 153ms
tests/responses-compact.test.ts ✅ Passed 186ms
tests/responses-create-advanced-stream.test.ts ✅ Passed 151ms
tests/responses-create-advanced.test.ts ✅ Passed 141ms
tests/responses-create-disconnect.test.ts ✅ Passed 971ms
tests/responses-create-errors.test.ts ✅ Passed 460ms
tests/responses-create-malformed-api-responses.test.ts ✅ Passed 105ms
tests/responses-create-retries.test.ts ✅ Passed 356ms
tests/responses-create-stream-failures.test.ts ✅ Passed 117ms
tests/responses-create-stream-timeout.test.ts ✅ Passed 199ms
tests/responses-create-stream-wire.test.ts ✅ Passed 2.232s
tests/responses-create-stream.test.ts ✅ Passed 93ms
tests/responses-create-terminal-states.test.ts ✅ Passed 253ms
tests/responses-create-timeout.test.ts ✅ Passed 230ms
tests/responses-create.test.ts ✅ Passed 203ms
tests/responses-delete.test.ts ✅ Passed 207ms
tests/responses-input-items-errors.test.ts ✅ Passed 147ms
tests/responses-input-items-list.test.ts ✅ Passed 143ms
tests/responses-input-items-options.test.ts ✅ Passed 126ms
tests/responses-input-tokens-count-timeout.test.ts ✅ Passed 246ms
tests/responses-input-tokens-count.test.ts ✅ Passed 247ms
tests/responses-malformed-inputs.test.ts ✅ Passed 1.85s
tests/responses-not-found-errors.test.ts ✅ Passed 237ms
tests/responses-parse.test.ts ✅ Passed 130ms
tests/responses-retrieve-retries.test.ts ✅ Passed 208ms
tests/responses-retrieve.test.ts ✅ Passed 195ms
tests/responses-stored-method-errors.test.ts ✅ Passed 537ms
tests/retry-behavior.test.ts ✅ Passed 3.109s
tests/sdk-error-shape.test.ts ✅ Passed 384ms

View OkTest run #32446590181

SDK merge (45d14afcced4) · head (b518c5c1b3ac) · base (851f53438f5a) · OkTest (2b1bdfd25e98)

@jbeckwith-oai jbeckwith-oai changed the title fix(examples): validate E2E manifest mappings examples: validate E2E manifest mappings Aug 21, 2026

@apcha-oai apcha-oai left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[sdk codex review] Approved after two independent reviews of b518c5c. Root, examples-collection, and per-example mapping guards reject malformed shapes before access and preserve valid-manifest behavior, safe YAML loading, subprocess controls, and report redaction. Both reviewers independently passed the 17-test/199-assertion harness suite, additional malformed-shape CLI probes, syntax checks, and offline inventory. Local tests used available mock-library fallbacks rather than the exact locked bundle; no live APIs were used. All current required checks pass. Head/base/author/draft/mergeability and existing feedback were rechecked; no actionable findings.

@jbeckwith-oai
jbeckwith-oai added this pull request to the merge queue Aug 21, 2026
Merged via the queue into main with commit 86eb8e2 Aug 21, 2026
17 checks passed
@jbeckwith-oai
jbeckwith-oai deleted the codex/validate-examples-e2e-manifest-mappings branch August 21, 2026 19:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

codex-maintenance Low-risk maintenance changes created by Codex

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants