fix: handle pydantic ValidationError in beta.chat.completions.parse (fixes #1763)#2917
Open
giulio-leone wants to merge 10 commits intoopenai:mainfrom
Open
fix: handle pydantic ValidationError in beta.chat.completions.parse (fixes #1763)#2917giulio-leone wants to merge 10 commits intoopenai:mainfrom
giulio-leone wants to merge 10 commits intoopenai:mainfrom
Conversation
When beta.chat.completions.parse() receives malformed or truncated JSON from the API, pydantic.ValidationError was raised directly without any context. Now catches pydantic.ValidationError and json.JSONDecodeError in _parse_content() and wraps them in a new ContentFormatError that includes the raw content string for debugging. Fixes openai#1763
There was a problem hiding this comment.
Pull request overview
Adds a dedicated SDK error for structured-output parsing failures so users get a clear, actionable exception (with access to the raw model content) instead of a bare pydantic.ValidationError when chat.completions.parse() receives malformed/truncated JSON.
Changes:
- Introduces
ContentFormatError(subclass ofOpenAIError) to represent response-format parsing failures. - Wraps Pydantic parsing in
_parse_content()to raiseContentFormatErroronpydantic.ValidationError/json.JSONDecodeError. - Adds regression tests covering malformed JSON and schema mismatch cases, and exports the new exception from
openai.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 4 comments.
| File | Description |
|---|---|
src/openai/lib/_parsing/_completions.py |
Wraps response-format parsing to raise a consistent SDK exception instead of leaking raw Pydantic errors. |
src/openai/_exceptions.py |
Adds ContentFormatError carrying raw_content for debugging. |
src/openai/__init__.py |
Exports ContentFormatError from the top-level package. |
tests/lib/chat/test_completions.py |
Adds tests asserting ContentFormatError and raw_content preservation for malformed JSON and schema mismatch. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Truncate raw_content in error message to 500 chars to prevent unbounded messages and reduce sensitive data exposure - Remove redundant self.__cause__ assignment (raise...from sets it); store error on dedicated .error attribute instead - Update docstring to cover both Pydantic models and dataclass-like types validated via TypeAdapter Refs: openai#1763
Author
|
Thanks for the thorough review! All feedback has been addressed in 81f3e5b:
|
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Include truncated raw_content (first 500 chars) in exception message for better debugging while avoiding unbounded output. Full content remains accessible via the raw_content attribute. Update tests to reflect the new message format and add truncation coverage test. Refs: openai#2917
121799e to
4290c9a
Compare
Apply reviewer code suggestions from PR review.
…response_format parameter - Remove dangling code block from _exceptions.py ContentFormatError class that caused IndentationError - Fix _completions.py to pass response_format= instead of expected_type= to ContentFormatError
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When
beta.chat.completions.parse()receives malformed or truncated JSON from the API (withfinish_reason: "stop"), a rawpydantic.ValidationErroris raised directly to the user with no context about what went wrong or what the original content was.This is confusing because:
Solution
Added a new
ContentFormatErrorexception (extendsOpenAIError, following the pattern ofLengthFinishReasonError) that wraps bothpydantic.ValidationErrorandjson.JSONDecodeErrorin_parse_content().The new error:
raw_contentattribute for debugging__cause__so the full traceback is availableChanges
src/openai/_exceptions.py: AddedContentFormatErrorclasssrc/openai/__init__.py: ExportedContentFormatErrorsrc/openai/lib/_parsing/_completions.py: Wrapped_parse_content()with try/excepttests/lib/chat/test_completions.py: Added tests for malformed JSON and schema mismatchExample
Fixes #1763