feat(chat-completions): widen reasoning_effort, add usage.completion_tokens_details and service_tier - #4842
Open
lvhan028 wants to merge 3 commits into
Open
feat(chat-completions): widen reasoning_effort, add usage.completion_tokens_details and service_tier#4842lvhan028 wants to merge 3 commits into
lvhan028 wants to merge 3 commits into
Conversation
Contributor
There was a problem hiding this comment.
Pull request overview
This PR advances OpenAI /v1/chat/completions parity by expanding support for reasoning_effort values, and by extending response usage metadata and response shape to include OpenAI-aligned placeholders.
Changes:
- Widen
ChatCompletionRequest.reasoning_effortaccepted values and forward any non-Nonevalue intochat_template_kwargs. - Add
usage.completion_tokens_details(viaCompletionTokensDetails) and addservice_tiertoChatCompletionResponse. - Refactor chat-completions endpoint internals into a package layout with separated protocol/validation/logprobs/logits-processor helpers, plus migration/regression tests.
Reviewed changes
Copilot reviewed 12 out of 12 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
lmdeploy/serve/parsers/response_parser.py |
Forwards any non-None reasoning_effort into chat_template_kwargs. |
lmdeploy/serve/openai/protocol.py |
Adds CompletionTokensDetails, wires it into UsageInfo, and extends UsageInfo.build() with optional reasoning_tokens. |
lmdeploy/serve/openai/endpoints/chat_completions/protocol.py |
Moves chat-completions-specific Pydantic models into endpoint package; widens reasoning_effort; adds service_tier on response. |
lmdeploy/serve/openai/endpoints/chat_completions/serving.py |
Updates serving module to use refactored helpers and the updated UsageInfo model. |
lmdeploy/serve/openai/endpoints/chat_completions/validation.py |
Extracts request validation into a dedicated module. |
lmdeploy/serve/openai/endpoints/chat_completions/logprobs.py |
Extracts logprobs construction helpers. |
lmdeploy/serve/openai/endpoints/chat_completions/logits_processors.py |
Extracts logit_bias logits processor helper. |
lmdeploy/serve/openai/endpoints/chat_completions/__init__.py |
Adds lazy register export to avoid circular imports. |
lmdeploy/serve/openai/endpoints/__init__.py |
Makes create_openai_router a lazy export to avoid circular imports. |
tests/test_lmdeploy/serve/openai/test_reasoning_effort.py |
Adds coverage for widened reasoning_effort validation + forwarding behavior. |
tests/test_lmdeploy/serve/openai/test_usage_details.py |
Adds coverage for completion_tokens_details and service_tier presence/defaults. |
tests/test_chat_completions_package_migration.py |
Adds structural migration tests for the chat-completions package refactor invariants. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Comment on lines
+90
to
+93
| ``reasoning_tokens`` is only populated when the engine exposes a | ||
| reasoning token count; otherwise ``completion_tokens_details`` is left | ||
| ``None`` to match the OpenAI shape (omitted when empty). | ||
| """ |
lvhan028
force-pushed
the
feat/chat-p0-misc
branch
3 times, most recently
from
August 10, 2026 02:40
5342e97 to
108edda
Compare
Aligns with the responses/ package layout. Splits the 633-line chat_completions.py into protocol/validation/logprobs/logits_processors/ serving modules. Chat-specific models move to endpoints/chat_completions/protocol.py; shared models stay in the top-level protocol.py with backward-compat re-exports. No behavior change. Co-Authored-By: Claude <noreply@anthropic.com>
Aligns with OpenAI/vLLM value sets and forwards all non-None values to chat_template_kwargs (previously only high/max were forwarded). Co-Authored-By: Claude <noreply@anthropic.com>
…ponse Adds CompletionTokensDetails (reasoning_tokens etc.) and a service_tier response field to align the response shape with the OpenAI spec. Co-Authored-By: Claude <noreply@anthropic.com>
lvhan028
force-pushed
the
feat/chat-p0-misc
branch
from
August 10, 2026 03:02
108edda to
287014c
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Two P0 chat-completions parity fixes:
reasoning_effortwidened tonone | minimal | low | medium | high | xhigh | max(OpenAI only definesminimal/low/medium/high; the extra values forward to models that support them). The forwarding guard now forwards any non-Nonevalue (previously onlyhigh/max).usage.completion_tokens_details(reasoning_tokens+ reserved fields) andChatCompletionResponse.service_tier.UsageInfo.buildgained an optionalreasoning_tokens(defaults toNone, never fabricated).Tasks
Tasks 3 + 5 of the chat-completions feature plan.
Files
lmdeploy/serve/openai/endpoints/chat_completions/protocol.py— widenedreasoning_effortLiteral;ChatCompletionResponse.service_tier.lmdeploy/serve/openai/protocol.py—CompletionTokensDetails,UsageInfo.completion_tokens_details,UsageInfo.build(reasoning_tokens=None).lmdeploy/serve/parsers/response_parser.py— reasoning_effort forwarding guard (if request.reasoning_effort is not None);BaseResponseParser.chat_template_kwargs_from_requestclassmethod (real prod path).tests/test_lmdeploy/serve/openai/test_reasoning_effort.py— 10 tests;test_usage_details.py— 4 tests.Tests
pytest tests/test_lmdeploy/serve/openai/test_reasoning_effort.py tests/test_lmdeploy/serve/openai/test_usage_details.py -v→ 14 passed.Dependency
Depends on #4840 (
refactor/chat-completions-package). Merge #4840 first, then rebase ontomain.Notes
deepseek_v4_encoding.py/model.pywere left untouched (out of scope).--no-verifylocally (env lacks python3.10 for the docformatter pre-commit hook); CI runs the hook with the correct interpreter.🤖 Generated with Claude Code