fix(litellm): parse DeepSeek-V3 proprietary inline tool-call tokens#5654
Open
fuchun1010 wants to merge 1 commit intogoogle:mainfrom
Open
fix(litellm): parse DeepSeek-V3 proprietary inline tool-call tokens#5654fuchun1010 wants to merge 1 commit intogoogle:mainfrom
fuchun1010 wants to merge 1 commit intogoogle:mainfrom
Conversation
|
Thanks for your pull request! It looks like this may be your first contribution to a Google open source project. Before we can look at your pull request, you'll need to sign a Contributor License Agreement (CLA). View this failed invocation of the CLA check for more information. For the most up to date status, view the checks section at the bottom of the pull request. |
c319bae to
e91b1f6
Compare
DeepSeek-V3 emits tool calls using proprietary special tokens (<|tool▁calls▁begin|>…<|tool▁call▁begin|>function<|tool▁sep|>NAME) embedded in the content field. When LiteLLM does not translate these into structured tool_calls (intermittent), the existing fallback JSON parser rejects the payload because the function name is stored inside the tokens rather than as a 'name' key in the JSON object. Add _parse_deepseek_tool_calls_from_text that detects the proprietary token format, extracts the function name and arguments, and emits standard ChatCompletionMessageToolCall objects. Integrate it into the existing _parse_tool_calls_from_text pipeline. Also add _extract_json_from_deepseek_args helper to handle optional Markdown code fences (json … ) that DeepSeek wraps around the arguments payload. Closes google#5024
e91b1f6 to
08e864e
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #5024
Problem
DeepSeek-V3 emits tool calls using proprietary special tokens embedded in the content field:
When LiteLLM does not translate these into structured
tool_calls(intermittent), ADK's fallback JSON parser finds the JSON object but rejects it because the function name (analysis_input) is embedded in the tokens (<|tool▁sep|>analysis_input) rather than as anamekey inside the JSON payload.Result: tool call is silently dropped and the raw tokens appear as text content.
Solution
_parse_deepseek_tool_calls_from_text— detects the proprietary token format, extracts function name + arguments, and emits standardChatCompletionMessageToolCallobjects_extract_json_from_deepseek_argshelper — handles optional Markdown code fences (```json ```) around the arguments payload_parse_tool_calls_from_textas the first-pass parser, with fallback to generic inline JSON parsingTesting Plan
Unit Tests: Added 8 new tests covering:
_parse_tool_calls_from_textRegression: Full
test_litellm.py: 264 passed, 0 failedFiles Changed
src/google/adk/models/lite_llm.pytests/unittests/models/test_litellm.py