Skip to content

fix: preserve Gemini thought_signature in LiteLLM multi-turn tool calls#1888

Open
giulio-leone wants to merge 1 commit intostrands-agents:mainfrom
giulio-leone:fix/litellm-thought-signature-preservation
Open

fix: preserve Gemini thought_signature in LiteLLM multi-turn tool calls#1888
giulio-leone wants to merge 1 commit intostrands-agents:mainfrom
giulio-leone:fix/litellm-thought-signature-preservation

Conversation

@giulio-leone
Copy link
Contributor

Issue

Closes #1764

Description

When using Gemini thinking models (e.g., gemini-2.5-flash, gemini-3-flash) through the LiteLLM model provider, multi-turn conversations with tool calls fail with:

function call 'current_time' in the 2. content block is missing a thought_signature

Root Cause

Gemini thinking models require a thought_signature on each function call in multi-turn conversations. LiteLLM encodes this signature into the tool call ID using a __thought__ separator (e.g., call_abc__thought__base64sig). However, Strands' OpenAIModel.format_chunk() passes the encoded ID through without extracting the signature into the reasoningSignature field that the streaming layer already supports.

While the encoded ID survives the round-trip in simple cases, the signature must also be explicitly stored in reasoningSignature so that:

  1. The streaming infrastructure can properly track it through content block lifecycle
  2. The signature can be re-encoded into the tool call ID even when the ID is cleaned or regenerated
  3. provider_specific_fields.thought_signature (an alternative LiteLLM transport) is also captured

Changes

src/strands/models/litellm.py:

  • Override format_chunk to detect __thought__ in tool call IDs and/or provider_specific_fields.thought_signature, extracting the signature into reasoningSignature on the contentBlockStart event
  • Override format_request_message_tool_call to re-encode reasoningSignature back into the tool call ID when not already present, ensuring LiteLLM can reconstruct the Gemini-native format

tests/strands/models/test_litellm.py:

  • 7 new unit tests covering: extraction from encoded ID, extraction from provider_specific_fields, no-signature passthrough, signature encoding, double-encode prevention, and full round-trip preservation

Checklist

  • I've read CONTRIBUTING.md
  • I've updated or added tests to cover changes
  • I've verified all existing tests pass (48/48)
  • This change is backward compatible (no breaking changes)

When using Gemini thinking models (e.g., gemini-2.5-flash) through the
LiteLLM model provider, multi-turn conversations with tool calls fail
because thought_signature is lost during the response-to-request
round trip.

LiteLLM encodes Gemini's thought_signature into the tool call ID using
a __thought__ separator. The OpenAI parent format_chunk passes this
through as-is, but the signature is never extracted into Strands'
reasoningSignature field, which the streaming layer already supports.

Changes:
- Override format_chunk in LiteLLMModel to detect __thought__ in tool
  call IDs and provider_specific_fields, extracting the signature into
  reasoningSignature for proper streaming layer storage
- Override format_request_message_tool_call to re-encode
  reasoningSignature back into the tool call ID when it is not already
  present, ensuring LiteLLM can reconstruct the Gemini-native format
- Add 7 unit tests covering extraction from ID, extraction from
  provider_specific_fields, no-signature passthrough, encoding,
  double-encode prevention, and full round-trip preservation

Closes strands-agents#1764
@giulio-leone giulio-leone force-pushed the fix/litellm-thought-signature-preservation branch from 5d7f638 to 5fe1504 Compare March 15, 2026 16:12
@github-actions github-actions bot added size/m and removed size/m labels Mar 15, 2026
@giulio-leone
Copy link
Contributor Author

Friendly ping — preserves the Gemini thought_signature field through LiteLLM multi-turn tool call cycles, fixing broken thinking chains.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[BUG] missing thought signature preservation for thinking models with LiteLLM Model Provider

1 participant