fix(llm): strip cache-breakpoint marker in the litellm path (#6789)

[Anai-Guo]

Problem

Fixes #6789.

A mistral/* LLM works via a direct llm.call("Say hello"), but the same LLM attached to an Agent fails during crew.kickoff() with:

litellm.BadRequestError: MistralException
{"type":"extra_forbidden","loc":["body","messages",0,"system","cache_breakpoint"],
 "msg":"Extra inputs are not permitted"}

Root cause

The agent executor marks the stable-prefix messages with a provider-agnostic cache_breakpoint flag (crew_agent_executor.py, via mark_cache_breakpoint). Adapters are expected to translate that marker into their own cache directive, or strip it:

• Native providers (is_litellm=False) strip it in BaseLLM._format_messages.
• The litellm fallback path (crewai.llm.LLM, used for mistral/*, and anything not in SUPPORTED_NATIVE_PROVIDERS) does not go through _format_messages — it builds the wire payload in _format_messages_for_provider, which never removed the marker.

So for litellm-routed models the unknown cache_breakpoint key leaked straight into the request. Providers that tolerate extra message fields (OpenAI) silently ignore it; Mistral validates strictly and rejects it. Direct llm.call() doesn't hit it because it doesn't carry the executor's markers.

Fix

Strip CACHE_BREAKPOINT_KEY at the top of LLM._format_messages_for_provider, mirroring what the native path already does. Only the messages that actually carry the marker are copied, so the executor's reused message buffer keeps its markers for later ReAct-loop iterations (the same invariant TestBaseFormatDoesNotMutate guards for the native path).

Tests

New TestLiteLLMStripsMarker in test_prompt_cache.py:

• test_provider_format_strips_marker — marker is gone from the wire payload, and the caller's buffer still has it. Fails on main, passes with the fix.
• test_unmarked_messages_pass_through — no marker ⇒ payload unchanged.

Full test_prompt_cache.py passes (11 passed).

🤖 Generated with Claude Code

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 36fa6127-e279-4cb0-b6fa-1542add2d6ec

📥 Commits

Reviewing files that changed from the base of the PR and between c8f441c and 6d9c800.

📒 Files selected for processing (2)

• lib/crewai/src/crewai/llm.py
• lib/crewai/tests/llms/test_prompt_cache.py

---

📝 Walkthrough

Walkthrough

Changes

LiteLLM cache marker handling

Layer / File(s)	Summary
Strip cache markers before provider formatting lib/crewai/src/crewai/llm.py	Messages containing CACHE_BREAKPOINT_KEY are copied without that field before LiteLLM processing. Unaffected messages are reused.
Validate marker preservation and removal lib/crewai/tests/llms/test_prompt_cache.py	Tests verify that transmitted messages omit the marker, caller messages retain it, and unmarked messages remain unchanged.

Suggested reviewers: lorenzejay

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title clearly describes stripping the cache-breakpoint marker in the LiteLLM path.
Description check	✅ Passed	The description explains the Mistral failure, root cause, fix, and test coverage covered by the changeset.
Linked Issues check	✅ Passed	The changes strip unsupported cache-breakpoint metadata for LiteLLM requests and preserve input markers, resolving issue #6789.
Out of Scope Changes check	✅ Passed	The code and tests are limited to the LiteLLM marker-handling fix described in issue #6789.
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}