fix(llm): strip cache_breakpoint marker on the litellm provider path

[HumphreySun98]

Problem

The prompt-cache breakpoint marker is injected provider-agnostically by the agent executors (mark_cache_breakpoint), but only the native Anthropic adapter consumes it. Native providers route through BaseLLM._format_messages, which already strips the marker. The default litellm-based LLM does not: _format_messages_for_provider returned messages untouched for non-Anthropic providers, so the raw cache_breakpoint key reached the wire and was rejected — e.g. Groq:

GroqException - 'messages.0': property 'cache_breakpoint' is unsupported

This breaks any crew using Groq or other OpenAI-compatible models on the litellm path.

Fix

Strip the marker in LLM._format_messages_for_provider, mirroring the base class. It copies rather than mutates, so the executor's reused message buffer keeps its markers across tool-loop iterations (preserving Anthropic prompt caching).

Tests

Added TestLiteLLMStripsMarker in test_prompt_cache.py covering the litellm path (the existing tests only exercised native providers). Verified the new tests fail without the fix and pass with it; ruff and mypy clean.

Fixes #5886

---

This PR was authored with Claude Code. Per CONTRIBUTING.md, AI-generated contributions require the llm-generated label — I don't have triage permission to set it, so could a maintainer please add it? 🤖 Generated with Claude Code

Summary by CodeRabbit

• Bug Fixes

  • Improved validation of message structure to ensure compatibility across providers.
  • Enhanced handling of internal caching metadata during message formatting.

• Tests

  • Added regression tests to verify message formatting and caching behavior.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: f01c55a7-fed6-4f26-af01-043dd093e3ae

📥 Commits

Reviewing files that changed from the base of the PR and between e9d568d and 867ec4e.

📒 Files selected for processing (2)

• lib/crewai/src/crewai/llm.py
• lib/crewai/tests/llms/test_prompt_cache.py

---

📝 Walkthrough

Walkthrough

LLM._format_messages_for_provider now imports CACHE_BREAKPOINT_KEY, validates each message dict has role and content, and builds a cleaned copy list with the cache_breakpoint key removed before any provider-specific formatting runs. Two regression tests are added to verify the marker is absent in Groq-formatted output and that the caller's original message list is not mutated.

Changes

Strip cache_breakpoint marker for non-Anthropic providers

Layer / File(s)	Summary
Validate and strip cache_breakpoint from messages lib/crewai/src/crewai/llm.py	_format_messages_for_provider imports CACHE_BREAKPOINT_KEY, validates each message has role and content (raising TypeError otherwise), copies each message dict with the marker key removed, and reassigns messages to the cleaned list before provider-specific branches execute.
Regression tests for stripping and non-mutation lib/crewai/tests/llms/test_prompt_cache.py	Adds LLM import and TestLiteLLMStripsMarker with two tests: one asserting no cache_breakpoint key is present in Groq-formatted output, and one asserting the caller-supplied message dicts still carry the key after formatting (no in-place mutation).

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~8 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title accurately describes the main change: stripping the cache_breakpoint marker on the litellm provider path.
Linked Issues check	✅ Passed	The implementation addresses issue #5886 by stripping cache_breakpoint in LLM._format_messages_for_provider and testing the litellm provider path.
Out of Scope Changes check	✅ Passed	All changes are directly related to fixing the cache_breakpoint marker handling for litellm providers as specified in the objectives.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}