feat: add LiteLLM plugin for 100+ LLM providers

[senthilkumaranT]

Description

This pull request adds livekit-plugins-litellm as a new plugin package. It integrates the LiteLLM SDK directly into LiveKit Agents, enabling developers to connect to 100+ LLM providers (Anthropic, Bedrock, Vertex AI, Cohere, etc.) through a single, unified interface.

Key Changes

• Plugin Implementation (livekit-plugins-litellm):
  • llm.py: Implements the LLM class inheriting from OpenAILLM to reuse existing streaming, chunking, and tool-calling infrastructure.
  • _LiteLLMClientShim: An AsyncOpenAI-compatible shim client that routes all completion requests to litellm.acompletion without requiring an external proxy server.
  • Exposes config overrides like drop_params (defaulting to True) to prevent provider-specific parameters from causing API errors.
• Root Configuration (pyproject.toml):
  • Registered livekit-plugins-litellm in the workspace under [tool.uv.sources].
• Unit Tests (tests/test_litellm_llm.py):
  • Added unit tests covering shim dispatching logic, drop_params overrides, defaults, and model instantiations.

---

Verification

Run the unit test suite:
python -m unittest livekit-plugins/livekit-plugins-litellm/tests/test_litellm_llm.py

Results:
Ran 8 tests in 4.133s - OK

[CLAassistant]

Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
_{You have signed the CLA already but the status is still pending? Let us recheck it.}

[devin-ai-integration]

Devin Review found 1 potential issue.

View 2 additional findings in Devin Review.

[devin-ai-integration]

🟡 Explicitly provided API key is silently ignored

An API key passed to the LiteLLM model is stored (os.environ.setdefault("LITELLM_API_KEY", ...) at livekit-plugins/livekit-plugins-litellm/livekit/plugins/litellm/llm.py:105) under a name the LiteLLM SDK never reads for provider authentication, so the key is never actually used and requests can fail to authenticate.
Impact: A user who passes an API key directly instead of using environment variables gets authentication failures at request time even though the key was accepted without error.

Prompt for agents

The constructor accepts an api_key argument but only writes it to os.environ under LITELLM_API_KEY via os.environ.setdefault (llm.py:103-105). This has two problems: (1) the LiteLLM SDK's acompletion() does not read a generic LITELLM_API_KEY env var to authenticate to the underlying provider — it reads provider-specific vars (OPENAI_API_KEY, ANTHROPIC_API_KEY, etc.) or accepts an explicit api_key= keyword argument on the completion call; and (2) setdefault silently does nothing if LITELLM_API_KEY already exists in the environment. As a result, an explicitly supplied api_key is effectively dropped and provider auth may fail. Consider storing the api_key on the instance and threading it through the shim so it is passed as api_key= into litellm.acompletion() (the shim's create() could inject a default api_key into kwargs), rather than setting an unused env var. Ensure the value flows to the actual completion call for the chosen provider.

---

Was this helpful? React with 👍 or 👎 to provide feedback.