Skip to content

feat: add litellm as AI provider gateway - #13151

Open
prodmanpd wants to merge 3 commits into
continuedev:mainfrom
prodmanpd:feat/add-litellm-provider
Open

feat: add litellm as AI provider gateway#13151
prodmanpd wants to merge 3 commits into
continuedev:mainfrom
prodmanpd:feat/add-litellm-provider

Conversation

@prodmanpd

Copy link
Copy Markdown

Description

Adds LiteLLM as a first-class model provider (provider: litellm). LiteLLM is an open-source, self-hosted AI gateway that fronts 100+ LLM providers behind one OpenAI-compatible API. LiteLLM extends the OpenAI class (mirroring the existing OpenRouter / ClawRouter gateway providers) and defaults apiBase to the local proxy (http://localhost:4000/v1/). A user points apiBase at their proxy with a key and either names a model or uses model: AUTODETECT to discover everything the proxy serves via GET /v1/models.

Changes:

  • core/llm/llms/LiteLLM.ts - new provider (providerName = "litellm", default proxy apiBase, reasoning-field support); registered in core/llm/llms/index.ts.
  • core/llm/autodetect.ts - added to PROVIDER_HANDLES_TEMPLATING and PROVIDER_SUPPORTS_IMAGES (a proxy can route to vision models).
  • core/llm/toolSupport.ts - tool-support detection by routed model name.
  • gui/src/pages/AddNewModel/configs/providers.ts - "Add Model" onboarding entry (API-key + Base URL inputs, then AUTODETECT).
  • Docs: more/litellm.mdx (incl. a model: AUTODETECT example) + overview table row and nav entry.

AI Code Review

  • Team members only: AI review runs automatically when PR is opened or marked ready for review
  • Team members can also trigger a review by commenting @continue-review

Checklist

  • I've read the contributing guide
  • The relevant docs, if any, have been updated or created
  • The relevant tests, if any, have been updated or created

Tests

  • Provider unit tests: core/llm/llms/LiteLLM.vitest.ts (5 passed) and a litellm block added to core/llm/toolSupport.test.ts.
  • Whole gui package: npm run tsc:check -> 0 errors (including the new Add-Model entry); npm run test (vitest) -> 37 files / 372 tests passed. npx tsc --noEmit on core/ is clean.
  • Live end-to-end through a running LiteLLM proxy - drove the real LiteLLM.chat(...) (-> streamChat -> OpenAI adapter -> proxy):
    provider=litellm  model=gpt-4.1-mini  RESPONSE: LITELLM_OK   (proxy -> Azure OpenAI)
    
  • Model discovery: LiteLLM extends OpenAI, and OpenAI.listModels() fetches GET /v1/models, so model: AUTODETECT auto-populates the models the proxy serves.

@prodmanpd
prodmanpd requested a review from a team as a code owner August 17, 2026 16:03
@prodmanpd
prodmanpd requested review from sestinj and removed request for a team August 17, 2026 16:03
@github-actions

github-actions Bot commented Aug 17, 2026

Copy link
Copy Markdown
Contributor

All contributors have signed the CLA ✍️ ✅
Posted by the CLA Assistant Lite bot.

@prodmanpd

Copy link
Copy Markdown
Author

I have read the CLA Document and I hereby sign the CLA

@prodmanpd prodmanpd changed the title Feat/add litellm provider feat: add litellm as AI provider gateway Aug 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant