Skip to content

AIServicePlugin should auto-detect and initialize LLM provider based on environment variables #1070

@hotlong

Description

@hotlong

Problem

Currently, only the CLI server (packages/cli/src/commands/serve.ts) contains logic to auto-detect LLM providers (OpenAI, Anthropic, Google, Vercel AI Gateway) via environment variables and construct a VercelLLMAdapter accordingly. In non-CLI deployments (e.g., Vercel via apps/studio/server/index.ts), AIServicePlugin is always constructed with no adapter supplied, so it falls back to the test-only MemoryLLMAdapter.

This leads to inconsistent behavior: production deployments risk silently using only the echo AI, unless every entrypoint duplicates the CLI detection logic.

Proposal

Refactor and move the entire LLM provider auto-detection and adapter selection logic into the core AIServicePlugin. When no adapter is supplied in options, AIServicePlugin should:

  1. Detect LLM provider preference and settings from standard environment variables, in priority order:
    • AI_GATEWAY_MODEL → use @ai-sdk/gateway via VercelLLMAdapter
    • OPENAI_API_KEY → use @ai-sdk/openai via VercelLLMAdapter
    • ANTHROPIC_API_KEY → use @ai-sdk/anthropic via VercelLLMAdapter
    • GOOGLE_GENERATIVE_AI_API_KEY → use @ai-sdk/google via VercelLLMAdapter
  2. Fallback to the MemoryLLMAdapter only if no config is available
  3. Log which provider is selected or a warning if falling back to memory adapter

Accept dynamic import failures as a soft error: optional peer dependencies should not break boot, just emit a warning and try the next provider.

Benefits

  • No need to repeat auto-detect logic for every serverless or containerized entrypoint
  • All use cases (CLI, Vercel, Docker, custom node) consistently get best-available real AI
  • One place to extend with new providers in the future
  • Reduced operational risk from silent fallback

Tasks

  • Refactor LLM adapter auto-detection into AIServicePlugin (private method during init)
  • Remove redundant LLM detection logic from CLI serve.ts
  • Add logging to report which adapter/provider is active
  • Add unit test (mock env) for provider detection logic
  • Update roadmap and changelog

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions