Problem
Currently, only the CLI server (packages/cli/src/commands/serve.ts) contains logic to auto-detect LLM providers (OpenAI, Anthropic, Google, Vercel AI Gateway) via environment variables and construct a VercelLLMAdapter accordingly. In non-CLI deployments (e.g., Vercel via apps/studio/server/index.ts), AIServicePlugin is always constructed with no adapter supplied, so it falls back to the test-only MemoryLLMAdapter.
This leads to inconsistent behavior: production deployments risk silently using only the echo AI, unless every entrypoint duplicates the CLI detection logic.
Proposal
Refactor and move the entire LLM provider auto-detection and adapter selection logic into the core AIServicePlugin. When no adapter is supplied in options, AIServicePlugin should:
- Detect LLM provider preference and settings from standard environment variables, in priority order:
- AI_GATEWAY_MODEL → use @ai-sdk/gateway via VercelLLMAdapter
- OPENAI_API_KEY → use @ai-sdk/openai via VercelLLMAdapter
- ANTHROPIC_API_KEY → use @ai-sdk/anthropic via VercelLLMAdapter
- GOOGLE_GENERATIVE_AI_API_KEY → use @ai-sdk/google via VercelLLMAdapter
- Fallback to the MemoryLLMAdapter only if no config is available
- Log which provider is selected or a warning if falling back to memory adapter
Accept dynamic import failures as a soft error: optional peer dependencies should not break boot, just emit a warning and try the next provider.
Benefits
- No need to repeat auto-detect logic for every serverless or containerized entrypoint
- All use cases (CLI, Vercel, Docker, custom node) consistently get best-available real AI
- One place to extend with new providers in the future
- Reduced operational risk from silent fallback
Tasks
Problem
Currently, only the CLI server (packages/cli/src/commands/serve.ts) contains logic to auto-detect LLM providers (OpenAI, Anthropic, Google, Vercel AI Gateway) via environment variables and construct a VercelLLMAdapter accordingly. In non-CLI deployments (e.g., Vercel via apps/studio/server/index.ts), AIServicePlugin is always constructed with no
adaptersupplied, so it falls back to the test-only MemoryLLMAdapter.This leads to inconsistent behavior: production deployments risk silently using only the echo AI, unless every entrypoint duplicates the CLI detection logic.
Proposal
Refactor and move the entire LLM provider auto-detection and adapter selection logic into the core AIServicePlugin. When no
adapteris supplied in options, AIServicePlugin should:Accept dynamic import failures as a soft error: optional peer dependencies should not break boot, just emit a warning and try the next provider.
Benefits
Tasks