Skip to content

feat(llm): add LLMSpy with smart routing (rc.4)#156

Closed
bussyjd wants to merge 1 commit intomainfrom
chore/bump-llmspy-rc4
Closed

feat(llm): add LLMSpy with smart routing (rc.4)#156
bussyjd wants to merge 1 commit intomainfrom
chore/bump-llmspy-rc4

Conversation

@bussyjd
Copy link
Collaborator

@bussyjd bussyjd commented Feb 13, 2026

Summary

  • Adds LLMSpy deployment manifest to the base infrastructure templates
  • Uses Obol fork image ghcr.io/obolnetwork/llms:3.0.32-obol.1-rc.4 which includes:
    • Smart routing extension — automatic model selection based on 14-dimension prompt complexity scoring (ported from ClawRouter)
    • Streaming SSE passthrough fix for chat handler
    • DB writer startup race fix for lock contention

What's deployed

Resource Namespace Details
Namespace llm Dedicated namespace for LLM services
ExternalName Service llm Routes ollama.llm.svc.cluster.local to host Ollama
ConfigMap llm llms.json + providers.json (Ollama default, Anthropic/OpenAI optional)
Secret llm API keys (empty by default, patched via obol llm configure)
Deployment llm llmspy on port 8000, init container seeds config
Service llm ClusterIP llmspy:8000

Test plan

  • Deploy to local k3d/k3s cluster
  • Verify llmspy pod is running and healthy
  • Test model: "auto" routing via smart routing extension
  • Verify Ollama passthrough works

Deploy LLMSpy (ghcr.io/obolnetwork/llms:3.0.32-obol.1-rc.4) as an
OpenAI-compatible gateway in the llm namespace. Includes smart routing
extension for automatic model selection, streaming SSE passthrough,
and db writer race fix.
@bussyjd bussyjd closed this Feb 13, 2026
@bussyjd bussyjd deleted the chore/bump-llmspy-rc4 branch February 13, 2026 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant