Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 20 additions & 7 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -616,14 +616,14 @@ The stack uses a two-tier architecture for LLM routing. A cluster-wide proxy (ll
When a cloud provider is selected during setup, two things happen simultaneously:

1. **Global tier**: `llm.ConfigureLLMSpy()` patches the cluster-wide llmspy gateway with the API key and enables the provider
2. **Instance tier**: `buildLLMSpyRoutedOverlay()` creates an overlay where a single "ollama" provider points at llmspy, the cloud model is listed under that provider, and `api` is set to `openai-completions`
2. **Instance tier**: `buildLLMSpyRoutedOverlay()` creates an overlay where a "llmspy" provider points at the llmspy gateway, the cloud model is listed under that provider with a `llmspy/` prefix, and `api` is set to `openai-completions`. The default "ollama" provider is disabled.

**Result**: The application never talks directly to cloud APIs. All traffic is routed through llmspy.

**Data flow**:
```
Application (openclaw.json)
│ model: "ollama/claude-sonnet-4-5-20250929"
│ model: "llmspy/claude-sonnet-4-5-20250929"
│ api: "openai-completions"
│ baseUrl: http://llmspy.llm.svc.cluster.local:8000/v1
Expand All @@ -636,24 +636,28 @@ llmspy (llm namespace, port 8000)
Anthropic API (or Ollama, OpenAI — depending on provider)
```

**Overlay example** (`values-obol.yaml`):
**Overlay example** (`values-obol.yaml` for cloud provider path):
```yaml
models:
ollama:
llmspy:
enabled: true
baseUrl: http://llmspy.llm.svc.cluster.local:8000/v1
api: openai-completions
apiKeyEnvVar: OLLAMA_API_KEY
apiKeyValue: ollama-local
apiKeyEnvVar: LLMSPY_API_KEY
apiKeyValue: llmspy-default
models:
- id: claude-sonnet-4-5-20250929
name: Claude Sonnet 4.5
ollama:
enabled: false
anthropic:
enabled: false
openai:
enabled: false
```

**Note**: The default Ollama path (no cloud provider) still uses the "ollama" provider name pointing at llmspy, since it genuinely routes Ollama model traffic.

### Summary Table

| Aspect | Tier 1 (llmspy) | Tier 2 (Application instance) |
Expand All @@ -663,7 +667,7 @@ models:
| **Config storage** | ConfigMap `llmspy-config` | ConfigMap `<release>-config` |
| **Secrets** | Secret `llms-secrets` | Secret `<release>-secrets` |
| **Configure via** | `obol llm configure` | `obol openclaw setup <id>` |
| **Providers** | Real (Ollama, Anthropic, OpenAI) | Virtual: everything appears as "ollama" pointing at llmspy |
| **Providers** | Real (Ollama, Anthropic, OpenAI) | Cloud: "llmspy" virtual provider; Default: "ollama" pointing at llmspy |
| **API field** | N/A (provider-native) | Must be `openai-completions` for llmspy routing |

### Key Source Files
Expand Down Expand Up @@ -972,3 +976,12 @@ This file should be updated when:
- New workflows or development practices are established

Always confirm with the user before making updates to maintain accuracy and relevance.

## Related Codebases (External Resources)

| Resource | Path | Description |
|----------|------|-------------|
| obol-stack-front-end | `/Users/bussyjd/Development/Obol_Workbench/obol-stack-front-end` | Next.js web dashboard |
| obol-stack-docs | `/Users/bussyjd/Development/Obol_Workbench/obol-stack-docs` | MkDocs documentation site |
| OpenClaw | `/Users/bussyjd/Development/Obol_Workbench/openclaw` | OpenClaw AI assistant (upstream) |
| llmspy | `/Users/bussyjd/Development/R&D/llmspy` | LLM proxy/router (upstream) |
16 changes: 9 additions & 7 deletions internal/openclaw/openclaw.go
Original file line number Diff line number Diff line change
Expand Up @@ -1147,23 +1147,25 @@ func promptForCloudProvider(reader *bufio.Reader, name, display, modelID, modelN
}

// buildLLMSpyRoutedOverlay creates an ImportResult that routes a cloud model
// through the llmspy proxy. OpenClaw sees a single "ollama" provider pointing
// at llmspy, with the cloud model in its model list. The actual cloud providers
// are disabled in OpenClaw — llmspy handles the routing.
// through the llmspy proxy. OpenClaw sees a "llmspy" provider pointing at the
// cluster-wide llmspy gateway, with the cloud model in its model list. The
// actual cloud providers (and default ollama) are disabled in OpenClaw — llmspy
// handles upstream routing based on the bare model ID.
func buildLLMSpyRoutedOverlay(cloud *CloudProviderInfo) *ImportResult {
return &ImportResult{
AgentModel: cloud.ModelID,
AgentModel: "llmspy/" + cloud.ModelID,
Providers: []ImportedProvider{
{
Name: "ollama",
Name: "llmspy",
BaseURL: "http://llmspy.llm.svc.cluster.local:8000/v1",
API: "openai-completions",
APIKeyEnvVar: "OLLAMA_API_KEY",
APIKey: "ollama-local",
APIKeyEnvVar: "LLMSPY_API_KEY",
APIKey: "llmspy-default",
Models: []ImportedModel{
{ID: cloud.ModelID, Name: cloud.Display},
},
},
{Name: "ollama", Disabled: true},
{Name: "anthropic", Disabled: true},
{Name: "openai", Disabled: true},
},
Expand Down
90 changes: 52 additions & 38 deletions internal/openclaw/overlay_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -15,42 +15,50 @@ func TestBuildLLMSpyRoutedOverlay_Anthropic(t *testing.T) {

result := buildLLMSpyRoutedOverlay(cloud)

// Check agent model uses bare model ID (no provider/ prefix)
if result.AgentModel != "claude-sonnet-4-5-20250929" {
t.Errorf("AgentModel = %q, want %q", result.AgentModel, "claude-sonnet-4-5-20250929")
// Check agent model uses llmspy/ prefix for correct OpenClaw provider routing
if result.AgentModel != "llmspy/claude-sonnet-4-5-20250929" {
t.Errorf("AgentModel = %q, want %q", result.AgentModel, "llmspy/claude-sonnet-4-5-20250929")
}

// Check 3 providers: ollama (enabled), anthropic (disabled), openai (disabled)
if len(result.Providers) != 3 {
t.Fatalf("len(Providers) = %d, want 3", len(result.Providers))
// Check 4 providers: llmspy (enabled), ollama (disabled), anthropic (disabled), openai (disabled)
if len(result.Providers) != 4 {
t.Fatalf("len(Providers) = %d, want 4", len(result.Providers))
}

ollama := result.Providers[0]
if ollama.Name != "ollama" || ollama.Disabled {
t.Errorf("ollama: name=%q disabled=%v, want ollama/false", ollama.Name, ollama.Disabled)
llmspy := result.Providers[0]
if llmspy.Name != "llmspy" || llmspy.Disabled {
t.Errorf("llmspy: name=%q disabled=%v, want llmspy/false", llmspy.Name, llmspy.Disabled)
}
if ollama.BaseURL != "http://llmspy.llm.svc.cluster.local:8000/v1" {
t.Errorf("ollama.BaseURL = %q", ollama.BaseURL)
if llmspy.BaseURL != "http://llmspy.llm.svc.cluster.local:8000/v1" {
t.Errorf("llmspy.BaseURL = %q", llmspy.BaseURL)
}
if ollama.APIKeyEnvVar != "OLLAMA_API_KEY" {
t.Errorf("ollama.APIKeyEnvVar = %q, want OLLAMA_API_KEY", ollama.APIKeyEnvVar)
if llmspy.APIKeyEnvVar != "LLMSPY_API_KEY" {
t.Errorf("llmspy.APIKeyEnvVar = %q, want LLMSPY_API_KEY", llmspy.APIKeyEnvVar)
}
if ollama.APIKey != "ollama-local" {
t.Errorf("ollama.APIKey = %q, want ollama-local", ollama.APIKey)
if llmspy.APIKey != "llmspy-default" {
t.Errorf("llmspy.APIKey = %q, want llmspy-default", llmspy.APIKey)
}
if ollama.API != "openai-completions" {
t.Errorf("ollama.API = %q, want openai-completions", ollama.API)
if llmspy.API != "openai-completions" {
t.Errorf("llmspy.API = %q, want openai-completions", llmspy.API)
}
if len(ollama.Models) != 1 || ollama.Models[0].ID != "claude-sonnet-4-5-20250929" {
t.Errorf("ollama.Models = %v", ollama.Models)
if len(llmspy.Models) != 1 || llmspy.Models[0].ID != "claude-sonnet-4-5-20250929" {
t.Errorf("llmspy.Models = %v", llmspy.Models)
}

// anthropic and openai should be disabled
if !result.Providers[1].Disabled || result.Providers[1].Name != "anthropic" {
t.Errorf("anthropic: disabled=%v name=%q", result.Providers[1].Disabled, result.Providers[1].Name)
// ollama, anthropic and openai should be disabled
for _, idx := range []int{1, 2, 3} {
if !result.Providers[idx].Disabled {
t.Errorf("Providers[%d] (%s) should be disabled", idx, result.Providers[idx].Name)
}
}
if result.Providers[1].Name != "ollama" {
t.Errorf("Providers[1].Name = %q, want ollama", result.Providers[1].Name)
}
if result.Providers[2].Name != "anthropic" {
t.Errorf("Providers[2].Name = %q, want anthropic", result.Providers[2].Name)
}
if !result.Providers[2].Disabled || result.Providers[2].Name != "openai" {
t.Errorf("openai: disabled=%v name=%q", result.Providers[2].Disabled, result.Providers[2].Name)
if result.Providers[3].Name != "openai" {
t.Errorf("Providers[3].Name = %q, want openai", result.Providers[3].Name)
}
}

Expand All @@ -64,13 +72,13 @@ func TestBuildLLMSpyRoutedOverlay_OpenAI(t *testing.T) {

result := buildLLMSpyRoutedOverlay(cloud)

if result.AgentModel != "gpt-5.2" {
t.Errorf("AgentModel = %q, want %q", result.AgentModel, "gpt-5.2")
if result.AgentModel != "llmspy/gpt-5.2" {
t.Errorf("AgentModel = %q, want %q", result.AgentModel, "llmspy/gpt-5.2")
}

ollama := result.Providers[0]
if len(ollama.Models) != 1 || ollama.Models[0].ID != "gpt-5.2" {
t.Errorf("ollama model = %v, want gpt-5.2", ollama.Models)
llmspy := result.Providers[0]
if len(llmspy.Models) != 1 || llmspy.Models[0].ID != "gpt-5.2" {
t.Errorf("llmspy model = %v, want gpt-5.2", llmspy.Models)
}
}

Expand All @@ -84,23 +92,26 @@ func TestOverlayYAML_LLMSpyRouted(t *testing.T) {
result := buildLLMSpyRoutedOverlay(cloud)
yaml := TranslateToOverlayYAML(result)

// Agent model should be the bare model ID
if !strings.Contains(yaml, "agentModel: claude-sonnet-4-5-20250929") {
// Agent model should have llmspy/ prefix
if !strings.Contains(yaml, "agentModel: llmspy/claude-sonnet-4-5-20250929") {
t.Errorf("YAML missing agentModel, got:\n%s", yaml)
}

// ollama should be enabled with llmspy baseUrl
// llmspy should be enabled with llmspy baseUrl
if !strings.Contains(yaml, "llmspy:\n enabled: true") {
t.Errorf("YAML missing enabled llmspy provider, got:\n%s", yaml)
}
if !strings.Contains(yaml, "baseUrl: http://llmspy.llm.svc.cluster.local:8000/v1") {
t.Errorf("YAML missing llmspy baseUrl, got:\n%s", yaml)
}

// apiKeyEnvVar should be set
if !strings.Contains(yaml, "apiKeyEnvVar: OLLAMA_API_KEY") {
// apiKeyEnvVar should be LLMSPY_API_KEY
if !strings.Contains(yaml, "apiKeyEnvVar: LLMSPY_API_KEY") {
t.Errorf("YAML missing apiKeyEnvVar, got:\n%s", yaml)
}

// apiKeyValue should be ollama-local
if !strings.Contains(yaml, "apiKeyValue: ollama-local") {
// apiKeyValue should be llmspy-default
if !strings.Contains(yaml, "apiKeyValue: llmspy-default") {
t.Errorf("YAML missing apiKeyValue, got:\n%s", yaml)
}

Expand All @@ -109,12 +120,15 @@ func TestOverlayYAML_LLMSpyRouted(t *testing.T) {
t.Errorf("YAML missing api: openai-completions, got:\n%s", yaml)
}

// Cloud model should appear in ollama's model list
// Cloud model should appear in llmspy's model list
if !strings.Contains(yaml, "- id: claude-sonnet-4-5-20250929") {
t.Errorf("YAML missing cloud model ID, got:\n%s", yaml)
}

// anthropic and openai should be disabled
// ollama, anthropic and openai should be disabled
if !strings.Contains(yaml, "ollama:\n enabled: false") {
t.Errorf("YAML missing disabled ollama, got:\n%s", yaml)
}
if !strings.Contains(yaml, "anthropic:\n enabled: false") {
t.Errorf("YAML missing disabled anthropic, got:\n%s", yaml)
}
Expand Down