revert(llm): back out OpenRouter managed-text routing until a key exists - #11360
Conversation
…okens (#11284)" This reverts merge commit 52ed1b8 (PR #11284) via -m 1. The catalog reads OPENROUTER_API_KEY to clamp completion budgets against OpenRouter's model limits. With managed text going back to direct OpenAI it has no caller, and no OpenRouter key exists to serve it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…11283)" This reverts merge commit d7858c8 (PR #11283) via -m 1. No OPENROUTER_API_KEY is provisioned, so every managed product text feature this repointed — chat, memories, knowledge graph, conversation processing, goals, notifications and the rest — would fail at the provider, and the gateway readiness probe now requires that key before reporting ready. Managed text returns to its previous direct-provider routing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_8cf3e907-e78d-46eb-9bfd-b33bb720bea8) |
undivisible
added
backend
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3b0aa79909
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| detail='llm gateway managed chat provider is not configured', | ||
| ) | ||
| if openai_chat and not os.getenv('OPENAI_API_KEY', '').strip(): | ||
| if _managed_openai_chat_enabled(config) and not os.getenv('OPENAI_API_KEY', '').strip(): |
There was a problem hiding this comment.
With OPENROUTER_API_KEY absent—the deployment scenario this revert targets—this OpenAI-only check lets /ready return 200 even though model_config.py still generates omi:auto:wrapped-analysis as openrouter/google/gemini-3-flash-preview. When global gateway feature mode sends wrapped generation through that lane, provider authentication fails after the serving gate has admitted the workload; retain the OpenRouter credential check until this remaining lane is repointed, or repoint it in this revert.
AGENTS.md reference: backend/AGENTS.md:L30-L30
Useful? React with 👍 / 👎.
| def test_premium_profile_has_expected_variant_count(self): | ||
| distinct = {model for model, _provider in MODEL_QOS_PROFILES['premium'].values()} | ||
| expected = { | ||
| 'gpt-5.6-luna', | ||
| 'sonar-pro', | ||
| } | ||
| assert distinct == expected, f"Unexpected premium variants: {distinct}" | ||
| assert len(distinct) == 7, f"Expected 7 variants in premium, got {len(distinct)}: {distinct}" |
There was a problem hiding this comment.
All three MODEL_QOS_PROFILES are constructed from the same restored map, which contains six distinct model IDs, so the new assertions for 7, 9, and 9 variants deterministically fail whenever this documented live integration suite is run. Assert the actual model set or the shared count instead; because integration tests are intentionally excluded from selector CI, this otherwise remains broken until a manual run.
AGENTS.md reference: backend/AGENTS.md:L227-L227
Useful? React with 👍 / 👎.
| def test_byok_mostly_openai(self): | ||
| """byok profile should use OpenAI for most features (chat_agent/web_search are exceptions).""" | ||
| exceptions = {'chat_agent': 'anthropic', 'web_search': 'perplexity', 'wrapped_analysis': 'openrouter'} |
There was a problem hiding this comment.
The restored BYOK profile still assigns session_titles, followup, onboarding, app_integration, trends, and translation to gemini, but this exception map lists only Anthropic, Perplexity, and OpenRouter. The following loop therefore always fails by asserting those Gemini entries are OpenAI; include the Gemini specialties or derive the expected mapping from the restored profile contract. This live suite is outside selector CI, so the regression will not be caught automatically.
AGENTS.md reference: backend/AGENTS.md:L227-L227
Useful? React with 👍 / 👎.
| 'conv_app_select', | ||
| 'external_structure', | ||
| 'trends', | ||
| 'translation', | ||
| 'what_matters_now', | ||
| } |
There was a problem hiding this comment.
The production _STRUCTURED_OUTPUT_FEATURES set still includes both translation and what_matters_now, but this rewritten expectation removes them, so test_structured_output_features_set deterministically fails even without making a provider call. Restore those two entries or assert against the intended complete contract; this integration directory is deliberately excluded from selector CI.
AGENTS.md reference: backend/AGENTS.md:L227-L227
Useful? React with 👍 / 👎.
| 'memories': ('gpt-5.6-luna', 'openai'), | ||
| 'learnings': ('gpt-5.6-luna', 'openai'), | ||
| 'memory_conflict': ('gpt-5.6-luna', 'openai'), | ||
| 'knowledge_graph': ('gpt-5.6-luna', 'openai'), |
There was a problem hiding this comment.
Restoring memories, knowledge_graph, and conv_structure to direct OpenAI leaves the retained SSOT endpoint rows in model_endpoint_inventory.yaml claiming that knowledge-graph extraction, memory extraction, connector synthesis, conversation topics, and AI-profile synthesis still use openrouter/openai/gpt-5.6-luna. Update those rows alongside this routing change so the inventory does not direct operators and future migrations to a provider these endpoints no longer use.
AGENTS.md reference: AGENTS.md:L33-L33
Useful? React with 👍 / 👎.
The extract/synthesis/topic endpoints stay, but reverting the OpenRouter routing puts knowledge_graph, memories and conv_structure back on direct OpenAI. The inventory rows still claimed openrouter/openai/gpt-5.6-luna, which would send operators and any future migration at a provider these endpoints no longer use. Verified: backend/.venv/bin/python -m pytest tests/unit/test_inventory_whitelist_honesty.py tests/unit/test_llm_gateway_coverage_guardrails.py tests/unit/test_route_policy_inventory.py — 36 passed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 participant
Summary
Reverts the two OpenRouter PRs: #11283 (managed product text → OpenRouter Luna) and #11284 (dynamic OpenRouter model catalog).
No
OPENROUTER_API_KEYis provisioned. As merged, every managed product-text feature — chat, memories, knowledge graph, conversation processing, goals, notifications, wrapped — resolves to an OpenRouter route and would fail at the provider, andllm_gateway/routers/health.pyrequires that key before reporting ready, so the gateway would never pass readiness.Routing config returns exactly to its pre-#11283 state (
git diffagainst the commit before that merge is empty formodel_config.py,llm_gateway/config/,health.pyandclients.py).Kept
The SSOT work from #11286 and #11325 stays:
/v1/knowledge-graph/extract,/v1/memories/extract,/v1/connectors/synthesize,/v1/conversations/topic,/v1/users/ai-profile/synthesizeand deterministic KG ids. Those route throughget_llm(feature), so they follow whatever providermodel_confignames — now direct OpenAI/Anthropic again — and keep working.Product invariants affected
Failure class (fixes)
Failure-Class: none
Test plan
backend/test.shover the 41 gateway/qos/openrouter/SSOT-endpoint test files — all pass file-by-file (one pre-existing fast-unit CPU-time guard trip ontest_llm_gateway_deploy_contract.py, 13/13 assertions pass, unrelated to this diff)model_config.py,llm_gateway/config/,health.py,clients.pyOPENROUTER_API_KEYNote
High Risk
Touches core LLM routing, gateway provider execution, readiness, and BYOK behavior for all managed text features; wrong config would break chat and background LLM workloads at scale.
Overview
Reverts managed product text routing from OpenRouter back to direct providers so the stack can run without
OPENROUTER_API_KEYand gateway/readyno longer blocks on that credential.model_configand gateway generated route overrides again send most features to direct OpenAI (gpt-5.6-luna/gpt-5-nano), Gemini for former flash-lite workloads, Anthropic forchat_agent, and OpenRouter only forwrapped_analysis. Inventory, route artifacts, and cost cards are aligned with that map; OpenRouter Luna/nano rate cards are removed.The gateway executor drops OpenRouter-specific request shaping: no BYOK vendor remapping on OpenRouter routes, no OpenRouter completion clamp, and GPT-5.6 sanitization applies only to
openaiprovider refs. Health reportsmanaged_chat_provider: openaiand requiresOPENAI_API_KEYinstead of OpenRouter.Deleted the dynamic OpenRouter model catalog and shared vendor-prefix helpers; synthetics/replay harnesses register
openaifakes directly again. QoS and gateway unit/integration tests are updated to match the pre–OpenRouter managed-text expectations.Reviewed by Cursor Bugbot for commit 3b0aa79. Configure here.