Skip to content

feat(router): discover route-target model lists live - #1305

Merged
edwin-zvs merged 1 commit into
mainfrom
dynamic-model-discovery
Aug 22, 2026
Merged

feat(router): discover route-target model lists live#1305
edwin-zvs merged 1 commit into
mainfrom
dynamic-model-discovery

Conversation

@edwin-zvs

Copy link
Copy Markdown
Contributor

What

Route pickers no longer depend solely on the hand-curated model catalog: every route target whose provider exposes a listing endpoint now gets its model rows fetched live, additively.

  • Listing dialects (router/discovery.rs): OpenAI-compatible GET /models (OpenAI, Grok, DeepSeek, OpenRouter, Responses backends), Anthropic GET /models (x-api-key + version header), Gemini ListModels (filtered to generateContent-capable, models/ prefix stripped). Azure (deployment-scoped listing) and Meta (no public surface) deliberately keep curated lists; OAuth/subscription targets keep their seed/configured lists.
  • Additive merge: profile_model_list order is declared model → curated catalog → discovered ids (deduped, endpoint order — OpenRouter returns newest-first, so stealth ids surface near the top of the tail). Applies to both the route menu and the native-harness published catalogs, which share the list builder.
  • TTL cache, bounded fetches: refreshed when the route menu is requested (fresh entries = no-op) and warmed in the background at router start; 5 min TTL for successes, 60 s for failures, 3 s per request under a 4 s whole-refresh budget — the menu can be seconds late once, never hang, and failures degrade to exactly the curated behavior.
  • Chat filter: OpenAI-compatible listings mix in embeddings/whisper/tts/image/moderation ids; a small conservative denylist hides those rows (typed ids always still work).
  • Config: [router] discover_models = true (default) — set false to suppress the listing requests entirely; hot-swappable via the config supervisor like the rest of [router].
  • Spec 0209 records the decision: additive over curated, best-effort/never load-bearing, bounded+cached, switchable off; curated stays authoritative for ordering and unsupported providers.

Testing

  • cargo test --workspace: all 57 suites green.
  • New: parse tests for all three listing shapes (incl. real-world OpenRouter slash-id ordering), chat-filter, cache success/failure semantics, an end-to-end refresh test against a local stub server proving TTL prevents refetch, and a router-level merge-order test.
  • Verified the parser against the live OpenRouter catalog (421 models, data[].id, recency-ordered, stealth/ox-alpha present).

No recording: the visible change is extra rows in the existing scrollable model column, driven by whichever keys are present in the running daemon's env.

🤖 Generated with Claude Code

Route targets whose provider exposes a model-listing endpoint (OpenAI-
compatible /models incl. Grok/DeepSeek/OpenRouter, Anthropic /models,
Gemini ListModels) get their picker rows fetched live through a TTL
cache, refreshed on route-menu opens and warmed at router start.
Discovered ids append after the declared model and curated catalog,
deduped; failures cache short and degrade to curated-only. New
[router] discover_models toggle (default true), hot-swappable.
@edwin-zvs
edwin-zvs merged commit 7052725 into main Aug 22, 2026
1 check passed
@edwin-zvs
edwin-zvs deleted the dynamic-model-discovery branch August 22, 2026 18:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant