Skip to content

Fix router model selection: filter available_models, remove same-provider override, iterate candidates#310210

Open
aashna wants to merge 3 commits intomainfrom
aashnagarg/iterate-candidate-models-fallback
Open

Fix router model selection: filter available_models, remove same-provider override, iterate candidates#310210
aashna wants to merge 3 commits intomainfrom
aashnagarg/iterate-candidate-models-fallback

Conversation

@aashna
Copy link
Copy Markdown
Contributor

@aashna aashna commented Apr 15, 2026

Problem

Three issues cause the router's model recommendation to be silently overridden on the client, sending simple queries to expensive models:

1. available_models / knownEndpoints out of sync

knownEndpoints (from GET /models) and available_models (from POST /auto_models) are separate CAPI calls with independent caches (10-min vs 5-min refresh). When a model like claude-haiku-4.5 appears in available_models before the Models API returns it, the router recommends it but the client can't resolve it to an endpoint — causing a silent fallback to the first available (often the most expensive) model.

Observed in logs: Router returns no_reasoning at 96% confidence with candidate_models: [claude-haiku-4.5, gpt-4.1, ...] for "what day is today", but the actual request goes to claude-sonnet-4.6.

2. Same-provider override negates router decisions

When the router re-runs after /compact or background summarization, _findSameProviderModel overrides the router's recommendation with a same-provider model, often switching to a more expensive one.

3. Only candidate_models[0] attempted

The fallback only tried the first candidate model. If that one model had no matching endpoint, all remaining candidates were discarded.

Fix

Filter available_models before routing

Intersect available_models with knownEndpoints before sending to the router. The router now only sees models the client can actually serve, eliminating the sync gap at the source.

const knownModelIds = new Set(knownEndpoints.map(e => e.model));
const routableModels = token.available_models.filter(m => knownModelIds.has(m));

Remove same-provider override from router path

When the router runs, trust its ranked candidate list directly. Same-provider preference is retained in _selectDefaultModel (the non-router fallback path).

Iterate all candidates via _findFirstAvailableModel

Walk the full candidate_models list instead of only trying [0], as defense-in-depth.

// Before:
const selectedModel = (entry?.endpoint && this._findSameProviderModel(...))
    ?? knownEndpoints.find(e => e.model === result.candidate_models[0]);

// After:
const selectedModel = this._findFirstAvailableModel(result.candidate_models, knownEndpoints);

Forward has_image to router instead of early-returning

Image requests no longer bypass the router. Instead, has_image: true is sent so the server can factor it in. Vision fallback (_applyVisionFallback) still handles downstream model selection if the routed model doesn't support vision.

Emit automode.routerModelSelection telemetry (from #310238)

New event reports candidateModel vs actualModel with overrideReason and filteredModelCount so analysts can detect overrides and CAPI sync gaps without fragile telemetry joins.

Add RouterDecisionError with server error code parsing

Non-OK router responses now parse the JSON error body and surface the error code (e.g. no_vision_models) as the fallbackReason instead of a generic routerError.

Warn-level logs for debuggability

All sync mismatches and fallbacks now emit warn-level logs with full context instead of being silently swallowed.

Behavioral changes

  • Image requests now go through the router (with has_image: true) instead of being skipped entirely. Vision fallback still applies after routing.
  • conversationId passed to router uses sessionResource ?? sessionId ?? 'unknown' (matching cache key) instead of raw chatRequest?.sessionId.

Supersedes #310238.

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Updates Copilot’s automode router model selection so that when the router returns a ranked candidate_models list, the service selects the first candidate that actually exists in the user’s configured endpoint list (instead of only trying candidate_models[0]).

Changes:

  • Iterate through candidate_models to find the first model with a matching known endpoint.
  • Keep same-provider preference, but apply it against the candidate list before falling back to the first available candidate endpoint.
Show a summary per file
File Description
extensions/copilot/src/platform/endpoint/node/automodeService.ts Adjusts router-result handling to choose the first viable endpoint from the router’s ordered candidates.

Copilot's findings

  • Files reviewed: 1/1 changed files
  • Comments generated: 2

Comment thread extensions/copilot/src/platform/endpoint/node/automodeService.ts Outdated
Comment thread extensions/copilot/src/platform/endpoint/node/automodeService.ts Outdated
@github-actions
Copy link
Copy Markdown
Contributor

github-actions bot commented Apr 15, 2026

Screenshot Changes

Base: 4ab3ca55 Current: e5dff969

Changed (3)

chat/aiCustomizations/aiCustomizationManagementEditor/McpBrowseMode/Light
Before After
before after
editor/inlineChatAffordance/InlineChatOverlay/Light
Before After
before after
editor/inlineCompletions/other/JumpToHint/Dark
Before After
before after

blocks-ci screenshots changed

Replace the contents of test/componentFixtures/blocks-ci-screenshots.md with:

Updated blocks-ci-screenshots.md
<!-- auto-generated by CI — do not edit manually -->

#### editor/codeEditor/CodeEditor/Dark
![screenshot](https://hediet-screenshots.azurewebsites.net/images/cb32a3e854b5734fe5aaca2318f2e0a42ee821b05ea97883ea42c5ba95edb3c3)

#### editor/codeEditor/CodeEditor/Light
![screenshot](https://hediet-screenshots.azurewebsites.net/images/42624fbba5e0db7f32c224b5eb9c5dd3b08245697ae2e7d2a88be0d7c287129b)

@aashna aashna force-pushed the aashnagarg/iterate-candidate-models-fallback branch 2 times, most recently from f5ce3f6 to 3183de3 Compare April 15, 2026 19:40
@aashna aashna force-pushed the aashnagarg/iterate-candidate-models-fallback branch 2 times, most recently from c25e8dd to 8535d04 Compare April 15, 2026 20:38
@aashna aashna changed the title Iterate candidate_models list instead of only trying [0] Remove same-provider override from router path, iterate candidate list Apr 15, 2026
@aashna aashna changed the title Remove same-provider override from router path, iterate candidate list Fix router model selection: filter available_models, remove same-provider override, iterate candidates Apr 16, 2026
@aashna aashna enabled auto-merge April 16, 2026 04:12
@aashna aashna force-pushed the aashnagarg/iterate-candidate-models-fallback branch 3 times, most recently from 2108023 to 1c38157 Compare April 16, 2026 04:21
Two changes to how the router's model recommendation is applied:

1. Remove _findSameProviderModel from _tryRouterSelection: When the
   router re-runs after /compact or background summarization, the
   same-provider preference was overriding the router's cost-saving
   decision. E.g., router picks claude-haiku-4.5 (cheap) but same-
   provider forces claude-sonnet-4.6 (expensive) because the prior
   turn used Anthropic. Same-provider is retained in _selectDefaultModel
   (the non-router fallback path) where it makes sense.

2. Use _findFirstAvailableModel to iterate the full candidate_models
   list instead of only trying [0], as a defensive fallback.

Telemetry from candidateModel field (#308597) shows 63% switch rate
where the router's pick is overridden — most switches go toward more
expensive models, erasing cost savings.
@aashna aashna force-pushed the aashnagarg/iterate-candidate-models-fallback branch from 1c38157 to 051b047 Compare April 16, 2026 04:25
Aashna Garg added 2 commits April 15, 2026 21:28
…ndidate-models-fallback

# Conflicts:
#	extensions/copilot/src/platform/endpoint/node/automodeService.ts
#	extensions/copilot/src/platform/endpoint/node/test/automodeService.spec.ts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants