fix(server): report tool_calling and context_window in /v1/models

[elyasmnvidian]

GET /v1/models from the Rust libsy server reported tool_calling and context_window as null for every model. This fills them in: tool_calling: true, and context_window inferred from the model id using the same known-model table as the Python listing (deepseek-v4 and nemotron-3-super = 1,000,000; nemotron-3-nano = 262,000; kimi-k2 = 256,000; claude = 200,000; anything else = 128,000).

Rebased onto main after #98 ("serve libsy algorithms without profiles") landed. #98 already maps a context-window overflow to HTTP 400 context_length_exceeded, so that half of the original PR is dropped — this PR is now just the /v1/models capability fields.

Test models_advertise_capabilities_from_the_model_name asserts a claude route id advertises 200000 and an unknown id the 128000 default, both with tool_calling: true.

[coderabbitai]

Walkthrough

The server maps context exhaustion errors to HTTP 400 responses with an OpenAI-compatible error code. Model listings now expose tool-calling support and inferred context-window values, with integration tests covering both changes.

Changes

Server behavior

Layer / File(s)	Summary
Context overflow error mapping crates/switchyard-server/src/lib.rs, crates/switchyard-server/tests/server.rs	Context window and pool exhaustion errors return HTTP 400 with context_length_exceeded; an overflow profile and integration test validate the response.
Model capability metadata crates/switchyard-server/src/lib.rs, crates/switchyard-server/tests/server.rs	Model listings report tool_calling: true and infer context_window from display-name substrings with a 128_000 fallback; endpoint assertions cover the values.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Poem

I’m a rabbit with tests in my burrow,
Errors now hop client-side in a hurry.
Models show tools and windows bright,
Context clues make their numbers right.
400 says, “Your prompt’s too furry!”

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title is clear and relevant, summarizing a major part of the PR’s /v1/models capability metadata changes.

---

_{Comment @coderabbitai help to get the list of available commands.}

[coderabbitai]

🧹 Nitpick comments (1)

crates/switchyard-server/tests/server.rs (1)

217-242: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Cover ContextPoolExhausted separately.

This test exercises only ContextWindowExceeded, while the production mapping also handles ContextPoolExhausted. Add a profile/test for the second variant so regressions in that branch cannot pass unnoticed.

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@crates/switchyard-server/tests/server.rs` around lines 217 - 242, Add a
separate test alongside context_window_overflow_maps_to_client_error using a
profile that triggers ContextPoolExhausted. Send the same chat completion
request through build_switchyard_router, then assert the response is BAD_REQUEST
with error.code set to the expected client-facing code for pool exhaustion,
covering the distinct production mapping branch.

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Nitpick comments:
In `@crates/switchyard-server/tests/server.rs`:
- Around line 217-242: Add a separate test alongside
context_window_overflow_maps_to_client_error using a profile that triggers
ContextPoolExhausted. Send the same chat completion request through
build_switchyard_router, then assert the response is BAD_REQUEST with error.code
set to the expected client-facing code for pool exhaustion, covering the
distinct production mapping branch.

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 4f5c9536-37c6-407c-a66f-fcadc9be076c

📥 Commits

Reviewing files that changed from the base of the PR and between 2fa6b87 and 01a361b.

📒 Files selected for processing (2)

• crates/switchyard-server/src/lib.rs
• crates/switchyard-server/tests/server.rs

[elyasmnvidian]

Live before/after — /v1/models capabilities and 400 on context overflow

Served the same config on main and this branch. Endpoints point at the live NVIDIA gateway; the config has a claude target, a small-context llama target, and a passthrough profile:

endpoints:
  nvidia: { base_url: https://inference-api.nvidia.com/v1, api_key: ${NVIDIA_API_KEY} }
targets:
  azure/anthropic/claude-opus-4-8:   { endpoint: nvidia, model: azure/anthropic/claude-opus-4-8, format: openai }
  nvidia/meta/llama-3.2-1b-instruct: { endpoint: nvidia, model: nvidia/meta/llama-3.2-1b-instruct, format: openai }
profiles:
  fast: { type: passthrough, target: nvidia/meta/llama-3.2-1b-instruct }

GET /v1/models

Before (main) — capabilities are null:

fast                                tool_calling=None  context_window=None
azure/anthropic/claude-opus-4-8     tool_calling=None  context_window=None
nvidia/meta/llama-3.2-1b-instruct   tool_calling=None  context_window=None

After (this branch) — tool_calling is set, and the claude target alias infers its real 200k window (the profile route reports the 128k default, since it carries the profile type, not a model name):

fast                                tool_calling=True  context_window=128000
azure/anthropic/claude-opus-4-8     tool_calling=True  context_window=200000
nvidia/meta/llama-3.2-1b-instruct   tool_calling=True  context_window=128000

Context overflow — POST a ~300k-token prompt to fast (routes to the 131k-context llama):

• Before (main): HTTP 500 {"error":{"code":"server_error", ...}}
• After (this branch): HTTP 400 {"error":{"code":"context_length_exceeded", ...}}

The upstream error (ContextWindowExceededError: ...maximum context length is 131072 tokens) is now mapped to a client error instead of a server error.

[elyasmnvidian]

Reworked onto the post-#98 libsy server — live before/after

#98 ("serve libsy algorithms without profiles") merged into main and replaced the profile server this PR originally targeted. Rebased onto it: dropped the context-overflow → 400 mapping (#98 already does that) and kept the /v1/models capability fields, re-applied to the new model_entry_json.

Ran the Rust libsy server on main and this branch against the live NVIDIA gateway:

switchyard-server \
  --route "azure/anthropic/claude-opus-4-8=azure/anthropic/claude-opus-4-8" \
  --route "acme/small-model=azure/anthropic/claude-haiku-4-5" \
  --base-url https://inference-api.nvidia.com/v1 --api-key $NVIDIA_API_KEY

GET /v1/models:

# main
acme/small-model                 tool_calling=None  context_window=None
azure/anthropic/claude-opus-4-8  tool_calling=None  context_window=None

# this branch
acme/small-model                 tool_calling=True  context_window=128000
azure/anthropic/claude-opus-4-8  tool_calling=True  context_window=200000

The claude route id infers its real 200k window; an unknown id falls back to the 128k default.