feat(prefill-probe): add learned prefill complexity router

[mich-xu]

What

Adds a learned prefill-probe profile that routes requests between configured weak and strong completion targets using prompt hidden states from a separate vLLM probe model.

The profile:

• Requests all-layer prompt hidden states from a dedicated Qwen3.6 vLLM endpoint.
• Mean-pools each layer across prompt tokens and concatenates the layer vectors.
• Applies an artifact-provided scaler and PCA-200 projection.
• Runs a five-member shared-trunk ensemble that produces four correctness probabilities.
• Uses only the two checkpoint heads mapped to the configured weak and strong targets.
• Selects between those targets using a cost-aware utility policy controlled by lambda.
• Caches successful decisions by resolved probe input and falls back to strong on probe or scoring failures.
• Loads the learned checkpoint from an external inference-artifact directory; no training or online learning occurs in Switchyard.

This also adds a Terminal-Bench Terminus 2 adapter that supplies the exact raw task instruction through the private _switchyard_prefill_probe_input field. The profile uses that value for scoring and caching, then removes it before calling the selected completion backend.

A sample local profile and vLLM deployment documentation are included.

Why

The trained complexity router expects prompt-only hidden-state features, while Terminal-Bench's standard Terminus 2 agent wraps the raw task instruction with its command protocol and terminal state.

Passing the raw task instruction explicitly reproduces the checkpoint's expected representation without changing the conversation received by the selected completion model.

The probe model is intentionally independent of the completion targets. It only needs to match the encoder metadata stored in the inference artifact.

Closes #

How tested

• uv run ruff check . clean
• uv run mypy switchyard clean: 162 source files checked
• uv run pytest tests/ -v -m "not integration" green: 1,978 passed, 12 skipped, 43 deselected
• Python SPDX header gate
• cargo fmt --all --check
• cargo clippy --workspace --all-targets -- -D warnings
• cargo test --workspace
• mkdocs build --strict
• Manual Qwen3.6 hidden-state probe startup and artifact inference
• Manual three-task verifier smoke: all three tasks completed successfully
• Two 89-task routing runs with explicit raw input:
  • lambda = 0.7: 46 weak, 43 strong, 0 probe failures
  • lambda = 0.8: 6 weak, 83 strong, 0 probe failures

The full live runs encountered upstream provider rate limits, but all 89 routing decisions completed in each run and no probe failures were observed.

Checklist

• One class per file; filename = snake_case of the primary class.
• New public symbols exported from switchyard/__init__.py.__all__ if intended for downstream use.
• Unit tests added for new components / bug fixes.
• README / --help updated if customer-facing surface changed.
• Commits signed off (Signed-off-by: Your Name <email>) per the DCO.

Notes for reviewers

This is a stacked PR based on and targeting #13. It can be retargeted to main after #13 merges.

The router artifact is deployment data and is not packaged or downloaded by Switchyard. At startup, the profile validates the artifact metadata and tensor shapes, including encoder identity, layer count, hidden width, PCA dimension, output heads, and ensemble architecture.

The learned profile uses a fixed public threshold of 0.5: score 1.0 selects weak and score 0.0 selects strong. Routing should be tuned through lambda, not through an additional confidence threshold.

The private Terminal-Bench probe-input field is rejected when malformed and is always removed before target execution.

[sabhatinas]

Curious what this field translates to in the algorithm.

[sabhatinas]

are hidden_states_dir and inference_artifact_dir the same thing?

[sabhatinas]

we have baseline benchmarking scripts in the repo to run closed book setup. can you update docs to use that instead ?

[sabhatinas]

LGTM overall, we just need e2e testing done using the benchmarks.