feat(stage_router): corroborative tanh-scored routing with windowed self-clearing signals

[sabhatinas]

Summary

Reworks the Python stage_router scorer / picker / signals into a clean, corroborative, honestly-documented router, then adds two follow-on capabilities on top: context-compaction handling and optional tier-transition handoff notes. Decisions are corroborative across two axes, signals self-clear over a recent window, and the confidence score spreads smoothly across the range.

Tracked in Linear: SWITCH-939 (scorer refactor), SWITCH-983 (compaction), SWITCH-982 (handoff notes).

1. Signal + scorer refactor (SWITCH-939)

• Dimensions cleaned — removed dead/redundant dims (write_intensity, edit_intensity, tests_passed-as-dim, pure_bash_intensity, planning_active, stuck_exploring/no_progress). spinning and exploring are now mutually exclusive (spinning = deep pure churn; exploring = deep read/plan without producing). exploring is promoted to a full escalation latch — trace-verified that treating it as neutral routed struggling agents to the weak tier and lost the task.
• Symmetric two-axis scorer — WRONG (severity, spinning, exploring) → CAPABLE (+); PROGRESS (recent_production_intensity) → EFFICIENT (−). tanh(gain·raw), confidence = |score|. Dropped no_error_streak (a clean-result streak was cancelling the windowed error signal and releasing escalations too early).
• Honest threshold docs — removed the earlier false claim that signals-to-clear = threshold / unit. The real dial is smooth: t≈0.3 escalates on ~1 signal, ~0.5 needs ~1.5, ~0.7 needs 2 corroborating; reachable range (0, ~0.76) for the two axes.
• Picker — capable_first/efficient_first differ only in the low-confidence default tier; both driven by the same scorer. Removed the old ef_escalate path (it had bypassed the scorer's strong side). Decision-source order: no_signal → override (CRITICAL severity) → tests_passed (settled-run, severity-gated) → dimensions → llm-classifier → fall_open.
• Rust signal extractor — DEFAULT_RECENT_WINDOW 3→5 for stickiness; detect_tests_passed honours the window; trace-mined error patterns from ~1006 local trajectories (added permission-denied, TypeError/KeyError, segfault, linker/compile, npm/cargo/go build failures, "file does not exist", etc. at correct severity).

2. Context-compaction handling (SWITCH-983)

Auto-compaction resets the stateless per-request signals, de-escalating hard tasks back to the weak tier at the worst moment (observed: video-processing strong-share 87%→0%, sam-cell-seg 48%→18% after the compaction marker).

• Rust — new compacted: bool on ToolResultSignal; DimensionCollector scans the full history for the compaction marker, so it self-latches (the summary stays pinned in the prefix every subsequent turn).
• Picker — _apply_overrides forces (and holds) CAPABLE when compacted.

3. repeated_cmd_ratio — added then removed (honest)

Added a repeated_cmd_ratio WRONG signal (catch a weak model looping one command), then removed it from the scorer after an ablation over 178 task-runs found 0/71 of its escalations were real loops: its max_count / window normalisation makes a single Bash call read as ratio 0.20, firing a low-grade CAPABLE bias on any shell turn (~8% spurious Opus, no accuracy payoff). Still computed for observability; re-add only with a max_count ≥ 3 gate.

4. Optional tier-transition handoff notes (SWITCH-982)

HandoffNoteInjector appends a short ephemeral guidance note to the outgoing request on a weak↔strong transition, so the model taking over knows why. Off by default, fully configurable (HandoffNoteConfig). Ephemeral (never persisted → notes don't accumulate), transition-gated (stays out of the cache prefix on steady-state turns), truthful escalation gate, API-correct injection for Anthropic Messages (append a text block after the trailing tool_result) and OpenAI Chat. Prototype in the v1 Python path; not yet A/B-tested.

Validation

• All stage_router unit tests pass (scorer, pickers, dimensions, handoff notes), plus Rust signal tests; ruff + mypy clean.
• On the full TB2.1 sweeps the scorer routes the known-hard tasks to Opus early and 100% via signals on efficient_first. Weak-tier accuracy/cost trade-offs (and the finding that no weak tier beats native Opus on this benchmark) are tracked in SWITCH-951; this PR is the routing machinery, not a benchmark-win claim.

Out of scope (follow-up)

• Port to the components-v2 stage_router profile (crates/switchyard-components-v2/src/profiles/stage_router/) for parity — intentionally separate.
• Stickiness/hysteresis and whipsaw suppression for the handoff/de-escalation path (SWITCH-982); A/B the handoff notes.

🤖 Generated with Claude Code

[coderabbitai]

Walkthrough

The stage router now separates tests_passed decisions, uses recent-window progress gates, routes settled runs directly to EFFICIENT, preserves critical overrides, and computes tanh-squashed scores with updated tests.

Changes

Stage router behavior

Layer / File(s)	Summary
Parameterized tanh scoring switchyard/lib/processors/stage_router/scorer.py, tests/test_stage_router_scorer.py	Grouped signal weights and tanh scoring replace fixed weights and hard clipping; tests cover direction, saturation, confidence, and raw contributions.
Routing gates and settled-run precedence switchyard/lib/processors/stage_router/decision_log.py, switchyard/lib/processors/stage_router/dimensions.py, switchyard/lib/processors/stage_router/picker.py, tests/test_stage_router_pickers.py	Recent-window gates and the dedicated tests_passed source support early EFFICIENT routing, while critical severity continues to force CAPABLE; picker scenarios are updated accordingly.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Poem

I’m a bunny with tests in my ear,
Hopping through scores that grow smooth and clear.
Settled runs dash to EFFICIENT light,
Critical thumps keep CAPABLE in sight.
Tanh curls the signals, soft as spring—
New paths bloom while green checks sing.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 88.24% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly reflects the main stage_router changes: tanh-based scoring and windowed self-clearing signals.

---

_{Comment @coderabbitai help to get the list of available commands.}