Skip to content

feat: libsy components#93

Draft
messiaen wants to merge 9 commits into
mainfrom
grclark/libsy-components
Draft

feat: libsy components#93
messiaen wants to merge 9 commits into
mainfrom
grclark/libsy-components

Conversation

@messiaen

@messiaen messiaen commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

libsy: composable routing components

Adds a small SDK for LLM traffic routing to libsy, built from two roles —
Processor (lightweight, accumulates session State) and Classifier
(scores routing targets, possibly via its own LLM call) — plus a shared
orchestrator that runs them. Every router in this PR is a configuration of the
same pieces rather than a bespoke implementation.

Core

  • Processor / Classifier / State / Score (core.rs) — the role
    traits. Processors run first and write facts into a shared, per-session
    State; classifiers then run in a cascade, each returning a Score per
    target or an empty vec to abstain (defer to the next, heavier classifier).
  • FallThroughClassification (fall_through.rs) — the orchestrator
    (Algorithm). Runs the processor chain, falls through the classifier cascade
    (first non-empty result wins, by argmax), dispatches the routed call via the
    Driver, then replays the decision to the processors so stateful ones bind
    it. Owns the session State (persists across turns behind a lock).

Components

  • StagedRouter — heuristic per-turn router ported from the cascade
    router's dimension collector + scorer (tool-result signals → capable/efficient,
    reranged to a [0,1] capability confidence; abstains in an ambiguous band).
  • LlmClassifier — LLM-backed classifier that offloads one strict
    tool-call
    through the Driver and emits a score per target. Trimmed request
    summary, temperature 0 + max_tokens, optional fail-open. Three Policy
    prompt presets (general / coding-agent / openclaw).
  • AffinityRouter — session latch (retain a model per identity);
    with_latch_only restricts which models latch.
  • TurnGate — routes a fixed target before min_turn, then abstains.
  • MetadataProcessor + Metadata::from_headers — normalizes harness
    headers into correlation metadata in State.

Composed routers (all one skeleton)

Router Cascade
LLM classifier [LlmClassifier]
Staged [StagedRouter, fallback]
Escalation [latch, TurnGate, ConfirmingJudge(LlmClassifier)] + latch processor
Escalation + staged …, TurnGate, StagedRouter, judge]

escalation_router(...) / escalation_router_with_staged(...) assemble the
judge-latched escalation flow: start weak, escalate to strong when the judge
(or the cheap staged heuristic) sees trouble, then latch. ConfirmingJudge
requires N escalate verdicts before latching — trivial here because State is
per-session (the reference needs a keyed streak store).

Notes

  • Classifier::score takes an optional &Driver (driver-backed classifiers use
    it, error on None; local ones ignore it).
  • Chain-shape changes required to run processors inside a Send algorithm
    future: Processor now yields Send futures, and Event::ModelResponse
    carries the buffered &AggLlmResponse (a live stream can't cross the spawn).

Testing

121 unit/integration tests, cargo clippy clean. Ports the reference test
suites (signal extraction, scorer, tier selection) and covers the cascade,
latch persistence, confirmations, and end-to-end escalation with a stubbed
judge over the driver stream.

Checklist

  • One class per file; filename = snake_case of the primary class.
  • New public symbols exported from switchyard/__init__.py.__all__ if intended for downstream use.
  • Unit tests added for new components / bug fixes.
  • README / --help updated if customer-facing surface changed.
  • Commits signed off (Signed-off-by: Your Name <email>) per the DCO.

Notes for reviewers

Anything reviewers should pay extra attention to — risky paths, follow-up tickets, intentional trade-offs.

messiaen added 9 commits July 18, 2026 13:10
…lization

Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
@messiaen messiaen changed the title Grclark/libsy components feat: libsy components Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant