feat: libsy components#93
Draft
messiaen wants to merge 9 commits into
Draft
Conversation
…lization Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
Signed-off-by: Greg Clark <grclark@nvidia.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
libsy: composable routing components
Adds a small SDK for LLM traffic routing to
libsy, built from two roles —Processor(lightweight, accumulates sessionState) andClassifier(scores routing targets, possibly via its own LLM call) — plus a shared
orchestrator that runs them. Every router in this PR is a configuration of the
same pieces rather than a bespoke implementation.
Core
Processor/Classifier/State/Score(core.rs) — the roletraits. Processors run first and write facts into a shared, per-session
State; classifiers then run in a cascade, each returning aScorepertarget or an empty vec to abstain (defer to the next, heavier classifier).
FallThroughClassification(fall_through.rs) — the orchestrator(
Algorithm). Runs the processor chain, falls through the classifier cascade(first non-empty result wins, by argmax), dispatches the routed call via the
Driver, then replays the decision to the processors so stateful ones bindit. Owns the session
State(persists across turns behind a lock).Components
StagedRouter— heuristic per-turn router ported from the cascaderouter's dimension collector + scorer (tool-result signals → capable/efficient,
reranged to a
[0,1]capability confidence; abstains in an ambiguous band).LlmClassifier— LLM-backed classifier that offloads one stricttool-call through the
Driverand emits a score per target. Trimmed requestsummary,
temperature 0+max_tokens, optional fail-open. ThreePolicyprompt presets (general / coding-agent / openclaw).
AffinityRouter— session latch (retain a model per identity);with_latch_onlyrestricts which models latch.TurnGate— routes a fixed target beforemin_turn, then abstains.MetadataProcessor+Metadata::from_headers— normalizes harnessheaders into correlation metadata in
State.Composed routers (all one skeleton)
[LlmClassifier][StagedRouter, fallback][latch, TurnGate, ConfirmingJudge(LlmClassifier)]+ latch processor…, TurnGate, StagedRouter, judge]escalation_router(...)/escalation_router_with_staged(...)assemble thejudge-latched escalation flow: start
weak, escalate tostrongwhen the judge(or the cheap staged heuristic) sees trouble, then latch.
ConfirmingJudgerequires N escalate verdicts before latching — trivial here because
Stateisper-session (the reference needs a keyed streak store).
Notes
Classifier::scoretakes an optional&Driver(driver-backed classifiers useit, error on
None; local ones ignore it).Sendalgorithmfuture:
Processornow yieldsSendfutures, andEvent::ModelResponsecarries the buffered
&AggLlmResponse(a live stream can't cross the spawn).Testing
121 unit/integration tests,
cargo clippyclean. Ports the reference testsuites (signal extraction, scorer, tier selection) and covers the cascade,
latch persistence, confirmations, and end-to-end escalation with a stubbed
judge over the driver stream.
Checklist
snake_caseof the primary class.switchyard/__init__.py.__all__if intended for downstream use.--helpupdated if customer-facing surface changed.Signed-off-by: Your Name <email>) per the DCO.Notes for reviewers
Anything reviewers should pay extra attention to — risky paths, follow-up tickets, intentional trade-offs.