Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ jobs:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
version: "0.11.29"
python-version: "3.12"
enable-cache: true
cache-dependency-glob: "uv.lock"
Expand Down Expand Up @@ -92,6 +93,7 @@ jobs:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
version: "0.11.29"
python-version: "3.12"
enable-cache: true
cache-dependency-glob: "uv.lock"
Expand All @@ -113,6 +115,7 @@ jobs:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
version: "0.11.29"
python-version: ${{ matrix.python-version }}
enable-cache: true
cache-dependency-glob: "uv.lock"
Expand All @@ -136,6 +139,7 @@ jobs:
- uses: actions/checkout@v4
- uses: astral-sh/setup-uv@v6
with:
version: "0.11.29"
python-version: "3.12"
enable-cache: true
cache-dependency-glob: "uv.lock"
Expand Down
1 change: 1 addition & 0 deletions .github/workflows/perf.yml
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ jobs:

- uses: astral-sh/setup-uv@v6
with:
version: "0.11.29"
python-version: "3.12"
enable-cache: true
cache-dependency-glob: "uv.lock"
Expand Down
18 changes: 18 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ members = [
"crates/switchyard-server",
"crates/switchyard-skill-distillation",
"crates/switchyard-translation",
"demo/libsy-proxy",
]

[workspace.package]
Expand Down
21 changes: 20 additions & 1 deletion crates/libsy-protocol/src/envelope.rs
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,16 @@ pub struct Context {
/// All fields are optional; algorithms and observers use whichever are present
/// (e.g. to key per-session state or emit correlated telemetry). `extra_metadata`
/// is a free-form escape hatch for host-specific keys.
#[derive(Clone)]
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct Metadata {
/// Stable id for a multi-request session/conversation.
pub session_id: Option<String>,
/// Id of the agent making the request.
pub agent_id: Option<String>,
/// Id of the task the request belongs to.
pub task_id: Option<String>,
/// Agent-specific lineage and semantic routing signals.
pub agent_context: Option<Box<AgentContext>>,
/// External trace/request id for joining with the host's telemetry.
pub correlation_id: Option<String>,
/// Arbitrary host-defined key/value metadata.
Expand All @@ -38,6 +40,23 @@ pub struct Metadata {
pub wire_format: Option<WireFormat>,
}

/// Optional lineage and semantic signals for affinity-aware routing.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct AgentContext {
/// Whether the harness explicitly identified this request as coming from a child agent.
pub is_subagent: bool,
/// Id of the parent agent, when this request comes from a child agent.
pub parent_agent_id: Option<String>,
/// Harness-defined kind of agent call, such as `collab_spawn` or `review`.
pub agent_kind: Option<String>,
/// Semantic agent role, such as `explorer`, `worker`, or `reviewer`.
pub agent_role: Option<String>,
/// Semantic task class supplied by the harness.
pub task_kind: Option<String>,
/// Id of the current agent turn.
pub turn_id: Option<String>,
}

/// A request an algorithm routes: the normalized [`LlmRequest`] plus the original
/// provider payload and correlation [`Metadata`].
#[derive(Clone)]
Expand Down
1 change: 1 addition & 0 deletions crates/libsy/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ async-trait = "0.1"
serde_json = "1"
futures = "0.3"
rand = "0.8"
serde = { version = "1", features = ["derive"] }
switchyard-protocol = { path = "../libsy-protocol" }
tokio = { version = "1", features = ["full"] }
tokio-stream = "0.1"
Expand Down
36 changes: 34 additions & 2 deletions crates/libsy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,38 @@ println!("answer: {}", completion_text(&response.llm_response.aggregate().await?

Runnable: [`research_agent`](../libsy-examples/examples/research_agent.rs) (in the `libsy-examples` crate).

## Composable affinity

Routing algorithms can consume any `affinity::Affinity` policy at the point where
their final model is known. `SessionAffinity` retains one model per session;
`SubAgentAffinity` retains one model per explicitly identified child agent while
ordinary requests continue through the algorithm on every turn.

```rust
use libsy::affinity::SubAgentAffinity;
use libsy::RandomAlgo;

let algo: Arc<dyn Algorithm> = Arc::new(
RandomAlgo::new(targets)
.with_affinity(Arc::new(SubAgentAffinity::new())),
);
```

The first request from a child agent runs the random algorithm and atomically retains
its choice; later requests for the same `(session_id, agent_id)` reuse that target.
With one configured target, that target is retained directly. `metadata_from_headers`
converts harness-specific identity into neutral `Metadata`.
Supported inputs include Codex `session-id`, `thread-id`,
`x-codex-parent-thread-id`, `x-openai-subagent`, and `x-codex-turn-metadata`;
NeMo Relay `x-nemo-relay-session-id` / `x-nemo-relay-subagent-id`; Dynamo
`x-dynamo-session-id` / `x-dynamo-parent-session-id`; and explicit
`x-switchyard-*` overrides, including `x-switchyard-is-subagent`. Header parsing is
separate from affinity keying, so non-HTTP embedders can populate the same metadata
directly. The adapter stays local and dependency-free because Relay's matching gateway
normalizer is not exposed through its public Rust library API.

See [`demo/libsy-proxy`](../../demo/libsy-proxy) for a runnable Switchyard HTTP proxy.

## Requests & responses

```rust
Expand Down Expand Up @@ -257,8 +289,8 @@ with `cargo test -p libsy-examples`).

**Reference algorithms** — implementations to read and route with:

- [`RandomAlgo`](src/algorithms/rand.rs) — uniform random over the set
(one call).
- [`RandomAlgo`](src/algorithms/rand.rs) — uniform random over the set, optionally
retaining selections through an `Affinity` policy (one call).
- [`LlmClassifierOrchAlgo`](../libsy-examples/src/llm_class.rs) — classify, then route
strong/weak; fail open to strong.
- [`EnsembleOrchAlgo`](../libsy-examples/src/ensemble.rs) — stateful: fan out to
Expand Down
Loading
Loading