NVIDIA · GPUtester · Jul 20, 2026 · Jul 20, 2026
@@ -15,6 +15,9 @@ Decision API. It adds routing-aware LLM execution intercepts to the Relay
 runtime while preserving Relay ownership of provider credentials, target
 bindings, dispatch, retries, fallbacks, and observability.
 
+NeMo Relay 0.6.0 uses a separately running Switchyard Decision API from the
+`topic/nemo-relay-integration` branch.
+
 Install it from crates.io, or build it from the NeMo Relay source checkout with
 the optional CLI feature while the Switchyard Decision API contract and
 service/library boundary are still evolving.
@@ -79,9 +82,7 @@ The current service setup is documented in
 the pinned topic-branch commit, local configuration, compatibility smoke test,
 and trajectory workflow.
 
-Translation is already in-process through Switchyard's Rust translation
-library. A future in-process `DecisionProvider` can replace the HTTP Decision API
-call without changing the Relay-owned dispatch and observability boundary.
+Translation runs in-process through Switchyard's Rust translation library.
 
 ## Configuration and Registration
 
@@ -118,6 +119,7 @@ outside tracked configuration files.
 
 For more information, refer to the following resources:
 
+- [Switchyard 0.6.0 setup and validation guide](https://docs.nvidia.com/nemo/relay/v0.6.0/configure-plugins/switchyard/about)
+- [Switchyard configuration reference](https://docs.nvidia.com/nemo/relay/v0.6.0/configure-plugins/switchyard/configuration)
 - [Switchyard integration examples](../../examples/switchyard/README.md)
-- [NeMo Relay documentation](https://docs.nvidia.com/nemo/relay)
 - [Switchyard repository](https://github.com/NVIDIA-NeMo/Switchyard)
@@ -1,9 +1,11 @@
 ---
 title: "Switchyard (Experimental)"
 sidebar-title: "Switchyard (Experimental)"
-description: "Route Relay LLM requests through the experimental Switchyard Decision API integration."
+description: "Set up and validate the experimental Switchyard Decision API integration for NeMo Relay 0.6.0."
 position: 5
 ---
+import { MermaidStyles } from "@/components/MermaidStyles";
+
 {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 SPDX-License-Identifier: Apache-2.0 */}
 
@@ -12,48 +14,169 @@ SPDX-License-Identifier: Apache-2.0 */}
 > change, and the current deployment requires a separately running Switchyard
 > Decision API service.
 
-The `nemo-relay-switchyard` plugin uses the
-[Switchyard](https://github.com/NVIDIA-NeMo/Switchyard) Decision API to select
-backends for supported LLM requests. Relay validates each decision, translates
-requests and responses in process through Switchyard's
-`switchyard-translation` library, and dispatches directly to the selected
-Relay-owned target. Relay also owns provider credentials, retries, trusted
-fallbacks, and observability. The separately running Switchyard service owns
-routing decisions.
+> **NeMo Relay 0.6.0 architecture:** This release uses the external Switchyard
+> Decision API from the
+> [`topic/nemo-relay-integration`](https://github.com/NVIDIA-NeMo/Switchyard/tree/topic/nemo-relay-integration)
+> branch.
+
+Switchyard is an LLM routing decision engine. The `nemo-relay-switchyard`
+plugin asks Switchyard which configured target should handle an LLM request.
+Relay validates the decision and performs the authenticated provider request.
+For example, a routing profile can send a simple prompt to a lower-cost model
+and a complex prompt to a more capable model.
+
+## Architecture
+
+The following diagram shows the NeMo Relay 0.6.0 service boundary:
+
+<MermaidStyles />
+
+```mermaid
+flowchart LR
+    Agent["Agent"] -->|LLM request| Relay["NeMo Relay"]
+    Relay -->|routing request| Decision["Switchyard Decision API"]
+    Decision -->|backend decision| Relay
+    Relay -->|authenticated provider request| Backend["Selected LLM backend"]
+    Relay -->|ATOF events| History["Switchyard ATOF accumulator"]
+    History -. routing history .-> Decision
+
+    class Agent grey-lightest;
+    class Relay green-lightest;
+    class Decision yellow-lightest;
+    class History yellow-lightest;
+    class Backend teal-lightest;
+```
+
+The components divide responsibility as follows:
+
+| Component | Responsibilities |
+| --- | --- |
+| Switchyard | Selects a backend and accumulates ATOF routing history for history-based profiles. |
+| NeMo Relay | Owns provider credentials, target bindings, decision validation, protocol translation, dispatch, retries, trusted fallback, and observability. |
+
+Relay does not start or supervise the Switchyard service. Start Switchyard
+before Relay activates the plugin. Relay derives the service's root `/health`
+URL from `decision_api_url` and fails activation unless the endpoint returns
+`{"status":"ok"}`.
 
 ## Prerequisites
 
-The current integration has two runtime pieces:
+Install the following prerequisites before you start:
 
-1. A Relay build with the optional `switchyard` feature.
-2. A Switchyard service built from the pinned
-   [`topic/nemo-relay-integration` revision](https://github.com/NVIDIA-NeMo/Switchyard/tree/topic/nemo-relay-integration).
+| Requirement | Version or Value |
+| --- | --- |
+| NeMo Relay | Tag `0.6.0` |
+| Rust | `1.96.1` |
+| Switchyard branch | `topic/nemo-relay-integration` |
+| Switchyard commit | `8f9db9a6a47f848cdff1d262276ba25a8ae9cbc8` |
+| Local commands | `git`, `python3`, and `curl` |
 
-The current Relay examples pin Switchyard commit
-[`8f9db9a6`](https://github.com/NVIDIA-NeMo/Switchyard/commit/8f9db9a6a47f848cdff1d262276ba25a8ae9cbc8).
-Treat that revision as part of this experimental compatibility boundary; update
-it deliberately when testing a different Switchyard topic commit.
+Keep the repositories in the following sibling layout. The validation script
+uses this layout by default:
 
-The plugin does not start or supervise the Switchyard service. Start a compatible
-service before Relay activates the plugin. Relay derives the service's root
-`/health` URL from `decision_api_url` and fails activation unless the endpoint
-returns `{"status":"ok"}`. After activation, unavailable Decision API calls fail
-open to the configured same-protocol fallback.
+```text
+<workspace>/
+├── NeMo-Relay/
+└── Switchyard-topic-nemo-relay-integration/
+```
 
-Profiles that depend on ATOF history, including StageRouter, must use
-`context_mode = "atof_required"` and name an enabled Relay ATOF HTTP stream sink
-with `atof_endpoint_name`. Relay fails startup validation when that sink is
-missing or invalid. Relay pushes ATOF events to the sink; a local ATOF file sink
-alone does not populate Switchyard's accumulator.
+## Build and Run the Compatibility Test
 
-Build the opt-in CLI feature with:
+Use the following procedure to clone and pin both repositories:
+
+```bash
+mkdir relay-switchyard-0.6
+cd relay-switchyard-0.6
+
+git clone --branch 0.6.0 \
+  https://github.com/NVIDIA/NeMo-Relay.git \
+  NeMo-Relay
+
+git clone --branch topic/nemo-relay-integration \
+  https://github.com/NVIDIA-NeMo/Switchyard.git \
+  Switchyard-topic-nemo-relay-integration
+
+git -C Switchyard-topic-nemo-relay-integration checkout --detach \
+  8f9db9a6a47f848cdff1d262276ba25a8ae9cbc8
+
+cd NeMo-Relay
+```
+
+The Switchyard plugin is excluded from default Relay CLI builds. Build the CLI
+with the optional feature explicitly:
 
 ```bash
 cargo build -p nemo-relay-cli --features switchyard
 ```
 
-The default CLI build does not include the plugin or the Switchyard translation
-library.
+Run the real-service compatibility test from the NeMo Relay repository root:
+
+```bash
+examples/switchyard/run-real-e2e.sh
+```
+
+The script prints the verified Switchyard revision and ends with the following
+line when routing succeeds:
+
+```text
+real Switchyard E2E passed: ['provider/weak', 'provider/strong', 'provider/strong']
+```
+
+The test starts these local processes:
+
+| Process | Port | Readiness Check or Purpose |
+| --- | --- | --- |
+| Switchyard Decision API | `4000` | Serves `/health`, routing decisions, and ATOF ingestion. |
+| NeMo Relay | `4041` | Serves `/healthz` and the OpenAI-compatible gateway. |
+| Deterministic provider | `4101` | Records the model selected for each provider request. |
+
+The first request has no accumulated ATOF history and routes to
+`provider/weak`. The script then sends session and tool events to Relay. Relay
+exports those events to Switchyard's ATOF accumulator, so the next buffered
+request and the final streaming request route to `provider/strong`. The test
+also validates the streamed response before reporting success.
+
+## Configuration Walkthrough
+
+The compatibility test uses
+[`real-e2e-plugins.toml`](https://github.com/NVIDIA/NeMo-Relay/blob/release/0.6/examples/switchyard/real-e2e-plugins.toml)
+and
+[`real-e2e-profiles.yaml`](https://github.com/NVIDIA/NeMo-Relay/blob/release/0.6/examples/switchyard/real-e2e-profiles.yaml).
+The following settings establish the routing boundary:
+
+| Setting | Purpose |
+| --- | --- |
+| `decision_api_url` | Points Relay to the separately running Switchyard Decision API. |
+| `decision_profile_id` | Selects the Switchyard routing profile. The profile maps its semantic targets to backend IDs. |
+| `targets.<backend_id>` | Binds each returned backend ID to a Relay-owned model, protocol, endpoint, base URL, and credentials. |
+| `mode = "enforce"` | Applies a valid Switchyard decision and dispatches to the selected target. |
+| `mode = "observe_only"` | Records the hypothetical decision but dispatches once to the trusted same-protocol default. |
+| `default_targets` | Defines trusted same-protocol fallbacks for unavailable or invalid decisions and provider failures. |
+| `context_mode = "atof_required"` | Requires stable identity and accumulated ATOF history for the selected profile. |
+| `atof_endpoint_name` | Selects exactly one named Relay ATOF HTTP stream sink that sends events to Switchyard. |
+
+For history-based routing, a local ATOF JSONL file is not sufficient.
+Switchyard must receive events through the named HTTP sink at
+`/v1/atof/events` so its accumulator can provide the history used by the
+routing profile. Relay rejects a missing, duplicate, disabled, or invalid
+named sink during startup validation.
+
+For the complete option reference, refer to
+[Switchyard Configuration](./configuration.mdx).
+
+## Troubleshooting
+
+Use the following table to diagnose common compatibility-test failures:
+
+| Symptom | Cause and Resolution |
+| --- | --- |
+| `Switchyard worktree not found: ...` | Place the Switchyard checkout next to `NeMo-Relay` as `Switchyard-topic-nemo-relay-integration`, or set `SWITCHYARD_ROOT` to its absolute path. |
+| `Switchyard checkout mismatch: expected ..., found ...` | Check out commit `8f9db9a6a47f848cdff1d262276ba25a8ae9cbc8`. Override `SWITCHYARD_EXPECTED_COMMIT` only when deliberately testing a different contract. |
+| A build uses Rust `1.96.0` or reports an unsupported toolchain | Run `rustc --version`, install Rust `1.96.1`, and allow the repository's `rust-toolchain.toml` to select it. |
+| `timed out waiting for http://127.0.0.1:4000/health` | Review `switchyard.log`. Confirm the pinned checkout builds and port `4000` is free. |
+| `timed out waiting for http://127.0.0.1:4041/healthz` or `process ... exited before http://127.0.0.1:4041/healthz became ready` | Review `relay.log`. Confirm the CLI was built with `--features switchyard`, the Switchyard health check passed, and port `4041` is free. |
+| The provider cannot start or requests fail | Review `upstream.log` and confirm port `4101` is free. |
+| `E2E logs preserved in ...` | The script preserves its temporary directory after a runtime failure and prints the exact path. Inspect `switchyard.log`, `relay.log`, `upstream.log`, and the captured request outputs there. Successful runs remove the directory. |
 
 ## Capabilities
 
@@ -73,8 +196,9 @@ The integration provides the following capabilities:
 For more information, refer to the following pages:
 
 - [Switchyard Configuration](./configuration.mdx)
-  is the complete option and deployment reference. Review its
+  provides the complete option and deployment reference. Review its
   [experimental limitations](./configuration.mdx#experimental-limitations)
   before adopting the integration.
-- [Switchyard integration examples](https://github.com/NVIDIA/NeMo-Relay/tree/main/examples/switchyard)
-  documents the pinned service worktree and manual compatibility smoke tests.
+- [Switchyard integration examples](https://github.com/NVIDIA/NeMo-Relay/tree/release/0.6/examples/switchyard)
+  contains the versioned configuration and validation scripts used by this
+  guide.
@@ -7,9 +7,12 @@ position: 2
 {/* SPDX-FileCopyrightText: Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 SPDX-License-Identifier: Apache-2.0 */}
 
-> **Experimental:** This page describes the current service-based Switchyard
-> integration. Its configuration and service boundary can change as Switchyard
-> develops an in-process Decision Provider.
+> **Experimental NeMo Relay 0.6.0 integration:** This release calls a separately
+> running Switchyard Decision API.
+
+Start with the
+[Switchyard 0.6.0 setup and validation guide](./about.mdx)
+before using this page as the complete configuration reference.
 
 The Switchyard component connects Relay's CLI gateway to a separately running
 [Switchyard](https://github.com/NVIDIA-NeMo/Switchyard) Decision API. Relay
@@ -341,8 +344,7 @@ Consider the following experimental limitations before adopting the plugin:
   [`8f9db9a6`](https://github.com/NVIDIA-NeMo/Switchyard/commit/8f9db9a6a47f848cdff1d262276ba25a8ae9cbc8)
   on `topic/nemo-relay-integration`. Testing another revision is deliberate and
   can expose contract drift.
-- The Decision API and service/library boundary are experimental and can change
-  when Switchyard introduces an in-process Decision Provider.
+- The Decision API and service/library boundary are experimental.
 - Nonportable provider extensions in configurations that can translate across
   protocols, malformed decisions, service failures, pre-dispatch translation
   failures, buffered-response translation failures, and target drift fail open
@@ -353,4 +355,4 @@ Consider the following experimental limitations before adopting the plugin:
 
 Run `nemo-relay doctor` to validate component configuration before startup. For
 the pinned local service and manual compatibility workflow, refer to the
-[Switchyard integration examples](https://github.com/NVIDIA/NeMo-Relay/tree/main/examples/switchyard).
+[Switchyard integration examples](https://github.com/NVIDIA/NeMo-Relay/tree/release/0.6/examples/switchyard).
@@ -5,13 +5,17 @@ SPDX-License-Identifier: Apache-2.0
 
 # Switchyard Integration Examples
 
-These examples exercise the experimental Relay integration with a separately running Switchyard
-Decision API service and the in-process Switchyard translation library. They are manual, local
-validation workflows rather than production startup orchestration.
+These examples exercise the experimental NeMo Relay 0.6.0 integration with a separately running
+Switchyard Decision API service and the in-process Switchyard translation library. They are
+manual, local validation workflows rather than production startup orchestration.
+
+For the canonical architecture, setup, configuration, validation, and troubleshooting workflow,
+refer to the
+[Switchyard 0.6.0 setup and validation guide](https://docs.nvidia.com/nemo/relay/v0.6.0/configure-plugins/switchyard/about).
 
 ## Required Switchyard Revision
 
-The scripts default to the latest commit currently pinned for the public topic branch:
+The NeMo Relay 0.6.0 scripts require the following public topic branch and commit:
 
 ```text
 https://github.com/NVIDIA-NeMo/Switchyard/tree/topic/nemo-relay-integration
@@ -58,6 +62,12 @@ a successful run.
 examples/switchyard/run-real-e2e.sh
 ```
 
+A successful run ends with:
+
+```text
+real Switchyard E2E passed: ['provider/weak', 'provider/strong', 'provider/strong']
+```
+
 ### Hermes and Ollama Trajectory
 
 `run-hermes-ollama-smoke.sh` runs a fixed multi-query trajectory through Hermes, Relay, Ollama,
@@ -99,4 +109,6 @@ configuration and refuses activation unless it reports `{"status":"ok"}`.
 Trajectory scripts write to `artifacts/` by default. Set `SWITCHYARD_TRAJECTORY_DIR` to choose a
 shareable output directory. On failure, logs are preserved and include the verified Switchyard
 revision. Do not place API keys or bearer tokens in configuration files; use environment variables
-or an untracked secrets file.
+or an untracked secrets file. For service readiness failures, port conflicts, and the exact
+temporary log locations used by `run-real-e2e.sh`, refer to the
+[canonical Switchyard 0.6.0 guide](https://docs.nvidia.com/nemo/relay/v0.6.0/configure-plugins/switchyard/about#troubleshooting).