From abc7731e3ac111d7783642390809777f0442ed31 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sat, 11 Jul 2026 21:25:44 +0200 Subject: [PATCH 01/18] empty From 2357149cbfb11a0d592de5c508d3fbd3b84ae612 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sat, 11 Jul 2026 21:37:23 +0200 Subject: [PATCH 02/18] docs/rshell-networks.md --- docs/rshell-networks.md | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 docs/rshell-networks.md diff --git a/docs/rshell-networks.md b/docs/rshell-networks.md new file mode 100644 index 00000000..f847bfee --- /dev/null +++ b/docs/rshell-networks.md @@ -0,0 +1,9 @@ + +# Rough idea + +Goal: Use rshell as tool for Networks Investigations and Remediation, especially for Network Devices (routers, switches, etc). + +Rshell will run in private action runner, which is itself Datadog Agent (https://github.com/DataDog/datadog-agent). + +Prior Art: +- NCM (Netework Config Mangement) already have a mechanism to communicate with Network Devices. \ No newline at end of file From 09fa665968c302035f8b317f464c5267d092337a Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sat, 11 Jul 2026 21:53:54 +0200 Subject: [PATCH 03/18] docs: define network remediation problem --- docs/rshell-networks.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/rshell-networks.md b/docs/rshell-networks.md index f847bfee..34fe8811 100644 --- a/docs/rshell-networks.md +++ b/docs/rshell-networks.md @@ -1,9 +1,11 @@ +# Network Investigation and Remediation with rshell -# Rough idea +## Problem Statement -Goal: Use rshell as tool for Networks Investigations and Remediation, especially for Network Devices (routers, switches, etc). +Datadog AI agents can detect and reason about network incidents using telemetry, topology, and configuration history, but they lack a secure way to actively investigate and remediate the underlying network from within a customer's private environment. -Rshell will run in private action runner, which is itself Datadog Agent (https://github.com/DataDog/datadog-agent). +Existing capabilities address parts of this problem: the Datadog Agent provides a private execution environment through the Private Action Runner, rshell safely executes constrained operations, and Network Configuration Management (NCM) communicates with network devices and supports configuration collection and rollback. However, there is no general execution interface that lets an AI agent perform open-ended host diagnostics, issue native vendor-specific commands, coordinate operations across multiple network devices, and verify remediation outcomes. -Prior Art: -- NCM (Netework Config Mangement) already have a mechanism to communicate with Network Devices. \ No newline at end of file +Without this capability, investigations must fall back to predefined actions or human operators with direct device access. This limits an AI agent's ability to diagnose dynamic operational state, correlate observations across devices, respond to the long tail of vendor-specific failures, and complete remediation workflows. + +The challenge is to enable bounded, auditable, protocol-agnostic network investigation and remediation from a Private Action Runner while preserving the customer's security boundary. Operations must support heterogeneous devices and multi-device workflows, keep credentials hidden from AI agents, remain constrained by backend policy and independent local safeguards, and execute as isolated, time-bounded actions rather than unrestricted interactive access. From 7f701e41cab4804ec1bc79b51524454f59f3f8e2 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sat, 11 Jul 2026 21:57:09 +0200 Subject: [PATCH 04/18] docs: clarify Agent execution context --- docs/rshell-networks.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/rshell-networks.md b/docs/rshell-networks.md index 34fe8811..5fb513ab 100644 --- a/docs/rshell-networks.md +++ b/docs/rshell-networks.md @@ -4,8 +4,8 @@ Datadog AI agents can detect and reason about network incidents using telemetry, topology, and configuration history, but they lack a secure way to actively investigate and remediate the underlying network from within a customer's private environment. -Existing capabilities address parts of this problem: the Datadog Agent provides a private execution environment through the Private Action Runner, rshell safely executes constrained operations, and Network Configuration Management (NCM) communicates with network devices and supports configuration collection and rollback. However, there is no general execution interface that lets an AI agent perform open-ended host diagnostics, issue native vendor-specific commands, coordinate operations across multiple network devices, and verify remediation outcomes. +Existing capabilities address parts of this problem: the Datadog Agent provides a customer-side execution environment, rshell safely executes constrained operations, and Network Configuration Management (NCM) communicates with network devices and supports configuration collection and rollback. The Private Action Runner is an existing Datadog Agent mechanism for receiving and executing actions from Datadog within a customer's private network. However, there is no general execution interface that lets an AI agent perform open-ended host diagnostics, issue native vendor-specific commands, coordinate operations across multiple network devices, and verify remediation outcomes. Without this capability, investigations must fall back to predefined actions or human operators with direct device access. This limits an AI agent's ability to diagnose dynamic operational state, correlate observations across devices, respond to the long tail of vendor-specific failures, and complete remediation workflows. -The challenge is to enable bounded, auditable, protocol-agnostic network investigation and remediation from a Private Action Runner while preserving the customer's security boundary. Operations must support heterogeneous devices and multi-device workflows, keep credentials hidden from AI agents, remain constrained by backend policy and independent local safeguards, and execute as isolated, time-bounded actions rather than unrestricted interactive access. +The challenge is to enable bounded, auditable, protocol-agnostic network investigation and remediation from a Datadog Agent while preserving the customer's security boundary. Operations must support heterogeneous devices and multi-device workflows, keep credentials hidden from AI agents, remain constrained by backend policy and independent local safeguards, and execute as isolated, time-bounded actions rather than unrestricted interactive access. From 63c3aa9469e860011f598df1e3f9643689b9d4c6 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sat, 11 Jul 2026 22:55:07 +0200 Subject: [PATCH 05/18] docs: propose rshell Agent extension RFC --- docs/rshell-networks.md | 392 +++++++++++++++++++++++++++++++++++++++- 1 file changed, 390 insertions(+), 2 deletions(-) diff --git a/docs/rshell-networks.md b/docs/rshell-networks.md index 5fb513ab..aa580a44 100644 --- a/docs/rshell-networks.md +++ b/docs/rshell-networks.md @@ -1,11 +1,399 @@ -# Network Investigation and Remediation with rshell +# RFC: Network Investigation and Remediation with rshell + +| Field | Value | +|---|---| +| Status | Draft | +| Scope | Datadog Agent and rshell execution layer | +| Audience | rshell, Datadog Agent, Network Device Monitoring, and Agent Health maintainers | + +## Summary + +This RFC proposes a general extension framework that lets the Datadog Agent register typed, capability-oriented commands with rshell. The first implementation will use the framework to expose a small set of network-device investigation and remediation operations to AI agents. Agent Health is a second candidate that validates the framework's generality, but its implementation is outside the scope of this RFC. + +rshell will own command parsing, argument validation, help, allowlisting, execution-mode enforcement, output rendering, cancellation, and resource limits. The Datadog Agent will own target resolution, credentials, connectivity, protocol handling, vendor translation, and the execution of registered operations. + +The proposal does not expose arbitrary native device commands. Instead, reviewed command leaves such as `device interfaces show` and `device interface set-state` map to typed Agent handlers. This preserves rshell's restricted execution model while giving AI agents composable capabilities for investigation and remediation. ## Problem Statement Datadog AI agents can detect and reason about network incidents using telemetry, topology, and configuration history, but they lack a secure way to actively investigate and remediate the underlying network from within a customer's private environment. -Existing capabilities address parts of this problem: the Datadog Agent provides a customer-side execution environment, rshell safely executes constrained operations, and Network Configuration Management (NCM) communicates with network devices and supports configuration collection and rollback. The Private Action Runner is an existing Datadog Agent mechanism for receiving and executing actions from Datadog within a customer's private network. However, there is no general execution interface that lets an AI agent perform open-ended host diagnostics, issue native vendor-specific commands, coordinate operations across multiple network devices, and verify remediation outcomes. +Existing capabilities address parts of this problem: the Datadog Agent provides a customer-side execution environment, rshell safely executes constrained operations, and Network Configuration Management (NCM) communicates with network devices and supports configuration collection and rollback. The Private Action Runner is an existing Datadog Agent mechanism for receiving and executing actions from Datadog within a customer's private network. However, there is no general execution interface that lets an AI agent perform open-ended host diagnostics, invoke reviewed vendor-aware device operations, coordinate operations across multiple network devices, and verify remediation outcomes. Without this capability, investigations must fall back to predefined actions or human operators with direct device access. This limits an AI agent's ability to diagnose dynamic operational state, correlate observations across devices, respond to the long tail of vendor-specific failures, and complete remediation workflows. The challenge is to enable bounded, auditable, protocol-agnostic network investigation and remediation from a Datadog Agent while preserving the customer's security boundary. Operations must support heterogeneous devices and multi-device workflows, keep credentials hidden from AI agents, remain constrained by backend policy and independent local safeguards, and execute as isolated, time-bounded actions rather than unrestricted interactive access. + +## Goals + +- Let the Datadog Agent expose reviewed investigation and remediation capabilities as rshell commands. +- Preserve rshell's default-deny model and local defense-in-depth controls. +- Support natural command trees with precise authorization of executable leaves. +- Keep credentials, connections, protocols, and vendor-specific behavior inside the Datadog Agent. +- Provide stable, composable output across device vendors. +- Support workflows spanning multiple devices through rshell composition or backend orchestration. +- Make the extension mechanism reusable by other Agent domains, with Agent Health as a validating use case. + +## Non-Goals + +- Designing backend authorization, approval, or AI orchestration policy. +- Selecting or implementing the final device credential provider. +- Giving AI agents access to device credentials. +- Providing arbitrary native device command execution. +- Reproducing complete vendor CLI grammars in rshell. +- Providing persistent or interactive device sessions. +- Supporting multi-target execution within one extension command. +- Implementing Agent Health commands as part of this RFC. +- Defining a complete network-device command catalog. + +## Background + +### rshell + +rshell is a restricted shell interpreter for AI agents. It provides a default-deny command allowlist, a filesystem sandbox, bounded execution, an empty environment by default, and separate read-only and remediation modes. Its builtins own their parsing and validation, and external command execution is blocked unless the embedding application explicitly provides a handler. + +Today, `AllowedCommands` authorizes top-level command identities such as `rshell:cat`, `rshell:ping`, and `rshell:ip`. Some builtins have subcommands, but authorization does not distinguish between those subcommands. + +### Datadog Agent and Private Action Runner + +The Datadog Agent is the customer-side environment in which the proposed capabilities execute. The Private Action Runner is one existing mechanism for delivering authorized actions from Datadog to an Agent in a private network. This RFC does not require the Private Action Runner to be the only future delivery mechanism. + +### Network Configuration Management + +NCM already provides device registration, SSH connectivity, credential use, host verification, profile matching, per-device locking, configuration retrieval, configuration storage, and configuration rollback. Its current remote interface is oriented around configuration collection and push operations rather than a general catalog of typed troubleshooting capabilities. + +NCM is a possible source of device identities, connections, and credentials for the proposed design. Backend-provided connection information is another possibility. The extension interface must not depend on either source. + +### Agent Health + +Agent Health may need to expose Agent-owned diagnostics and remediation capabilities through rshell. It has different domain behavior from network-device management but needs the same command registration, validation, authorization, mode, help, telemetry, and execution contracts. It therefore serves as a second use case against which to evaluate the extension framework. + +## Use Cases and Requirements + +### Investigation + +AI agents should be able to inspect operational state such as: + +- Interface status, addresses, errors, and descriptions. +- Routing tables and selected routes. +- Routing-protocol neighbor state. +- Reachability from the Agent host and from supported devices. +- Relevant differences between observed state and known configuration. + +### Remediation + +AI agents should be able to invoke reviewed state-changing capabilities such as: + +- Enabling or disabling an interface. +- Restoring a known-good configuration through an existing NCM operation. +- Applying future narrowly scoped, typed remediation operations. +- Verifying device state after a remediation. + +### Multi-device workflows + +Each extension command operates on exactly one target. Workflows spanning multiple devices are composed by the backend or through rshell control flow: + +```sh +for target in edge-router-1 edge-router-2; do + device interfaces show "$target" +done +``` + +This keeps connection handling, exit status, retries, and audit records scoped to a single device operation. Concurrent orchestration remains a backend responsibility for the first version. + +### Execution requirements + +- Commands are non-interactive and bounded by context cancellation and timeouts. +- Targets are explicit; commands cannot discover and operate on an unrestricted device fleet. +- Arguments are parsed and validated before an Agent handler is invoked. +- Investigation and remediation operations are classified separately. +- Credentials never enter the rshell program, output, or AI-agent context. +- Output and errors have deterministic contracts suitable for automation. + +## Proposed Architecture + +### Overview + +The Datadog Agent embeds rshell and registers one or more extension command trees. A registration describes the command hierarchy and executable leaves. Each leaf provides metadata, an argument schema, an execution-mode requirement, and a typed handler. + +For a device operation, execution proceeds as follows: + +1. rshell parses the shell program and identifies the top-level extension command. +2. The extension framework walks the registered command tree and resolves the longest executable leaf. +3. rshell authorizes the leaf using `AllowedCommands`. +4. rshell validates arguments and verifies that the current execution mode permits the operation. +5. rshell invokes the registered Agent handler with a bounded context and typed inputs. +6. The Agent resolves the opaque target through a device provider. +7. The provider obtains an authenticated connection without exposing credentials to rshell. +8. A vendor or protocol adapter performs the typed operation and returns a structured result. +9. rshell renders the result and records the exit status and telemetry. + +### Responsibility boundary + +rshell owns: + +- Command-tree registration and resolution. +- Command identity and `AllowedCommands` enforcement. +- Argument-schema validation. +- Read-only and remediation-mode enforcement. +- Help and usage generation. +- Context cancellation, timeout propagation, and output bounds. +- Stable text and JSON rendering contracts. +- Framework-level telemetry and errors. + +The Datadog Agent owns: + +- Extension handlers and domain behavior. +- Opaque target resolution. +- Credential acquisition and isolation. +- Device connection lifecycle and locking. +- Protocol selection and vendor translation. +- Parsing native responses into structured results. +- Domain-level telemetry and audit context. + +### Extension registration + +The exact Go API is left to implementation, but it must express a tree of namespaces and executable leaves. An illustrative registration is: + +```go +ExtensionCommand{ + Path: []string{"device", "interfaces", "show"}, + Description: "Show network interfaces on a device.", + Mode: ModeReadOnly, + Arguments: []Argument{{Name: "target", Type: String, Required: true}}, + Handler: showInterfaces, +} +``` + +Handlers receive only their typed inputs, standard output contract, and a cancellable context. Registering an extension must not implicitly grant access to the host filesystem, process execution, the host environment, credentials, or other rshell internals. + +The framework must reject duplicate paths, ambiguous paths, incomplete registrations, invalid command identities, and handlers without required metadata. + +## Command and Authorization Model + +### Executable command paths + +Current rshell commands have one-segment paths: + +```text +rshell:cat +rshell:ip +``` + +Extension commands may have multi-segment paths. The canonical identity is formed from the complete executable path: + +```sh +device interfaces show edge-router-1 +``` + +resolves to: + +```text +rshell:device.interfaces.show +``` + +This remains one `AllowedCommands` model: existing builtins are one-segment leaves, while extension operations may be deeper leaves. Namespace nodes such as `rshell:device` and `rshell:device.interfaces` are not executable or valid exact grants unless separately registered as executable operations. + +### Wildcards + +The framework may support a terminal namespace wildcard: + +```text +rshell:device.interfaces.* +``` + +It matches registered descendant leaves such as: + +```text +rshell:device.interfaces.show +rshell:device.interfaces.set-state +``` + +It does not match other namespaces such as `rshell:device.routes.show`. Arbitrary or partial glob patterns are not supported. + +Operator configuration may use terminal wildcards for convenience. Backend tasks should authorize exact leaf identities. The effective allowlist remains the intersection of backend and operator policy, so an operator wildcard does not independently authorize a new operation. + +### Execution modes + +Every executable leaf declares its minimum mode: + +- Investigation leaves run in read-only and remediation modes. +- Remediation leaves run only in remediation mode. + +Mode classification is mandatory registration metadata and is enforced by rshell before dispatch. An extension handler cannot downgrade or bypass this check. + +## Device Provider and Adapter Boundary + +The Agent-side device provider accepts an opaque target reference and returns access to typed device capabilities. It is responsible for resolving the device, locating credentials, applying connection policy, selecting a protocol or vendor adapter, and closing the connection. + +The provider must support different credential sources without changing rshell commands. Candidate implementations include: + +- Reusing NCM-registered devices and credentials. +- Resolving backend-provided connection information inside the Agent. +- Supporting a future Agent credential provider. + +The AI agent and rshell program operate only on opaque target references. Raw usernames, passwords, private keys, and tokens are not valid command arguments or handler results. + +Vendor adapters translate typed operations into the appropriate native protocol operations and normalize responses. SSH is a likely first transport, but the interface must permit NETCONF, REST APIs, controllers, or other protocols. + +## V1 Device Command Catalog + +The first version implements a small vertical slice rather than a complete network automation surface. + +### Investigation commands + +```text +device interfaces show TARGET +device routes show TARGET +device routing-neighbors show TARGET +``` + +These commands validate typed reads, normalized cross-vendor results, command-tree authorization, and the Agent provider boundary. + +### Remediation command + +```text +device interface set-state TARGET INTERFACE up|down +``` + +This command validates remediation-mode enforcement, typed state changes, argument validation, and post-operation error reporting. Configuration rollback may continue to use NCM's existing dedicated action in v1. + +The final names and detailed schemas may change during implementation, but the v1 catalog must retain equivalent coverage of read-only and state-changing operations. + +## Output and Error Contract + +Agent handlers return structured domain results. rshell renders those results rather than passing through raw native device output. + +The default format is deterministic, native-inspired text. For example: + +```text +INTERFACE STATUS PROTOCOL ADDRESS DESCRIPTION +Ethernet1 up up 10.0.0.1/30 uplink-core +Ethernet2 down down - unused +``` + +Commands also support a stable JSON representation for machine consumption: + +```sh +device interfaces show --output json edge-router-1 +``` + +Raw native output is not exposed in v1 because it is vendor-dependent, difficult to validate, and may contain unexpected or sensitive content. + +The output contract follows normal shell conventions: + +- Successful results are written to stdout. +- Diagnostics are written to stderr. +- Exit code `0` means the operation succeeded. +- Exit code `1` means a target or domain operation failed. +- Invocation errors use the standard rshell parsing and usage behavior. +- Framework failures that cannot be represented as normal command failures are returned through the embedding API. + +## Security Model + +The backend is responsible for deciding which operation should be requested and for applying product-level authorization and approval policy. The Agent and rshell provide an independent enforcement boundary. + +Local protections include: + +- Intersection of backend and operator `AllowedCommands` policies. +- Exact resolution and authorization of executable command leaves. +- Mandatory investigation or remediation classification. +- Typed argument validation before dispatch. +- Explicit, opaque targets. +- No arbitrary native command payload. +- No credential arguments or results. +- Context cancellation and execution timeouts. +- Bounded output and resource use. +- Agent-owned connection policy and device-side least-privilege credentials. +- Auditable command identity, target identity, mode, outcome, and duration. + +Extension handlers are privileged Agent code and require the same security review as builtins. The framework must not present extension registration as a way to execute arbitrary binaries or bypass rshell filesystem, environment, or process restrictions. + +## Alternatives Considered + +### A dedicated injected `DeviceExecutor` builtin + +rshell could implement a fixed `device` builtin backed by a domain-specific `DeviceExecutor` interface. This is simpler for one use case and keeps parsing close to existing builtins. It was not selected because Agent Health provides a second credible extension domain, and adding one injected interface per Agent domain would not scale cleanly. + +### A generic `device exec` command + +A generic command could forward arbitrary native CLI text to a target. This offers maximum vendor coverage but turns one allowed rshell command into an unrestricted device-command tunnel. It weakens local defense in depth and makes investigation/remediation classification unreliable. It is explicitly rejected. + +### Reimplement native device CLIs as rshell commands + +rshell could parse vendor-native commands directly. This would require dynamic vendor grammars, stateful CLI contexts, and significant ongoing compatibility work. It would also create command collisions and couple rshell to network vendors. It is rejected in favor of typed capabilities and Agent adapters. + +### Separate `AllowedCapabilities` + +rshell could authorize the top-level `rshell:device` command and use a second capability allowlist for subcommands. This preserves the current first-token authorization model but introduces overlapping policy concepts. The proposal instead generalizes `AllowedCommands` to canonical executable command paths. + +### Multi-target command handlers + +Each command could accept a set of devices and define concurrency and partial-success behavior. This duplicates orchestration already available in rshell and the backend, complicates results and retries, and broadens blast radius. V1 uses single-target operations. + +### Exact native output + +Returning raw device output would resemble familiar CLI sessions but creates unstable, vendor-specific contracts and may expose unexpected content. V1 uses normalized structured results with native-inspired rendering. + +## Compatibility and Migration + +Existing rshell builtins, command identities, and allowlists remain valid. A traditional builtin such as `rshell:ip` is a one-segment executable leaf. Only registered extension trees use deeper canonical identities. + +The Agent and rshell versions must agree on the extension registration API. Unsupported or duplicate registrations fail during runner construction rather than at command execution time. An Agent that does not register extensions exposes no extension command leaves. + +Help output must list only registered and enabled extension operations. Detailed help must be available for namespaces and executable leaves without treating a namespace help request as execution authorization. + +## Testing and Validation + +### rshell framework tests + +- Registration validation and duplicate detection. +- Longest executable-path resolution. +- Exact and terminal-wildcard `AllowedCommands` behavior. +- Backend/operator allowlist intersection. +- Mode enforcement before handler dispatch. +- Argument-schema validation and generated help. +- Context cancellation, timeout, and output limits. +- Panic containment and deterministic error mapping. +- Confirmation that extensions receive no implicit filesystem, environment, credential, or process access. + +### Agent tests + +- Target resolution through each supported provider. +- Credential non-disclosure. +- Connection lifecycle and per-device locking. +- Vendor translation and normalized results. +- Investigation and remediation handlers. +- Unsupported vendor, protocol, and capability behavior. +- End-to-end Private Action Runner execution where applicable. + +### Compatibility tests + +- Existing builtin allowlists remain unchanged. +- Existing help and command behavior remain unchanged when no extensions are registered. +- Agent and rshell version mismatches fail safely. + +## Rollout and Observability + +The proposed rollout is staged: + +1. Land the extension registration and authorization framework with test-only handlers. +2. Add the Agent device-provider boundary and read-only v1 commands. +3. Pilot read-only investigation with a limited set of vendors and protocols. +4. Add the v1 remediation command behind explicit remediation-mode and allowlist configuration. +5. Expand the command catalog and adapters based on demonstrated use cases. +6. Evaluate Agent Health against the framework before declaring the extension API stable. + +Telemetry should record the canonical command identity, execution mode, duration, outcome, provider and adapter type, and a non-secret target identifier. It must not record credentials, native command payloads, or unredacted device output. + +The Agent must be able to disable all extensions or individual command leaves through local configuration. Rolling back the Agent or removing a registration removes the corresponding operations without changing rshell's core builtin behavior. + +## Dependencies and Open Questions + +- Which device and credential provider should be implemented first: NCM, backend-provided connection information, or another Agent provider? +- Which vendors and protocols are required for the initial pilot? +- Should terminal namespace wildcards be included in v1, or should v1 accept exact leaf identities only? +- What stable structured schemas should each v1 device operation return? +- How should extension API compatibility be versioned between rshell and the Datadog Agent? +- Which telemetry and audit events must be emitted by rshell, the Agent handler, and the backend? +- What evidence is required before remediation capabilities are enabled beyond a limited pilot? From 7734b4f223e9fff725f3febdb1153dc2a02045cd Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sat, 11 Jul 2026 23:00:16 +0200 Subject: [PATCH 06/18] docs: separate network and extension RFC scope --- docs/rshell-networks.md | 185 +++++++++++++++++++++++----------------- 1 file changed, 106 insertions(+), 79 deletions(-) diff --git a/docs/rshell-networks.md b/docs/rshell-networks.md index aa580a44..80a966ed 100644 --- a/docs/rshell-networks.md +++ b/docs/rshell-networks.md @@ -4,13 +4,13 @@ |---|---| | Status | Draft | | Scope | Datadog Agent and rshell execution layer | -| Audience | rshell, Datadog Agent, Network Device Monitoring, and Agent Health maintainers | +| Audience | rshell, Datadog Agent, and Network Device Monitoring maintainers | ## Summary -This RFC proposes a general extension framework that lets the Datadog Agent register typed, capability-oriented commands with rshell. The first implementation will use the framework to expose a small set of network-device investigation and remediation operations to AI agents. Agent Health is a second candidate that validates the framework's generality, but its implementation is outside the scope of this RFC. +This RFC proposes a safe way for AI agents to investigate and remediate network devices from a Datadog Agent with rshell. The first version exposes a small set of reviewed, capability-oriented device operations while keeping target resolution, credentials, connectivity, protocols, and vendor translation inside the Datadog Agent. -rshell will own command parsing, argument validation, help, allowlisting, execution-mode enforcement, output rendering, cancellation, and resource limits. The Datadog Agent will own target resolution, credentials, connectivity, protocol handling, vendor translation, and the execution of registered operations. +A requirement of the network solution is a general rshell extension framework through which the Datadog Agent registers typed command trees. That framework requires a separate RFC and is a dependency of this proposal. This RFC defines the requirements network operations place on the framework, but does not define its API or general authorization model. Agent Health is a second candidate that demonstrates why this boundary must be reusable, but Agent Health itself is outside the scope of this RFC. The proposal does not expose arbitrary native device commands. Instead, reviewed command leaves such as `device interfaces show` and `device interface set-state` map to typed Agent handlers. This preserves rshell's restricted execution model while giving AI agents composable capabilities for investigation and remediation. @@ -32,7 +32,7 @@ The challenge is to enable bounded, auditable, protocol-agnostic network investi - Keep credentials, connections, protocols, and vendor-specific behavior inside the Datadog Agent. - Provide stable, composable output across device vendors. - Support workflows spanning multiple devices through rshell composition or backend orchestration. -- Make the extension mechanism reusable by other Agent domains, with Agent Health as a validating use case. +- Use a general extension mechanism rather than introducing a network-specific embedding API in rshell. ## Non-Goals @@ -44,6 +44,7 @@ The challenge is to enable bounded, auditable, protocol-agnostic network investi - Providing persistent or interactive device sessions. - Supporting multi-target execution within one extension command. - Implementing Agent Health commands as part of this RFC. +- Designing or implementing the general rshell extension framework. - Defining a complete network-device command catalog. ## Background @@ -64,9 +65,11 @@ NCM already provides device registration, SSH connectivity, credential use, host NCM is a possible source of device identities, connections, and credentials for the proposed design. Backend-provided connection information is another possibility. The extension interface must not depend on either source. -### Agent Health +### Requirement for a general extension mechanism -Agent Health may need to expose Agent-owned diagnostics and remediation capabilities through rshell. It has different domain behavior from network-device management but needs the same command registration, validation, authorization, mode, help, telemetry, and execution contracts. It therefore serves as a second use case against which to evaluate the extension framework. +The network solution requires the Datadog Agent to expose Agent-owned operations without moving target resolution, credentials, protocols, or vendor behavior into rshell. This boundary must be general rather than network-specific and will be proposed in a separate RFC. Agent Health is another likely consumer: it may need to expose Agent-owned diagnostics and remediation through rshell and would need the same registration, validation, authorization, mode, help, telemetry, and execution contracts. + +Agent Health provides evidence for the generality requirement, not a second subject for this RFC. The proposed design and v1 validation remain centered on network investigation and remediation. ## Use Cases and Requirements @@ -110,17 +113,87 @@ This keeps connection handling, exit status, retries, and audit records scoped t - Credentials never enter the rshell program, output, or AI-agent context. - Output and errors have deterministic contracts suitable for automation. -## Proposed Architecture +### Extension requirement + +The implementation depends on a general rshell extension framework rather than a network-specific `DeviceExecutor` option. A separate RFC must define that framework. To support this proposal, the framework must provide: + +- Typed command trees with independently authorizable executable operations. +- Read-only and remediation classification enforced before handler dispatch. +- Argument validation and generated help. +- Bounded, cancellable Agent handlers. +- Deterministic text and JSON output contracts. +- Safe registration that grants no implicit filesystem, environment, process, or credential access. + +Network-device commands are the first implementation delivered by this RFC. The companion framework RFC owns the general API, command-identity encoding, allowlist syntax, wildcard semantics, compatibility, and behavior for other extension domains. + +### Design insights for the companion RFC + +The discussion behind this RFC produced several framework-level design insights. They are retained here as recommendations and input to the companion RFC, not as framework-wide decisions made by this RFC. + +#### Prefer one command policy model + +The framework should extend the existing `AllowedCommands` concept instead of adding a separate `AllowedCapabilities` policy. A registered executable operation should resolve to one canonical rshell command identity and pass through the same backend/operator policy intersection as builtins. + +Existing builtins can remain one-segment executable paths: + +```text +rshell:cat +rshell:ip +``` + +Extension operations can use deeper executable paths: + +```sh +device interfaces show edge-router-1 +``` + +with a canonical identity such as: + +```text +rshell:device.interfaces.show +``` + +Under this model, `rshell:cat` and `rshell:device.interfaces.show` follow the same rule: each identifies a registered executable path. Intermediate namespace nodes such as `rshell:device` or `rshell:device.interfaces` are not exact grants unless they are themselves registered as executable operations. + +#### Authorize executable leaves precisely + +Each executable leaf should have an independent policy and audit identity. Adding a future remediation operation must not silently expand an exact grant for an existing investigation operation. + +The companion RFC should consider a deliberately limited terminal namespace wildcard for operator convenience: + +```text +rshell:device.interfaces.* +``` + +Arbitrary patterns such as `device.*.show` or partial-name globbing should not be supported. Backend tasks should authorize exact leaves even when local operator configuration uses a terminal wildcard. The effective policy remains the intersection of both sides, so an operator wildcard cannot authorize an operation by itself. + +#### Register typed command trees + +Extensions should declare command paths, descriptions, typed arguments, execution modes, and handlers. The framework should reject duplicate or ambiguous paths, incomplete registrations, invalid identities, and missing required metadata during runner construction. + +Command resolution should happen before authorization and dispatch. The framework should resolve the complete executable path, validate its arguments, check its mode and effective command policy, and only then invoke the Agent handler. + +#### Keep handlers narrow and bounded + +Handlers should receive typed inputs and a cancellable context rather than unrestricted access to shell internals. Registration must not implicitly grant filesystem, environment, process-execution, credential, or network access. Any privileged capability must be supplied deliberately by the embedding application. + +The framework should provide consistent help, telemetry, timeout propagation, output bounds, panic containment, and error mapping for all extension domains. + +#### Validate the abstraction with another domain + +Agent Health should be used as a design test before the general API is declared stable. The framework should be able to represent both network-device operations and Agent-owned health diagnostics without embedding domain-specific types in rshell. Implementing Agent Health remains outside this RFC. + +## Proposed Network Architecture ### Overview -The Datadog Agent embeds rshell and registers one or more extension command trees. A registration describes the command hierarchy and executable leaves. Each leaf provides metadata, an argument schema, an execution-mode requirement, and a typed handler. +Assuming the companion extension framework, the Datadog Agent registers a network-device command tree with rshell. Each executable operation provides network-specific metadata, arguments, an execution-mode requirement, and an Agent handler. For a device operation, execution proceeds as follows: 1. rshell parses the shell program and identifies the top-level extension command. -2. The extension framework walks the registered command tree and resolves the longest executable leaf. -3. rshell authorizes the leaf using `AllowedCommands`. +2. The extension framework resolves the requested executable operation. +3. rshell verifies that the operation is enabled by the effective command policy. 4. rshell validates arguments and verifies that the current execution mode permits the operation. 5. rshell invokes the registered Agent handler with a bounded context and typed inputs. 6. The Agent resolves the opaque target through a device provider. @@ -130,10 +203,10 @@ For a device operation, execution proceeds as follows: ### Responsibility boundary -rshell owns: +The companion rshell extension framework owns: - Command-tree registration and resolution. -- Command identity and `AllowedCommands` enforcement. +- Command identity and command-policy enforcement. - Argument-schema validation. - Read-only and remediation-mode enforcement. - Help and usage generation. @@ -151,9 +224,9 @@ The Datadog Agent owns: - Parsing native responses into structured results. - Domain-level telemetry and audit context. -### Extension registration +### Network command registration -The exact Go API is left to implementation, but it must express a tree of namespaces and executable leaves. An illustrative registration is: +The companion RFC defines the registration API. The network implementation needs to register operations equivalent to the following illustrative definition: ```go ExtensionCommand{ @@ -167,60 +240,28 @@ ExtensionCommand{ Handlers receive only their typed inputs, standard output contract, and a cancellable context. Registering an extension must not implicitly grant access to the host filesystem, process execution, the host environment, credentials, or other rshell internals. -The framework must reject duplicate paths, ambiguous paths, incomplete registrations, invalid command identities, and handlers without required metadata. +Registration failures are handled according to the companion framework RFC and must prevent unsafe or ambiguous network operations from becoming available. -## Command and Authorization Model +## Network Command and Authorization Requirements ### Executable command paths -Current rshell commands have one-segment paths: - -```text -rshell:cat -rshell:ip -``` - -Extension commands may have multi-segment paths. The canonical identity is formed from the complete executable path: +Network operations use natural command paths: ```sh device interfaces show edge-router-1 ``` -resolves to: - -```text -rshell:device.interfaces.show -``` - -This remains one `AllowedCommands` model: existing builtins are one-segment leaves, while extension operations may be deeper leaves. Namespace nodes such as `rshell:device` and `rshell:device.interfaces` are not executable or valid exact grants unless separately registered as executable operations. - -### Wildcards - -The framework may support a terminal namespace wildcard: - -```text -rshell:device.interfaces.* -``` - -It matches registered descendant leaves such as: - -```text -rshell:device.interfaces.show -rshell:device.interfaces.set-state -``` - -It does not match other namespaces such as `rshell:device.routes.show`. Arbitrary or partial glob patterns are not supported. - -Operator configuration may use terminal wildcards for convenience. Backend tasks should authorize exact leaf identities. The effective allowlist remains the intersection of backend and operator policy, so an operator wildcard does not independently authorize a new operation. +Each executable network operation must have a distinct policy and audit identity. Allowing an investigation operation must not implicitly allow a remediation operation, and adding a future operation must not silently expand an existing exact grant. The companion framework RFC decides how those identities are encoded in `AllowedCommands`, whether namespace wildcards exist, and how backend and operator policy are expressed and intersected. ### Execution modes -Every executable leaf declares its minimum mode: +Every executable network operation declares its minimum mode: - Investigation leaves run in read-only and remediation modes. - Remediation leaves run only in remediation mode. -Mode classification is mandatory registration metadata and is enforced by rshell before dispatch. An extension handler cannot downgrade or bypass this check. +Mode classification is mandatory registration metadata and is enforced by the extension framework before dispatch. A network handler cannot downgrade or bypass this check. ## Device Provider and Adapter Boundary @@ -313,7 +354,7 @@ Extension handlers are privileged Agent code and require the same security revie ### A dedicated injected `DeviceExecutor` builtin -rshell could implement a fixed `device` builtin backed by a domain-specific `DeviceExecutor` interface. This is simpler for one use case and keeps parsing close to existing builtins. It was not selected because Agent Health provides a second credible extension domain, and adding one injected interface per Agent domain would not scale cleanly. +rshell could implement a fixed `device` builtin backed by a domain-specific `DeviceExecutor` interface. This is simpler for one use case and keeps parsing close to existing builtins. It was not selected because the solution requires a reusable Agent extension boundary; adding one injected interface per Agent domain would not scale cleanly. Agent Health is one known candidate that would otherwise require another domain-specific interface. ### A generic `device exec` command @@ -323,10 +364,6 @@ A generic command could forward arbitrary native CLI text to a target. This offe rshell could parse vendor-native commands directly. This would require dynamic vendor grammars, stateful CLI contexts, and significant ongoing compatibility work. It would also create command collisions and couple rshell to network vendors. It is rejected in favor of typed capabilities and Agent adapters. -### Separate `AllowedCapabilities` - -rshell could authorize the top-level `rshell:device` command and use a second capability allowlist for subcommands. This preserves the current first-token authorization model but introduces overlapping policy concepts. The proposal instead generalizes `AllowedCommands` to canonical executable command paths. - ### Multi-target command handlers Each command could accept a set of devices and define concurrency and partial-success behavior. This duplicates orchestration already available in rshell and the backend, complicates results and retries, and broadens blast radius. V1 uses single-target operations. @@ -337,25 +374,17 @@ Returning raw device output would resemble familiar CLI sessions but creates uns ## Compatibility and Migration -Existing rshell builtins, command identities, and allowlists remain valid. A traditional builtin such as `rshell:ip` is a one-segment executable leaf. Only registered extension trees use deeper canonical identities. +The companion framework RFC owns compatibility for existing rshell builtins, command identities, allowlists, and extension registrations. This RFC requires the network implementation to introduce no behavior changes when its commands are not registered or enabled. -The Agent and rshell versions must agree on the extension registration API. Unsupported or duplicate registrations fail during runner construction rather than at command execution time. An Agent that does not register extensions exposes no extension command leaves. +The Agent and rshell versions must agree on the extension API version selected by the companion RFC. An Agent without the network implementation exposes no network-device operations. -Help output must list only registered and enabled extension operations. Detailed help must be available for namespaces and executable leaves without treating a namespace help request as execution authorization. +Help output must list only registered and enabled network operations. Detailed help must not expose or authorize disabled remediation operations. ## Testing and Validation -### rshell framework tests +### Framework dependency tests -- Registration validation and duplicate detection. -- Longest executable-path resolution. -- Exact and terminal-wildcard `AllowedCommands` behavior. -- Backend/operator allowlist intersection. -- Mode enforcement before handler dispatch. -- Argument-schema validation and generated help. -- Context cancellation, timeout, and output limits. -- Panic containment and deterministic error mapping. -- Confirmation that extensions receive no implicit filesystem, environment, credential, or process access. +The companion RFC owns the general extension-framework test suite. Before the network implementation can depend on it, that suite must demonstrate the registration, authorization, mode enforcement, argument validation, help, cancellation, output bounds, error mapping, and isolation guarantees required by this RFC. ### Agent tests @@ -369,20 +398,19 @@ Help output must list only registered and enabled extension operations. Detailed ### Compatibility tests -- Existing builtin allowlists remain unchanged. -- Existing help and command behavior remain unchanged when no extensions are registered. +- Existing help and command behavior remain unchanged when network commands are not registered. +- Network command identities and schemas remain stable across supported Agent upgrades. - Agent and rshell version mismatches fail safely. ## Rollout and Observability The proposed rollout is staged: -1. Land the extension registration and authorization framework with test-only handlers. +1. Approve and implement the companion rshell extension-framework RFC. 2. Add the Agent device-provider boundary and read-only v1 commands. -3. Pilot read-only investigation with a limited set of vendors and protocols. -4. Add the v1 remediation command behind explicit remediation-mode and allowlist configuration. -5. Expand the command catalog and adapters based on demonstrated use cases. -6. Evaluate Agent Health against the framework before declaring the extension API stable. +3. Pilot read-only network investigation with a limited set of vendors and protocols. +4. Add the v1 network remediation command behind explicit remediation-mode and command-policy configuration. +5. Expand the network command catalog and adapters based on demonstrated use cases. Telemetry should record the canonical command identity, execution mode, duration, outcome, provider and adapter type, and a non-secret target identifier. It must not record credentials, native command payloads, or unredacted device output. @@ -390,10 +418,9 @@ The Agent must be able to disable all extensions or individual command leaves th ## Dependencies and Open Questions +- The companion rshell extension-framework RFC must be approved and implemented before this proposal can ship. - Which device and credential provider should be implemented first: NCM, backend-provided connection information, or another Agent provider? - Which vendors and protocols are required for the initial pilot? -- Should terminal namespace wildcards be included in v1, or should v1 accept exact leaf identities only? - What stable structured schemas should each v1 device operation return? -- How should extension API compatibility be versioned between rshell and the Datadog Agent? - Which telemetry and audit events must be emitted by rshell, the Agent handler, and the backend? - What evidence is required before remediation capabilities are enabled beyond a limited pilot? From cc1885eddeac315d7f827c66d66b893586ca12d5 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 22:43:02 +0200 Subject: [PATCH 07/18] docs: make network RFC implementation-neutral --- docs/rshell-networks.md | 226 +++++++++++++++++++++------------------- 1 file changed, 118 insertions(+), 108 deletions(-) diff --git a/docs/rshell-networks.md b/docs/rshell-networks.md index 80a966ed..76659d4f 100644 --- a/docs/rshell-networks.md +++ b/docs/rshell-networks.md @@ -8,11 +8,11 @@ ## Summary -This RFC proposes a safe way for AI agents to investigate and remediate network devices from a Datadog Agent with rshell. The first version exposes a small set of reviewed, capability-oriented device operations while keeping target resolution, credentials, connectivity, protocols, and vendor translation inside the Datadog Agent. +This RFC defines a safe way for AI agents to investigate and remediate network devices from a Datadog Agent with rshell. The first version exposes a small set of reviewed, capability-oriented device operations while keeping target resolution, credentials, connectivity, protocols, and vendor translation inside the Datadog Agent. -A requirement of the network solution is a general rshell extension framework through which the Datadog Agent registers typed command trees. That framework requires a separate RFC and is a dependency of this proposal. This RFC defines the requirements network operations place on the framework, but does not define its API or general authorization model. Agent Health is a second candidate that demonstrates why this boundary must be reusable, but Agent Health itself is outside the scope of this RFC. +The mechanism that connects rshell commands to Agent-owned network operations is intentionally undecided. This RFC compares a general rshell extension framework, a network-specific `device` builtin backed by an injected `DeviceExecutor`, and an adaptation of rshell's existing external-command handler. Review of this RFC should select one approach. If the general extension framework is selected, its detailed API requires a companion RFC. -The proposal does not expose arbitrary native device commands. Instead, reviewed command leaves such as `device interfaces show` and `device interface set-state` map to typed Agent handlers. This preserves rshell's restricted execution model while giving AI agents composable capabilities for investigation and remediation. +None of the options exposes arbitrary native device commands. Reviewed operations such as `device interfaces show` and `device interface set-state` map to typed Agent-owned capabilities. This preserves rshell's restricted execution model while giving AI agents composable capabilities for investigation and remediation. ## Problem Statement @@ -28,11 +28,11 @@ The challenge is to enable bounded, auditable, protocol-agnostic network investi - Let the Datadog Agent expose reviewed investigation and remediation capabilities as rshell commands. - Preserve rshell's default-deny model and local defense-in-depth controls. -- Support natural command trees with precise authorization of executable leaves. +- Provide natural command syntax with precise authorization of individual operations. - Keep credentials, connections, protocols, and vendor-specific behavior inside the Datadog Agent. - Provide stable, composable output across device vendors. - Support workflows spanning multiple devices through rshell composition or backend orchestration. -- Use a general extension mechanism rather than introducing a network-specific embedding API in rshell. +- Select an rshell integration mechanism based on security, maintainability, implementation cost, and reuse. ## Non-Goals @@ -42,9 +42,9 @@ The challenge is to enable bounded, auditable, protocol-agnostic network investi - Providing arbitrary native device command execution. - Reproducing complete vendor CLI grammars in rshell. - Providing persistent or interactive device sessions. -- Supporting multi-target execution within one extension command. +- Supporting multi-target execution within one device command. - Implementing Agent Health commands as part of this RFC. -- Designing or implementing the general rshell extension framework. +- Defining the detailed API of a general rshell extension framework, if that option is selected. - Defining a complete network-device command catalog. ## Background @@ -63,13 +63,13 @@ The Datadog Agent is the customer-side environment in which the proposed capabil NCM already provides device registration, SSH connectivity, credential use, host verification, profile matching, per-device locking, configuration retrieval, configuration storage, and configuration rollback. Its current remote interface is oriented around configuration collection and push operations rather than a general catalog of typed troubleshooting capabilities. -NCM is a possible source of device identities, connections, and credentials for the proposed design. Backend-provided connection information is another possibility. The extension interface must not depend on either source. +NCM is a possible source of device identities, connections, and credentials for the proposed design. Backend-provided connection information is another possibility. The rshell integration must not depend on either source. -### Requirement for a general extension mechanism +### Agent-owned rshell capabilities -The network solution requires the Datadog Agent to expose Agent-owned operations without moving target resolution, credentials, protocols, or vendor behavior into rshell. This boundary must be general rather than network-specific and will be proposed in a separate RFC. Agent Health is another likely consumer: it may need to expose Agent-owned diagnostics and remediation through rshell and would need the same registration, validation, authorization, mode, help, telemetry, and execution contracts. +The network solution requires the Datadog Agent to expose Agent-owned operations without moving target resolution, credentials, protocols, or vendor behavior into rshell. Several implementation mechanisms can provide this boundary. This RFC does not assume one before comparing them. -Agent Health provides evidence for the generality requirement, not a second subject for this RFC. The proposed design and v1 validation remain centered on network investigation and remediation. +Agent Health is another likely consumer of Agent-owned rshell capabilities. It provides evidence that a reusable mechanism may be valuable, but does not by itself require the network implementation to use a general framework. Implementing Agent Health remains outside this RFC. ## Use Cases and Requirements @@ -94,7 +94,7 @@ AI agents should be able to invoke reviewed state-changing capabilities such as: ### Multi-device workflows -Each extension command operates on exactly one target. Workflows spanning multiple devices are composed by the backend or through rshell control flow: +Each device command operates on exactly one target. Workflows spanning multiple devices are composed by the backend or through rshell control flow: ```sh for target in edge-router-1 edge-router-2; do @@ -108,115 +108,134 @@ This keeps connection handling, exit status, retries, and audit records scoped t - Commands are non-interactive and bounded by context cancellation and timeouts. - Targets are explicit; commands cannot discover and operate on an unrestricted device fleet. -- Arguments are parsed and validated before an Agent handler is invoked. +- Arguments are parsed and validated before an Agent-owned operation is invoked. - Investigation and remediation operations are classified separately. - Credentials never enter the rshell program, output, or AI-agent context. - Output and errors have deterministic contracts suitable for automation. -### Extension requirement +### rshell integration requirements -The implementation depends on a general rshell extension framework rather than a network-specific `DeviceExecutor` option. A separate RFC must define that framework. To support this proposal, the framework must provide: +Whichever integration option is selected must provide: -- Typed command trees with independently authorizable executable operations. -- Read-only and remediation classification enforced before handler dispatch. -- Argument validation and generated help. -- Bounded, cancellable Agent handlers. +- Independently authorizable network operations. +- Read-only and remediation classification enforced before device execution. +- Argument validation and useful command help. +- Bounded, cancellable Agent-owned operations. - Deterministic text and JSON output contracts. -- Safe registration that grants no implicit filesystem, environment, process, or credential access. +- No implicit filesystem, environment, process, network, or credential access beyond capabilities deliberately supplied by the embedding application. +- Stable telemetry, error mapping, and compatibility behavior. -Network-device commands are the first implementation delivered by this RFC. The companion framework RFC owns the general API, command-identity encoding, allowlist syntax, wildcard semantics, compatibility, and behavior for other extension domains. +## Decision Required: rshell Integration -### Design insights for the companion RFC +Review of this RFC should select one of the following implementation options. The remainder of the RFC describes solution-independent network behavior. -The discussion behind this RFC produced several framework-level design insights. They are retained here as recommendations and input to the companion RFC, not as framework-wide decisions made by this RFC. +### Option A: General rshell extension framework -#### Prefer one command policy model +rshell provides a reusable mechanism for the Datadog Agent to register typed command trees, metadata, execution modes, and bounded handlers. Network-device commands are its first implementation; Agent Health is a second candidate validating that the abstraction is not network-specific. -The framework should extend the existing `AllowedCommands` concept instead of adding a separate `AllowedCapabilities` policy. A registered executable operation should resolve to one canonical rshell command identity and pass through the same backend/operator policy intersection as builtins. +Advantages: -Existing builtins can remain one-segment executable paths: +- Provides one reusable boundary for Agent-owned rshell capabilities. +- Keeps network and Agent Health implementations in the Datadog Agent. +- Can provide uniform help, authorization, telemetry, cancellation, and output behavior. +- Avoids adding one injected interface to rshell for every Agent domain. -```text -rshell:cat -rshell:ip -``` +Disadvantages: -Extension operations can use deeper executable paths: +- Requires a separate RFC and a larger initial rshell API design. +- Introduces a generic privileged extension surface that needs strong isolation guarantees. +- Increases initial implementation and compatibility work before delivering network operations. -```sh -device interfaces show edge-router-1 -``` +Useful design insights if this option is selected: -with a canonical identity such as: +- Prefer extending `AllowedCommands` instead of adding a separate `AllowedCapabilities` policy. +- Resolve each executable operation to an independently authorizable identity, such as `rshell:device.interfaces.show`. +- Treat intermediate command namespaces as non-executable unless explicitly registered. +- Consider only terminal namespace wildcards, with backend tasks authorizing exact operations. +- Resolve, authorize, validate, and check execution mode before invoking a handler. +- Give handlers typed inputs and a cancellable context, not unrestricted shell internals. +- Validate the abstraction against Agent Health before declaring the API stable. -```text -rshell:device.interfaces.show -``` +### Option B: Injected `DeviceExecutor` -Under this model, `rshell:cat` and `rshell:device.interfaces.show` follow the same rule: each identifies a registered executable path. Intermediate namespace nodes such as `rshell:device` or `rshell:device.interfaces` are not exact grants unless they are themselves registered as executable operations. +rshell implements a network-specific `device` builtin and accepts an injected, typed `DeviceExecutor` from the embedding Datadog Agent. rshell owns the device command syntax and validation; the Agent implementation owns targets, credentials, connections, protocols, and vendor translation. -#### Authorize executable leaves precisely +Advantages: -Each executable leaf should have an independent policy and audit identity. Adding a future remediation operation must not silently expand an exact grant for an existing investigation operation. +- Small, explicit API tailored to the known network operations. +- Keeps security-sensitive parsing and mode checks alongside rshell builtins. +- Requires less generic infrastructure and can deliver a vertical slice sooner. +- Is straightforward to test and audit for the network use case. -The companion RFC should consider a deliberately limited terminal namespace wildcard for operator convenience: +Disadvantages: -```text -rshell:device.interfaces.* -``` +- Adds network-domain concepts to a general-purpose rshell library. +- Couples rshell and Agent releases when device commands evolve. +- Does not help Agent Health or future Agent-owned domains. +- May lead to one specialized injected interface per domain. -Arbitrary patterns such as `device.*.show` or partial-name globbing should not be supported. Backend tasks should authorize exact leaves even when local operator configuration uses a terminal wildcard. The effective policy remains the intersection of both sides, so an operator wildcard cannot authorize an operation by itself. +### Option C: Adapt the external-command handler -#### Register typed command trees +The Datadog Agent exposes reviewed network operations through an enhanced form of rshell's existing external-command handler. The enhancement supplies the metadata and enforcement needed for allowlisting, help, modes, argument validation, and bounded output. -Extensions should declare command paths, descriptions, typed arguments, execution modes, and handlers. The framework should reject duplicate or ambiguous paths, incomplete registrations, invalid identities, and missing required metadata during runner construction. +Advantages: -Command resolution should happen before authorization and dispatch. The framework should resolve the complete executable path, validate its arguments, check its mode and effective command policy, and only then invoke the Agent handler. +- Builds on an existing rshell embedding mechanism. +- Keeps network command implementations and release cadence in the Agent. +- May require less new public API than a complete extension framework. -#### Keep handlers narrow and bounded +Disadvantages: -Handlers should receive typed inputs and a cancellable context rather than unrestricted access to shell internals. Registration must not implicitly grant filesystem, environment, process-execution, credential, or network access. Any privileged capability must be supplied deliberately by the embedding application. +- The current handler is intentionally generic and lacks typed registration, help, and operation-level mode metadata. +- Extending it may blur the boundary between reviewed Agent operations and arbitrary external execution. +- Static analysis and security review are harder when behavior lives outside rshell. +- It may evolve into a less explicit version of the general extension framework. -The framework should provide consistent help, telemetry, timeout propagation, output bounds, panic containment, and error mapping for all extension domains. +### Evaluation criteria -#### Validate the abstraction with another domain +The decision should weigh: -Agent Health should be used as a design test before the general API is declared stable. The framework should be able to represent both network-device operations and Agent-owned health diagnostics without embedding domain-specific types in rshell. Implementing Agent Health remains outside this RFC. +- Preservation of rshell's default-deny security model. +- Ability to authorize investigation and remediation operations independently. +- Handler isolation and auditability. +- Implementation time for the v1 network command catalog. +- Release and compatibility coupling between rshell and the Agent. +- Reuse by Agent Health and other credible domains. +- Long-term clarity of ownership and testing. -## Proposed Network Architecture +## Solution-independent Network Architecture ### Overview -Assuming the companion extension framework, the Datadog Agent registers a network-device command tree with rshell. Each executable operation provides network-specific metadata, arguments, an execution-mode requirement, and an Agent handler. +rshell exposes reviewed network-device operations implemented using the selected integration option. Each operation has defined syntax, typed arguments, an execution-mode requirement, and an Agent-owned implementation. For a device operation, execution proceeds as follows: -1. rshell parses the shell program and identifies the top-level extension command. -2. The extension framework resolves the requested executable operation. -3. rshell verifies that the operation is enabled by the effective command policy. -4. rshell validates arguments and verifies that the current execution mode permits the operation. -5. rshell invokes the registered Agent handler with a bounded context and typed inputs. -6. The Agent resolves the opaque target through a device provider. -7. The provider obtains an authenticated connection without exposing credentials to rshell. -8. A vendor or protocol adapter performs the typed operation and returns a structured result. -9. rshell renders the result and records the exit status and telemetry. +1. rshell parses the shell program and resolves the requested network operation. +2. rshell verifies that the operation is enabled by the effective command policy. +3. The integration validates arguments and verifies that the current execution mode permits the operation. +4. The integration invokes the Agent-owned implementation with a bounded context and typed inputs. +5. The Agent resolves the opaque target through a device provider. +6. The provider obtains an authenticated connection without exposing credentials to rshell. +7. A vendor or protocol adapter performs the typed operation and returns a structured result. +8. The integration renders the result and records the exit status and telemetry. ### Responsibility boundary -The companion rshell extension framework owns: +rshell and the selected integration mechanism own: -- Command-tree registration and resolution. -- Command identity and command-policy enforcement. +- Command parsing and operation resolution. +- Command-policy enforcement. - Argument-schema validation. - Read-only and remediation-mode enforcement. - Help and usage generation. - Context cancellation, timeout propagation, and output bounds. - Stable text and JSON rendering contracts. -- Framework-level telemetry and errors. +- Integration-level telemetry and errors. The Datadog Agent owns: -- Extension handlers and domain behavior. +- Network-operation implementations and domain behavior. - Opaque target resolution. - Credential acquisition and isolation. - Device connection lifecycle and locking. @@ -224,23 +243,18 @@ The Datadog Agent owns: - Parsing native responses into structured results. - Domain-level telemetry and audit context. -### Network command registration - -The companion RFC defines the registration API. The network implementation needs to register operations equivalent to the following illustrative definition: +### Network operation contract -```go -ExtensionCommand{ - Path: []string{"device", "interfaces", "show"}, - Description: "Show network interfaces on a device.", - Mode: ModeReadOnly, - Arguments: []Argument{{Name: "target", Type: String, Required: true}}, - Handler: showInterfaces, -} -``` +The selected integration must represent, for every operation: -Handlers receive only their typed inputs, standard output contract, and a cancellable context. Registering an extension must not implicitly grant access to the host filesystem, process execution, the host environment, credentials, or other rshell internals. +- Its command syntax and description. +- Its typed arguments and validation rules. +- Whether it is investigative or remedial. +- Its command-policy and audit identity. +- Its bounded Agent-owned implementation. +- Its text, JSON, error, and exit-status contracts. -Registration failures are handled according to the companion framework RFC and must prevent unsafe or ambiguous network operations from becoming available. +Invalid, duplicate, or ambiguous operation definitions must fail safely before the operation becomes available. ## Network Command and Authorization Requirements @@ -252,16 +266,16 @@ Network operations use natural command paths: device interfaces show edge-router-1 ``` -Each executable network operation must have a distinct policy and audit identity. Allowing an investigation operation must not implicitly allow a remediation operation, and adding a future operation must not silently expand an existing exact grant. The companion framework RFC decides how those identities are encoded in `AllowedCommands`, whether namespace wildcards exist, and how backend and operator policy are expressed and intersected. +Each executable network operation must have a distinct policy and audit identity. Allowing an investigation operation must not implicitly allow a remediation operation, and adding a future operation must not silently expand an existing exact grant. The selected integration must define how those identities use or interact with `AllowedCommands` and how backend and operator policy are expressed and intersected. ### Execution modes Every executable network operation declares its minimum mode: -- Investigation leaves run in read-only and remediation modes. -- Remediation leaves run only in remediation mode. +- Investigation operations run in read-only and remediation modes. +- Remediation operations run only in remediation mode. -Mode classification is mandatory registration metadata and is enforced by the extension framework before dispatch. A network handler cannot downgrade or bypass this check. +Mode classification is mandatory and is enforced before device execution. An Agent-owned network implementation cannot downgrade or bypass this check. ## Device Provider and Adapter Boundary @@ -289,7 +303,7 @@ device routes show TARGET device routing-neighbors show TARGET ``` -These commands validate typed reads, normalized cross-vendor results, command-tree authorization, and the Agent provider boundary. +These commands validate typed reads, normalized cross-vendor results, operation-level authorization, and the Agent provider boundary. ### Remediation command @@ -303,7 +317,7 @@ The final names and detailed schemas may change during implementation, but the v ## Output and Error Contract -Agent handlers return structured domain results. rshell renders those results rather than passing through raw native device output. +Agent-owned operations return structured domain results. The selected rshell integration renders those results rather than passing through raw native device output. The default format is deterministic, native-inspired text. For example: @@ -328,7 +342,7 @@ The output contract follows normal shell conventions: - Exit code `0` means the operation succeeded. - Exit code `1` means a target or domain operation failed. - Invocation errors use the standard rshell parsing and usage behavior. -- Framework failures that cannot be represented as normal command failures are returned through the embedding API. +- Integration failures that cannot be represented as normal command failures are returned through the embedding API. ## Security Model @@ -337,7 +351,7 @@ The backend is responsible for deciding which operation should be requested and Local protections include: - Intersection of backend and operator `AllowedCommands` policies. -- Exact resolution and authorization of executable command leaves. +- Precise resolution and authorization of executable network operations. - Mandatory investigation or remediation classification. - Typed argument validation before dispatch. - Explicit, opaque targets. @@ -348,14 +362,10 @@ Local protections include: - Agent-owned connection policy and device-side least-privilege credentials. - Auditable command identity, target identity, mode, outcome, and duration. -Extension handlers are privileged Agent code and require the same security review as builtins. The framework must not present extension registration as a way to execute arbitrary binaries or bypass rshell filesystem, environment, or process restrictions. +Agent-owned network implementations are privileged code and require the same security scrutiny as rshell builtins. The selected integration must not provide a way to execute arbitrary binaries or bypass rshell filesystem, environment, or process restrictions. ## Alternatives Considered -### A dedicated injected `DeviceExecutor` builtin - -rshell could implement a fixed `device` builtin backed by a domain-specific `DeviceExecutor` interface. This is simpler for one use case and keeps parsing close to existing builtins. It was not selected because the solution requires a reusable Agent extension boundary; adding one injected interface per Agent domain would not scale cleanly. Agent Health is one known candidate that would otherwise require another domain-specific interface. - ### A generic `device exec` command A generic command could forward arbitrary native CLI text to a target. This offers maximum vendor coverage but turns one allowed rshell command into an unrestricted device-command tunnel. It weakens local defense in depth and makes investigation/remediation classification unreliable. It is explicitly rejected. @@ -374,17 +384,17 @@ Returning raw device output would resemble familiar CLI sessions but creates uns ## Compatibility and Migration -The companion framework RFC owns compatibility for existing rshell builtins, command identities, allowlists, and extension registrations. This RFC requires the network implementation to introduce no behavior changes when its commands are not registered or enabled. +The selected integration must preserve existing rshell builtins, command identities, and allowlists. The network implementation introduces no behavior changes when its commands are not configured or enabled. -The Agent and rshell versions must agree on the extension API version selected by the companion RFC. An Agent without the network implementation exposes no network-device operations. +The Agent and rshell versions must agree on the selected integration contract. An Agent without the network implementation exposes no network-device operations. -Help output must list only registered and enabled network operations. Detailed help must not expose or authorize disabled remediation operations. +Help output must list only enabled network operations. Detailed help must not expose or authorize disabled remediation operations. ## Testing and Validation -### Framework dependency tests +### rshell integration contract tests -The companion RFC owns the general extension-framework test suite. Before the network implementation can depend on it, that suite must demonstrate the registration, authorization, mode enforcement, argument validation, help, cancellation, output bounds, error mapping, and isolation guarantees required by this RFC. +The selected integration must demonstrate operation resolution, authorization, mode enforcement, argument validation, help, cancellation, output bounds, error mapping, and isolation. Option-specific tests are defined when the integration decision is made. ### Agent tests @@ -392,13 +402,13 @@ The companion RFC owns the general extension-framework test suite. Before the ne - Credential non-disclosure. - Connection lifecycle and per-device locking. - Vendor translation and normalized results. -- Investigation and remediation handlers. +- Investigation and remediation implementations. - Unsupported vendor, protocol, and capability behavior. - End-to-end Private Action Runner execution where applicable. ### Compatibility tests -- Existing help and command behavior remain unchanged when network commands are not registered. +- Existing help and command behavior remain unchanged when network commands are not enabled. - Network command identities and schemas remain stable across supported Agent upgrades. - Agent and rshell version mismatches fail safely. @@ -406,7 +416,7 @@ The companion RFC owns the general extension-framework test suite. Before the ne The proposed rollout is staged: -1. Approve and implement the companion rshell extension-framework RFC. +1. Select and implement the rshell integration option. If Option A is selected, approve its companion framework RFC first. 2. Add the Agent device-provider boundary and read-only v1 commands. 3. Pilot read-only network investigation with a limited set of vendors and protocols. 4. Add the v1 network remediation command behind explicit remediation-mode and command-policy configuration. @@ -414,13 +424,13 @@ The proposed rollout is staged: Telemetry should record the canonical command identity, execution mode, duration, outcome, provider and adapter type, and a non-secret target identifier. It must not record credentials, native command payloads, or unredacted device output. -The Agent must be able to disable all extensions or individual command leaves through local configuration. Rolling back the Agent or removing a registration removes the corresponding operations without changing rshell's core builtin behavior. +The Agent must be able to disable all network operations or individual operations through local configuration. Rolling back the Agent or removing the network integration removes the corresponding operations without changing rshell's core builtin behavior. ## Dependencies and Open Questions -- The companion rshell extension-framework RFC must be approved and implemented before this proposal can ship. +- Which rshell integration option should this RFC select? - Which device and credential provider should be implemented first: NCM, backend-provided connection information, or another Agent provider? - Which vendors and protocols are required for the initial pilot? - What stable structured schemas should each v1 device operation return? -- Which telemetry and audit events must be emitted by rshell, the Agent handler, and the backend? +- Which telemetry and audit events must be emitted by rshell, the Agent-owned implementation, and the backend? - What evidence is required before remediation capabilities are enabled beyond a limited pilot? From 235420d40ed8d73ebba8e82d307ebcfb7c2e869e Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 22:56:15 +0200 Subject: [PATCH 08/18] docs: focus network RFC summary on problem --- docs/rshell-networks.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/docs/rshell-networks.md b/docs/rshell-networks.md index 76659d4f..9e2ed052 100644 --- a/docs/rshell-networks.md +++ b/docs/rshell-networks.md @@ -8,11 +8,11 @@ ## Summary -This RFC defines a safe way for AI agents to investigate and remediate network devices from a Datadog Agent with rshell. The first version exposes a small set of reviewed, capability-oriented device operations while keeping target resolution, credentials, connectivity, protocols, and vendor translation inside the Datadog Agent. +Datadog AI agents can use telemetry, topology, and configuration history to detect and reason about network incidents, but they cannot actively inspect operational state or remediate the affected network devices from within a customer's private environment. As a result, investigations must fall back to narrow predefined actions or human operators with direct device access. -The mechanism that connects rshell commands to Agent-owned network operations is intentionally undecided. This RFC compares a general rshell extension framework, a network-specific `device` builtin backed by an injected `DeviceExecutor`, and an adaptation of rshell's existing external-command handler. Review of this RFC should select one approach. If the general extension framework is selected, its detailed API requires a companion RFC. +This RFC seeks to enable AI agents to use rshell from a Datadog Agent to investigate heterogeneous network devices, invoke reviewed remediation operations, compose workflows across devices, and verify outcomes. The capability must support the long tail of vendor-specific network behavior without granting unrestricted or interactive device access. -None of the options exposes arbitrary native device commands. Reviewed operations such as `device interfaces show` and `device interface set-state` map to typed Agent-owned capabilities. This preserves rshell's restricted execution model while giving AI agents composable capabilities for investigation and remediation. +The solution must preserve the customer's security boundary: credentials remain hidden from AI agents, operations are explicit and independently authorized, execution is bounded and auditable, and backend policy is reinforced by local safeguards in the Agent. The rshell implementation mechanism remains a decision for this RFC and is evaluated after the problem, goals, and requirements. ## Problem Statement From 180845909f47c0792cd35375d8f585bcae4f98d7 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 23:01:17 +0200 Subject: [PATCH 09/18] docs: add network RFC one-pager --- docs/rshell-networks-one-pager.md | 67 +++++++++++++++++++++++++++++++ 1 file changed, 67 insertions(+) create mode 100644 docs/rshell-networks-one-pager.md diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md new file mode 100644 index 00000000..522f4920 --- /dev/null +++ b/docs/rshell-networks-one-pager.md @@ -0,0 +1,67 @@ +# Network Investigation and Remediation with rshell + +## The problem + +Datadog AI agents can use telemetry, topology, and configuration history to detect and reason about network incidents, but they cannot actively inspect operational state or remediate affected network devices from within a customer's private environment. Investigations therefore fall back to narrow predefined actions or human operators with direct device access. This limits an AI agent's ability to diagnose dynamic failures, correlate evidence across devices, respond to vendor-specific behavior, and complete remediation workflows. + +The missing capability is not basic device connectivity. The Datadog Agent already runs inside customer networks, rshell provides a restricted execution environment, and Network Configuration Management (NCM) can connect to supported devices, collect configurations, and roll back changes. The gap is a safe, composable interface through which AI agents can perform bounded network investigation and remediation. + +## Desired outcome + +AI agents should be able to use rshell from a Datadog Agent to: + +- Inspect interface, routing, neighbor, reachability, and configuration state. +- Invoke reviewed remediation operations and verify their outcome. +- Work across heterogeneous devices without depending on one vendor CLI or protocol. +- Compose multi-device workflows through rshell or backend orchestration. +- Receive stable, machine-readable results suitable for further reasoning. + +The capability should cover both Agent-host diagnostics and operations performed on network devices. Individual device commands remain single-target and non-interactive; workflows coordinate multiple bounded operations rather than opening persistent sessions. + +## Safety boundary + +The solution must preserve the customer's security boundary: + +- Credentials remain hidden from AI agents and rshell programs. +- Every operation is explicit, reviewed, and independently authorizable. +- Investigation and remediation are distinct execution modes. +- The Agent enforces local safeguards in addition to backend policy. +- Execution is time-bounded, cancellable, resource-bounded, and auditable. +- Targets are explicit; commands cannot discover and operate on an unrestricted fleet. +- Arbitrary native device-command passthrough is not supported. + +Target resolution, credentials, connectivity, protocol handling, and vendor translation remain Agent-owned responsibilities. NCM-managed devices and backend-provided connection information are both possible sources and remain an open design choice. + +## Proposed v1 scope + +V1 should validate the end-to-end boundary with a deliberately small catalog: + +- Show interfaces. +- Show routes. +- Show routing neighbors. +- Set an interface administratively up or down. + +These operations exercise cross-vendor reads, structured results, precise authorization, read-only versus remediation enforcement, and Agent-owned device access. Configuration rollback can continue through NCM's existing dedicated action initially. + +Results should default to stable, native-inspired text and optionally support JSON. Raw native output is excluded because it is vendor-dependent, difficult to validate, and may expose unexpected content. + +## rshell integration decision + +The network behavior above does not require one specific rshell implementation. This RFC review should select among three options: + +| Option | Advantages | Tradeoffs | +|---|---|---| +| General extension framework | Reusable by Network Devices, Agent Health, and future Agent domains; uniform authorization, help, telemetry, and execution contracts | Largest initial design; requires a companion RFC; creates a generic privileged extension surface | +| Injected `DeviceExecutor` | Small, explicit, easier to audit, and likely the fastest network vertical slice | Adds network concepts to rshell; couples rshell and Agent releases; does not serve other domains | +| Adapt external-command handling | Builds on an existing rshell embedding point and keeps implementations in the Agent | Existing handler lacks typed metadata and mode semantics; risks blurring reviewed capabilities with arbitrary execution | + +The decision should prioritize preservation of rshell's default-deny model, independent authorization of remediation operations, handler isolation, implementation time, release coupling, auditability, and credible reuse by Agent Health. + +## Decisions requested + +1. Confirm that bounded network investigation and remediation from the Datadog Agent is a capability worth pursuing. +2. Confirm the safety boundary and deliberately small v1 command catalog. +3. Select the rshell integration approach for detailed design and implementation. +4. Identify the initial credential/device provider and the vendors and protocols required for a pilot. + +The full design and supporting analysis are in [the engineering RFC](rshell-networks.md). From b05e75d3647afd7ccbaf98825b8cf79bef5effd9 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 23:06:47 +0200 Subject: [PATCH 10/18] docs: refine network one-pager problem --- docs/rshell-networks-one-pager.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md index 522f4920..22010035 100644 --- a/docs/rshell-networks-one-pager.md +++ b/docs/rshell-networks-one-pager.md @@ -2,7 +2,7 @@ ## The problem -Datadog AI agents can use telemetry, topology, and configuration history to detect and reason about network incidents, but they cannot actively inspect operational state or remediate affected network devices from within a customer's private environment. Investigations therefore fall back to narrow predefined actions or human operators with direct device access. This limits an AI agent's ability to diagnose dynamic failures, correlate evidence across devices, respond to vendor-specific behavior, and complete remediation workflows. +Datadog AI agents can use telemetry to detect and reason about network issues, but they cannot actively inspect operational state or remediate affected network devices from within a customer's private environment. This limits an AI agent's ability to diagnose dynamic failures, correlate evidence across devices, respond to vendor-specific behavior, and complete remediation workflows. The missing capability is not basic device connectivity. The Datadog Agent already runs inside customer networks, rshell provides a restricted execution environment, and Network Configuration Management (NCM) can connect to supported devices, collect configurations, and roll back changes. The gap is a safe, composable interface through which AI agents can perform bounded network investigation and remediation. From 4de76e6283bd3f575fd80bbf77eff4f85aedda46 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 23:09:13 +0200 Subject: [PATCH 11/18] docs: position rshell as network execution interface --- docs/rshell-networks-one-pager.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md index 22010035..a8cecdeb 100644 --- a/docs/rshell-networks-one-pager.md +++ b/docs/rshell-networks-one-pager.md @@ -4,7 +4,7 @@ Datadog AI agents can use telemetry to detect and reason about network issues, but they cannot actively inspect operational state or remediate affected network devices from within a customer's private environment. This limits an AI agent's ability to diagnose dynamic failures, correlate evidence across devices, respond to vendor-specific behavior, and complete remediation workflows. -The missing capability is not basic device connectivity. The Datadog Agent already runs inside customer networks, rshell provides a restricted execution environment, and Network Configuration Management (NCM) can connect to supported devices, collect configurations, and roll back changes. The gap is a safe, composable interface through which AI agents can perform bounded network investigation and remediation. +We propose using rshell as the safe, composable execution interface for network-device investigation and remediation. Running within the Datadog Agent, rshell would expose reviewed operations while preserving bounded execution and local safeguards. ## Desired outcome From 2e242d78daf81ffa2cadbbdedc429f3715181f0c Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 23:12:14 +0200 Subject: [PATCH 12/18] docs: simplify network one-pager --- docs/rshell-networks-one-pager.md | 62 +------------------------------ 1 file changed, 1 insertion(+), 61 deletions(-) diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md index a8cecdeb..da720ea2 100644 --- a/docs/rshell-networks-one-pager.md +++ b/docs/rshell-networks-one-pager.md @@ -1,67 +1,7 @@ # Network Investigation and Remediation with rshell -## The problem +## Overview Datadog AI agents can use telemetry to detect and reason about network issues, but they cannot actively inspect operational state or remediate affected network devices from within a customer's private environment. This limits an AI agent's ability to diagnose dynamic failures, correlate evidence across devices, respond to vendor-specific behavior, and complete remediation workflows. We propose using rshell as the safe, composable execution interface for network-device investigation and remediation. Running within the Datadog Agent, rshell would expose reviewed operations while preserving bounded execution and local safeguards. - -## Desired outcome - -AI agents should be able to use rshell from a Datadog Agent to: - -- Inspect interface, routing, neighbor, reachability, and configuration state. -- Invoke reviewed remediation operations and verify their outcome. -- Work across heterogeneous devices without depending on one vendor CLI or protocol. -- Compose multi-device workflows through rshell or backend orchestration. -- Receive stable, machine-readable results suitable for further reasoning. - -The capability should cover both Agent-host diagnostics and operations performed on network devices. Individual device commands remain single-target and non-interactive; workflows coordinate multiple bounded operations rather than opening persistent sessions. - -## Safety boundary - -The solution must preserve the customer's security boundary: - -- Credentials remain hidden from AI agents and rshell programs. -- Every operation is explicit, reviewed, and independently authorizable. -- Investigation and remediation are distinct execution modes. -- The Agent enforces local safeguards in addition to backend policy. -- Execution is time-bounded, cancellable, resource-bounded, and auditable. -- Targets are explicit; commands cannot discover and operate on an unrestricted fleet. -- Arbitrary native device-command passthrough is not supported. - -Target resolution, credentials, connectivity, protocol handling, and vendor translation remain Agent-owned responsibilities. NCM-managed devices and backend-provided connection information are both possible sources and remain an open design choice. - -## Proposed v1 scope - -V1 should validate the end-to-end boundary with a deliberately small catalog: - -- Show interfaces. -- Show routes. -- Show routing neighbors. -- Set an interface administratively up or down. - -These operations exercise cross-vendor reads, structured results, precise authorization, read-only versus remediation enforcement, and Agent-owned device access. Configuration rollback can continue through NCM's existing dedicated action initially. - -Results should default to stable, native-inspired text and optionally support JSON. Raw native output is excluded because it is vendor-dependent, difficult to validate, and may expose unexpected content. - -## rshell integration decision - -The network behavior above does not require one specific rshell implementation. This RFC review should select among three options: - -| Option | Advantages | Tradeoffs | -|---|---|---| -| General extension framework | Reusable by Network Devices, Agent Health, and future Agent domains; uniform authorization, help, telemetry, and execution contracts | Largest initial design; requires a companion RFC; creates a generic privileged extension surface | -| Injected `DeviceExecutor` | Small, explicit, easier to audit, and likely the fastest network vertical slice | Adds network concepts to rshell; couples rshell and Agent releases; does not serve other domains | -| Adapt external-command handling | Builds on an existing rshell embedding point and keeps implementations in the Agent | Existing handler lacks typed metadata and mode semantics; risks blurring reviewed capabilities with arbitrary execution | - -The decision should prioritize preservation of rshell's default-deny model, independent authorization of remediation operations, handler isolation, implementation time, release coupling, auditability, and credible reuse by Agent Health. - -## Decisions requested - -1. Confirm that bounded network investigation and remediation from the Datadog Agent is a capability worth pursuing. -2. Confirm the safety boundary and deliberately small v1 command catalog. -3. Select the rshell integration approach for detailed design and implementation. -4. Identify the initial credential/device provider and the vendors and protocols required for a pilot. - -The full design and supporting analysis are in [the engineering RFC](rshell-networks.md). From 69c455e33f5df4e93f1153ad56f11ba91c7bf328 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 23:14:05 +0200 Subject: [PATCH 13/18] docs: complete network one-pager --- docs/rshell-networks-one-pager.md | 39 +++++++++++++++++++++++++++++++ 1 file changed, 39 insertions(+) diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md index da720ea2..65a68463 100644 --- a/docs/rshell-networks-one-pager.md +++ b/docs/rshell-networks-one-pager.md @@ -5,3 +5,42 @@ Datadog AI agents can use telemetry to detect and reason about network issues, but they cannot actively inspect operational state or remediate affected network devices from within a customer's private environment. This limits an AI agent's ability to diagnose dynamic failures, correlate evidence across devices, respond to vendor-specific behavior, and complete remediation workflows. We propose using rshell as the safe, composable execution interface for network-device investigation and remediation. Running within the Datadog Agent, rshell would expose reviewed operations while preserving bounded execution and local safeguards. + +## What this enables + +An AI agent could investigate an incident by inspecting interface state and errors, routing tables, routing-protocol neighbors, reachability, and relevant configuration state. It could correlate those observations across devices and with Datadog telemetry, then verify whether the network has recovered. + +The longer-term capability would also expose narrowly scoped remediation operations, such as enabling or disabling an interface or restoring a known-good configuration. AI agents would compose reviewed operations rather than receive unrestricted, interactive access to native device CLIs. + +## Why rshell and the Datadog Agent + +rshell already provides a restricted, composable shell designed for AI agents. The Datadog Agent already runs inside customer networks and provides the natural customer-side execution environment. Network Configuration Management (NCM) also demonstrates that the Agent can connect to supported devices, collect configurations, and perform rollback operations. + +Together, these components provide a credible foundation: rshell can define and constrain what an AI agent may request, while the Agent owns private-network access, device connectivity, credentials, protocol handling, and vendor-specific behavior. The exact integration between rshell and Agent-owned operations remains an engineering decision covered by the full RFC. + +## Safety principles + +The capability must preserve the customer's security boundary: + +- Credentials remain hidden from AI agents and rshell programs. +- Operations are explicit, reviewed, and independently authorizable. +- Execution is non-interactive, time-bounded, cancellable, resource-bounded, and auditable. +- Targets are explicit; commands cannot discover and operate on an unrestricted device fleet. +- The Agent enforces local safeguards in addition to backend policy. +- Arbitrary native device-command passthrough is not supported. + +## First milestone + +The first milestone should prove read-only investigation with a small set of representative operations: + +- Show interfaces and interface health. +- Show routes. +- Show routing-protocol neighbors. + +This milestone validates device access, cross-vendor normalization, authorization, structured results, and safe composition without introducing state-changing behavior. After that boundary is proven, a narrowly scoped remediation operation can validate the stronger controls required for changes. + +## Ask + +Align on pursuing rshell-based network investigation and remediation, starting with a read-only investigation milestone. With that alignment, the next step is to select the rshell integration approach, initial device-access mechanism, and vendors and protocols for a pilot. + +The detailed requirements, options, and tradeoffs are in [the engineering RFC](rshell-networks.md). From 83756da376eb73b24554e711b9a081a85fe81965 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 23:24:59 +0200 Subject: [PATCH 14/18] docs: refine network one-pager milestones --- docs/rshell-networks-one-pager.md | 27 +++++++++++---------------- 1 file changed, 11 insertions(+), 16 deletions(-) diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md index 65a68463..678cde19 100644 --- a/docs/rshell-networks-one-pager.md +++ b/docs/rshell-networks-one-pager.md @@ -4,17 +4,17 @@ Datadog AI agents can use telemetry to detect and reason about network issues, but they cannot actively inspect operational state or remediate affected network devices from within a customer's private environment. This limits an AI agent's ability to diagnose dynamic failures, correlate evidence across devices, respond to vendor-specific behavior, and complete remediation workflows. -We propose using rshell as the safe, composable execution interface for network-device investigation and remediation. Running within the Datadog Agent, rshell would expose reviewed operations while preserving bounded execution and local safeguards. +We propose using rshell as the safe, composable execution interface for network-device investigation and remediation. Running within the Datadog Agent, rshell would expose restricted operations while preserving bounded execution and local safeguards. ## What this enables An AI agent could investigate an incident by inspecting interface state and errors, routing tables, routing-protocol neighbors, reachability, and relevant configuration state. It could correlate those observations across devices and with Datadog telemetry, then verify whether the network has recovered. -The longer-term capability would also expose narrowly scoped remediation operations, such as enabling or disabling an interface or restoring a known-good configuration. AI agents would compose reviewed operations rather than receive unrestricted, interactive access to native device CLIs. +The longer-term capability would also expose narrowly scoped remediation operations, such as enabling or disabling an interface or changing configurations. AI agents would compose restricted operations rather than receive unrestricted, interactive access to native device CLIs. ## Why rshell and the Datadog Agent -rshell already provides a restricted, composable shell designed for AI agents. The Datadog Agent already runs inside customer networks and provides the natural customer-side execution environment. Network Configuration Management (NCM) also demonstrates that the Agent can connect to supported devices, collect configurations, and perform rollback operations. +rshell already provides a restricted, composable shell designed for AI agents. The Datadog Agent already runs inside customer networks and provides the natural customer-side execution environment. Network Configuration Management (NCM) also demonstrates that the Agent can connect to supported devices. Together, these components provide a credible foundation: rshell can define and constrain what an AI agent may request, while the Agent owns private-network access, device connectivity, credentials, protocol handling, and vendor-specific behavior. The exact integration between rshell and Agent-owned operations remains an engineering decision covered by the full RFC. @@ -24,23 +24,18 @@ The capability must preserve the customer's security boundary: - Credentials remain hidden from AI agents and rshell programs. - Operations are explicit, reviewed, and independently authorizable. -- Execution is non-interactive, time-bounded, cancellable, resource-bounded, and auditable. -- Targets are explicit; commands cannot discover and operate on an unrestricted device fleet. -- The Agent enforces local safeguards in addition to backend policy. +- The Agent enforces local safeguards (only expose subset of device commands) in addition to backend policy. - Arbitrary native device-command passthrough is not supported. -## First milestone +## Milestones -The first milestone should prove read-only investigation with a small set of representative operations: +### M1: Read-only device investigation via rshell -- Show interfaces and interface health. -- Show routes. -- Show routing-protocol neighbors. +Example operations: +- Show interfaces +- Show routes -This milestone validates device access, cross-vendor normalization, authorization, structured results, and safe composition without introducing state-changing behavior. After that boundary is proven, a narrowly scoped remediation operation can validate the stronger controls required for changes. +This milestone validates device access, cross-vendor normalization and safe composition via rshell. -## Ask +### M2: Remediation device investigation via rshell -Align on pursuing rshell-based network investigation and remediation, starting with a read-only investigation milestone. With that alignment, the next step is to select the rshell integration approach, initial device-access mechanism, and vendors and protocols for a pilot. - -The detailed requirements, options, and tradeoffs are in [the engineering RFC](rshell-networks.md). From 53fe5a4a6f4b0e377a41bfff739d9321dba0c164 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 23:32:15 +0200 Subject: [PATCH 15/18] docs: expand read-only network examples --- docs/rshell-networks-one-pager.md | 9 ++++++--- 1 file changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md index 678cde19..051a4d55 100644 --- a/docs/rshell-networks-one-pager.md +++ b/docs/rshell-networks-one-pager.md @@ -32,10 +32,13 @@ The capability must preserve the customer's security boundary: ### M1: Read-only device investigation via rshell Example operations: -- Show interfaces -- Show routes + +- Show interfaces and errors. +- Show routes. +- Show routing-protocol neighbors. +- Show ARP and MAC address tables. +- Test reachability from the device. This milestone validates device access, cross-vendor normalization and safe composition via rshell. ### M2: Remediation device investigation via rshell - From fa2a5a8dbe003463f829125fa0c8da9284b9831a Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 23:36:58 +0200 Subject: [PATCH 16/18] docs: add network use case scripts --- docs/rshell-networks-one-pager.md | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+) diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md index 051a4d55..28d4c44e 100644 --- a/docs/rshell-networks-one-pager.md +++ b/docs/rshell-networks-one-pager.md @@ -6,6 +6,33 @@ Datadog AI agents can use telemetry to detect and reason about network issues, b We propose using rshell as the safe, composable execution interface for network-device investigation and remediation. Running within the Datadog Agent, rshell would expose restricted operations while preserving bounded execution and local safeguards. +## Use cases + +The device-command syntax below is illustrative; the examples show how rshell composition supports the workflow. + +### Investigation + +An AI agent follows up on a Datadog alert by inspecting live state across relevant devices to identify the failing interface, route, neighbor, or network path. + +```sh +for device in edge-router-1 core-router-1; do + echo "== $device ==" + device interfaces show "$device" | grep -E 'down|error|drop' + device routing-neighbors show "$device" | grep -v established + device routes show "$device" --destination 10.20.0.0/16 +done +``` + +### Remediation + +An AI agent applies an authorized, narrowly scoped change to the affected device, then verifies that connectivity and device state have recovered. + +```sh +device interface set-state edge-router-1 Ethernet1 up && + device interfaces show edge-router-1 --interface Ethernet1 && + device reachability test edge-router-1 10.20.0.1 +``` + ## What this enables An AI agent could investigate an incident by inspecting interface state and errors, routing tables, routing-protocol neighbors, reachability, and relevant configuration state. It could correlate those observations across devices and with Datadog telemetry, then verify whether the network has recovered. From 85792a00b064be28400c709c7b92109e4c5af158 Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Sun, 12 Jul 2026 23:55:55 +0200 Subject: [PATCH 17/18] docs: use target-first device syntax --- docs/rshell-networks-one-pager.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md index 28d4c44e..615f4768 100644 --- a/docs/rshell-networks-one-pager.md +++ b/docs/rshell-networks-one-pager.md @@ -17,9 +17,9 @@ An AI agent follows up on a Datadog alert by inspecting live state across releva ```sh for device in edge-router-1 core-router-1; do echo "== $device ==" - device interfaces show "$device" | grep -E 'down|error|drop' - device routing-neighbors show "$device" | grep -v established - device routes show "$device" --destination 10.20.0.0/16 + device --target "$device" show interfaces | grep -E 'down|error|drop' + device --target "$device" show routing-neighbors | grep -v established + device --target "$device" show routes --destination 10.20.0.0/16 done ``` @@ -28,9 +28,9 @@ done An AI agent applies an authorized, narrowly scoped change to the affected device, then verifies that connectivity and device state have recovered. ```sh -device interface set-state edge-router-1 Ethernet1 up && - device interfaces show edge-router-1 --interface Ethernet1 && - device reachability test edge-router-1 10.20.0.1 +device --target edge-router-1 set interface-state Ethernet1 up && + device --target edge-router-1 show interfaces --interface Ethernet1 && + device --target edge-router-1 test reachability 10.20.0.1 ``` ## What this enables From 1c06a7f97b368f986a93d4f0605922576b51e32f Mon Sep 17 00:00:00 2001 From: Alexandre Yang Date: Mon, 13 Jul 2026 00:03:34 +0200 Subject: [PATCH 18/18] docs: improve network use case examples --- docs/rshell-networks-one-pager.md | 24 +++++++++++++++++------- 1 file changed, 17 insertions(+), 7 deletions(-) diff --git a/docs/rshell-networks-one-pager.md b/docs/rshell-networks-one-pager.md index 615f4768..6bebb45e 100644 --- a/docs/rshell-networks-one-pager.md +++ b/docs/rshell-networks-one-pager.md @@ -16,10 +16,10 @@ An AI agent follows up on a Datadog alert by inspecting live state across releva ```sh for device in edge-router-1 core-router-1; do - echo "== $device ==" - device --target "$device" show interfaces | grep -E 'down|error|drop' - device --target "$device" show routing-neighbors | grep -v established - device --target "$device" show routes --destination 10.20.0.0/16 + echo == $device == + device --target $device show interfaces | grep -E 'down|error|drop' + device --target $device show routing-neighbors | grep -v established + device --target $device show routes --destination 10.20.0.0/16 done ``` @@ -28,9 +28,19 @@ done An AI agent applies an authorized, narrowly scoped change to the affected device, then verifies that connectivity and device state have recovered. ```sh -device --target edge-router-1 set interface-state Ethernet1 up && - device --target edge-router-1 show interfaces --interface Ethernet1 && - device --target edge-router-1 test reachability 10.20.0.1 +target=edge-router-1 +interface=Ethernet1 +destination=10.20.0.1 + +echo Enabling $interface on $target +if device --target $target set interface-state $interface up && + device --target $target show interfaces --interface $interface | grep -q 'up' && + device --target $target test reachability $destination; then + echo Remediation succeeded +else + echo Remediation failed + exit 1 +fi ``` ## What this enables