From c902cbe57461e540fdbac0d9a6d2cd984c0c5eb0 Mon Sep 17 00:00:00 2001 From: Paul Reilly Date: Thu, 2 Jul 2026 12:57:16 -0400 Subject: [PATCH 1/8] Update the skills to support creating and using markers --- .claude-plugin/marketplace.json | 2 +- dtwo/.claude-plugin/plugin.json | 2 +- dtwo/skills/dtwo-gateway-policy/SKILL.md | 209 +++++++++++++++++++++-- dtwo/skills/dtwo-policy-rego/SKILL.md | 124 +++++++++++++- 4 files changed, 319 insertions(+), 18 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index 11bb679..b621aea 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -9,7 +9,7 @@ "name": "dtwo", "source": "./dtwo", "description": "DTwo MCP gateway management — bundles the DTwo MCP server and skills for managing gateway configs, deploys, and policies", - "version": "1.0.2", + "version": "1.0.3", "category": "security", "keywords": [ "agentic", diff --git a/dtwo/.claude-plugin/plugin.json b/dtwo/.claude-plugin/plugin.json index 3616378..d7de6fc 100644 --- a/dtwo/.claude-plugin/plugin.json +++ b/dtwo/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "dtwo", - "version": "1.0.2", + "version": "1.0.3", "description": "Manage DTwo gateways, policies, and Rego with the DTwo MCP server.", "author": { "name": "DTwo", diff --git a/dtwo/skills/dtwo-gateway-policy/SKILL.md b/dtwo/skills/dtwo-gateway-policy/SKILL.md index 457488b..9b15991 100644 --- a/dtwo/skills/dtwo-gateway-policy/SKILL.md +++ b/dtwo/skills/dtwo-gateway-policy/SKILL.md @@ -4,10 +4,13 @@ description: | Create, validate, attach, publish, deploy, verify, and roll back DTwo policies and their pipeline attachments. This is the system-of-record skill: it owns the DTwo MCP tools for policy lifecycle (create, update, publish, revert) and pipeline lifecycle (attach, deploy, verify), and is responsible for confirming - pipeline attachments before and after deployment. + pipeline attachments before and after deployment. Also manages the session-state marker registry, and + the intent registry when the intent tools are enabled. TRIGGER when: user says "create/add a policy", "modify/update a policy", "block/allow/redact something", "attach/detach policy", "set/update pipeline", "publish/pin policy version", "deploy gateway" after a - policy change. Always pair with dtwo-policy-rego for the Rego authoring/modification step. + policy change; also "register/create/manage a marker", "marker registry", or (only when the intent tools + are enabled) "intent capture", "intent registry", "intent transitions", "intent/marker compatibility". + Always pair with dtwo-policy-rego for the Rego authoring/modification step. SKIP when: task is purely *explaining* existing Rego with no save/attach/deploy intent (use dtwo-policy-rego alone); task is editing gateway YAML or MCP server entries (use dtwo-gateway-config). --- @@ -25,6 +28,41 @@ This skill is typically used alongside others. Invoke them via the `Skill` tool - **dtwo-policy-rego** — load at the start of any task that requires writing, modifying, or explaining Rego. Almost always load this together with `dtwo-gateway-policy` unless the task is pure pipeline attachment of an already-authored policy. - **dtwo-gateway-config** — load when the task also involves editing gateway YAML or adding/removing MCP server entries. +## Core Concepts + +Before choosing an approach, understand how the pieces relate. From smallest to largest: + +- **Policy** — a single unit of Rego that makes one decision about one tool call: allow it, deny it, transform the request/response, and/or write a marker. Policies are authored with the companion `dtwo-policy-rego` skill and stored as records with a draft plus published versions. **A policy on its own is inert** — it does nothing until it is attached to a gateway and deployed. +- **Pipeline** — the ordered list of policies attached to a gateway in one direction. Each gateway has two: an **ingress** pipeline that runs *before* a tool call reaches the upstream server (inspect arguments, identity; block or rewrite the request), and an **egress** pipeline that runs *after* the tool returns (inspect the response; block or redact it). Steps run in array order, and an earlier deny short-circuits later steps — so ordering matters. +- **Gateway** — the runtime that fronts one or more upstream MCP servers and enforces its pipelines. Attaching or editing policies changes only stored state; a **deploy** is what makes the current pipelines live. +- **Marker** — a session-state flag one policy writes and another reads, letting policies coordinate *across* tool calls, directions, and upstream servers within a session (e.g. "PII was seen in an earlier response → block outbound sends now"). A marker is defined once in the registry, then used by a writer policy and a reader policy. See Managing Markers. +- **Intent** *(only when the intent tools are enabled)* — a declared session *purpose* captured into session state and gated on by policies. Built on the same session-state mechanism as markers, with its own registry. See Intent Capture, including its availability gate. + +**Mental model:** *policies* are the decision logic; the *pipeline* is where and when they run; the *gateway* is what enforces them once deployed; *markers* and *intent* are how policies share context beyond a single call. + +**Which skill does what:** this skill (`dtwo-gateway-policy`) owns the lifecycle and orchestration — policy records, pipelines, marker/intent registries, deploy, and verify. The companion `dtwo-policy-rego` skill owns the Rego logic *inside* a policy. Most authoring tasks use both. + +## Choosing the Right Approach + +Match the user's goal to the **smallest** mechanism that solves it, then follow that section: + +| The user wants to… | Use | Where | +|---|---|---| +| Block, allow, or restrict a tool call based on the call itself (tool name, arguments, caller identity) | One **ingress policy** (deny) | Creating a New Policy | +| Block or redact a response based on its content | One **egress policy** (deny or transform) | Creating a New Policy | +| Rewrite a tool's arguments before it runs (e.g. force a filter) | One **ingress transform policy** | Creating a New Policy + `dtwo-policy-rego` | +| Make a decision that depends on something earlier in the session (a prior tool, a prior response, another upstream server) | A **marker** — a writer policy stamps it, a reader policy gates on it | Managing Markers | +| Gate tools on *what the agent is currently doing* | **Intent** *(only if the intent tools are enabled — otherwise not available; do not offer it)* | Intent Capture | +| Turn a policy on/off, reorder it, or pin a version at runtime | **Pipeline attachment** + deploy | Pipeline Attachment | +| Stop a policy's runtime effect, or remove it entirely | **Detach** + redeploy (then `dtwo-delete-policy` only if the record should also go) | Deleting a Policy | + +Guidance: + +- **Prefer one policy.** If a single call carries everything needed to decide, a single ingress or egress policy is the answer — don't reach for markers. +- **Markers are for cross-call state**, not for anything decidable from the current call alone. They add a second policy and a registry entry, so use them only when the decision genuinely depends on earlier session activity. +- **Intent is for session-purpose gating** and is only available when the intent tools are present. If they are not, solve the request with policies and markers and do not mention intent. +- **Compose small policies over one large one** — the companion `dtwo-policy-rego` skill explains why (single-concern policies are easier to test, order, and debug). + ## Prerequisites This skill requires the DTwo MCP server to be connected (`dtwo-*` tools must be loaded). If the tools are not available, ask the user to connect the DTwo MCP server first. @@ -60,34 +98,63 @@ The tools listed below reflect the initial set. The DTwo MCP server may add new | `dtwo-get-policy` | Fetch a single policy by UID (includes draft Rego code) | | `dtwo-get-policy-versions` | List published versions for a policy | | `dtwo-validate-policy-rego` | Validate Rego code without saving — useful for dry-run checks before committing changes (note: `dtwo-add-policy` and `dtwo-update-policy` also validate automatically) | -| `dtwo-add-policy` | Validate and create a new policy (requires name, description, policy, packageName, direction) | -| `dtwo-update-policy` | Update an existing policy's draft — any field (policy, packageName, name, description, direction, tags). Validates Rego when both policy and packageName are provided | +| `dtwo-add-policy` | Validate and create a new policy (requires name, description, policy, packageName, direction). Optionally pass `writableKeySchema` to declare the session-state keys the policy is authorized to write — required for any policy that emits a marker (see Managing Markers) | +| `dtwo-update-policy` | Update an existing policy's draft — any field (policy, packageName, name, description, direction, tags, `writableKeySchema`). Validates Rego when both policy and packageName are provided. `writableKeySchema` is tri-state: omit → leave unchanged, `null` → clear, `[]` → explicit-empty, `[...]` → set | | `dtwo-publish-policy` | Publish the current draft as a new version | | `dtwo-revert-policy` | Restore a published version back into the draft | +| `dtwo-delete-policy` | Permanently delete a policy by UID. Fails if the policy is still attached to one or more gateways — detach it from every gateway first (see Deleting a Policy). Distinct from `dtwo-revert-policy`, which only restores a prior version | | `dtwo-list-claims` | Return the union of JWT claim names observed across the tenant, plus the issuers seen. Defaults to tenant-wide; pass `gatewayUid` to scope to a single gateway when the user asks. Call this when authoring or modifying identity-aware policies so rules can reference claims that actually exist; skip for policies that don't read `input.subject.claims`. | +### Marker Registry Tools + +Markers are session-state flags that one policy writes and other policies read to gate on (see Managing Markers). These tools are **always registered** on the DTwo MCP server — they do not depend on any feature flag. + +| Tool | Purpose | +|------|---------| +| `dtwo-list-markers` | List markers in the registry (optional filters: `name` for exact FQID, `tag`). Returns the marker *vocabulary*, not which markers are currently active on a session | +| `dtwo-get-marker` | Fetch a single marker by UID | +| `dtwo-create-marker` | Create a customer-tier marker (requires `namespace`, `markerId`, `description`, `minimumTtlSeconds`; optional `tags`). Full key is `marker::`. `internal` and `dtwo` namespaces are reserved for platform markers | +| `dtwo-update-marker` | Update mutable fields on a customer-tier marker (`description`, `tags`, `minimumTtlSeconds`) | +| `dtwo-delete-marker` | Delete a customer-tier marker. Platform-managed entries cannot be deleted | + +### Intent Registry Tools (conditional — feature-gated) + +> **Availability gate — read this before surfacing anything about intents.** The intent tools below are only registered when the DTwo MCP server is deployed with `enable_intent_tools: true`. **Marker tools (above) are always available; intent tools are not.** Before mentioning intent capture, intent registries, transitions, or intent/marker compatibility to the user, confirm the relevant `dtwo-*-intent*` tools are actually present in your available tool list. **If they are absent, the server is not configured for intent capture — do not present intent capture, the intent registry, transitions, or compatibility to the user, and do not attempt to call these tools.** Treat this subsection and the "Intent Capture" section below as inert in that case. Markers work fully without intent capture, so continue to use them normally. + +When present, these tools manage the intent vocabulary and the rules that govern it. See the Intent Capture section for the workflow. + +| Tool | Purpose | +|------|---------| +| `dtwo-set-intent` | Declare the current session intent (the working intent captured by the gateway's egress capture policy). Accepts only intents registered in the registry | +| `dtwo-list-intents` | List intents in the registry (platform `system=true` entries are read-only; customer-tier entries are tenant-scoped) | +| `dtwo-get-intent` | Fetch a single intent by UID | +| `dtwo-create-intent` / `dtwo-update-intent` / `dtwo-delete-intent` | Manage customer-tier intents in the registry vocabulary | +| `dtwo-list-intent-transitions` / `dtwo-set-intent-transition-mode` / `dtwo-add-intent-transition` / `dtwo-delete-intent-transition` | Govern which intent→intent moves are allowed | +| `dtwo-list-intent-compatibility` / `dtwo-create-intent-compatibility` / `dtwo-delete-intent-compatibility` | Govern which markers block which intents at `set_intent` time. `dtwo-create-intent-compatibility` takes `intentUid` + `excludedMarkerUid` | + ### Pipeline & Gateway Tools | Tool | Purpose | |------|---------| | `dtwo-list-gateways` | List gateways with optional filters (name, status, uid) | | `dtwo-get-gateway` | Fetch a single gateway by UID | +| `dtwo-get-gateway-config` | Fetch the gateway's YAML configuration. Used here **read-only** to discover `mcp_servers[].name` and tool names when authoring policies (see Tool Discovery). Editing gateway YAML belongs to the companion `dtwo-gateway-config` skill | | `dtwo-set-gateway-pipelines` | Attach policies to ingress/egress pipelines | | `dtwo-get-gateway-pipelines` | Fetch ingress and egress pipeline steps for a gateway, including policy details | | `dtwo-deploy-gateway` | Queue a deployment for the gateway | | `dtwo-get-gateway-deployments` | List deployment tasks for a gateway | | `dtwo-get-deployment` | Check status of a specific deployment | -### Deletion (not supported via MCP) +### Deleting a Policy -The DTwo MCP server does not expose a `delete-policy` tool. `revert-policy` restores a prior version — it does **not** delete. +`dtwo-delete-policy` performs a **permanent** delete by UID. This is different from `dtwo-revert-policy`, which only restores a prior version into the draft — it does **not** delete. -When the user asks to delete a policy: +The delete **fails if the policy is still attached to any gateway**, so detach it everywhere first: -1. **Detach first** — remove the policy from all pipelines with `dtwo-set-gateway-pipelines` (pass `[]` to clear the relevant direction), then redeploy. A detached policy remains in the policy list but has no runtime effect. -2. **Delete via the DTwo web UI** — the MCP surface does not offer deletion. +1. **Detach first** — remove the policy from all pipelines with `dtwo-set-gateway-pipelines` (pass `[]` to clear the relevant direction, or re-send the direction's steps without this policy), then redeploy each affected gateway. A detached policy remains in the policy list but has no runtime effect. +2. **Delete** — call `dtwo-delete-policy` with the `uid`. If it errors that the policy is still attached, a detach was missed (or a deploy hasn't landed) — re-check attachments with `dtwo-get-gateway-pipelines` before retrying. -If a `dtwo-delete-policy` tool later appears (see the tool-discovery note under Prerequisites), prefer it over this workaround. +Deletion is irreversible and confirmation-worthy — confirm with the user before calling `dtwo-delete-policy`. If they only want to stop the policy's runtime effect (not remove the record), detaching and redeploying is sufficient; leave the policy in place. ## Identifying the Target Gateway @@ -364,10 +431,130 @@ After publishing, call `dtwo-set-gateway-pipelines` again with `policyVersion: 1 **Do not skip step 14.** Leaving the attachment on the draft does not take effect immediately, but the current draft state will be bundled into the *next* deploy of that gateway, whoever triggers it and whatever the reason. A later `dtwo-update-policy` edit — even an experimental one — will then go live on a deploy that was meant for an unrelated change. Pinning to a published version freezes runtime behavior against future draft edits. +## Managing Markers + +Markers are session-state flags that policies write and later policies read to gate on. They give the gateway a shared, tenant-scoped, TTL-bounded "notepad" that survives across tool calls and across upstream MCP servers — a marker written during a Slack call is visible during the next Jira call in the same session. Use them to compose small single-purpose policies that signal to each other without shared code: a **writer** policy stamps a marker when it observes something (PII in a response, a production resource touched), and a **reader** policy on a different tool/pipeline/server gates on it. + +Marker tools are always available (they do not require `enable_intent_tools`). The full lifecycle — register, author writer + reader, attach, deploy — runs through this skill plus `dtwo-policy-rego` for the Rego. The Rego authoring patterns (emitting `session_writes["marker::"]`, walking `input.context.session.policies` to read, and the `writableKeySchema` gotchas) live in the companion `dtwo-policy-rego` instructions — load that skill for the writer/reader bodies. + +### Registering a marker + +Register the marker in the vocabulary before any policy references it: + +``` +dtwo-create-marker( + namespace = "acme", + markerId = "pii_detected", + description = "Session received PII in a tool response", + minimumTtlSeconds = 3600 +) +``` + +- The full key is `marker:acme:pii_detected`. Customer markers live under any namespace except the reserved `internal` and `dtwo`. +- `minimumTtlSeconds` is a **floor** — a writer policy may declare any TTL ≥ this. Raise it later with `dtwo-update-marker` if a writer needs a longer minimum. + +### Authoring the writer policy — `writableKeySchema` + +A policy that emits a marker must declare the key in its `writableKeySchema` (on `dtwo-add-policy` / `dtwo-update-policy`), or the gateway drops the write. Each entry is: + +- `name` — the session-state key, matching the registered marker exactly (e.g. `marker:acme:pii_detected`). +- **Marker key format enforcement.** The tool boundary rejects `writableKeySchema[].name` values starting with `marker:` unless they match `^marker:[A-Za-z0-9][A-Za-z0-9_-]*:[A-Za-z0-9][A-Za-z0-9_-]*$` — exactly two colon-separated segments of alphanumerics, underscores, or hyphens, each segment starting with an alphanumeric (case-insensitive). Bare keys (no `marker:` prefix) are structurally validated by the backend instead. Registry *existence* — that the key is actually in the marker registry — is enforced separately at deploy time by the `marker-not-in-registry` rule (the deploy fails with `UnknownMarkerReferences`; see the Deploy-time validator note below). +- `jsonSchema` — a **stringified JSON object** (a JSON Schema) for the value the policy writes. Rejected at the tool boundary if it doesn't parse as a JSON object (arrays and primitives fail). Use it strictly (`additionalProperties: false`, `required` lists) so drift is caught. Add `"x-d2-is-marker": true` for marker keys. +- `ttlSeconds` — per-key TTL. For a marker key this must be **≥ the marker's registered `minimumTtlSeconds`**; a lower value is rejected at policy save time. +- `onDrop` — behavior when a write fails the schema: `"drop"` (default) silently drops the write (best-effort markers); `"deny_request"` hard-denies the tool call (use for security-critical writes so bugs surface loudly instead of silently letting the call through). + +``` +dtwo-add-policy( + name = "acme-pii-detector", + direction = "egress", + packageName = "acme.egress.pii_detector", + policy = , + writableKeySchema = [{ + name: "marker:acme:pii_detected", + jsonSchema: "{\"type\":\"object\",\"required\":[\"marked_at\",\"source_action\"],\"properties\":{\"marked_at\":{\"type\":\"integer\",\"minimum\":0},\"source_action\":{\"type\":\"string\",\"minLength\":1}},\"x-d2-is-marker\":true,\"additionalProperties\":false}", + ttlSeconds: 3600, + onDrop: "deny_request" + }] +) +``` + +### Attaching, deploying, and reading + +1. Author the reader policy (walks `input.context.session.policies` for the marker key — see `dtwo-policy-rego`). The reader needs no `writableKeySchema`; it only reads. +2. Attach both with `dtwo-set-gateway-pipelines` — the writer on the direction that observes the signal (often egress), the reader on the direction that gates (often ingress). Preserve existing steps. +3. Deploy with `dtwo-deploy-gateway`. This is a **policy-only deploy** — hot-reloaded, no gateway restart, no MCP client disconnect (see Deploying). + +**Deploy-time validator.** The deploy hard-rejects (`UnknownMarkerReferences`, the `marker-not-in-registry` rule) if any attached policy declares a `writableKeySchema` marker key that isn't in the registry. This is separate from the tool-boundary *format* check on the key (see Authoring the writer policy) — the format check runs at save time, the registry-existence check at deploy time. Register the marker *before* attaching a policy that writes it. + +### Verifying a marker pipeline + +Markers can't be verified the way a single policy can — there is no tool to read a session's active markers (see Marker constraints today), so verification is **behavioral, session-scoped, and order-dependent**. A marker does nothing until its writer fires, and its effect is only visible through the reader's decision: + +1. **Confirm the deploy and attachment** as for any policy — poll `dtwo-get-deployment` to `completed`, then `dtwo-get-gateway-pipelines` to confirm both the writer and the reader landed with the expected `evalNamespace` and version pins. +2. **Trigger the writer first, in one session.** Make the tool call that satisfies the writer's condition (e.g. a response containing PII). This is what stamps the marker — nothing is active until the writer fires. +3. **Then exercise the reader in that same session.** Call the reader's guarded tool and confirm it now denies (or transforms) as intended. Because markers are session-scoped, this only proves out within the session where the writer fired. +4. **Confirm the negative case in a fresh session.** With no writer having fired (or after the TTL expires), the reader's tool should succeed — proving the reader blocks only when the marker is active, not unconditionally. + +**Tip — validate with a short TTL, then raise it.** A production-length TTL (say an hour) makes iterating painful: a marker stamped in one test lingers and masks the next. During validation, give the marker a short TTL (e.g. 30–60s) so it self-clears between iterations and you can retest in the same session without waiting it out or starting fresh. The written TTL is the writer policy's `writableKeySchema.ttlSeconds`, and it must be ≥ the marker's registered `minimumTtlSeconds` — so lower **both** for testing: set the marker's `minimumTtlSeconds` low (`dtwo-create-marker`, or `dtwo-update-marker` if it already exists) and set the writer's `ttlSeconds` to match. Once the pipeline is validated, raise the writer's `ttlSeconds` to the desired length (`dtwo-update-policy`) and the marker's `minimumTtlSeconds` floor if you want to enforce it (`dtwo-update-marker`), then republish/redeploy. + +Watch for these: + +- **Order and session matter.** Calling the reader before the writer has fired, or in a different session, shows the marker as absent and the reader allowing — that is correct behavior, not a bug. Sequence the calls within one session. +- **A stale marker can mask a result.** If an earlier call already stamped the marker and its TTL hasn't expired, the reader keeps denying. Use a fresh session for a clean negative test, or test with a short TTL (see the tip above) so it clears on its own between iterations. +- **`onDrop: "deny_request"` surfaces schema problems as a denied *writer* call.** If the tool that should stamp the marker is itself denied, the written value likely failed its `writableKeySchema` (e.g. a float timestamp against a `type: integer` field — see the `time.now_ns()` gotcha in `dtwo-policy-rego`). Fix the value shape, not the reader. +- **To see the marker directly while debugging,** attach a temporary reader-side debug policy that dumps `input.context.session.policies` in a deny reason — the marker analog of the dump-input technique in `dtwo-policy-rego` (Debugging Policies). Detach it when done. + +### Cleanup order (reverse of setup) + +Skipping a step makes the next deploy fail with `UnknownMarkerReferences` (a policy still claims to write a marker that no longer exists). Tear down in reverse: + +1. Update/remove the **writer policy** so it no longer references the marker in `writableKeySchema`; redeploy so the write contract leaves the bundle. +2. Delete any **intent/marker compatibility** rows that reference the marker (only relevant when intent tools are enabled — `dtwo-delete-intent-compatibility`); redeploy. +3. `dtwo-delete-marker` — nothing references it now. (`dtwo-delete-marker` does **not** currently check for policy references, so it can leave the bundle inconsistent if you skip step 1.) + +### Marker constraints today + +- **No "list active markers" tool.** `dtwo-list-markers` returns the registry *vocabulary* (the markers that are defined), not which markers are currently set on a given session. A policy can read active markers at evaluation time via `input.context.session.policies` (that's how reader policies work), but there is no MCP tool to query a session's live marker state on demand. +- **No "clear marker" tool.** Markers lift on their own when their TTL expires; there is no MCP tool to unset one mid-session. To recover from a marker that is blocking a session, wait out the TTL or start a new session. +- **Multiple writers land in separate per-writer slots.** If two policies declare and emit the same marker key, each write lands under its own writer UID; readers get "any-writer" semantics by walking `session.policies.*`. Prefer one canonical writer per marker. + +## Intent Capture (conditional — feature-gated) + +> **Availability gate — read this first.** Everything in this section depends on the DTwo MCP server being deployed with `enable_intent_tools: true`, which registers the `dtwo-set-intent` / `dtwo-*-intent*` tools listed under Intent Registry Tools. **Before presenting any of this to the user, confirm those tools are in your available tool list. If they are not, do not surface intent capture, the intent registry, transitions, or intent/marker compatibility — the deployment is not configured for it. Say only that intent capture is not enabled on this server if the user asks; do not walk them through a workflow they cannot run.** Markers (above) are unaffected and remain fully usable. + +Intent capture lets the agent declare *what it's trying to do* (`dtwo-set-intent`), captures that into session state via an egress policy, and lets ingress policies gate downstream tools on the current intent. It builds on the same session-state mechanism as markers. + +**Status.** This ships as **starter policies**, not an auto-injected platform feature — you attach them to a pipeline as drafts; nothing is auto-deployed. Registry-management tools are expected to stay on this server; `dtwo-set-intent` itself may move to a dedicated server later. + +### The two starter policies + +- **Egress capture** (package `set_intent.egress.intent_capture`, egress) — captures the declared intent into session state when `dtwo-set-intent` is invoked. Validates the proposal against the intent registry, normalizes the stored category, denies disallowed transitions, and denies when the registry marks the proposed intent incompatible with a marker currently active in the session (`intent_marker_incompatible`). +- **Intent-required gate** (package `set_intent.ingress.intent_required`, ingress) — **optional.** Denies every tool call until an intent has been set; `dtwo-set-intent` itself is always allowed so the agent can declare. Attach only when you want the gate enforced. + +The Rego bodies and the two coupling requirements below are detailed in `dtwo-policy-rego` (Intent-capture policies). The two requirements that block them silently if missed: + +- **Upstream server must be named `Dtwo`.** Both policies fire on tool names case-folding to `dtwo-set-intent` / `dtwo-set_intent`. The DTwo MCP upstream entry in the gateway's `mcp_servers` config **must** be named `Dtwo` (anything case-folding to `dtwo`). A different name silently bypasses capture and deadlocks the ingress gate. +- **UID placeholder swap.** Both policies share the capture policy's own UID (`REPLACE-WITH-INTENT-CAPTURE-POLICY-UID`). After `dtwo-add-policy` returns the real UID, replace the placeholder in both bodies and re-save. Until swapped, the ingress gate denies every non-`set_intent` call and the egress capture treats every set as first-set (no transition enforcement). + +### Intent/marker compatibility + +If a marker should block switching into a given intent, register a compatibility row so the egress capture denies `set_intent` while that marker is active: + +``` +dtwo-create-intent-compatibility( + intentUid = , + excludedMarkerUid = +) +``` + +Example: once `marker:acme:pii_detected` is set, a `set_intent` to `incident_response` is blocked — the session already touched sensitive data. + +**Set-time enforcement only.** The check runs at `set_intent` time. A marker raised *after* an intent is set does **not** retroactively invalidate the current intent. Markers accumulate; intents are validated at the decision point. Tell users this plainly so they don't design around a symmetric re-check that doesn't exist. + ## Limitations - This skill cannot author or modify Rego policies — see the companion `dtwo-policy-rego` instructions - This skill cannot edit gateway YAML or add/remove MCP server entries — see the companion `dtwo-gateway-config` instructions -- This skill cannot delete a policy via the MCP surface — detach via `dtwo-set-gateway-pipelines`, then delete in the DTwo web UI +- This skill cannot delete a policy that is still attached to a gateway — detach via `dtwo-set-gateway-pipelines` and redeploy first, then delete with `dtwo-delete-policy` (see Deleting a Policy) - This skill cannot evaluate policies outside a deployed gateway — verification requires live tool calls against the running gateway - This skill cannot retrieve runtime evaluation logs or OPA decision history from the MCP surface diff --git a/dtwo/skills/dtwo-policy-rego/SKILL.md b/dtwo/skills/dtwo-policy-rego/SKILL.md index 423be3f..e0be013 100644 --- a/dtwo/skills/dtwo-policy-rego/SKILL.md +++ b/dtwo/skills/dtwo-policy-rego/SKILL.md @@ -6,10 +6,11 @@ description: | catalog contribution structure. TRIGGER when: user asks to write/modify/explain/debug a Rego policy, says "block/allow/redact/transform" a tool call or response, mentions OPA, package paths, `input.payload`, `default allow`, or pastes Rego - for review; asks to contribute to `dtwoai/policy-store`, create catalog policy files, or update app, - industry, bundle, manifest, or tests files for reusable DTwo policies; also when diagnosing blanket denies - or transform conflicts. Pair with dtwo-gateway-policy whenever the resulting Rego must be saved, attached, - or deployed. + for review; asks to write a marker writer/reader policy, `session_writes`, session state, or (only when the + intent tools are enabled) an intent-capture policy; asks to contribute to `dtwoai/policy-store`, create + catalog policy files, or update app, industry, bundle, manifest, or tests files for reusable DTwo policies; + also when diagnosing blanket denies or transform conflicts. Pair with dtwo-gateway-policy whenever the + resulting Rego must be saved, attached, or deployed. SKIP when: task is policy CRUD or pipeline attachment that does not change Rego (use dtwo-gateway-policy alone); task is general OPA usage outside the MCP Gateway; task is editing gateway YAML (use dtwo-gateway-config). --- @@ -20,6 +21,15 @@ description: | You are a Rego policy expert for the DTwo MCP Gateway. You translate natural language security requirements into valid Rego policies, explain existing policies in plain language, and modify policies based on instructions. +## Where a policy fits + +This skill owns the Rego *inside* a policy — the allow / deny / transform / `session_writes` logic for a single tool call. The surrounding pieces it plugs into (a **pipeline** is the ordered list of policies on a gateway direction; a **gateway** enforces them once deployed; **markers** and **intent** let policies share session state) are owned by the companion `dtwo-gateway-policy` skill — see its Core Concepts and Choosing the Right Approach sections to pick the right mechanism before writing Rego. A quick orientation for what you write here: + +- **Deny / allow** — decide a single call from the call itself (ingress) or its response (egress). +- **Transform** — rewrite request arguments (ingress) or redact response content (egress). +- **Marker write / read** — coordinate across calls: one policy stamps a session-state flag (`session_writes`), another reads it (see Session State & Markers). +- **Intent** *(only when the intent tools are enabled)* — session-purpose capture and gating (see Intent-capture policies, including its availability gate). + ## Companion skills This skill is typically used alongside others. Invoke them via the `Skill` tool when relevant (in other agents, use your host's equivalent skill-loading mechanism): @@ -227,6 +237,7 @@ transform := { | `reasons` | `set` | Collects human-readable denial messages. Multiple reasons can fire. | | `reason` | `string` | Joins `reasons` into a single semicolon-delimited string. | | `transform` | `object` | Transformation instructions (redaction, payload replacement). | +| `session_writes` | `object` | Optional. Session-state keys this policy writes (e.g. markers: `session_writes["marker::"] := {...}`). Each written key must be declared in the policy's `writableKeySchema`. See Session State & Markers. | ### Transform Object Fields @@ -332,7 +343,7 @@ Older fields (`input.user`, `input.kind`, `input.payload.name`) are **deprecated }, "payload": {}, // Hook-specific data (see Payload by Hook Type) "tool_metadata": {} | null, // Tool definition (name, url, auth_type, gateway_id, input_schema, ...) - "context": {}, // Mirror of top-level fields (correlation_id, payload, tool_metadata, ...) + "context": {}, // Mirror of top-level fields (correlation_id, payload, tool_metadata, ...) plus context.session.policies[][] for reading markers / session state — see Session State & Markers "correlation_id": "", // Request trace ID "request_ip": "", // Client IP address or "unknown" "headers": {}, // Filtered HTTP headers (dict) @@ -922,6 +933,109 @@ reasons contains reason if { - When the aggregator ANDs multiple step policies (`allow if { policy_a.allow; policy_b.allow }`), **all** must evaluate to `true`. If any step policy's package path is wrong, its `allow` is `undefined`, and the AND fails. - Transform-only step policies should use `default allow := true` so they don't block requests when their transform doesn't apply. +## Session State & Markers + +Beyond allow/deny/transform, a policy can **write to session state** and other policies can **read it** — giving the gateway a shared, tenant-scoped, TTL-bounded "notepad" that survives across tool calls and across upstream MCP servers. The main use is **markers**: session-state flags one policy stamps and another gates on. A marker written during a Slack egress call is visible during the next Jira ingress call in the same session, because markers are scoped by tenant and user (not by server). + +This lets you compose small single-purpose policies that signal to each other without shared code, instead of one giant policy that observes everything: + +- **Writer policy** — inspects the current call (arguments, response text, identity, whatever) and stamps a marker when a condition is met. +- **Reader policy** — checks whether a marker is set and allows/denies/transforms accordingly. The writer and reader can attach to different tools, different directions, and different upstream servers. + +Registering the marker vocabulary, attaching the `writableKeySchema`, and deploying are lifecycle operations owned by the companion `dtwo-gateway-policy` instructions (Managing Markers). This section covers only the **Rego**. + +### Writing a marker (`session_writes`) + +A writer emits `session_writes["marker::"] := ` when its trigger fires. The value shape must satisfy the `writableKeySchema` attached to the policy (see below), and the policy **must** declare that key in its `writableKeySchema` or the gateway drops the write. + +```rego +package acme.egress.pii_detector + +import future.keywords.if +import future.keywords.in + +default allow := true # writer only observes and stamps; it does not deny + +_email_pattern := `[a-zA-Z0-9._%+-]+@[a-zA-Z0-9.-]+\.[a-zA-Z]{2,}` + +_pii_email_found if { + some text in input.payload.text + is_string(text) + regex.match(_email_pattern, text) +} + +# Stamp the marker when PII is observed in the response +session_writes["marker:acme:pii_detected"] := marker_value if { + _pii_email_found + marker_value := { + "marked_at": time.now_ns(), + "source_action": input.resource.name, + } +} +``` + +A writer is usually a `default allow := true` policy — it observes and stamps, it does not block. (A single policy *can* both deny and write, but prefer separate concerns.) + +### Reading a marker + +Active session state is exposed to a policy at `input.context.session.policies`, shaped as `policies[][] = ` — an outer object keyed by the UID of the policy that wrote each key, and under each writer the keys it wrote (a marker key `marker::` maps to the object the writer emitted). A marker is stored under the **writer policy's UID**, so the read walks all writer slots and treats the key as truthy if any writer set it: + +```rego +package acme.ingress.pii_gate + +import future.keywords.if + +default allow := true + +_pii_active if { + some writer_uid + input.context.session.policies[writer_uid]["marker:acme:pii_detected"] +} + +# Block outbound Slack sends once PII was seen anywhere in this session +allow := false if { + lower(input.resource.name) == "slack-mcp-slack-send-message" + _pii_active +} + +reason := "PII was detected earlier in this session; outbound Slack sends are blocked. Wait for the marker TTL to expire or start a new session." if { + lower(input.resource.name) == "slack-mcp-slack-send-message" + _pii_active +} +``` + +- The walk-all-writers pattern (`some writer_uid; input.context.session.policies[writer_uid][key]`) is "present under *any* writer is truthy." To trust only a specific writer, filter on `writer_uid == ""`. +- Deny reasons are user-visible — explain what to do about the block ("wait for TTL", "start a new session"). + +### `writableKeySchema` (attached via `dtwo-add-policy` / `dtwo-update-policy`) + +The Rego emits the write; the `writableKeySchema` on the policy record tells the gateway what shape the write must have. It is set through the lifecycle tools (see `dtwo-gateway-policy`), not inside the Rego, but the Rego author owns getting the value shape right. Each entry has `name` (the marker key, matching the registry exactly), `jsonSchema` (a stringified JSON object — a JSON Schema — for the value), `ttlSeconds` (≥ the marker's registered `minimumTtlSeconds`), and `onDrop` (`"drop"` — silently drop a schema-failing write; `"deny_request"` — hard-deny the tool call). + +The tool boundary rejects malformed marker keys: each `marker::` segment must be alphanumerics, underscores, or hyphens and start with an alphanumeric (see `dtwo-gateway-policy` → Managing Markers for the exact pattern; lowercase is conventional). It also rejects a `jsonSchema` that doesn't parse as a JSON object. + +### Marker Rego gotchas + +- **`time.now_ns()` must stay an integer.** Use `time.now_ns()` raw for timestamp fields typed `integer` in the schema. Dividing in Rego (e.g. `time.now_ns() / 1000000`) produces a **float**, which fails a `"type": "integer"` schema — and with `onDrop: "deny_request"` that silently-authored bug will block the tool call. +- **Match the key exactly.** The `session_writes` key, the `writableKeySchema` `name`, and the registered marker FQID must all be the identical `marker::` string. A mismatch drops the write. +- **Multiple writers land in separate slots.** If two policies declare and emit the same key, each write lands under its own writer UID; the walk-all-writers read finds either. Prefer one canonical writer per marker. +- **Reads fail open on absent state.** If `input.context.session.policies` is missing or the marker was never written, the `_active` helper simply doesn't match — the reader allows. Structure high-sensitivity gates so the *presence* of the marker is what denies, not its absence (that's the intended semantics: no signal → nothing to block). + +## Intent-capture policies (conditional — feature-gated) + +> **Availability gate — read this first.** The intent-capture surface (the `dtwo-set-intent` tool and the intent registry) only exists when the DTwo MCP server is deployed with `enable_intent_tools: true`. **Do not present intent-capture policies, `set_intent`, or intent/marker compatibility to the user unless those tools are actually available** — check for `dtwo-set-intent` / `dtwo-*-intent*` in your tool list, or confirm via the companion `dtwo-gateway-policy` instructions. If they are absent, this section is inert; markers (above) still work fully. This is Rego guidance only — registry management lives in `dtwo-gateway-policy` (Intent Capture). + +Intent capture builds on the same session-state mechanism as markers. It ships as two **starter policies** (attach as drafts; nothing is auto-injected): + +- **Egress capture** — package `set_intent.egress.intent_capture`, egress. Captures the declared intent into session state when `set_intent` is invoked, validates it against `data.dtwo.intent_registry`, denies disallowed transitions, and denies when the proposed intent is registered incompatible with a marker currently active in the session (`intent_marker_incompatible`). +- **Intent-required gate** — package `set_intent.ingress.intent_required`, ingress, **optional**. Denies every tool call until an intent has been set; the `set_intent` tool itself is always allowed. + +Two coupling requirements that fail silently if missed: + +- **Upstream server must be named `Dtwo`.** Both policies trigger on tool names case-folding to `dtwo-set-intent` / `dtwo-set_intent` (names arrive as `-`). The DTwo MCP upstream `mcp_servers[].name` **must** case-fold to `dtwo`. Any other name silently bypasses the egress capture and deadlocks the ingress gate. +- **Shared UID placeholder.** Both files ship with `REPLACE-WITH-INTENT-CAPTURE-POLICY-UID` — the ingress gate trusts only writes from the canonical capture policy, so both must reference the capture policy's own UID. After `dtwo-add-policy` returns the real UID, replace the placeholder in **both** bodies and re-save. Until swapped, the ingress gate denies every non-`set_intent` call and the egress capture treats every set as first-set. + +Compatibility is **one-directional and set-time only**: the egress checks `intent → active markers` at `set_intent` time. A marker raised *after* an intent is set does not retroactively deny. + ## Handling Parse and Access Failures Rego rules fail **silently** — if any expression in a rule body fails (e.g., `json.unmarshal` on non-JSON text, or accessing a missing key), the entire rule body does not match. No error is raised. This means the policy's behavior on bad data depends on how the rules are structured. From 3a39caa804c285e2edfb821bf5ad010da5dffd86 Mon Sep 17 00:00:00 2001 From: Paul Reilly Date: Thu, 2 Jul 2026 13:43:22 -0400 Subject: [PATCH 2/8] Update to match d2 changes --- dtwo/skills/dtwo-gateway-policy/SKILL.md | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/dtwo/skills/dtwo-gateway-policy/SKILL.md b/dtwo/skills/dtwo-gateway-policy/SKILL.md index 9b15991..1b19f2f 100644 --- a/dtwo/skills/dtwo-gateway-policy/SKILL.md +++ b/dtwo/skills/dtwo-gateway-policy/SKILL.md @@ -113,7 +113,7 @@ Markers are session-state flags that one policy writes and other policies read t |------|---------| | `dtwo-list-markers` | List markers in the registry (optional filters: `name` for exact FQID, `tag`). Returns the marker *vocabulary*, not which markers are currently active on a session | | `dtwo-get-marker` | Fetch a single marker by UID | -| `dtwo-create-marker` | Create a customer-tier marker (requires `namespace`, `markerId`, `description`, `minimumTtlSeconds`; optional `tags`). Full key is `marker::`. `internal` and `dtwo` namespaces are reserved for platform markers | +| `dtwo-create-marker` | Create a customer-tier marker (requires `namespace`, `markerId`, `description`, `minimumTtlSeconds`; optional `tags`). Full key is `marker::`. `namespace` and `markerId` are each validated at the tool boundary (must start with an alphanumeric; alphanumerics, underscores, or hyphens only). `internal` and `dtwo` namespaces are reserved for platform markers | | `dtwo-update-marker` | Update mutable fields on a customer-tier marker (`description`, `tags`, `minimumTtlSeconds`) | | `dtwo-delete-marker` | Delete a customer-tier marker. Platform-managed entries cannot be deleted | @@ -451,6 +451,7 @@ dtwo-create-marker( ``` - The full key is `marker:acme:pii_detected`. Customer markers live under any namespace except the reserved `internal` and `dtwo`. +- **`namespace` and `markerId` are validated here** — each must start with an alphanumeric and contain only alphanumerics, underscores, or hyphens (no dots, colons, spaces, etc.). This is the same per-segment rule enforced on `writableKeySchema[].name` (see Authoring the writer policy), so a marker you register is always writable via a policy's `writableKeySchema` — you can't register a key you then can't write to. - `minimumTtlSeconds` is a **floor** — a writer policy may declare any TTL ≥ this. Raise it later with `dtwo-update-marker` if a writer needs a longer minimum. ### Authoring the writer policy — `writableKeySchema` From 3415925c3a957a3497de0cb3c5b69cd02080c5ab Mon Sep 17 00:00:00 2001 From: Paul Reilly Date: Thu, 2 Jul 2026 15:28:45 -0400 Subject: [PATCH 3/8] Review updates --- dtwo/README.md | 4 +-- dtwo/skills/dtwo-gateway-policy/SKILL.md | 40 ++++++++++++------------ dtwo/skills/dtwo-policy-rego/SKILL.md | 14 ++++----- 3 files changed, 29 insertions(+), 29 deletions(-) diff --git a/dtwo/README.md b/dtwo/README.md index 768a2c9..4dbd36f 100644 --- a/dtwo/README.md +++ b/dtwo/README.md @@ -40,8 +40,8 @@ Naming the connector `dtwo` is required — it has to match the plugin's MCP ser | Skill | Use when | | --------------------- | -------------------------------------------------------------------------------------- | | `dtwo-gateway-config` | Editing gateway YAML, adding/removing MCP servers, publishing or rolling back configs. | -| `dtwo-gateway-policy` | Creating, attaching, publishing, deploying, or verifying policies and pipelines. | -| `dtwo-policy-rego` | Authoring, modifying, explaining, or debugging Rego policy code for the DTwo Gateway. | +| `dtwo-gateway-policy` | Creating, attaching, publishing, deploying, or verifying policies and pipelines; managing markers (and the intent registry when those tools are enabled). | +| `dtwo-policy-rego` | Authoring, modifying, explaining, or debugging Rego policy code for the DTwo Gateway, including marker writer/reader policies. | The skills load each other on demand via Claude Code's `Skill` tool — most real tasks pull in two or three together. diff --git a/dtwo/skills/dtwo-gateway-policy/SKILL.md b/dtwo/skills/dtwo-gateway-policy/SKILL.md index 1b19f2f..e9aa223 100644 --- a/dtwo/skills/dtwo-gateway-policy/SKILL.md +++ b/dtwo/skills/dtwo-gateway-policy/SKILL.md @@ -113,7 +113,7 @@ Markers are session-state flags that one policy writes and other policies read t |------|---------| | `dtwo-list-markers` | List markers in the registry (optional filters: `name` for exact FQID, `tag`). Returns the marker *vocabulary*, not which markers are currently active on a session | | `dtwo-get-marker` | Fetch a single marker by UID | -| `dtwo-create-marker` | Create a customer-tier marker (requires `namespace`, `markerId`, `description`, `minimumTtlSeconds`; optional `tags`). Full key is `marker::`. `namespace` and `markerId` are each validated at the tool boundary (must start with an alphanumeric; alphanumerics, underscores, or hyphens only). `internal` and `dtwo` namespaces are reserved for platform markers | +| `dtwo-create-marker` | Create a customer-tier marker (requires `namespace`, `markerId`, `description`, `minimumTtlSeconds`; optional `tags`). Full key is `marker::`. The tool requires only non-empty `namespace`/`markerId`; character-shape rules are validated server-side, not at the tool boundary. `internal` and `dtwo` namespaces are reserved for platform markers | | `dtwo-update-marker` | Update mutable fields on a customer-tier marker (`description`, `tags`, `minimumTtlSeconds`) | | `dtwo-delete-marker` | Delete a customer-tier marker. Platform-managed entries cannot be deleted | @@ -125,7 +125,7 @@ When present, these tools manage the intent vocabulary and the rules that govern | Tool | Purpose | |------|---------| -| `dtwo-set-intent` | Declare the current session intent (the working intent captured by the gateway's egress capture policy). Accepts only intents registered in the registry | +| `dtwo-set-intent` | Declare the current session intent (captured by the gateway's egress capture policy). Resolves against the tool's **built-in** intent vocabulary, not the registry (registry-driven resolution is planned). Not customer-available yet — stays behind `enable_intent_tools` until registry-driven resolution ships | | `dtwo-list-intents` | List intents in the registry (platform `system=true` entries are read-only; customer-tier entries are tenant-scoped) | | `dtwo-get-intent` | Fetch a single intent by UID | | `dtwo-create-intent` / `dtwo-update-intent` / `dtwo-delete-intent` | Manage customer-tier intents in the registry vocabulary | @@ -138,7 +138,7 @@ When present, these tools manage the intent vocabulary and the rules that govern |------|---------| | `dtwo-list-gateways` | List gateways with optional filters (name, status, uid) | | `dtwo-get-gateway` | Fetch a single gateway by UID | -| `dtwo-get-gateway-config` | Fetch the gateway's YAML configuration. Used here **read-only** to discover `mcp_servers[].name` and tool names when authoring policies (see Tool Discovery). Editing gateway YAML belongs to the companion `dtwo-gateway-config` skill | +| `dtwo-get-gateway-config` | Fetch the gateway's YAML configuration. Used here **read-only** to discover `mcp_servers[].name` and tool names when authoring policies (see Tool Discovery). Returns the **draft** config, which can diverge from what's deployed — confirm names against the deployed config before relying on them. Editing gateway YAML belongs to the companion `dtwo-gateway-config` skill | | `dtwo-set-gateway-pipelines` | Attach policies to ingress/egress pipelines | | `dtwo-get-gateway-pipelines` | Fetch ingress and egress pipeline steps for a gateway, including policy details | | `dtwo-deploy-gateway` | Queue a deployment for the gateway | @@ -433,7 +433,7 @@ After publishing, call `dtwo-set-gateway-pipelines` again with `policyVersion: 1 ## Managing Markers -Markers are session-state flags that policies write and later policies read to gate on. They give the gateway a shared, tenant-scoped, TTL-bounded "notepad" that survives across tool calls and across upstream MCP servers — a marker written during a Slack call is visible during the next Jira call in the same session. Use them to compose small single-purpose policies that signal to each other without shared code: a **writer** policy stamps a marker when it observes something (PII in a response, a production resource touched), and a **reader** policy on a different tool/pipeline/server gates on it. +Markers are session-state flags that policies write and later policies read to gate on. They give the gateway a shared, tenant- and user-scoped, TTL-bounded "notepad" that survives across tool calls and across upstream MCP servers — a marker written during a Slack call is visible during a later Jira call for the same user (until its TTL expires). Use them to compose small single-purpose policies that signal to each other without shared code: a **writer** policy stamps a marker when it observes something (PII in a response, a production resource touched), and a **reader** policy on a different tool/pipeline/server gates on it. Marker tools are always available (they do not require `enable_intent_tools`). The full lifecycle — register, author writer + reader, attach, deploy — runs through this skill plus `dtwo-policy-rego` for the Rego. The Rego authoring patterns (emitting `session_writes["marker::"]`, walking `input.context.session.policies` to read, and the `writableKeySchema` gotchas) live in the companion `dtwo-policy-rego` instructions — load that skill for the writer/reader bodies. @@ -451,17 +451,17 @@ dtwo-create-marker( ``` - The full key is `marker:acme:pii_detected`. Customer markers live under any namespace except the reserved `internal` and `dtwo`. -- **`namespace` and `markerId` are validated here** — each must start with an alphanumeric and contain only alphanumerics, underscores, or hyphens (no dots, colons, spaces, etc.). This is the same per-segment rule enforced on `writableKeySchema[].name` (see Authoring the writer policy), so a marker you register is always writable via a policy's `writableKeySchema` — you can't register a key you then can't write to. -- `minimumTtlSeconds` is a **floor** — a writer policy may declare any TTL ≥ this. Raise it later with `dtwo-update-marker` if a writer needs a longer minimum. +- **Keep the key simple.** The tool requires only non-empty `namespace`/`markerId`; the character-shape rules are validated server-side (when a writer policy is saved and at deploy), not at this tool boundary. In practice, use lowercase alphanumerics with underscores or hyphens and avoid dots, slashes, and spaces, so the key is accepted everywhere it's referenced (registry entry, `writableKeySchema` name, and `session_writes` key must all match exactly — see Authoring the writer policy). +- `minimumTtlSeconds` is the **intended floor** for a writer policy's `ttlSeconds`. **Not yet enforced** — a writer can currently declare a lower `ttlSeconds` and it will still save and deploy — so treat it as documentation of intent and keep the two in sync manually. Adjust later with `dtwo-update-marker`. ### Authoring the writer policy — `writableKeySchema` A policy that emits a marker must declare the key in its `writableKeySchema` (on `dtwo-add-policy` / `dtwo-update-policy`), or the gateway drops the write. Each entry is: - `name` — the session-state key, matching the registered marker exactly (e.g. `marker:acme:pii_detected`). -- **Marker key format enforcement.** The tool boundary rejects `writableKeySchema[].name` values starting with `marker:` unless they match `^marker:[A-Za-z0-9][A-Za-z0-9_-]*:[A-Za-z0-9][A-Za-z0-9_-]*$` — exactly two colon-separated segments of alphanumerics, underscores, or hyphens, each segment starting with an alphanumeric (case-insensitive). Bare keys (no `marker:` prefix) are structurally validated by the backend instead. Registry *existence* — that the key is actually in the marker registry — is enforced separately at deploy time by the `marker-not-in-registry` rule (the deploy fails with `UnknownMarkerReferences`; see the Deploy-time validator note below). +- **Key shape is validated server-side, not at the MCP boundary.** The tool accepts any non-empty `name`; the marker-key shape (allowed characters, reserved prefixes) is checked by the backend when the policy is saved and at deploy — the MCP layer does not shape-check it. Keys are **exact-match strings and are never normalized**, so the `session_writes` key, this `name`, and the registered marker FQID must be byte-identical (including case) or the write silently drops. Registry *existence* — that the key is actually in the marker registry — is enforced at deploy time by the `marker-not-in-registry` rule (the deploy fails with `UnknownMarkerReferences`; see the Deploy-time validator note below). - `jsonSchema` — a **stringified JSON object** (a JSON Schema) for the value the policy writes. Rejected at the tool boundary if it doesn't parse as a JSON object (arrays and primitives fail). Use it strictly (`additionalProperties: false`, `required` lists) so drift is caught. Add `"x-d2-is-marker": true` for marker keys. -- `ttlSeconds` — per-key TTL. For a marker key this must be **≥ the marker's registered `minimumTtlSeconds`**; a lower value is rejected at policy save time. +- `ttlSeconds` — per-key TTL. For a marker key this **should** be ≥ the marker's registered `minimumTtlSeconds`, but keep them in sync manually: the floor is **not yet enforced** (no save-time or deploy-time check today), so a lower value currently saves, deploys, and simply expires early. - `onDrop` — behavior when a write fails the schema: `"drop"` (default) silently drops the write (best-effort markers); `"deny_request"` hard-denies the tool call (use for security-critical writes so bugs surface loudly instead of silently letting the call through). ``` @@ -485,23 +485,23 @@ dtwo-add-policy( 2. Attach both with `dtwo-set-gateway-pipelines` — the writer on the direction that observes the signal (often egress), the reader on the direction that gates (often ingress). Preserve existing steps. 3. Deploy with `dtwo-deploy-gateway`. This is a **policy-only deploy** — hot-reloaded, no gateway restart, no MCP client disconnect (see Deploying). -**Deploy-time validator.** The deploy hard-rejects (`UnknownMarkerReferences`, the `marker-not-in-registry` rule) if any attached policy declares a `writableKeySchema` marker key that isn't in the registry. This is separate from the tool-boundary *format* check on the key (see Authoring the writer policy) — the format check runs at save time, the registry-existence check at deploy time. Register the marker *before* attaching a policy that writes it. +**Deploy-time validator.** The deploy hard-rejects (`UnknownMarkerReferences`, the `marker-not-in-registry` rule) if any attached policy declares a `writableKeySchema` marker key that isn't in the registry. This is separate from the key's structural validation (allowed characters, reserved prefixes), which the backend applies when the policy is saved — the registry-existence check runs at deploy time. Register the marker *before* attaching a policy that writes it. ### Verifying a marker pipeline -Markers can't be verified the way a single policy can — there is no tool to read a session's active markers (see Marker constraints today), so verification is **behavioral, session-scoped, and order-dependent**. A marker does nothing until its writer fires, and its effect is only visible through the reader's decision: +Markers can't be verified the way a single policy can — there is no tool to read active markers (see Marker constraints today), so verification is **behavioral and order-dependent**, and it hinges on scope: marker state is keyed to **tenant + user** and lives until its TTL expires — it is *not* reset by opening a new session or reconnecting. A marker does nothing until its writer fires, and its effect is only visible through the reader's decision: 1. **Confirm the deploy and attachment** as for any policy — poll `dtwo-get-deployment` to `completed`, then `dtwo-get-gateway-pipelines` to confirm both the writer and the reader landed with the expected `evalNamespace` and version pins. -2. **Trigger the writer first, in one session.** Make the tool call that satisfies the writer's condition (e.g. a response containing PII). This is what stamps the marker — nothing is active until the writer fires. -3. **Then exercise the reader in that same session.** Call the reader's guarded tool and confirm it now denies (or transforms) as intended. Because markers are session-scoped, this only proves out within the session where the writer fired. -4. **Confirm the negative case in a fresh session.** With no writer having fired (or after the TTL expires), the reader's tool should succeed — proving the reader blocks only when the marker is active, not unconditionally. +2. **Trigger the writer first.** Make the tool call that satisfies the writer's condition (e.g. a response containing PII). This is what stamps the marker — nothing is active until the writer fires. +3. **Then exercise the reader** (as the same user). Confirm the reader's guarded tool now denies (or transforms) as intended. The marker stays active for that user — across reconnects and new sessions — until its TTL expires. +4. **Confirm the negative case with a clean marker.** Either use a **short TTL and wait for it to expire**, or test as a **different user** who hasn't triggered the writer — then the reader's tool should succeed, proving it blocks only when the marker is active. Reopening the session as the *same* user does **not** clear the marker (tenant+user scope), so that is not a valid negative test. -**Tip — validate with a short TTL, then raise it.** A production-length TTL (say an hour) makes iterating painful: a marker stamped in one test lingers and masks the next. During validation, give the marker a short TTL (e.g. 30–60s) so it self-clears between iterations and you can retest in the same session without waiting it out or starting fresh. The written TTL is the writer policy's `writableKeySchema.ttlSeconds`, and it must be ≥ the marker's registered `minimumTtlSeconds` — so lower **both** for testing: set the marker's `minimumTtlSeconds` low (`dtwo-create-marker`, or `dtwo-update-marker` if it already exists) and set the writer's `ttlSeconds` to match. Once the pipeline is validated, raise the writer's `ttlSeconds` to the desired length (`dtwo-update-policy`) and the marker's `minimumTtlSeconds` floor if you want to enforce it (`dtwo-update-marker`), then republish/redeploy. +**Tip — validate with a short TTL.** A production-length TTL (say an hour) makes iterating painful: a marker stamped in one test stays set for that user until it expires and masks the next attempt. During validation, set the writer's `writableKeySchema.ttlSeconds` short (e.g. 30–60s) so it clears on its own between iterations. (The marker's `minimumTtlSeconds` floor is **not enforced** today — see below — so a short `ttlSeconds` deploys regardless of the floor; still, set the marker's `minimumTtlSeconds` to match via `dtwo-update-marker` so the registry reflects intent.) Once validated, raise the writer's `ttlSeconds` to the production length with `dtwo-update-policy` (and `minimumTtlSeconds` to match), then republish/redeploy. Watch for these: -- **Order and session matter.** Calling the reader before the writer has fired, or in a different session, shows the marker as absent and the reader allowing — that is correct behavior, not a bug. Sequence the calls within one session. -- **A stale marker can mask a result.** If an earlier call already stamped the marker and its TTL hasn't expired, the reader keeps denying. Use a fresh session for a clean negative test, or test with a short TTL (see the tip above) so it clears on its own between iterations. +- **Order and identity matter.** Calling the reader before the writer has fired, or as a different user, shows the marker absent and the reader allowing — correct behavior, not a bug. A new session for the *same* user does **not** reset the marker, so sequence writer-then-reader rather than reconnecting. +- **A stale marker can mask a result.** If an earlier call already stamped the marker and its TTL hasn't expired, the reader keeps denying for that user. Negative-test with a short TTL you can wait out (see the tip) or as a different user — not by reopening the session. - **`onDrop: "deny_request"` surfaces schema problems as a denied *writer* call.** If the tool that should stamp the marker is itself denied, the written value likely failed its `writableKeySchema` (e.g. a float timestamp against a `type: integer` field — see the `time.now_ns()` gotcha in `dtwo-policy-rego`). Fix the value shape, not the reader. - **To see the marker directly while debugging,** attach a temporary reader-side debug policy that dumps `input.context.session.policies` in a deny reason — the marker analog of the dump-input technique in `dtwo-policy-rego` (Debugging Policies). Detach it when done. @@ -516,7 +516,7 @@ Skipping a step makes the next deploy fail with `UnknownMarkerReferences` (a pol ### Marker constraints today - **No "list active markers" tool.** `dtwo-list-markers` returns the registry *vocabulary* (the markers that are defined), not which markers are currently set on a given session. A policy can read active markers at evaluation time via `input.context.session.policies` (that's how reader policies work), but there is no MCP tool to query a session's live marker state on demand. -- **No "clear marker" tool.** Markers lift on their own when their TTL expires; there is no MCP tool to unset one mid-session. To recover from a marker that is blocking a session, wait out the TTL or start a new session. +- **No "clear marker" tool.** Markers lift on their own when their TTL expires; there is no MCP tool to unset one. To recover from a marker that is blocking a user, wait out the TTL — a new session for the same user does **not** clear it (state is scoped to tenant + user, not per connection). - **Multiple writers land in separate per-writer slots.** If two policies declare and emit the same marker key, each write lands under its own writer UID; readers get "any-writer" semantics by walking `session.policies.*`. Prefer one canonical writer per marker. ## Intent Capture (conditional — feature-gated) @@ -525,7 +525,7 @@ Skipping a step makes the next deploy fail with `UnknownMarkerReferences` (a pol Intent capture lets the agent declare *what it's trying to do* (`dtwo-set-intent`), captures that into session state via an egress policy, and lets ingress policies gate downstream tools on the current intent. It builds on the same session-state mechanism as markers. -**Status.** This ships as **starter policies**, not an auto-injected platform feature — you attach them to a pipeline as drafts; nothing is auto-deployed. Registry-management tools are expected to stay on this server; `dtwo-set-intent` itself may move to a dedicated server later. +**Status.** Intent capture is **not customer-available yet.** The intent tools (including `dtwo-set-intent`) stay behind `enable_intent_tools` and will **not** be ungated until the user-intent registry actually drives resolution — today `set_intent` resolves against a built-in vocabulary, not the registry. It ships as **starter policies**, not an auto-injected platform feature — you attach them to a pipeline as drafts; nothing is auto-deployed. Registry-management tools are expected to stay on this server; `dtwo-set-intent` itself may move to a dedicated server later. ### The two starter policies @@ -534,8 +534,8 @@ Intent capture lets the agent declare *what it's trying to do* (`dtwo-set-intent The Rego bodies and the two coupling requirements below are detailed in `dtwo-policy-rego` (Intent-capture policies). The two requirements that block them silently if missed: -- **Upstream server must be named `Dtwo`.** Both policies fire on tool names case-folding to `dtwo-set-intent` / `dtwo-set_intent`. The DTwo MCP upstream entry in the gateway's `mcp_servers` config **must** be named `Dtwo` (anything case-folding to `dtwo`). A different name silently bypasses capture and deadlocks the ingress gate. -- **UID placeholder swap.** Both policies share the capture policy's own UID (`REPLACE-WITH-INTENT-CAPTURE-POLICY-UID`). After `dtwo-add-policy` returns the real UID, replace the placeholder in both bodies and re-save. Until swapped, the ingress gate denies every non-`set_intent` call and the egress capture treats every set as first-set (no transition enforcement). +- **Upstream server must be named `Dtwo`.** Both policies fire on tool names case-folding to `dtwo-set-intent` / `dtwo-set_intent`. The DTwo MCP upstream entry in the gateway's `mcp_servers` config **must** be named `Dtwo` (anything case-folding to `dtwo`). A different name silently bypasses capture and deadlocks the ingress gate — and since `dtwo-get-gateway-config` returns the *draft*, confirm the name against the deployed config. +- **UID placeholder swap.** Both policies share the capture policy's own UID (`REPLACE-WITH-INTENT-CAPTURE-POLICY-UID`). After `dtwo-add-policy` returns the real UID, replace the placeholder in both bodies and re-save. Until swapped, the egress capture **hard-denies every `set_intent` with a `config_error`** that names the missed swap (a deliberate fail-closed guard), and the ingress gate — if attached — denies every non-`set_intent` call. So a blanket `set_intent` deny right after setup almost always means the placeholder wasn't replaced. ### Intent/marker compatibility diff --git a/dtwo/skills/dtwo-policy-rego/SKILL.md b/dtwo/skills/dtwo-policy-rego/SKILL.md index e0be013..b0d2c48 100644 --- a/dtwo/skills/dtwo-policy-rego/SKILL.md +++ b/dtwo/skills/dtwo-policy-rego/SKILL.md @@ -343,7 +343,7 @@ Older fields (`input.user`, `input.kind`, `input.payload.name`) are **deprecated }, "payload": {}, // Hook-specific data (see Payload by Hook Type) "tool_metadata": {} | null, // Tool definition (name, url, auth_type, gateway_id, input_schema, ...) - "context": {}, // Mirror of top-level fields (correlation_id, payload, tool_metadata, ...) plus context.session.policies[][] for reading markers / session state — see Session State & Markers + "context": {}, // Mirror of top-level fields (correlation_id, payload, tool_metadata, ...); additionally carries context.session.policies[][] (context-only, no top-level equivalent) for reading markers / session state — see Session State & Markers "correlation_id": "", // Request trace ID "request_ip": "", // Client IP address or "unknown" "headers": {}, // Filtered HTTP headers (dict) @@ -935,7 +935,7 @@ reasons contains reason if { ## Session State & Markers -Beyond allow/deny/transform, a policy can **write to session state** and other policies can **read it** — giving the gateway a shared, tenant-scoped, TTL-bounded "notepad" that survives across tool calls and across upstream MCP servers. The main use is **markers**: session-state flags one policy stamps and another gates on. A marker written during a Slack egress call is visible during the next Jira ingress call in the same session, because markers are scoped by tenant and user (not by server). +Beyond allow/deny/transform, a policy can **write to session state** and other policies can **read it** — giving the gateway a shared, tenant- and user-scoped, TTL-bounded "notepad" that survives across tool calls and across upstream MCP servers. The main use is **markers**: session-state flags one policy stamps and another gates on. A marker written during a Slack egress call is visible during a later Jira ingress call for the same user, because markers are scoped by tenant and user (not by server) and persist until their TTL expires. This lets you compose small single-purpose policies that signal to each other without shared code, instead of one giant policy that observes everything: @@ -998,20 +998,20 @@ allow := false if { _pii_active } -reason := "PII was detected earlier in this session; outbound Slack sends are blocked. Wait for the marker TTL to expire or start a new session." if { +reason := "PII was detected earlier in this session; outbound Slack sends are blocked. Wait for the marker TTL to expire." if { lower(input.resource.name) == "slack-mcp-slack-send-message" _pii_active } ``` - The walk-all-writers pattern (`some writer_uid; input.context.session.policies[writer_uid][key]`) is "present under *any* writer is truthy." To trust only a specific writer, filter on `writer_uid == ""`. -- Deny reasons are user-visible — explain what to do about the block ("wait for TTL", "start a new session"). +- Deny reasons are user-visible — explain what to do about the block (e.g. "wait for the marker TTL to expire"). Avoid "start a new session": marker state is scoped to tenant + user and survives reconnecting, so a new session for the same user won't clear it. ### `writableKeySchema` (attached via `dtwo-add-policy` / `dtwo-update-policy`) -The Rego emits the write; the `writableKeySchema` on the policy record tells the gateway what shape the write must have. It is set through the lifecycle tools (see `dtwo-gateway-policy`), not inside the Rego, but the Rego author owns getting the value shape right. Each entry has `name` (the marker key, matching the registry exactly), `jsonSchema` (a stringified JSON object — a JSON Schema — for the value), `ttlSeconds` (≥ the marker's registered `minimumTtlSeconds`), and `onDrop` (`"drop"` — silently drop a schema-failing write; `"deny_request"` — hard-deny the tool call). +The Rego emits the write; the `writableKeySchema` on the policy record tells the gateway what shape the write must have. It is set through the lifecycle tools (see `dtwo-gateway-policy`), not inside the Rego, but the Rego author owns getting the value shape right. Each entry has `name` (the marker key, matching the registry exactly), `jsonSchema` (a stringified JSON object — a JSON Schema — for the value), `ttlSeconds` (should be ≥ the marker's registered `minimumTtlSeconds` — not enforced yet, so keep them in sync manually), and `onDrop` (`"drop"` — silently drop a schema-failing write; `"deny_request"` — hard-deny the tool call). -The tool boundary rejects malformed marker keys: each `marker::` segment must be alphanumerics, underscores, or hyphens and start with an alphanumeric (see `dtwo-gateway-policy` → Managing Markers for the exact pattern; lowercase is conventional). It also rejects a `jsonSchema` that doesn't parse as a JSON object. +Marker-key *shape* is validated server-side (the backend on save, and at deploy), not at the MCP tool boundary — so keep the key simple (lowercase alphanumerics with `_`/`-`; avoid dots, slashes, spaces) and reference it identically everywhere; keys are exact-match and never normalized (see `dtwo-gateway-policy` → Managing Markers). The `jsonSchema`, though, *is* checked at the tool boundary — it must parse as a JSON object. ### Marker Rego gotchas @@ -1032,7 +1032,7 @@ Intent capture builds on the same session-state mechanism as markers. It ships a Two coupling requirements that fail silently if missed: - **Upstream server must be named `Dtwo`.** Both policies trigger on tool names case-folding to `dtwo-set-intent` / `dtwo-set_intent` (names arrive as `-`). The DTwo MCP upstream `mcp_servers[].name` **must** case-fold to `dtwo`. Any other name silently bypasses the egress capture and deadlocks the ingress gate. -- **Shared UID placeholder.** Both files ship with `REPLACE-WITH-INTENT-CAPTURE-POLICY-UID` — the ingress gate trusts only writes from the canonical capture policy, so both must reference the capture policy's own UID. After `dtwo-add-policy` returns the real UID, replace the placeholder in **both** bodies and re-save. Until swapped, the ingress gate denies every non-`set_intent` call and the egress capture treats every set as first-set. +- **Shared UID placeholder.** Both files ship with `REPLACE-WITH-INTENT-CAPTURE-POLICY-UID` — the ingress gate trusts only writes from the canonical capture policy, so both must reference the capture policy's own UID. After `dtwo-add-policy` returns the real UID, replace the placeholder in **both** bodies and re-save. Until swapped, the egress capture **hard-denies every `set_intent` with a `config_error`** naming the missed swap (a deliberate fail-closed guard), and the ingress gate (if attached) denies every non-`set_intent` call — so a blanket `set_intent` deny right after setup points straight at the missed swap. Compatibility is **one-directional and set-time only**: the egress checks `intent → active markers` at `set_intent` time. A marker raised *after* an intent is set does not retroactively deny. From 2665f77ac246f9f72405c2dd04bc1690831364e0 Mon Sep 17 00:00:00 2001 From: Paul Reilly Date: Thu, 2 Jul 2026 16:15:45 -0400 Subject: [PATCH 4/8] Updates from reviews --- dtwo/skills/dtwo-gateway-policy/SKILL.md | 31 +++++++++++++++--------- dtwo/skills/dtwo-policy-rego/SKILL.md | 18 +++++--------- 2 files changed, 25 insertions(+), 24 deletions(-) diff --git a/dtwo/skills/dtwo-gateway-policy/SKILL.md b/dtwo/skills/dtwo-gateway-policy/SKILL.md index e9aa223..4f2b7c3 100644 --- a/dtwo/skills/dtwo-gateway-policy/SKILL.md +++ b/dtwo/skills/dtwo-gateway-policy/SKILL.md @@ -99,7 +99,7 @@ The tools listed below reflect the initial set. The DTwo MCP server may add new | `dtwo-get-policy-versions` | List published versions for a policy | | `dtwo-validate-policy-rego` | Validate Rego code without saving — useful for dry-run checks before committing changes (note: `dtwo-add-policy` and `dtwo-update-policy` also validate automatically) | | `dtwo-add-policy` | Validate and create a new policy (requires name, description, policy, packageName, direction). Optionally pass `writableKeySchema` to declare the session-state keys the policy is authorized to write — required for any policy that emits a marker (see Managing Markers) | -| `dtwo-update-policy` | Update an existing policy's draft — any field (policy, packageName, name, description, direction, tags, `writableKeySchema`). Validates Rego when both policy and packageName are provided. `writableKeySchema` is tri-state: omit → leave unchanged, `null` → clear, `[]` → explicit-empty, `[...]` → set | +| `dtwo-update-policy` | Update an existing policy's draft — any field (policy, packageName, name, description, direction, tags, `writableKeySchema`). Validates Rego when both policy and packageName are provided. `writableKeySchema` is tri-state: **omit** → leave unchanged; **`null`** → clear the field (policy keeps no writable keys); **`[]`** → set an explicit empty list (also leaves no writable keys — practically the same effect as `null`; use `null` as the reset); **`[...]`** → replace with that list | | `dtwo-publish-policy` | Publish the current draft as a new version | | `dtwo-revert-policy` | Restore a published version back into the draft | | `dtwo-delete-policy` | Permanently delete a policy by UID. Fails if the policy is still attached to one or more gateways — detach it from every gateway first (see Deleting a Policy). Distinct from `dtwo-revert-policy`, which only restores a prior version | @@ -152,7 +152,7 @@ When present, these tools manage the intent vocabulary and the rules that govern The delete **fails if the policy is still attached to any gateway**, so detach it everywhere first: 1. **Detach first** — remove the policy from all pipelines with `dtwo-set-gateway-pipelines` (pass `[]` to clear the relevant direction, or re-send the direction's steps without this policy), then redeploy each affected gateway. A detached policy remains in the policy list but has no runtime effect. -2. **Delete** — call `dtwo-delete-policy` with the `uid`. If it errors that the policy is still attached, a detach was missed (or a deploy hasn't landed) — re-check attachments with `dtwo-get-gateway-pipelines` before retrying. +2. **Delete** — call `dtwo-delete-policy` with the `uid`. If the policy is still attached, the call fails with an error naming the gateways still referencing it. Use those names (or `dtwo-get-gateway-pipelines`) to find the remaining attachments, detach them, redeploy, and retry. Deletion is irreversible and confirmation-worthy — confirm with the user before calling `dtwo-delete-policy`. If they only want to stop the policy's runtime effect (not remove the record), detaching and redeploying is sufficient; leave the policy in place. @@ -437,6 +437,14 @@ Markers are session-state flags that policies write and later policies read to g Marker tools are always available (they do not require `enable_intent_tools`). The full lifecycle — register, author writer + reader, attach, deploy — runs through this skill plus `dtwo-policy-rego` for the Rego. The Rego authoring patterns (emitting `session_writes["marker::"]`, walking `input.context.session.policies` to read, and the `writableKeySchema` gotchas) live in the companion `dtwo-policy-rego` instructions — load that skill for the writer/reader bodies. +**Start simple — the minimal marker is a boolean flag.** A writer stamps `marker::` when it observes a condition; a reader denies (or transforms) whenever that key is present. Presence *is* the signal — no value semantics needed. That flag pattern (the PII example used throughout this section) is the recommended starting point; reach for value-carrying markers only when a flag won't do. Counters and other read-modify-write markers are possible but more involved — a self-incrementing writer has to read its own prior value and re-emit on every call, which keeps refreshing (pinning) the TTL — so they aren't a good first marker. + +**Know these limits before you design** (full list under Marker constraints today): + +- **No runtime inspection** — no tool reads a session's active markers; you verify behaviorally (see Verifying a marker pipeline). +- **No manual clearing** — a marker lifts only when its TTL expires; there is no unset tool. +- **Tenant + user scope** — marker state persists for a user across reconnects and new sessions until the TTL expires; opening a fresh session does not clear it. + ### Registering a marker Register the marker in the vocabulary before any policy references it: @@ -459,7 +467,7 @@ dtwo-create-marker( A policy that emits a marker must declare the key in its `writableKeySchema` (on `dtwo-add-policy` / `dtwo-update-policy`), or the gateway drops the write. Each entry is: - `name` — the session-state key, matching the registered marker exactly (e.g. `marker:acme:pii_detected`). -- **Key shape is validated server-side, not at the MCP boundary.** The tool accepts any non-empty `name`; the marker-key shape (allowed characters, reserved prefixes) is checked by the backend when the policy is saved and at deploy — the MCP layer does not shape-check it. Keys are **exact-match strings and are never normalized**, so the `session_writes` key, this `name`, and the registered marker FQID must be byte-identical (including case) or the write silently drops. Registry *existence* — that the key is actually in the marker registry — is enforced at deploy time by the `marker-not-in-registry` rule (the deploy fails with `UnknownMarkerReferences`; see the Deploy-time validator note below). +- **Key shape is validated server-side, not at the MCP boundary.** The tool accepts any non-empty `name`; the marker-key shape (allowed characters, reserved prefixes) is checked by the backend when the policy is saved and at deploy — the MCP layer does not shape-check it. Keys are **exact-match strings and are never normalized**, so the `session_writes` key, this `name`, and the registered marker FQID must be byte-identical (including case) or the write silently drops. Registry *existence* — that the key is actually in the marker registry — is enforced at deploy time: the deploy fails if any attached policy declares a marker key that isn't registered (see the Deploy-time validator note below). - `jsonSchema` — a **stringified JSON object** (a JSON Schema) for the value the policy writes. Rejected at the tool boundary if it doesn't parse as a JSON object (arrays and primitives fail). Use it strictly (`additionalProperties: false`, `required` lists) so drift is caught. Add `"x-d2-is-marker": true` for marker keys. - `ttlSeconds` — per-key TTL. For a marker key this **should** be ≥ the marker's registered `minimumTtlSeconds`, but keep them in sync manually: the floor is **not yet enforced** (no save-time or deploy-time check today), so a lower value currently saves, deploys, and simply expires early. - `onDrop` — behavior when a write fails the schema: `"drop"` (default) silently drops the write (best-effort markers); `"deny_request"` hard-denies the tool call (use for security-critical writes so bugs surface loudly instead of silently letting the call through). @@ -485,7 +493,7 @@ dtwo-add-policy( 2. Attach both with `dtwo-set-gateway-pipelines` — the writer on the direction that observes the signal (often egress), the reader on the direction that gates (often ingress). Preserve existing steps. 3. Deploy with `dtwo-deploy-gateway`. This is a **policy-only deploy** — hot-reloaded, no gateway restart, no MCP client disconnect (see Deploying). -**Deploy-time validator.** The deploy hard-rejects (`UnknownMarkerReferences`, the `marker-not-in-registry` rule) if any attached policy declares a `writableKeySchema` marker key that isn't in the registry. This is separate from the key's structural validation (allowed characters, reserved prefixes), which the backend applies when the policy is saved — the registry-existence check runs at deploy time. Register the marker *before* attaching a policy that writes it. +**Deploy-time validator.** The deploy hard-rejects if any attached policy declares a `writableKeySchema` marker key that isn't in the registry, reporting which key is unregistered. This is separate from the key's structural validation (allowed characters, reserved prefixes), which the backend applies when the policy is saved — the registry-existence check runs at deploy time. Register the marker *before* attaching a policy that writes it. ### Verifying a marker pipeline @@ -507,7 +515,7 @@ Watch for these: ### Cleanup order (reverse of setup) -Skipping a step makes the next deploy fail with `UnknownMarkerReferences` (a policy still claims to write a marker that no longer exists). Tear down in reverse: +Skipping a step makes the next deploy fail (a policy still claims to write a marker that no longer exists in the registry). Tear down in reverse: 1. Update/remove the **writer policy** so it no longer references the marker in `writableKeySchema`; redeploy so the write contract leaves the bundle. 2. Delete any **intent/marker compatibility** rows that reference the marker (only relevant when intent tools are enabled — `dtwo-delete-intent-compatibility`); redeploy. @@ -525,17 +533,16 @@ Skipping a step makes the next deploy fail with `UnknownMarkerReferences` (a pol Intent capture lets the agent declare *what it's trying to do* (`dtwo-set-intent`), captures that into session state via an egress policy, and lets ingress policies gate downstream tools on the current intent. It builds on the same session-state mechanism as markers. -**Status.** Intent capture is **not customer-available yet.** The intent tools (including `dtwo-set-intent`) stay behind `enable_intent_tools` and will **not** be ungated until the user-intent registry actually drives resolution — today `set_intent` resolves against a built-in vocabulary, not the registry. It ships as **starter policies**, not an auto-injected platform feature — you attach them to a pipeline as drafts; nothing is auto-deployed. Registry-management tools are expected to stay on this server; `dtwo-set-intent` itself may move to a dedicated server later. +**Status.** Intent capture is **not customer-available yet.** The intent tools (including `dtwo-set-intent`) stay behind `enable_intent_tools` and will **not** be ungated until the user-intent registry actually drives resolution — today `set_intent` resolves against a built-in vocabulary, not the registry. The enforcement policies themselves are **platform-managed** (see below); `dtwo-set-intent` may also move to a dedicated server later. -### The two starter policies +### The enforcement policies are platform-managed -- **Egress capture** (package `set_intent.egress.intent_capture`, egress) — captures the declared intent into session state when `dtwo-set-intent` is invoked. Validates the proposal against the intent registry, normalizes the stored category, denies disallowed transitions, and denies when the registry marks the proposed intent incompatible with a marker currently active in the session (`intent_marker_incompatible`). -- **Intent-required gate** (package `set_intent.ingress.intent_required`, ingress) — **optional.** Denies every tool call until an intent has been set; `dtwo-set-intent` itself is always allowed so the agent can declare. Attach only when you want the gate enforced. +Two policies do the enforcement: -The Rego bodies and the two coupling requirements below are detailed in `dtwo-policy-rego` (Intent-capture policies). The two requirements that block them silently if missed: +- **Egress capture** — captures the declared intent into session state when `dtwo-set-intent` is invoked, validates it against the registry, normalizes the category, denies disallowed transitions, and denies when a currently-active marker is registered incompatible with the proposed intent (`intent_marker_incompatible`). +- **Intent-required gate** — optional: denies every tool call until an intent has been set (`dtwo-set-intent` itself is always allowed so the agent can declare). -- **Upstream server must be named `Dtwo`.** Both policies fire on tool names case-folding to `dtwo-set-intent` / `dtwo-set_intent`. The DTwo MCP upstream entry in the gateway's `mcp_servers` config **must** be named `Dtwo` (anything case-folding to `dtwo`). A different name silently bypasses capture and deadlocks the ingress gate — and since `dtwo-get-gateway-config` returns the *draft*, confirm the name against the deployed config. -- **UID placeholder swap.** Both policies share the capture policy's own UID (`REPLACE-WITH-INTENT-CAPTURE-POLICY-UID`). After `dtwo-add-policy` returns the real UID, replace the placeholder in both bodies and re-save. Until swapped, the egress capture **hard-denies every `set_intent` with a `config_error`** that names the missed swap (a deliberate fail-closed guard), and the ingress gate — if attached — denies every non-`set_intent` call. So a blanket `set_intent` deny right after setup almost always means the placeholder wasn't replaced. +**These are platform-managed policies — end users do not author, attach, copy, or modify them, and you should not offer to.** They are being moved to automatic injection when intent capture is enabled; the platform owns their bodies and wiring (upstream-server naming, internal UIDs), and their Rego may not be visible to users. If a user asks to write or change intent-capture Rego, decline and point them at the platform-managed feature rather than reconstructing it. The only intent surface users drive is the **registry** — the intent vocabulary, transitions, and marker compatibility (below), when the tools are enabled. ### Intent/marker compatibility diff --git a/dtwo/skills/dtwo-policy-rego/SKILL.md b/dtwo/skills/dtwo-policy-rego/SKILL.md index b0d2c48..9c6a8ee 100644 --- a/dtwo/skills/dtwo-policy-rego/SKILL.md +++ b/dtwo/skills/dtwo-policy-rego/SKILL.md @@ -7,7 +7,7 @@ description: | TRIGGER when: user asks to write/modify/explain/debug a Rego policy, says "block/allow/redact/transform" a tool call or response, mentions OPA, package paths, `input.payload`, `default allow`, or pastes Rego for review; asks to write a marker writer/reader policy, `session_writes`, session state, or (only when the - intent tools are enabled) an intent-capture policy; asks to contribute to `dtwoai/policy-store`, create + intent tools are enabled) to recognize/explain the platform-managed intent-capture policies (not user-authored); asks to contribute to `dtwoai/policy-store`, create catalog policy files, or update app, industry, bundle, manifest, or tests files for reusable DTwo policies; also when diagnosing blanket denies or transform conflicts. Pair with dtwo-gateway-policy whenever the resulting Rego must be saved, attached, or deployed. @@ -942,6 +942,8 @@ This lets you compose small single-purpose policies that signal to each other wi - **Writer policy** — inspects the current call (arguments, response text, identity, whatever) and stamps a marker when a condition is met. - **Reader policy** — checks whether a marker is set and allows/denies/transforms accordingly. The writer and reader can attach to different tools, different directions, and different upstream servers. +Start with the simplest shape — a **boolean flag** whose *presence* is the whole signal (the PII example below). Value-carrying markers (counters, structured payloads) are more advanced and rarely needed; a self-incrementing counter, for instance, has to read its own prior value and re-emit every call, which keeps refreshing the TTL. + Registering the marker vocabulary, attaching the `writableKeySchema`, and deploying are lifecycle operations owned by the companion `dtwo-gateway-policy` instructions (Managing Markers). This section covers only the **Rego**. ### Writing a marker (`session_writes`) @@ -1022,19 +1024,11 @@ Marker-key *shape* is validated server-side (the backend on save, and at deploy) ## Intent-capture policies (conditional — feature-gated) -> **Availability gate — read this first.** The intent-capture surface (the `dtwo-set-intent` tool and the intent registry) only exists when the DTwo MCP server is deployed with `enable_intent_tools: true`. **Do not present intent-capture policies, `set_intent`, or intent/marker compatibility to the user unless those tools are actually available** — check for `dtwo-set-intent` / `dtwo-*-intent*` in your tool list, or confirm via the companion `dtwo-gateway-policy` instructions. If they are absent, this section is inert; markers (above) still work fully. This is Rego guidance only — registry management lives in `dtwo-gateway-policy` (Intent Capture). - -Intent capture builds on the same session-state mechanism as markers. It ships as two **starter policies** (attach as drafts; nothing is auto-injected): - -- **Egress capture** — package `set_intent.egress.intent_capture`, egress. Captures the declared intent into session state when `set_intent` is invoked, validates it against `data.dtwo.intent_registry`, denies disallowed transitions, and denies when the proposed intent is registered incompatible with a marker currently active in the session (`intent_marker_incompatible`). -- **Intent-required gate** — package `set_intent.ingress.intent_required`, ingress, **optional**. Denies every tool call until an intent has been set; the `set_intent` tool itself is always allowed. - -Two coupling requirements that fail silently if missed: +> **Availability gate — read this first.** The intent-capture surface (the `dtwo-set-intent` tool and the intent registry) only exists when the DTwo MCP server is deployed with `enable_intent_tools: true`. **Do not present intent-capture policies, `set_intent`, or intent/marker compatibility to the user unless those tools are actually available** — check for `dtwo-set-intent` / `dtwo-*-intent*` in your tool list, or confirm via the companion `dtwo-gateway-policy` instructions. If they are absent, this section is inert; markers (above) still work fully. -- **Upstream server must be named `Dtwo`.** Both policies trigger on tool names case-folding to `dtwo-set-intent` / `dtwo-set_intent` (names arrive as `-`). The DTwo MCP upstream `mcp_servers[].name` **must** case-fold to `dtwo`. Any other name silently bypasses the egress capture and deadlocks the ingress gate. -- **Shared UID placeholder.** Both files ship with `REPLACE-WITH-INTENT-CAPTURE-POLICY-UID` — the ingress gate trusts only writes from the canonical capture policy, so both must reference the capture policy's own UID. After `dtwo-add-policy` returns the real UID, replace the placeholder in **both** bodies and re-save. Until swapped, the egress capture **hard-denies every `set_intent` with a `config_error`** naming the missed swap (a deliberate fail-closed guard), and the ingress gate (if attached) denies every non-`set_intent` call — so a blanket `set_intent` deny right after setup points straight at the missed swap. +**The intent-capture Rego is platform-managed — do not write or modify it, and do not offer to.** Two policies do the enforcement — an **egress capture** that records the declared intent into session state and denies disallowed transitions or intent/marker incompatibilities, and an optional **intent-required gate** that blocks tool calls until an intent is set. These are owned by the platform (being moved to automatic injection when intent capture is enabled); their bodies and wiring are not user-authored, and the Rego may not be visible to users. If asked to author or change intent-capture Rego, decline and point the user at the platform-managed feature (and the user-facing registry/compatibility tools in `dtwo-gateway-policy` → Intent Capture). This section exists only so you can *recognize and explain* the behavior, not reproduce it. -Compatibility is **one-directional and set-time only**: the egress checks `intent → active markers` at `set_intent` time. A marker raised *after* an intent is set does not retroactively deny. +One behavior worth knowing when explaining them: intent/marker compatibility is **one-directional and set-time only** — the check runs at `set_intent` time; a marker raised *after* an intent is set does not retroactively deny. ## Handling Parse and Access Failures From 72184d825aa570e4fdd9c311aadf194e3e678c0c Mon Sep 17 00:00:00 2001 From: Paul Reilly Date: Thu, 2 Jul 2026 16:42:12 -0400 Subject: [PATCH 5/8] Address review feedback --- dtwo/skills/dtwo-gateway-policy/SKILL.md | 15 +++++++------- dtwo/skills/dtwo-policy-rego/SKILL.md | 25 +++++++++++++++++++++++- 2 files changed, 31 insertions(+), 9 deletions(-) diff --git a/dtwo/skills/dtwo-gateway-policy/SKILL.md b/dtwo/skills/dtwo-gateway-policy/SKILL.md index 4f2b7c3..32656cd 100644 --- a/dtwo/skills/dtwo-gateway-policy/SKILL.md +++ b/dtwo/skills/dtwo-gateway-policy/SKILL.md @@ -460,14 +460,13 @@ dtwo-create-marker( - The full key is `marker:acme:pii_detected`. Customer markers live under any namespace except the reserved `internal` and `dtwo`. - **Keep the key simple.** The tool requires only non-empty `namespace`/`markerId`; the character-shape rules are validated server-side (when a writer policy is saved and at deploy), not at this tool boundary. In practice, use lowercase alphanumerics with underscores or hyphens and avoid dots, slashes, and spaces, so the key is accepted everywhere it's referenced (registry entry, `writableKeySchema` name, and `session_writes` key must all match exactly — see Authoring the writer policy). -- `minimumTtlSeconds` is the **intended floor** for a writer policy's `ttlSeconds`. **Not yet enforced** — a writer can currently declare a lower `ttlSeconds` and it will still save and deploy — so treat it as documentation of intent and keep the two in sync manually. Adjust later with `dtwo-update-marker`. +- `minimumTtlSeconds` is the **intended floor** for a writer policy's `ttlSeconds` — but it isn't enforced yet (see the `ttlSeconds` note under Authoring the writer policy), so keep the two in sync manually. Adjust later with `dtwo-update-marker`. ### Authoring the writer policy — `writableKeySchema` A policy that emits a marker must declare the key in its `writableKeySchema` (on `dtwo-add-policy` / `dtwo-update-policy`), or the gateway drops the write. Each entry is: -- `name` — the session-state key, matching the registered marker exactly (e.g. `marker:acme:pii_detected`). -- **Key shape is validated server-side, not at the MCP boundary.** The tool accepts any non-empty `name`; the marker-key shape (allowed characters, reserved prefixes) is checked by the backend when the policy is saved and at deploy — the MCP layer does not shape-check it. Keys are **exact-match strings and are never normalized**, so the `session_writes` key, this `name`, and the registered marker FQID must be byte-identical (including case) or the write silently drops. Registry *existence* — that the key is actually in the marker registry — is enforced at deploy time: the deploy fails if any attached policy declares a marker key that isn't registered (see the Deploy-time validator note below). +- `name` — the session-state key, matching the registered marker **exactly** (e.g. `marker:acme:pii_detected`). Keys are exact-match and never normalized, so this `name`, the `session_writes` key, and the registered marker FQID must be byte-identical (including case) or the write silently drops. The tool accepts any non-empty string; marker-key *shape* (allowed characters, reserved prefixes) is validated server-side on save/deploy, and registry *existence* is enforced at deploy time — the deploy fails on an unregistered key (see the Deploy-time validator note below). - `jsonSchema` — a **stringified JSON object** (a JSON Schema) for the value the policy writes. Rejected at the tool boundary if it doesn't parse as a JSON object (arrays and primitives fail). Use it strictly (`additionalProperties: false`, `required` lists) so drift is caught. Add `"x-d2-is-marker": true` for marker keys. - `ttlSeconds` — per-key TTL. For a marker key this **should** be ≥ the marker's registered `minimumTtlSeconds`, but keep them in sync manually: the floor is **not yet enforced** (no save-time or deploy-time check today), so a lower value currently saves, deploys, and simply expires early. - `onDrop` — behavior when a write fails the schema: `"drop"` (default) silently drops the write (best-effort markers); `"deny_request"` hard-denies the tool call (use for security-critical writes so bugs surface loudly instead of silently letting the call through). @@ -497,19 +496,19 @@ dtwo-add-policy( ### Verifying a marker pipeline -Markers can't be verified the way a single policy can — there is no tool to read active markers (see Marker constraints today), so verification is **behavioral and order-dependent**, and it hinges on scope: marker state is keyed to **tenant + user** and lives until its TTL expires — it is *not* reset by opening a new session or reconnecting. A marker does nothing until its writer fires, and its effect is only visible through the reader's decision: +Markers can't be verified the way a single policy can — there is no tool to read active markers (see Marker constraints today), so verification is **behavioral and order-dependent**: a marker does nothing until its writer fires, and its effect is only visible through the reader's decision. The tenant+user scope (below) is what makes the negative case tricky, so mind it: 1. **Confirm the deploy and attachment** as for any policy — poll `dtwo-get-deployment` to `completed`, then `dtwo-get-gateway-pipelines` to confirm both the writer and the reader landed with the expected `evalNamespace` and version pins. 2. **Trigger the writer first.** Make the tool call that satisfies the writer's condition (e.g. a response containing PII). This is what stamps the marker — nothing is active until the writer fires. -3. **Then exercise the reader** (as the same user). Confirm the reader's guarded tool now denies (or transforms) as intended. The marker stays active for that user — across reconnects and new sessions — until its TTL expires. +3. **Then exercise the reader** (as the same user). Confirm the reader's guarded tool now denies (or transforms) as intended. The marker stays active until its TTL expires. 4. **Confirm the negative case with a clean marker.** Either use a **short TTL and wait for it to expire**, or test as a **different user** who hasn't triggered the writer — then the reader's tool should succeed, proving it blocks only when the marker is active. Reopening the session as the *same* user does **not** clear the marker (tenant+user scope), so that is not a valid negative test. -**Tip — validate with a short TTL.** A production-length TTL (say an hour) makes iterating painful: a marker stamped in one test stays set for that user until it expires and masks the next attempt. During validation, set the writer's `writableKeySchema.ttlSeconds` short (e.g. 30–60s) so it clears on its own between iterations. (The marker's `minimumTtlSeconds` floor is **not enforced** today — see below — so a short `ttlSeconds` deploys regardless of the floor; still, set the marker's `minimumTtlSeconds` to match via `dtwo-update-marker` so the registry reflects intent.) Once validated, raise the writer's `ttlSeconds` to the production length with `dtwo-update-policy` (and `minimumTtlSeconds` to match), then republish/redeploy. +**Tip — validate with a short TTL.** A production-length TTL (say an hour) makes iterating painful: a marker stamped in one test stays set for that user until it expires and masks the next attempt. During validation, set the writer's `writableKeySchema.ttlSeconds` short (e.g. 30–60s) so it clears on its own between iterations. (Since the floor isn't enforced, a short `ttlSeconds` deploys regardless; set the marker's `minimumTtlSeconds` to match via `dtwo-update-marker` so the registry still reflects intent.) Once validated, raise the writer's `ttlSeconds` to the production length with `dtwo-update-policy` (and `minimumTtlSeconds` to match), then republish/redeploy. Watch for these: -- **Order and identity matter.** Calling the reader before the writer has fired, or as a different user, shows the marker absent and the reader allowing — correct behavior, not a bug. A new session for the *same* user does **not** reset the marker, so sequence writer-then-reader rather than reconnecting. -- **A stale marker can mask a result.** If an earlier call already stamped the marker and its TTL hasn't expired, the reader keeps denying for that user. Negative-test with a short TTL you can wait out (see the tip) or as a different user — not by reopening the session. +- **Order and identity matter.** Calling the reader before the writer has fired, or as a different user, shows the marker absent and the reader allowing — correct behavior, not a bug. Sequence writer-then-reader (reconnecting as the same user won't reset it). +- **A stale marker can mask a result.** If an earlier call already stamped the marker and its TTL hasn't expired, the reader keeps denying for that user. Negative-test with a short TTL you can wait out (see the tip) or as a different user. - **`onDrop: "deny_request"` surfaces schema problems as a denied *writer* call.** If the tool that should stamp the marker is itself denied, the written value likely failed its `writableKeySchema` (e.g. a float timestamp against a `type: integer` field — see the `time.now_ns()` gotcha in `dtwo-policy-rego`). Fix the value shape, not the reader. - **To see the marker directly while debugging,** attach a temporary reader-side debug policy that dumps `input.context.session.policies` in a deny reason — the marker analog of the dump-input technique in `dtwo-policy-rego` (Debugging Policies). Detach it when done. diff --git a/dtwo/skills/dtwo-policy-rego/SKILL.md b/dtwo/skills/dtwo-policy-rego/SKILL.md index 9c6a8ee..c925f0f 100644 --- a/dtwo/skills/dtwo-policy-rego/SKILL.md +++ b/dtwo/skills/dtwo-policy-rego/SKILL.md @@ -948,7 +948,11 @@ Registering the marker vocabulary, attaching the `writableKeySchema`, and deploy ### Writing a marker (`session_writes`) -A writer emits `session_writes["marker::"] := ` when its trigger fires. The value shape must satisfy the `writableKeySchema` attached to the policy (see below), and the policy **must** declare that key in its `writableKeySchema` or the gateway drops the write. +A writer emits `session_writes["marker::"] := ` when its trigger fires. The canonical shape: + +- **The value is stored verbatim — there is no envelope.** The object you assign *is* what's persisted for that key. The `writableKeySchema` entry's `jsonSchema` validates that object **at its top level** — it describes the value object directly, not a nested `value`/`data`/`payload` wrapper. So your object's keys must be exactly the properties the schema declares (typically a couple of flat fields like `marked_at` + `source_action`). +- **The key must be declared** in the policy's `writableKeySchema`, or the gateway drops the write. +- **TTL comes from the schema, never the value.** The entry's `ttlSeconds` is applied by the gateway as the key's expiry when the write lands; it is metadata attached to the key at runtime, not a field of the value, and there is no way to set expiry from within the Rego. ```rego package acme.egress.pii_detector @@ -978,6 +982,25 @@ session_writes["marker:acme:pii_detected"] := marker_value if { A writer is usually a `default allow := true` policy — it observes and stamps, it does not block. (A single policy *can* both deny and write, but prefer separate concerns.) +**The write only happens when the trigger matches.** The `if { ... }` body gates the write like any other rule, so it uses the same request-matching as the rest of this skill: compare tool names case-insensitively (`lower(input.resource.name) == "-"`, with the server-name prefix), read args via `object.get(input.payload.args, ...)`, and inspect response content via `input.payload.text` on egress. A writer whose trigger never matches (wrong case, missing server prefix, wrong direction) stamps nothing — and there's no error, just an absent marker. Confirm the exact tool name and payload shape with the dump-input technique (see Debugging Policies) before relying on the trigger. + +**Silent-drop trap.** Because the schema is strict (`additionalProperties: false`), slipping any undeclared field into the value fails validation — most commonly a `ttl`/`ttlSeconds`, which belongs on the schema (per the rule above), not the value. With the default `onDrop: "drop"` that failure is **silent**: the marker never lands and readers never see it. If a marker mysteriously isn't being read, check the written value against the schema first. + +```rego +# WRONG — `ttl_seconds` is not a declared schema field, so the entire write silently drops +session_writes["marker:acme:jira_access"] := { + "marked_at": time.now_ns(), + "source_action": input.resource.name, + "ttl_seconds": 3600, # ← remove; TTL belongs on the writableKeySchema, not the value +} + +# RIGHT — value carries only the declared fields; TTL is configured on writableKeySchema +session_writes["marker:acme:jira_access"] := { + "marked_at": time.now_ns(), + "source_action": input.resource.name, +} +``` + ### Reading a marker Active session state is exposed to a policy at `input.context.session.policies`, shaped as `policies[][] = ` — an outer object keyed by the UID of the policy that wrote each key, and under each writer the keys it wrote (a marker key `marker::` maps to the object the writer emitted). A marker is stored under the **writer policy's UID**, so the read walks all writer slots and treats the key as truthy if any writer set it: From e65b55eba7c44d9b0b8799fa6d787a5fe249c354 Mon Sep 17 00:00:00 2001 From: Paul Reilly Date: Wed, 8 Jul 2026 13:24:51 -0400 Subject: [PATCH 6/8] Add information about intents gated on the DTwo MCP server having the intent tools Cleanup references to old format instead of PARC --- dtwo/skills/dtwo-gateway-config/SKILL.md | 4 +- dtwo/skills/dtwo-gateway-policy/SKILL.md | 92 +++++-- dtwo/skills/dtwo-policy-rego/SKILL.md | 251 +++++++++--------- .../references/policy-store-catalog.md | 99 +++++++ 4 files changed, 296 insertions(+), 150 deletions(-) create mode 100644 dtwo/skills/dtwo-policy-rego/references/policy-store-catalog.md diff --git a/dtwo/skills/dtwo-gateway-config/SKILL.md b/dtwo/skills/dtwo-gateway-config/SKILL.md index 799f437..cb7fe78 100644 --- a/dtwo/skills/dtwo-gateway-config/SKILL.md +++ b/dtwo/skills/dtwo-gateway-config/SKILL.md @@ -58,7 +58,7 @@ The tools listed below reflect the initial set. The DTwo MCP server may add new | `dtwo-validate-gateway-config` | Validate YAML configuration without saving | | `dtwo-save-gateway-draft-config` | Validate and save YAML as the draft configuration | | `dtwo-publish-gateway-config` | Publish the gateway draft as a new version | -| `dtwo-revert-gateway-config` | Restore a published gateway version back into the draft | +| `dtwo-revert-gateway-config` | Restore a published `version` back into the draft. Pass `publish: true` to publish it immediately as well | ### Deploy & Status Tools @@ -215,7 +215,7 @@ Every field marked `secret: true` in the artifact. Emit a self-describing placeh ### Gateway Section -Controls authentication, SSRF protection, logging, CORS, and advanced flags. +Controls authentication, SSRF protection, `log_level`, and `advanced` flags — all documented in the Schema Digest above. (CORS is also modeled by the parser but is not detailed in the digest; configure it via the `advanced` escape hatch or confirm the field names with `dtwo-validate-gateway-config` before relying on them.) Authentication and SSRF are the load-bearing ones and are expanded below. - **Authentication** defaults to enabled when omitted. Supports JWKS-based JWT verification, SSO issuer, audience/issuer verification, JTI requirements, token expiration enforcement, and OAuth resource metadata. - **Gateway-side `jwks_info` is independent of any `mcp_servers[].authentication` block.** When the prompt supplies an IdP tenant and audience (e.g. Auth0), populate `gateway.authentication.jwks_info` (`jwt_algorithm`, `jwt_jwks_uri`, `jwt_issuer`, `jwt_audience`) — even when the upstream MCP server uses OAuth/DCR, and even when the prompt says the upstream server "only supports OAuth" or "does not accept bearer tokens." Those statements describe the outbound leg to the MCP server, not the inbound leg from clients to the gateway. diff --git a/dtwo/skills/dtwo-gateway-policy/SKILL.md b/dtwo/skills/dtwo-gateway-policy/SKILL.md index 32656cd..d1b7611 100644 --- a/dtwo/skills/dtwo-gateway-policy/SKILL.md +++ b/dtwo/skills/dtwo-gateway-policy/SKILL.md @@ -1,18 +1,16 @@ --- name: "dtwo-gateway-policy" description: | - Create, validate, attach, publish, deploy, verify, and roll back DTwo policies and their pipeline attachments. - This is the system-of-record skill: it owns the DTwo MCP tools for policy lifecycle (create, update, - publish, revert) and pipeline lifecycle (attach, deploy, verify), and is responsible for confirming - pipeline attachments before and after deployment. Also manages the session-state marker registry, and - the intent registry when the intent tools are enabled. - TRIGGER when: user says "create/add a policy", "modify/update a policy", "block/allow/redact something", - "attach/detach policy", "set/update pipeline", "publish/pin policy version", "deploy gateway" after a - policy change; also "register/create/manage a marker", "marker registry", or (only when the intent tools - are enabled) "intent capture", "intent registry", "intent transitions", "intent/marker compatibility". - Always pair with dtwo-policy-rego for the Rego authoring/modification step. - SKIP when: task is purely *explaining* existing Rego with no save/attach/deploy intent - (use dtwo-policy-rego alone); task is editing gateway YAML or MCP server entries (use dtwo-gateway-config). + Create, validate, attach, publish, deploy, verify, and roll back DTwo policies and their pipeline + attachments — the system-of-record skill for policy lifecycle (create/update/publish/revert), pipeline + lifecycle (attach/deploy/verify), the session-state marker registry, and (when intent tools are enabled) + the intent registry. + TRIGGER when: user says create/modify a policy, block/allow/redact something, attach/detach policy, + set/update pipeline, publish/pin a policy version, or deploy gateway after a policy change; also manage a + marker/marker registry; or (only when intent tools are enabled) intent capture/registry/transitions or + intent/marker compatibility. Always pair with dtwo-policy-rego for the Rego authoring step. + SKIP when: task is purely explaining existing Rego with no save/attach/deploy intent (use dtwo-policy-rego); + or editing gateway YAML / MCP server entries (use dtwo-gateway-config). --- @@ -38,9 +36,9 @@ Before choosing an approach, understand how the pieces relate. From smallest to - **Marker** — a session-state flag one policy writes and another reads, letting policies coordinate *across* tool calls, directions, and upstream servers within a session (e.g. "PII was seen in an earlier response → block outbound sends now"). A marker is defined once in the registry, then used by a writer policy and a reader policy. See Managing Markers. - **Intent** *(only when the intent tools are enabled)* — a declared session *purpose* captured into session state and gated on by policies. Built on the same session-state mechanism as markers, with its own registry. See Intent Capture, including its availability gate. -**Mental model:** *policies* are the decision logic; the *pipeline* is where and when they run; the *gateway* is what enforces them once deployed; *markers* and *intent* are how policies share context beyond a single call. +**Mental model:** *policies* are the decision logic; the *pipeline* is where and when they run; the *gateway* is what enforces them once deployed; *markers* (and *intent*, only when the intent tools are enabled — see the Intent gate below) are how policies share context beyond a single call. -**Which skill does what:** this skill (`dtwo-gateway-policy`) owns the lifecycle and orchestration — policy records, pipelines, marker/intent registries, deploy, and verify. The companion `dtwo-policy-rego` skill owns the Rego logic *inside* a policy. Most authoring tasks use both. +**Which skill does what:** this skill (`dtwo-gateway-policy`) owns the lifecycle and orchestration — policy records, pipelines, the marker registry (and, when the intent tools are enabled, the intent registry), deploy, and verify. The companion `dtwo-policy-rego` skill owns the Rego logic *inside* a policy. Most authoring tasks use both. ## Choosing the Right Approach @@ -63,6 +61,20 @@ Guidance: - **Intent is for session-purpose gating** and is only available when the intent tools are present. If they are not, solve the request with policies and markers and do not mention intent. - **Compose small policies over one large one** — the companion `dtwo-policy-rego` skill explains why (single-concern policies are easier to test, order, and debug). +## Quick start: create → attach → deploy a policy + +The common path, end to end. Each step links to its detailed section below; the fully worked version is the End-to-End Example. + +1. **Resolve the gateway** — `dtwo-list-gateways` (by name) → capture the UID. +2. **Author the Rego** — hand off to the `dtwo-policy-rego` skill; pull identity claims first (`dtwo-list-claims`) only if the policy gates on identity. +3. **Validate & create** — `dtwo-validate-policy-rego`, then `dtwo-add-policy` (name, description, policy, packageName, direction) → capture the policy UID. +4. **Attach as a draft** — `dtwo-set-gateway-pipelines`, **omitting** `policyVersion`, preserving existing steps. +5. **Deploy** — confirm with the user, then `dtwo-deploy-gateway`; poll `dtwo-get-deployment` to `completed`. (Policy-only deploys hot-reload — no gateway restart.) +6. **Verify** — `dtwo-get-gateway-pipelines` to confirm attachment, then test allow and deny paths. +7. **Publish & pin** — once verified and with user OK, `dtwo-publish-policy`, then re-attach with `policyVersion` pinned and redeploy. + +For markers, session state, or (feature-gated) intent gating, see Managing Markers / Intent Capture. For removing a policy, see Deleting a Policy. + ## Prerequisites This skill requires the DTwo MCP server to be connected (`dtwo-*` tools must be loaded). If the tools are not available, ask the user to connect the DTwo MCP server first. @@ -73,12 +85,7 @@ The tools listed below reflect the initial set. The DTwo MCP server may add new ## High-level workflow -1. If the policy reads identity (claims like `sub`, `email`, `org_id`), pull tenant claims with `dtwo-list-claims` (see Tool Discovery → Finding Identity Claims). Skip for policies that only gate on tool names, arguments, or other non-identity inputs. -2. Identify the target gateway and relevant policy or policies. -3. Inspect the current policy draft, published versions, and pipeline attachments. -4. For any Rego authoring or modification, invoke `Skill("dtwo-policy-rego")` (or your host's equivalent) to produce or revise the code before proceeding. -5. Validate before creating, updating, publishing, or attaching. -6. Only deploy after confirming with the user, then verify both deployment completion and policy behavior. +See **Quick start** above for the create→attach→deploy path and **Creating a New Policy** / **Modifying an Existing Policy** for the detailed steps. Two invariants that apply throughout: author/modify Rego via the `dtwo-policy-rego` skill, and **validate before every create/update/publish/attach**; only deploy after confirming with the user, then verify. ## Rules @@ -101,7 +108,7 @@ The tools listed below reflect the initial set. The DTwo MCP server may add new | `dtwo-add-policy` | Validate and create a new policy (requires name, description, policy, packageName, direction). Optionally pass `writableKeySchema` to declare the session-state keys the policy is authorized to write — required for any policy that emits a marker (see Managing Markers) | | `dtwo-update-policy` | Update an existing policy's draft — any field (policy, packageName, name, description, direction, tags, `writableKeySchema`). Validates Rego when both policy and packageName are provided. `writableKeySchema` is tri-state: **omit** → leave unchanged; **`null`** → clear the field (policy keeps no writable keys); **`[]`** → set an explicit empty list (also leaves no writable keys — practically the same effect as `null`; use `null` as the reset); **`[...]`** → replace with that list | | `dtwo-publish-policy` | Publish the current draft as a new version | -| `dtwo-revert-policy` | Restore a published version back into the draft | +| `dtwo-revert-policy` | Restore a published `version` back into the draft. Pass `publish: true` to publish it immediately as well | | `dtwo-delete-policy` | Permanently delete a policy by UID. Fails if the policy is still attached to one or more gateways — detach it from every gateway first (see Deleting a Policy). Distinct from `dtwo-revert-policy`, which only restores a prior version | | `dtwo-list-claims` | Return the union of JWT claim names observed across the tenant, plus the issuers seen. Defaults to tenant-wide; pass `gatewayUid` to scope to a single gateway when the user asks. Call this when authoring or modifying identity-aware policies so rules can reference claims that actually exist; skip for policies that don't read `input.subject.claims`. | @@ -123,6 +130,8 @@ Markers are session-state flags that one policy writes and other policies read t When present, these tools manage the intent vocabulary and the rules that govern it. See the Intent Capture section for the workflow. +> **Customer-created intents are not yet reachable.** `dtwo-set-intent` today resolves the declared intent against the tool's **built-in** vocabulary, not this registry. So an intent you create with `dtwo-create-intent` (and any transitions or compatibility rows referencing it) **cannot actually be declared/entered via `set_intent`** until registry-driven resolution ships — the vocabulary you build is inert for now. Only manage customer-tier intents when the user explicitly wants to pre-build that vocabulary; don't present it as immediately usable for gating. + | Tool | Purpose | |------|---------| | `dtwo-set-intent` | Declare the current session intent (captured by the gateway's egress capture policy). Resolves against the tool's **built-in** intent vocabulary, not the registry (registry-driven resolution is planned). Not customer-available yet — stays behind `enable_intent_tools` until registry-driven resolution ships | @@ -207,6 +216,8 @@ If `dtwo-get-gateway-config` shows an MCP server named `atlassian-jira-mcp`, and Every policy `description` is structured markdown with up to three sections. The field is rendered in a markdown editor that supports headings, bold, italic, lists, and code blocks. +> **Naming note:** the `## Intent` heading here is the policy-description *field* (a one-line statement of the policy's goal). It is unrelated to the **Intent Capture** feature (session-purpose declared via `set_intent`) covered later — every policy has this description field regardless of whether intent capture is enabled. + ```markdown ## Intent @@ -437,13 +448,11 @@ Markers are session-state flags that policies write and later policies read to g Marker tools are always available (they do not require `enable_intent_tools`). The full lifecycle — register, author writer + reader, attach, deploy — runs through this skill plus `dtwo-policy-rego` for the Rego. The Rego authoring patterns (emitting `session_writes["marker::"]`, walking `input.context.session.policies` to read, and the `writableKeySchema` gotchas) live in the companion `dtwo-policy-rego` instructions — load that skill for the writer/reader bodies. -**Start simple — the minimal marker is a boolean flag.** A writer stamps `marker::` when it observes a condition; a reader denies (or transforms) whenever that key is present. Presence *is* the signal — no value semantics needed. That flag pattern (the PII example used throughout this section) is the recommended starting point; reach for value-carrying markers only when a flag won't do. Counters and other read-modify-write markers are possible but more involved — a self-incrementing writer has to read its own prior value and re-emit on every call, which keeps refreshing (pinning) the TTL — so they aren't a good first marker. +**Marker vs. general session key.** A registered marker is the right tool when the signal is meant to be **shared across policies** (a different policy reads it) or wants to be a registered, discoverable, governed session-wide indicator. For state that is **targeted to a single policy or one coordinated ingress/egress set**, an unregistered *general session key* (a bare `session_writes` key, no `marker:` prefix, no registry entry) is the lighter choice. Neither is access-isolated, and the bare keyspace isn't namespaced — see the decision table in `dtwo-policy-rego` → Session State & Markers → "Marker vs. general session key" before choosing. -**Know these limits before you design** (full list under Marker constraints today): +**Start simple — the minimal marker is a boolean flag.** A writer stamps `marker::` when it observes a condition; a reader denies (or transforms) whenever that key is present. Presence *is* the signal — no value semantics needed. That flag pattern (the PII example used throughout this section) is the recommended starting point; reach for value-carrying markers only when a flag won't do. Counters and other read-modify-write markers are possible but more involved — a self-incrementing writer has to read its own prior value and re-emit on every call, which keeps refreshing (pinning) the TTL — so they aren't a good first marker. -- **No runtime inspection** — no tool reads a session's active markers; you verify behaviorally (see Verifying a marker pipeline). -- **No manual clearing** — a marker lifts only when its TTL expires; there is no unset tool. -- **Tenant + user scope** — marker state persists for a user across reconnects and new sessions until the TTL expires; opening a fresh session does not clear it. +**Know these limits before you design:** no runtime inspection (verify behaviorally), no manual clearing (a marker lifts only on TTL expiry), and tenant+user scope (state survives reconnects/new sessions until TTL). Full detail — and why each bites — is under **Marker constraints today** below. ### Registering a marker @@ -460,7 +469,7 @@ dtwo-create-marker( - The full key is `marker:acme:pii_detected`. Customer markers live under any namespace except the reserved `internal` and `dtwo`. - **Keep the key simple.** The tool requires only non-empty `namespace`/`markerId`; the character-shape rules are validated server-side (when a writer policy is saved and at deploy), not at this tool boundary. In practice, use lowercase alphanumerics with underscores or hyphens and avoid dots, slashes, and spaces, so the key is accepted everywhere it's referenced (registry entry, `writableKeySchema` name, and `session_writes` key must all match exactly — see Authoring the writer policy). -- `minimumTtlSeconds` is the **intended floor** for a writer policy's `ttlSeconds` — but it isn't enforced yet (see the `ttlSeconds` note under Authoring the writer policy), so keep the two in sync manually. Adjust later with `dtwo-update-marker`. +- `minimumTtlSeconds` is the **intended floor** for a writer policy's `ttlSeconds`. Adjust later with `dtwo-update-marker`. (Whether the floor is enforced, and the keep-in-sync caveat, are stated once under Authoring the writer policy → `ttlSeconds` — the canonical note.) ### Authoring the writer policy — `writableKeySchema` @@ -541,8 +550,37 @@ Two policies do the enforcement: - **Egress capture** — captures the declared intent into session state when `dtwo-set-intent` is invoked, validates it against the registry, normalizes the category, denies disallowed transitions, and denies when a currently-active marker is registered incompatible with the proposed intent (`intent_marker_incompatible`). - **Intent-required gate** — optional: denies every tool call until an intent has been set (`dtwo-set-intent` itself is always allowed so the agent can declare). + > **Management lockout (same shape as the deny-policy self-lock).** When the intent-required gate is on **and the DTwo MCP server is behind this gateway with your client routing `dtwo-*` through it**, your management calls are themselves denied with `intent_required` until you `set_intent` — you'll see it the moment you try to inspect or change the gateway. Declaring an intent clears it (`set_intent` is never gated). This only bites for DTwo-behind-the-gateway setups; if your DTwo MCP server runs *outside* this gateway, management traffic bypasses the gate and this does not apply. The intent can also lapse mid-session (TTL/clear), so you may need to re-declare — do so explicitly, don't auto-fire `set_intent` as a silent recovery. + **These are platform-managed policies — end users do not author, attach, copy, or modify them, and you should not offer to.** They are being moved to automatic injection when intent capture is enabled; the platform owns their bodies and wiring (upstream-server naming, internal UIDs), and their Rego may not be visible to users. If a user asks to write or change intent-capture Rego, decline and point them at the platform-managed feature rather than reconstructing it. The only intent surface users drive is the **registry** — the intent vocabulary, transitions, and marker compatibility (below), when the tools are enabled. +**Gating a tenant policy on the current intent** is allowed, though — a user policy may *read* the captured intent to decide access (e.g. "only allow this tool under the `internal:debug`/`internal:explore` intents" — compare against the full FQIDs, not the short form). When it does, it must read intent **only** through the platform helper `data.dtwo.lib.intent_match.*`, never via a direct `input.context.session.policies` read (a raw read is spoofable and couples to internals). The category values to compare against are the intent FQIDs from `dtwo-list-intents`. The Rego belongs to the companion `dtwo-policy-rego` skill — see its Intent-capture policies → Reading the session intent. + +### Intent transitions (discoverability) + +Moving between intents is itself governed, and a `set_intent` can be **denied for two independent reasons** — in both cases the current intent stays unchanged. This surprises authors mid-test: + +- **`intent_change_disallowed` — transition rules.** The registry forbids that from→to move. Each entry carries `transitionsFromMode` (`ALL` / `RESTRICTED` / `NONE`) and, when `RESTRICTED`, an `allowedTransitionsFrom` list of the intents you may arrive *from*. Inspect it with `dtwo-list-intents` (or `dtwo-list-intent-transitions` when present). To reach a restricted target you may need an intermediate hop (e.g. `explore → debug → deploy` when `deploy` only allows arrival from `debug`/`review`/`incident_response`). +- **`intent_marker_incompatible` — an active marker blocks the target.** If a currently-set marker is registered incompatible with the intent you're switching *to*, the capture policy denies the `set_intent` (see Intent/marker compatibility below). So a marker stamped earlier in the session can make an otherwise-legal transition fail — and because markers only lift on TTL expiry (no clear tool), the transition stays blocked until the marker ages out. If a `set_intent` fails and the transition rules allow it, check for an active incompatible marker. + +Both are distinct from any tenant gate you author on the intent value. + +### Verifying an intent gate + +An intent-gating tenant policy is verified behaviorally (there's no tool that reports the live intent). Mirror the marker verification flow, and mind the ordering traps: + +1. **Confirm the deploy + attachment** as for any policy — poll `dtwo-get-deployment` to `completed`, then `dtwo-get-gateway-pipelines`. +2. **No-intent case → deny.** With no intent set, call the gated tool; a well-formed gate denies (the helper returns false when no intent is set). Proves the gate is live. +3. **Permitted-intent case → allow.** `set_intent` to one of the policy's allowed categories, then call the gated tool; expect success. (Confirm before calling `set_intent` — treat it as a state change, not an automatic step.) +4. **Disallowed-intent case → deny.** Move to an intent *outside* the allow-set and confirm the tool denies again — but remember step (3)'s intent may restrict which target you can transition to (see Intent transitions), so pick a reachable one. + +Watch for: + +- **Reachability, not policy, may block a test.** A `set_intent` can fail for reasons unrelated to your gate: `intent_change_disallowed` (transition rules) or `intent_marker_incompatible` (an active marker blocks the target intent — see Intent transitions). Don't chase either as a policy bug; resolve the transition/marker first, then test the gate. +- **Intent can lapse.** TTL/clear can drop the intent between steps; if a previously-allowed call starts denying, re-check the current intent before suspecting the policy. +- **Compare against FQIDs.** The gate matches `internal:debug` etc. (registry `name`), not the short form echoed by `set_intent` — an allow-set of short forms silently never matches. +- **Fail-closed when intent is disabled.** If the gate is deployed to a gateway where intent capture is *not* enabled, the helper is always false and the gated tool denies for everyone — verify against a gateway with the feature on. + ### Intent/marker compatibility If a marker should block switching into a given intent, register a compatibility row so the egress capture denies `set_intent` while that marker is active: diff --git a/dtwo/skills/dtwo-policy-rego/SKILL.md b/dtwo/skills/dtwo-policy-rego/SKILL.md index c925f0f..6493dad 100644 --- a/dtwo/skills/dtwo-policy-rego/SKILL.md +++ b/dtwo/skills/dtwo-policy-rego/SKILL.md @@ -1,18 +1,16 @@ --- name: "dtwo-policy-rego" description: | - Generate, modify, explain, and debug Rego policy code for the DTwo MCP Gateway (ingress and egress), - including the input schema, allow/deny/transform patterns, debugging techniques, and policy-store - catalog contribution structure. - TRIGGER when: user asks to write/modify/explain/debug a Rego policy, says "block/allow/redact/transform" - a tool call or response, mentions OPA, package paths, `input.payload`, `default allow`, or pastes Rego - for review; asks to write a marker writer/reader policy, `session_writes`, session state, or (only when the - intent tools are enabled) to recognize/explain the platform-managed intent-capture policies (not user-authored); asks to contribute to `dtwoai/policy-store`, create - catalog policy files, or update app, industry, bundle, manifest, or tests files for reusable DTwo policies; - also when diagnosing blanket denies or transform conflicts. Pair with dtwo-gateway-policy whenever the - resulting Rego must be saved, attached, or deployed. - SKIP when: task is policy CRUD or pipeline attachment that does not change Rego (use dtwo-gateway-policy - alone); task is general OPA usage outside the MCP Gateway; task is editing gateway YAML (use dtwo-gateway-config). + Generate, modify, explain, and debug Rego policy code for the DTwo MCP Gateway (ingress and egress) — + input schema, allow/deny/transform patterns, markers/session-state, debugging, and policy-store catalog + contribution structure. + TRIGGER when: user asks to write/modify/explain/debug a Rego policy; says block/allow/redact/transform a + tool call or response; mentions OPA, package paths, input.payload, or default allow; pastes Rego for + review; writes a marker writer/reader or session_writes; (only when intent tools are enabled) recognizes + the platform intent-capture policies; contributes to the dtwoai/policy-store catalog; or diagnoses blanket + denies or transform conflicts. Pair with dtwo-gateway-policy when the resulting Rego must be saved or deployed. + SKIP when: policy CRUD or pipeline attachment that does not change Rego (use dtwo-gateway-policy); general + OPA usage outside the MCP Gateway; or editing gateway YAML (use dtwo-gateway-config). --- @@ -23,7 +21,7 @@ You are a Rego policy expert for the DTwo MCP Gateway. You translate natural lan ## Where a policy fits -This skill owns the Rego *inside* a policy — the allow / deny / transform / `session_writes` logic for a single tool call. The surrounding pieces it plugs into (a **pipeline** is the ordered list of policies on a gateway direction; a **gateway** enforces them once deployed; **markers** and **intent** let policies share session state) are owned by the companion `dtwo-gateway-policy` skill — see its Core Concepts and Choosing the Right Approach sections to pick the right mechanism before writing Rego. A quick orientation for what you write here: +This skill owns the Rego *inside* a policy — the allow / deny / transform / `session_writes` logic for a single tool call. The surrounding pieces it plugs into (a **pipeline** is the ordered list of policies on a gateway direction; a **gateway** enforces them once deployed; **markers** — and **intent**, only when the intent tools are enabled — let policies share session state) are owned by the companion `dtwo-gateway-policy` skill — see its Core Concepts and Choosing the Right Approach sections to pick the right mechanism before writing Rego. A quick orientation for what you write here: - **Deny / allow** — decide a single call from the call itself (ingress) or its response (egress). - **Transform** — rewrite request arguments (ingress) or redact response content (egress). @@ -55,7 +53,7 @@ This skill is typically used alongside others. Invoke them via the `Skill` tool - **Keep each policy simple and focused on a single concern.** A policy should do one thing — block a specific tool, rewrite a specific argument, redact a specific pattern. Multiple policies can be attached to a gateway, so prefer composing several small, single-purpose policies over one large policy that handles multiple concerns. This makes policies easier to understand, test, and debug independently. - Every policy must declare a `package` name following the convention `..` — where `` is the MCP server (e.g., `jira`, `slack`), `` is `ingress` or `egress`, and `` describes the policy's intent (e.g., `readonly`, `deny_comment`, `pii_redaction`). If the policy is not tool-specific, the tool segment can be omitted (e.g., `egress.pii_redaction`). If the user does not supply a name, choose one based on this convention -- Policies that **deny or block** requests must default to deny: `default allow := false`. Policies that only **transform** data (e.g., PII redaction, query rewriting) and never deny should use `default allow := true` — this avoids needing explicit allow rules for every unrelated tool +- Choose the `default allow` value by the policy's **primary job**. An **access-control** policy whose default stance is to block should default to deny (`default allow := false`) and enumerate `allow` rules. A policy that only **transforms** (PII redaction, query rewriting) or that **allows broadly and denies a specific condition** (e.g. a marker/intent reader that blocks one tool when a signal is set) should use `default allow := true` with targeted `allow := false if { ... }` rules — this avoids needing an explicit `allow` for every unrelated tool. Both are valid; see "Valid Rego Patterns (Not Bugs)" for the `default allow := true` + `allow := false if` shape - Use **separate top-level rules** for `allow`, `reasons`, `reason`, and `transform` — do NOT return structured decision objects - The `allow` rule is a boolean (`true`/`false`), not an object - The `reasons` rule is a set that collects human-readable denial messages @@ -71,101 +69,32 @@ This skill is typically used alongside others. Invoke them via the `Skill` tool - Always return generated or modified policies in a fenced `rego` code block - When generating or modifying a policy, include a brief explanation of what the policy does and its direction (ingress/egress) - When explaining an existing policy, no code block is needed unless referencing specific rules -- When contributing to the DTwo Policy Store repository, produce or edit the repository files described in "Policy Store Catalog Contributions" instead of returning only a standalone fenced Rego block. +- When contributing to the DTwo Policy Store repository, produce or edit the repository files described in `references/policy-store-catalog.md` (see the Policy Store Catalog Contributions pointer near the end) instead of returning only a standalone fenced Rego block. -## Policy Store Catalog Contributions - -Use this section when the target is the `dtwoai/policy-store` repository rather than a tenant-local gateway policy. The policy-store catalog is a curated source of reusable policies; this skill still owns Rego correctness, while the repository owns file layout, metadata, tests, and manifest generation. - -### Repository layout - -```text -apps/ - / - README.md - / - policy.md - tests.yaml +## Quick start: the three policy shapes -industries/ - / - README.md +Most policies are one of these. Copy the matching skeleton, then see the DTwo Gateway Input Schema for the fields and Examples for fuller, correct versions. Compare tool names case-insensitively against `input.resource.name`. -bundles/ - / - README.md - -manifest.json -schema.json +**Deny** (block a request/response; default-deny): +```rego +package .. +import future.keywords.if +default allow := false +allow if { lower(input.resource.name) != "-" } # pass everything else through +reason := "why this was blocked" if { not allow } ``` -- Put canonical policy bodies only under `apps///`. -- Never duplicate policy bodies under `industries/` or `bundles/`; those directories contain landing pages that link to canonical app policies. -- Use `` as the lowercase, hyphenated MCP server or SaaS app slug commonly configured on a gateway, such as `slack`, `jira`, `github`, or `postgres`. -- Use `` as the lowercase, hyphenated purpose, such as `block-secrets`, `readonly`, or `pii-redaction`. -- Do not add per-policy `README.md` or `metadata.json` files. Human documentation and all metadata live in `policy.md` frontmatter. -- Treat `manifest.json` as generated from policy frontmatter. Do not hand-edit it except through the manifest generator. - -### `policy.md` - -Each policy directory requires a `policy.md` file with YAML frontmatter followed by one fenced `rego` block. - -Frontmatter must include the required schema fields from `schema.json`: - -- `name` -- `tags` -- `publishedAt` -- `description` -- `direction` -- `apps` -- `schemaVersion` - -Include `industries`, `bundles`, and `minimumGatewayVersion` when they apply. Put the policy's human-facing documentation in `description`: what it does, when to use it, assumptions, limitations, examples, and relevant composition notes. - -Catalog policies are stricter than tenant-local examples: - -- Use PARC fields for action, resource, and identity: `input.action`, `input.resource`, `input.subject`, and `input.context`. -- Do not use deprecated legacy aliases in catalog policies: `input.kind`, `input.payload.name`, or `input.user`. -- Use `input.payload.args` and other hook-specific `input.payload` data for the actual request/response payload when needed. -- Compare tool names case-insensitively with `lower(input.resource.name)`. -- Use `object.get(obj, key, default)` for optional fields and missing claims. -- Use only IdP-supplied claims in `input.subject.claims`; do not authorize on stripped ContextForge-internal claims such as `is_admin`, `teams`, or nested `user`. +**Allow-list** (permit only specific tools; default-deny) — same shape, with an `allow if { lower(input.resource.name) == "-" }` rule per permitted tool. -### `tests.yaml` - -Each policy directory requires a `tests.yaml` file containing a top-level YAML array of test cases. Include at least one positive and one negative case. For deny policies, this usually means one allow case and one deny case; for transform-only policies, use a passthrough case and a transform-applied case. - -Each test case uses this shape: - -```yaml -- description: what this case demonstrates - input: - resource: { ... } - action: tool_pre_invoke - payload: { ... } - output: - expectedResult: deny - expectedReason: exact reason when relevant - transformApplied: true - transform: { replacement: "[REDACTED]" } - transformedArgsContain: { jql: "project != HR" } +**Transform** (rewrite a request arg on ingress, or redact a response on egress; never denies): +```rego +package .. +import future.keywords.if +default allow := true +transform := { "redact_patterns": ["..."], "replacement": "[REDACTED]" } if { input.mode == "output" } ``` -- The runner feeds only each case's `input` object to OPA. Do not nest the PARC decision object under another `input` key inside the test case input. -- `output.expectedResult` is required and must be `allow` or `deny`. -- `output.expectedReason`, `transformApplied`, `transform`, and `transformedArgsContain` are optional and should be included only when they assert relevant behavior. -- For identity-aware policies, document required IdP claim names and missing-claim defaults in the `policy.md` description. - -### Registering a policy - -1. Create `apps///policy.md` and `tests.yaml`. -2. Add a row or link to `apps//README.md`. -3. If the policy belongs to an industry or bundle, list the slug in `policy.md` frontmatter and add a link from the matching `industries//README.md` or `bundles//README.md`. -4. For new apps, industries, or bundles, create the corresponding directory and `README.md`. -5. Run `pnpm manifest`, commit the generated `manifest.json`, then run `pnpm manifest:check`. -6. Run `pnpm test`; it requires the OPA CLI on `PATH` or `OPA_BIN` set. - -Do not invent new manifest fields. Match existing policy frontmatter and open an issue first if the schema lacks a needed field. +Rules of thumb: **ingress** to decide from the request alone, **egress** to inspect the response; `default allow := false` for deny policies, `default allow := true` for transform-only; read optional fields with `object.get(...)`. For cross-call state see Session State & Markers; for the multi-condition deny form see Policy Structure. ## Handling Ambiguous Requests @@ -178,12 +107,14 @@ This skill has four primary modes: - **Generate** — produce a complete Rego policy from a natural language requirement. Before returning, verify all `input.*` paths used in the policy exist in the DTwo Gateway Input Schema below. - **Modify** — change an existing Rego policy based on instructions, preserving its logic and style. If the policy contains syntax errors or schema violations, flag them to the user before applying modifications. - **Explain** — describe an existing policy in plain language: what it permits/blocks/modifies, its direction (ingress vs egress), what data it inspects, what triggers allow/deny, and any transformations applied. Flag syntax errors or schema violations as part of the explanation. -- **Contribute** — create or update `dtwoai/policy-store` catalog artifacts (`policy.md`, `tests.yaml`, landing-page links, and generated manifest) using the repository layout and contribution flow above. +- **Contribute** — create or update `dtwoai/policy-store` catalog artifacts (`policy.md`, `tests.yaml`, landing-page links, and generated manifest) using the repository layout and contribution flow in `references/policy-store-catalog.md`. All modes must follow the Core Rules above. ## DTwo Policy Structure +> **OPA version & the `if`/`contains`/`in` keywords.** The gateway runs OPA v1.x (currently v1.17). On OPA ≥ 1.0 the `if`, `contains`, `in`, and `every` keywords are part of the language, so `import future.keywords.*` is **optional and a no-op** — it neither helps nor hurts. The worked Examples below omit the import; the marker/session-state snippets include it. Both compile and behave identically on this gateway. Follow whichever the surrounding policy uses; don't add the import to "fix" an example that lacks it. + Every DTwo policy follows this structure with separate top-level rules. Use `default allow := false` for policies that deny requests, or `default allow := true` for policies that only transform data. ```rego @@ -669,7 +600,7 @@ allow if { allow if { input.action == "tool_pre_invoke" lower(input.resource.name) == "atlassian-editjiraissue" - not disallowed_field + count(disallowed_fields) == 0 } # Extract the fields being edited in the request @@ -678,8 +609,11 @@ edited_fields := {k | k := object.keys(fields)[_] } -# Detect if any disallowed field is being modified -disallowed_field := f if { +# Collect ALL disallowed fields as a set. This must be a set comprehension, not a +# complete rule (`disallowed_field := f if { f := edited_fields[_]; ... }`): a +# complete rule assigns a single value, so editing two disallowed fields makes it +# derive two values and OPA raises eval_conflict_error instead of denying cleanly. +disallowed_fields := {f | f := edited_fields[_] not allowed_fields[f] } @@ -687,7 +621,7 @@ disallowed_field := f if { reasons contains msg if { input.action == "tool_pre_invoke" lower(input.resource.name) == "atlassian-editjiraissue" - f := disallowed_field + f := disallowed_fields[_] msg := sprintf("Editing the field '%s' is not allowed. Only description, labels, priority, environment, and comment may be edited.", [f]) } @@ -777,7 +711,7 @@ transform := { "replacement": "[PROFANITY]", } if { input.mode == "output" - lower(input.tool_metadata.name) == "echo-echo" + lower(input.resource.name) == "echo-echo" } ``` @@ -946,6 +880,24 @@ Start with the simplest shape — a **boolean flag** whose *presence* is the who Registering the marker vocabulary, attaching the `writableKeySchema`, and deploying are lifecycle operations owned by the companion `dtwo-gateway-policy` instructions (Managing Markers). This section covers only the **Rego**. +### Marker vs. general session key + +`session_writes` can write two flavors of key. Both are the same underlying `session_state_kv` mechanism — same `(tenant, subject, writer_id, key)` scoping, same TTL-from-schema, same read path — so the choice is about **design intent and governance**, not capability: + +| | **Marker** (`marker::`) | **General session key** (bare name, e.g. `pipeline_stage`) | +|---|---|---| +| Intended for | A **shared, named signal across policies** and a general session-wide indicator — one policy stamps it, *other* policies (different tools/directions/servers) gate on it. | State **targeted to a specific use, isolated to one policy or a coordinated ingress/egress set** — private working state, not a broadcast signal. | +| Registry | **Required** — register with `dtwo-create-marker`; the deploy validator rejects an unregistered marker key. | **None** — bare keys need no registry entry. | +| Governance | Discoverable via `dtwo-list-markers`, carries tags + a `minimumTtlSeconds` floor, and (when the intent tools are enabled) can participate in intent/marker compatibility rules. | No registry, tags, TTL floor, or compatibility hooks — lighter-weight, ungoverned. | +| Declare in `writableKeySchema` | `name: "marker::"`, add `"x-d2-is-marker": true` to the `jsonSchema`. | `name: ""`, omit the `marker:` prefix and `x-d2-is-marker`. | + +**Rule of thumb:** if another, separate policy is meant to read the signal — or you want it registered, discoverable, and governed — use a **marker**. If the state only coordinates within your own policy or your own ingress/egress pair and doesn't belong in a shared vocabulary, a **general session key** is the lighter choice. + +**Two honesty caveats:** + +- **Neither is access-isolated.** "Isolated to a policy" is a *design convention*, not an enforcement boundary — any policy in the session can read any key (marker or bare) via `session.policies[writer_uid][key]` if it knows the key. Don't treat a bare key as private/secret storage. +- **The bare keyspace isn't namespaced.** Two unrelated policies that pick the same bare key name will collide (each writes under its own `writer_id`, but a walk-all-writers read sees both). Markers avoid this by construction via `marker::`. Pick distinctive bare-key names, or use a marker when you need a collision-safe, shared name. + ### Writing a marker (`session_writes`) A writer emits `session_writes["marker::"] := ` when its trigger fires. The canonical shape: @@ -1030,6 +982,7 @@ reason := "PII was detected earlier in this session; outbound Slack sends are bl ``` - The walk-all-writers pattern (`some writer_uid; input.context.session.policies[writer_uid][key]`) is "present under *any* writer is truthy." To trust only a specific writer, filter on `writer_uid == ""`. +- **This pattern is for reading *marker* keys only.** Do **not** use it — or any direct `input.context.session.policies` read — to read the platform **intent** (see Intent-capture policies → Reading the session intent). "Present under any writer" is exactly wrong for intent: a tenant policy could stamp an intent-shaped value under its own writer slot and a walk-all-writers read would honour it, spoofing the session intent. Read intent only through the platform helper, which is pinned to the trusted intent-capture slot. - Deny reasons are user-visible — explain what to do about the block (e.g. "wait for the marker TTL to expire"). Avoid "start a new session": marker state is scoped to tenant + user and survives reconnecting, so a new session for the same user won't clear it. ### `writableKeySchema` (attached via `dtwo-add-policy` / `dtwo-update-policy`) @@ -1053,6 +1006,54 @@ Marker-key *shape* is validated server-side (the backend on save, and at deploy) One behavior worth knowing when explaining them: intent/marker compatibility is **one-directional and set-time only** — the check runs at `set_intent` time; a marker raised *after* an intent is set does not retroactively deny. +### Reading the session intent in a tenant policy + +Authoring the intent-*capture* Rego is off-limits (above), but a tenant policy may legitimately **gate a tool on the current session intent** (e.g. "only allow this tool under a `debug` or `explore` intent"). When you do, read the intent **only** through the platform helper library `data.dtwo.lib.intent_match`: + +| Helper | Returns | +|---|---| +| `data.dtwo.lib.intent_match.current_intent(input)` | The full intent object (`{category, description, set_at}`); **undefined** when no intent is set. | +| `data.dtwo.lib.intent_match.current_category(input)` | Just the category FQID (e.g. `internal:debug`); undefined when unset. | +| `data.dtwo.lib.intent_match.category_in(input, allowed)` | `true` when the current category is in the `allowed` set of FQIDs; `false` otherwise (including when no intent is set). | + +**Do not read the intent from `input.context.session.policies` directly**, and do not hand-roll a walk-all-writers read for it. The helper is pinned to the trusted platform intent-capture slot, which tenant policies cannot write; a direct read is spoofable (any policy's own writer slot could supply an intent-shaped value) and couples your policy to internal storage details. Reading intent is the one case where the marker walk-all-writers pattern is the wrong tool. + +The category values you compare against are the intent **FQIDs** — the `name` field returned by `dtwo-list-intents` (e.g. `internal:default`, `internal:debug`, `internal:explore`), **not** the short form echoed in the `set_intent` response. Pull the registry to confirm the exact FQIDs rather than guessing. + +Example — an ingress policy that permits one tool only under low-risk intents, and passes every other tool through untouched: + +```rego +package echo.ingress.intent_gate + +import future.keywords.if + +default allow := false + +# Intent FQIDs (from dtwo-list-intents `name`) permitted to call the gated tool. +_allowed_intents := {"internal:default", "internal:debug", "internal:explore"} + +# This policy only gates one tool — everything else passes through. +allow if { + lower(input.resource.name) != "echo-echo" +} + +allow if { + lower(input.resource.name) == "echo-echo" + data.dtwo.lib.intent_match.category_in(input, _allowed_intents) +} + +reason := "This tool is only permitted under the 'default', 'debug', or 'explore' intents. Declare one with set_intent and retry." if { + lower(input.resource.name) == "echo-echo" + not data.dtwo.lib.intent_match.category_in(input, _allowed_intents) +} +``` + +Notes: + +- **Availability — safe to reference on any gateway.** The `dtwo.lib.intent_match` library is shipped into **every** policy bundle unconditionally (independent of the intent flag), so a reference to `data.dtwo.lib.intent_match.*` always resolves and compiles — it will *not* cause an "undefined function" bundle failure when intent capture is off. It only returns real values when intent capture is enabled; with it off there's no captured intent, so `current_intent` is undefined and `category_in` is simply always `false` — meaning a gate like the one above would deny the gated tool on a no-intent gateway. Design the default accordingly (and see the availability gate at the top of this section before surfacing intent behavior at all). +- **Never echo the intent value into a deny `reason` or a `transform`.** The intent `description` is free text the caller supplied; use the intent for the *decision*, not for output. +- **A `default allow := false` gate still risks self-lock** if it fronts the DTwo MCP server — keep the non-gated-tool passthrough (as above) so `dtwo-*` management calls are unaffected. See the self-lock pitfall in Common Pitfalls. + ## Handling Parse and Access Failures Rego rules fail **silently** — if any expression in a rule body fails (e.g., `json.unmarshal` on non-JSON text, or accessing a missing key), the entire rule body does not match. No error is raised. This means the policy's behavior on bad data depends on how the rules are structured. @@ -1141,11 +1142,11 @@ When a policy isn't behaving as expected, use these techniques to inspect what t ### Finding the Correct Tool Name -Tool names in `input.payload.name` are constructed as `-`, where `` is the name given to the MCP server when it was configured on the gateway. For example, if an Atlassian MCP server is named `atlassian-jira-mcp` on the gateway, its `getjiraissue` tool appears as `atlassian-jira-mcp-getjiraissue`. A Slack server named `slack-mcp` would have tools like `slack-mcp-slack-send-message`. +Tool names appear in `input.resource.name` (PARC; the legacy alias `input.payload.name` carries the same value) and are constructed as `-`, where `` is the name given to the MCP server when it was configured on the gateway. For example, if an Atlassian MCP server is named `atlassian-jira-mcp` on the gateway, its `getjiraissue` tool appears as `atlassian-jira-mcp-getjiraissue`. A Slack server named `slack-mcp` would have tools like `slack-mcp-slack-send-message`. The client may display a different name (e.g., `mcp__dtwo__atlassian-jira-mcp-getjiraissue` in Claude Code) — the gateway prefix is stripped, but the server name prefix is preserved. -> Note: this is the gateway-to-OPA name (what `input.payload.name` contains inside a Rego policy), **not** the MCP client invocation name you call as a tool. The latter is covered in the companion `dtwo-gateway-config` and `dtwo-gateway-policy` instructions. +> Note: this is the gateway-to-OPA name (what `input.resource.name` contains inside a Rego policy), **not** the MCP client invocation name you call as a tool. The latter is covered in the companion `dtwo-gateway-config` and `dtwo-gateway-policy` instructions. **Do not guess tool names.** The server name is configured by the gateway admin and is not standardized. Use a debug policy to discover the exact name the gateway passes, or follow the tool-discovery guidance in the companion `dtwo-gateway-policy` instructions to discover tool names and argument schemas via the DTwo MCP server when available. @@ -1159,13 +1160,13 @@ allow := false if { true } -reasons contains sprintf("Debug - name: %s, args: %v", [input.payload.name, input.payload.args]) if { +reasons contains sprintf("Debug - name: %s, args: %v", [input.resource.name, input.payload.args]) if { true } # --- END DEBUG RULES --- ``` -This blocks all tool calls and returns the tool name and arguments in the denial message. +This blocks all tool calls and returns the tool name and arguments in the denial message. (`input.resource.name` is the PARC field; the legacy alias `input.payload.name` holds the same value if you see it in older policies. Arguments stay at `input.payload.args`.) To inspect **identity** (the `subject` block and the JWT-derived claims), add: @@ -1179,23 +1180,23 @@ reasons contains sprintf("Debug - user: %s, subject.sub: %s, claims: %v", [ Use this to confirm what specific claim *values* are present in `input.subject.claims` for the current caller. To enumerate just the *names* the tenant has observed (across all callers, without attaching a policy), prefer `dtwo-list-claims` from the DTwo MCP server — tenant-wide by default, optionally scoped with `gatewayUid`. -> **Self-lock warning.** An always-deny dump policy attached to a gateway will block **every** tool call on that gateway, including any DTwo MCP tools your client routes through it. If you manage policies via an MCP client that goes through the same gateway, you will lock yourself out and have to detach the policy from the DTwo web UI (or another client/gateway) to recover. Attach to a gateway you are **not** using for policy management, or be prepared to detach via the UI. +> **Self-lock warning.** An always-deny dump policy blocks **every** call on that gateway — including the `dtwo-*` management tools if your client routes through it — so you can lock yourself out. Attach it to a gateway you are *not* managing through, or be ready to detach via the DTwo web UI. Full mechanism and the management-bypass pattern: see Common Pitfalls → "Self-locking with an always-deny ingress policy". To scope the debug to a specific tool pattern (e.g., only Atlassian tools): ```rego # --- TEMPORARY DEBUG RULES — remove after debugging --- allow := false if { - startswith(lower(input.payload.name), "atlassian-") + startswith(lower(input.resource.name), "atlassian-") } -reasons contains sprintf("Debug - name: %s, args: %v", [input.payload.name, input.payload.args]) if { - startswith(lower(input.payload.name), "atlassian-") +reasons contains sprintf("Debug - name: %s, args: %v", [input.resource.name, input.payload.args]) if { + startswith(lower(input.resource.name), "atlassian-") } # --- END DEBUG RULES --- ``` -For **egress** policies, the input structure is different — use `input.tool_metadata.name` and `input.payload.text` instead: +For **egress** policies the tool output is in `input.payload.text`; the tool name is still `input.resource.name` (PARC populates it on egress too — the legacy `input.tool_metadata.name` holds the same value): ```rego # --- TEMPORARY EGRESS DEBUG RULES — remove after debugging --- @@ -1203,7 +1204,7 @@ allow := false if { true } -reasons contains sprintf("Debug - tool: %s, text: %v", [input.tool_metadata.name, input.payload.text]) if { +reasons contains sprintf("Debug - tool: %s, text: %v", [input.resource.name, input.payload.text]) if { true } # --- END EGRESS DEBUG RULES --- @@ -1225,7 +1226,7 @@ If all tool calls are being denied (including tools on unrelated MCP servers), t ### Debugging Checklist -1. **Verify tool names:** Use the debug policy above to confirm `input.payload.name` +1. **Verify tool names:** Use the debug policy above to confirm `input.resource.name` 2. **Verify argument keys:** Check the exact keys in `input.payload.args` (e.g., `issueIdOrKey` vs `issueKey`) 3. **Check compilation:** Ensure every `.rego` file has valid syntax and a `package` declaration 4. **Check package paths:** Ensure step policy packages match the aggregator's `data.*` references @@ -1244,7 +1245,7 @@ default allow := true # This is NOT a conflict — the default only fires when the condition below doesn't match allow := false if { - input.payload.name == "some-tool" + input.resource.name == "some-tool" some_blocking_condition } ``` @@ -1263,8 +1264,8 @@ Policies often contain hardcoded values like transition IDs, project keys, or to - **Assuming fields are always present:** External APIs don't always return the same fields. For example, Jira may expose `emailAddress` for some users but not others depending on privacy settings. Always use `object.get(obj, key, default)` and consider fallback checks (e.g., matching on `displayName` when `emailAddress` is missing). - **Forgetting the `default allow` declaration:** Without a default, `allow` is `undefined` when no rule matches, which OPA treats differently than `false`. Always include `default allow := false` (for deny policies) or `default allow := true` (for transform-only policies). - **Direct key access on optional fields:** `input.payload.args.someField` will cause the rule to silently fail if `someField` doesn't exist. Use `object.get(input.payload.args, "someField", "")` instead. -- **Guessing tool names:** Tool names in `input.payload.name` are prefixed with the MCP server name as configured on the gateway (e.g., `atlassian-jira-mcp-getjiraissue`, not just `atlassian-getjiraissue`). Examples in this skill use shortened names for readability, but real policies must match the exact name the gateway sends. Use the debug policy technique to confirm. -- **Case-sensitive tool name comparisons:** By default, compare tool names case-insensitively using `lower(input.payload.name) == "..."` (or `lower(input.tool_metadata.name)` for egress). Tool names are strings configured on the gateway, and a case mismatch will silently cause the policy to not match — which is a hard bug to debug. Use case-sensitive comparisons only when the user has an explicit reason to do so. +- **Guessing tool names:** Tool names in `input.resource.name` (legacy alias `input.payload.name`) are prefixed with the MCP server name as configured on the gateway (e.g., `atlassian-jira-mcp-getjiraissue`, not just `atlassian-getjiraissue`). Examples in this skill use shortened names for readability, but real policies must match the exact name the gateway sends. Use the debug policy technique to confirm. +- **Case-sensitive tool name comparisons:** By default, compare tool names case-insensitively using `lower(input.resource.name) == "..."` (works for both ingress and egress). Tool names are strings configured on the gateway, and a case mismatch will silently cause the policy to not match — which is a hard bug to debug. Use case-sensitive comparisons only when the user has an explicit reason to do so. - **Direct access on `subject.claims`:** `input.subject.claims.org_id` silently fails if the claim isn't present, making the entire rule body fail to match — easy to mistake for a different bug. Always use `object.get(input.subject.claims, "", "")`. Choose the default carefully so a missing claim produces a clean deny (e.g., `""` for string compares, `[]` for `some x in ...`). - **Assuming standard JWT claims are in `subject.claims`:** `aud`, `exp`, `iat`, `nbf`, `jti`, `azp`, `nonce`, and other validation-layer/OIDC claims are stripped before they reach the policy — Rego is not meant to be a parallel JWT validator. `iss` is the documented exception. See Identity (Subject and Claims) for the full strip set. - **Using `is_admin`, `teams`, or `user` from `subject.claims` for authorization:** These claims are stripped because they are ContextForge-internal RBAC plumbing minted by CF's own JWT issuer, not upstream IdP assertions. A policy that does `input.subject.claims.is_admin == true` is *always* false. For role-based authorization, use IdP-supplied claims like `groups`, `roles`, or namespaced custom claims. @@ -1282,6 +1283,14 @@ Policies often contain hardcoded values like transition IDs, project keys, or to Place this near the top of your policy, before the deny conditions. If your gateway also fronts a different management surface, add similar passthroughs for that prefix. Recovery if you skip the bypass and lock yourself out: detach the policy via the DTwo web UI, or via an MCP client that goes through a different gateway. +## Policy Store Catalog Contributions + +Contributing reusable policies to the **`dtwoai/policy-store`** repo (repository layout, `policy.md`/`tests.yaml` +frontmatter, landing-page links, manifest generation, and the registration flow) is a separate, lower-frequency +workflow. To keep this skill lean it lives in a reference file: **read `references/policy-store-catalog.md` in +this skill's directory before doing any policy-store contribution work.** Rego correctness for those catalog +policies still follows the rules in this skill (use PARC fields, compare tool names with `lower(input.resource.name)`, etc.). + ## Limitations - This skill cannot version, revert, or deploy policies — see the companion `dtwo-gateway-policy` instructions for lifecycle operations diff --git a/dtwo/skills/dtwo-policy-rego/references/policy-store-catalog.md b/dtwo/skills/dtwo-policy-rego/references/policy-store-catalog.md new file mode 100644 index 0000000..c87c717 --- /dev/null +++ b/dtwo/skills/dtwo-policy-rego/references/policy-store-catalog.md @@ -0,0 +1,99 @@ + + +# Policy Store Catalog Contributions (reference) + +> Reference file for the `dtwo-policy-rego` skill. Read this only when contributing reusable +> policies to the **`dtwoai/policy-store`** repo. Not needed for authoring tenant-local gateway +> policies. Rego correctness rules still come from the main skill (SKILL.md). + +Use this section when the target is the `dtwoai/policy-store` repository rather than a tenant-local gateway policy. The policy-store catalog is a curated source of reusable policies; this skill still owns Rego correctness, while the repository owns file layout, metadata, tests, and manifest generation. + +### Repository layout + +```text +apps/ + / + README.md + / + policy.md + tests.yaml + +industries/ + / + README.md + +bundles/ + / + README.md + +manifest.json +schema.json +``` + +- Put canonical policy bodies only under `apps///`. +- Never duplicate policy bodies under `industries/` or `bundles/`; those directories contain landing pages that link to canonical app policies. +- Use `` as the lowercase, hyphenated MCP server or SaaS app slug commonly configured on a gateway, such as `slack`, `jira`, `github`, or `postgres`. +- Use `` as the lowercase, hyphenated purpose, such as `block-secrets`, `readonly`, or `pii-redaction`. +- Do not add per-policy `README.md` or `metadata.json` files. Human documentation and all metadata live in `policy.md` frontmatter. +- Treat `manifest.json` as generated from policy frontmatter. Do not hand-edit it except through the manifest generator. + +### `policy.md` + +Each policy directory requires a `policy.md` file with YAML frontmatter followed by one fenced `rego` block. + +Frontmatter must include the required schema fields from `schema.json`: + +- `name` +- `tags` +- `publishedAt` +- `description` +- `direction` +- `apps` +- `schemaVersion` + +Include `industries`, `bundles`, and `minimumGatewayVersion` when they apply. Put the policy's human-facing documentation in `description`: what it does, when to use it, assumptions, limitations, examples, and relevant composition notes. + +Catalog policies are stricter than tenant-local examples: + +- Use PARC fields for action, resource, and identity: `input.action`, `input.resource`, `input.subject`, and `input.context`. +- Do not use deprecated legacy aliases in catalog policies: `input.kind`, `input.payload.name`, or `input.user`. +- Use `input.payload.args` and other hook-specific `input.payload` data for the actual request/response payload when needed. +- Compare tool names case-insensitively with `lower(input.resource.name)`. +- Use `object.get(obj, key, default)` for optional fields and missing claims. +- Use only IdP-supplied claims in `input.subject.claims`; do not authorize on stripped ContextForge-internal claims such as `is_admin`, `teams`, or nested `user`. + +### `tests.yaml` + +Each policy directory requires a `tests.yaml` file containing a top-level YAML array of test cases. Include at least one positive and one negative case. For deny policies, this usually means one allow case and one deny case; for transform-only policies, use a passthrough case and a transform-applied case. + +Each test case uses this shape: + +```yaml +- description: what this case demonstrates + input: + resource: { ... } + action: tool_pre_invoke + payload: { ... } + output: + expectedResult: deny + expectedReason: exact reason when relevant + transformApplied: true + transform: { replacement: "[REDACTED]" } + transformedArgsContain: { jql: "project != HR" } +``` + +- The runner feeds only each case's `input` object to OPA. Do not nest the PARC decision object under another `input` key inside the test case input. +- `output.expectedResult` is required and must be `allow` or `deny`. +- `output.expectedReason`, `transformApplied`, `transform`, and `transformedArgsContain` are optional and should be included only when they assert relevant behavior. +- For identity-aware policies, document required IdP claim names and missing-claim defaults in the `policy.md` description. + +### Registering a policy + +1. Create `apps///policy.md` and `tests.yaml`. +2. Add a row or link to `apps//README.md`. +3. If the policy belongs to an industry or bundle, list the slug in `policy.md` frontmatter and add a link from the matching `industries//README.md` or `bundles//README.md`. +4. For new apps, industries, or bundles, create the corresponding directory and `README.md`. +5. Run `pnpm manifest`, commit the generated `manifest.json`, then run `pnpm manifest:check`. +6. Run `pnpm test`; it requires the OPA CLI on `PATH` or `OPA_BIN` set. + +Do not invent new manifest fields. Match existing policy frontmatter and open an issue first if the schema lacks a needed field. From 57fcb59ad226744d7245354a3efb659c07866406 Mon Sep 17 00:00:00 2001 From: Paul Reilly Date: Wed, 8 Jul 2026 13:45:46 -0400 Subject: [PATCH 7/8] Bump versions (again) and use the documented reference pattern --- .claude-plugin/marketplace.json | 2 +- dtwo/.claude-plugin/plugin.json | 2 +- dtwo/skills/dtwo-policy-rego/SKILL.md | 9 +++++---- 3 files changed, 7 insertions(+), 6 deletions(-) diff --git a/.claude-plugin/marketplace.json b/.claude-plugin/marketplace.json index b621aea..a05544d 100644 --- a/.claude-plugin/marketplace.json +++ b/.claude-plugin/marketplace.json @@ -9,7 +9,7 @@ "name": "dtwo", "source": "./dtwo", "description": "DTwo MCP gateway management — bundles the DTwo MCP server and skills for managing gateway configs, deploys, and policies", - "version": "1.0.3", + "version": "1.0.4", "category": "security", "keywords": [ "agentic", diff --git a/dtwo/.claude-plugin/plugin.json b/dtwo/.claude-plugin/plugin.json index d7de6fc..daeaa15 100644 --- a/dtwo/.claude-plugin/plugin.json +++ b/dtwo/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "dtwo", - "version": "1.0.3", + "version": "1.0.4", "description": "Manage DTwo gateways, policies, and Rego with the DTwo MCP server.", "author": { "name": "DTwo", diff --git a/dtwo/skills/dtwo-policy-rego/SKILL.md b/dtwo/skills/dtwo-policy-rego/SKILL.md index 09f00c5..1263ddb 100644 --- a/dtwo/skills/dtwo-policy-rego/SKILL.md +++ b/dtwo/skills/dtwo-policy-rego/SKILL.md @@ -69,7 +69,7 @@ This skill is typically used alongside others. Invoke them via the `Skill` tool - Always return generated or modified policies in a fenced `rego` code block - When generating or modifying a policy, include a brief explanation of what the policy does and its direction (ingress/egress) - When explaining an existing policy, no code block is needed unless referencing specific rules -- When contributing to the DTwo Policy Store repository, produce or edit the repository files described in `references/policy-store-catalog.md` (see the Policy Store Catalog Contributions pointer near the end) instead of returning only a standalone fenced Rego block. +- When contributing to the DTwo Policy Store repository, produce or edit the repository files described in [`references/policy-store-catalog.md`](references/policy-store-catalog.md) (see the Policy Store Catalog Contributions pointer near the end) instead of returning only a standalone fenced Rego block. ## Quick start: the three policy shapes @@ -107,7 +107,7 @@ This skill has four primary modes: - **Generate** — produce a complete Rego policy from a natural language requirement. Before returning, verify all `input.*` paths used in the policy exist in the DTwo Gateway Input Schema below. - **Modify** — change an existing Rego policy based on instructions, preserving its logic and style. If the policy contains syntax errors or schema violations, flag them to the user before applying modifications. - **Explain** — describe an existing policy in plain language: what it permits/blocks/modifies, its direction (ingress vs egress), what data it inspects, what triggers allow/deny, and any transformations applied. Flag syntax errors or schema violations as part of the explanation. -- **Contribute** — create or update `dtwoai/policy-store` catalog artifacts (`policy.md`, `tests.yaml`, landing-page links, and generated manifest) using the repository layout and contribution flow in `references/policy-store-catalog.md`. +- **Contribute** — create or update `dtwoai/policy-store` catalog artifacts (`policy.md`, `tests.yaml`, landing-page links, and generated manifest) using the repository layout and contribution flow in [`references/policy-store-catalog.md`](references/policy-store-catalog.md). All modes must follow the Core Rules above. @@ -1296,8 +1296,9 @@ Policies often contain hardcoded values like transition IDs, project keys, or to Contributing reusable policies to the **`dtwoai/policy-store`** repo (repository layout, `policy.md`/`tests.yaml` frontmatter, landing-page links, manifest generation, and the registration flow) is a separate, lower-frequency -workflow. To keep this skill lean it lives in a reference file: **read `references/policy-store-catalog.md` in -this skill's directory before doing any policy-store contribution work.** Rego correctness for those catalog +workflow. To keep this skill lean it lives in a reference file: before doing any policy-store contribution work, +read [`references/policy-store-catalog.md`](references/policy-store-catalog.md) (bundled alongside this skill; if the +relative path doesn't resolve, read `${CLAUDE_SKILL_DIR}/references/policy-store-catalog.md`). Rego correctness for those catalog policies still follows the rules in this skill (use PARC fields, compare tool names with `lower(input.resource.name)`, etc.). ## Limitations From baf67556ff69de3b9ef6755bb8e8dd57a6e2686a Mon Sep 17 00:00:00 2001 From: Paul Reilly Date: Thu, 9 Jul 2026 10:38:47 -0400 Subject: [PATCH 8/8] Updates --- dtwo/skills/dtwo-gateway-config/SKILL.md | 63 ++++++++++++++++++++++++ dtwo/skills/dtwo-policy-rego/SKILL.md | 1 + 2 files changed, 64 insertions(+) diff --git a/dtwo/skills/dtwo-gateway-config/SKILL.md b/dtwo/skills/dtwo-gateway-config/SKILL.md index cb7fe78..be925dd 100644 --- a/dtwo/skills/dtwo-gateway-config/SKILL.md +++ b/dtwo/skills/dtwo-gateway-config/SKILL.md @@ -105,6 +105,7 @@ This subsection is generated from `schema-reference.json` by `scripts/generate-s | `gateway.authentication` | Inbound auth from clients to the gateway. `enabled` defaults to `true`; cross-field constraint requires `jwks_info` when enabled. | | `gateway.authentication.jwks_info` | Inbound JWT validation parameters. All four fields required when this object is present. | | `gateway.ssrf` | SSRF protection overrides. Strict defaults apply when omitted. | +| `gateway.intent` | Session-intent controls (conditional — feature-gated). Enables platform intent capture / gating; ties the platform policies to a specific `mcp_servers[]` entry. See its subsection below for the availability gate. | | `mcp_servers[]` | One entry per upstream MCP server. `name` and `url` required. | | `mcp_servers[].authentication` | Discriminated union keyed on `type`; outbound auth from gateway to the upstream server. Seven variants (see table). | @@ -138,6 +139,68 @@ Strict defaults block localhost, private networks, and fail-closed DNS when omit | `allow_private_networks` | boolean | `false` | Enable only when the gateway and MCP server share a private network (e.g. co-located on one EC2 host); prefer `allowed_networks` with a surgical CIDR allowlist in production, since a blanket private-range allow widens the gateway's outbound attack surface. | | `allowed_networks` | array | `[]` | Set to the specific CIDRs your MCP servers live on when the gateway must reach private hosts — prefer this surgical allowlist over the blanket `allow_private_networks=true`, since every range you add widens the gateway's outbound attack surface. | +#### `gateway.intent` — session-intent controls (conditional — feature-gated) + +> **Availability gate — read this first.** Session intent capture is not +> customer-available yet: the `dtwo-*-intent*` MCP tools stay behind the +> `enable_intent_tools` server flag until registry-driven resolution ships. +> **Before writing any `gateway.intent` block into a customer config, confirm +> intent capture is intended for this deployment.** For customer-facing config +> work today, treat this subsection as inert and do not surface it. It is +> documented here so agents driving the (currently internal) intent-capture +> workflow can author the config correctly, and so config validation errors +> mentioning `gateway.intent.*` have a schema reference. + +The `gateway.intent` block turns the platform intent-capture and +intent-required policies on. Three fields: + +| Field | Type | deployDefault | Notes | +|---|---|---|---| +| `enabled` | boolean | `false` | Injects the platform egress capture policy. Safe to enable without `required`. | +| `required` | boolean | `false` | Injects the platform ingress gate that denies non-`set_intent` calls until an intent is set. Requires `enabled: true`. | +| `server` | string | (none) | The `mcp_servers[].name` of the platform Dtwo MCP server that hosts `dtwo-set-intent`. **Required whenever `enabled: true`.** Constrained to alphanumeric segments joined by single `_` or `-` (no spaces, punctuation, Unicode, repeated separators, or leading/trailing separators). Matches an `mcp_servers[]` entry case-insensitively with `_↔-` interchangeable. | + +**Cross-field constraints** (both enforced at parse time): + +- `required: true` requires `enabled: true`. Enabling the gate without the + capture policy would deny every call forever — parse-time rejection prevents + shipping that config. +- `enabled: true` requires `server`. The platform policies identify the + set_intent tool by **exact tool name** (`-set_intent` / + `-set-intent`), not by suffix. Without the binding a lookalike + customer tool named `*-set_intent` would satisfy the check — the parse-time + refine keeps that from shipping. + +**Deploy-time validation** (state-machine, on top of parse-time): + +- `server` must resolve to an entry in `mcp_servers[]`. Case-insensitive match + with `_↔-` treated as equivalent (`server: dtwo` resolves to + `mcp_servers[].name: Dtwo`). +- Ambiguity — two `mcp_servers[]` entries whose names normalize identically + (e.g. `Dtwo` and `dtwo`) — is rejected at deploy with a message naming + both offending entries. Rename one so the intent target is unambiguous. + +**Example:** + +```yaml +gateway: + intent: + enabled: true + required: true + server: Dtwo +mcp_servers: + - name: Dtwo # must match intent.server under normalization + url: http://host.docker.internal:3000/mcp + transport_type: streamablehttp + authentication: + type: none +``` + +For the enforcement policies themselves, the intent/marker registries, and the +Rego library that user policies use to read the captured intent, see the +Intent Capture section in `dtwo-gateway-policy`. Those surfaces (and their +availability gate) are outside this skill's scope. + #### `gateway.advanced` and `gateway.log_level` - **`advanced`** — `array`, `targetKind: advanced`. Lines are appended verbatim to the deployed env file under systemd `EnvironmentFile` semantics (last-occurrence-wins). Validation rejects keys already emitted by a typed field, so it cannot shadow a typed field by accident. Keys here are case-sensitive — preserve exact casing. diff --git a/dtwo/skills/dtwo-policy-rego/SKILL.md b/dtwo/skills/dtwo-policy-rego/SKILL.md index 1263ddb..2c1b1d2 100644 --- a/dtwo/skills/dtwo-policy-rego/SKILL.md +++ b/dtwo/skills/dtwo-policy-rego/SKILL.md @@ -1023,6 +1023,7 @@ Authoring the intent-*capture* Rego is off-limits (above), but a tenant policy m | `data.dtwo.lib.intent_match.current_intent(input)` | The full intent object (`{category, description, set_at}`); **undefined** when no intent is set. | | `data.dtwo.lib.intent_match.current_category(input)` | Just the category FQID (e.g. `internal:debug`); undefined when unset. | | `data.dtwo.lib.intent_match.category_in(input, allowed)` | `true` when the current category is in the `allowed` set of FQIDs; `false` otherwise (including when no intent is set). | +| `data.dtwo.lib.intent_match.is_platform_set_intent(input)` | `true` when the incoming call is the platform `dtwo-set-intent` tool hosted by the operator-declared `gateway.intent.server`; `false` for anything else, including lookalike tools from other MCP servers. `false` when intent capture is off. Use to skip a gate for the platform set_intent call (same reason the platform policies use it) — an exact-match anchor that cannot be forged by tenant configuration. | **Do not read the intent from `input.context.session.policies` directly**, and do not hand-roll a walk-all-writers read for it. The helper is pinned to the trusted platform intent-capture slot, which tenant policies cannot write; a direct read is spoofable (any policy's own writer slot could supply an intent-shaped value) and couples your policy to internal storage details. Reading intent is the one case where the marker walk-all-writers pattern is the wrong tool.