feat: let tenants configure executor approval gates via the API

README.md

@@ -87,6 +87,32 @@ builder.Services
 
 `OrderWorkflow` must implement `IWorkflowDefinition` or `IWorkflowDefinition<TContext, TResult>`. Use `WorkflowBuildContext.Node(...)` to attach host executors and declare approval gates.
 
+### Declaring executor gates
+
+A node attached with no gate block runs autonomously. Pass a gate block to require a human decision, either always or under a predicate:
+
+```csharp
+public ValueTask<Workflow> BuildAsync(WorkflowBuildContext context, CancellationToken cancellationToken)
+{
+    ExecutorBinding validate = context.Node(new Validate("validate"));           // autonomous
+
+    ExecutorBinding notify = context.Node(new Notify("notify"), gate => gate
+        .Mode(ExecutionMode.RequireApproval)
+        .AssignTo("group:ops")
+        .ExpiresAfter(TimeSpan.FromHours(4)));
+
+    ExecutorBinding settle = context.Node(new Settle("settle"), gate => gate
+        .When<OrderContext>(order => order.Amount > 25_000m)
+        .Reason("RegulatedSettlement")
+        .RequireApprovers(2)
+        .Locked());                                                              // tenants may not weaken this
+
+    // ...
+}
+```
+
+Every gated node is configurable per tenant at run time unless the author calls `.Locked()`. A locked gate is a floor, not a freeze: a tenant may still tighten it. `RawNode(...)` bindings run outside the executor pipeline and cannot be gated at all.
+
 ## API Surface
 
 ### Workflow catalog
@@ -97,6 +123,58 @@ builder.Services
 
 The start endpoint accepts an optional `version` query parameter and supports `Idempotency-Key` and `Prefer: wait=<seconds>` headers.
 
+### Per-tenant executor configuration
+
+- `GET /workflows/{name}/versions/{version}/nodes`
+- `PUT /workflows/{name}/versions/{version}/nodes`
+- `PUT /workflows/{name}/versions/{version}/nodes/{executorId}`
+- `DELETE /workflows/{name}/versions/{version}/nodes/{executorId}`
+
+Tenants configure whether each executor of a workflow runs autonomously or waits for approval, without a redeploy. The tenant comes from `X-Tenant-Id` or the `tenant_id` claim; `version` accepts a concrete version or `latest`.
+
+`GET` lists the executor nodes the definition declares, each with its declared gate, the calling tenant's override, and the gate that will actually run:
+
+```json
+{
+  "workflowName": "order", "workflowVersion": "1.0.0", "tenantId": "acme",
+  "nodes": [{
+    "executorId": "settle", "executorType": "Settle",
+    "inputType": "OrderContext", "outputType": "OrderResult",
+    "configurable": true, "locked": false,
+    "declared":       { "mode": "autonomous",     "requiredApprovers": 1, "...": "..." },
+    "tenantOverride": { "mode": "requireApproval", "assignees": ["group:finance"], "...": "..." },
+    "effective":      { "mode": "requireApproval", "assignees": ["group:finance"], "...": "..." },
+    "effectiveSource": "tenant"
+  }]
+}
+```
+
+`PUT` writes the calling tenant's policy for one node, or for several at once via `{ "nodes": { "<executorId>": { ... } } }`. Only `mode` is required; every other field inherits the declared gate, so the body below keeps the author's assignees and expiry:
+
+```bash
+curl -X PUT http://localhost:5000/workflows/order/versions/1.0.0/nodes/settle \
+  -H 'X-Tenant-Id: acme' -H 'Content-Type: application/json' \
+  -d '{"mode":"requireApproval","requiredApprovers":2}'
+```
+
+| Field | Notes |
+| --- | --- |
+| `mode` | `autonomous` or `requireApproval`. `conditional` is code-only — its predicate cannot be expressed in JSON |
+| `reason` | Shown on the resulting approval request |
+| `assignees`, `escalationAssignees` | Principal or group identifiers |
+| `requiredApprovers` | Quorum; at least 1 |
+| `expirySeconds` | Approval window; greater than zero |
+| `onExpiry` | `deadStop`, `reject`, `autoApprove`, or `escalate` |
+| `allowModification`, `requireSegregationOfDuties` | Booleans |
+
+`DELETE` drops the override and restores the declared gate. A bulk `PUT` is all-or-nothing: if any node is unknown, not configurable, or refused, nothing is persisted.
+
+At run time the runner resolves gates for the instance's own tenant, in this order: per-instance override, tenant policy, host-wide policy (a policy written with no tenant), then the definition, then autonomous. Responses report which one applied as `effectiveSource`.
+
+A gate the author declared with `.Locked()` is a floor. Configuration may tighten it; an attempt to weaken it — downgrading `mode`, lowering `requiredApprovers`, enabling `allowModification`, disabling segregation of duties, or setting `onExpiry` to `autoApprove` — is refused with `409` and persists nothing. The runtime re-applies the floor on read as well, so a policy that reached the store by another route still cannot un-gate a locked executor.
+
+Policies are keyed by workflow **version**, since executor ids and gates change between versions. A tenant's configuration therefore does not carry forward to a new version, and `POST /instances/{id}/rerun` in restart mode creates the new instance at the *current* version — so a restart after a version bump runs under that version's configuration, or under declared defaults if the tenant has none.
+
 ### Instance operations
 
 - `GET /instances/{id}`

docs/wiki.md

@@ -17,6 +17,7 @@ This page is the repository-level technical wiki. It documents the implementatio
 - [Retries and failure classification](#retries-and-failure-classification)
 - [Checkpoints and resumption](#checkpoints-and-resumption)
 - [Human approval gates](#human-approval-gates)
+- [Tenant executor configuration](#tenant-executor-configuration)
 - [Events, history, and SSE](#events-history-and-sse)
 - [HTTP API](#http-api)
 - [Configuration](#configuration)
@@ -491,6 +492,48 @@ Decision outcomes are `Approve`, `Reject`, and `ApproveWithModification`. Modifi
 
 Expiry actions are `DeadStop`, `Reject`, `AutoApprove`, and `Escalate`. Approval decisions enforce assignees, quorum, and optional segregation of duties.
 
+A gate may also be declared with `.Locked()`, which prevents tenant configuration from weakening it. See [Tenant executor configuration](#tenant-executor-configuration).
+
+## Tenant executor configuration
+
+The gate declared in code is a default, not a fixed setting. Each tenant decides whether an executor runs autonomously or waits for approval, through the API and without a redeploy. An executor attached with `context.Node(executor)` and no gate block is autonomous for every tenant until someone changes it.
+
+### Resolving the effective gate
+
+The runner resolves each executor's gate for the instance's own tenant, highest precedence first:
+
+| Precedence | Scope | Written by |
+| --- | --- | --- |
+| 1 | Per-instance override | `IGatePolicyStore.SetInstanceOverrideAsync` |
+| 2 | Tenant policy | `PUT /workflows/{name}/versions/{version}/nodes` with a tenant |
+| 3 | Host-wide policy | `IGatePolicyStore.SetAsync` with a null tenant |
+| 4 | Workflow definition | The gate block in `BuildAsync` |
+| 5 | Host default | `ExecutionMode.Autonomous` |
+
+A policy-store failure falls back to the definition's gate, never to autonomous: a lookup error must not un-gate a protected executor. `GET` responses report which scope applied as `effectiveSource` (`tenant`, `host`, or `definition`).
+
+### Locked gates
+
+`.Locked()` marks a declared gate as the author's floor. Tenant configuration may still tighten it, but any of the following is refused with `409` and persists nothing:
+
+- Downgrading `Mode` (`RequireApproval` or `Conditional` to `Autonomous`, or `RequireApproval` to `Conditional`).
+- Lowering `RequiredApprovers` below the declared quorum.
+- Enabling `AllowModification` where the declaration disabled it.
+- Disabling `RequireSegregationOfDuties` where the declaration required it.
+- Setting `OnExpiry` to `AutoApprove` where the declaration did not.
+
+The same rules are re-applied when the gate is read at execution time, so a policy that reached the store before the gate was locked — or through a store client rather than the API — still cannot un-gate the executor. A locked `Conditional` gate also keeps its predicate, because a predicate is code and no stored policy can supply one.
+
+### Scope and lifetime
+
+Policies are keyed by tenant, workflow name, and workflow **version**, because executor ids and gates change between versions. A tenant's configuration does not carry forward when a new version is registered; instances of the new version run under its declared defaults until configured. `POST /instances/{id}/rerun` in restart mode creates the new instance at the current version, so a restart after a version bump uses that version's configuration.
+
+Configuration writes are recorded in the audit store as `gate.policy.set` and `gate.policy.reset`, with the actor, tenant, workflow, version, and executor.
+
+### Discovering nodes
+
+`GET /workflows/{name}/versions/{version}/nodes` builds the definition once against an inspection context — attaching nothing to a runtime and running no executor — and caches the result per version. Each node reports its executor id, implementation type, input and output types, whether it is configurable, whether it is locked, and its declared, tenant, and effective gates. `RawNode(...)` bindings are listed with `configurable: false`; they run outside the executor middleware pipeline and cannot be approval-gated.
+
 ## Events, history, and SSE
 
 Every instance event has a monotonically increasing per-instance `Sequence`. The same sequence is used as the SSE event ID, which lets clients reconnect with `Last-Event-ID` and request replay from the same cursor.
@@ -544,6 +587,27 @@ A start request body contains a workflow-specific context object. Optional reque
 
 A successful asynchronous start returns `202 Accepted` and a location such as `/instances/{id}`. Unknown workflows return `404`; invalid context returns `400` with field errors.
 
+### Executor nodes and tenant policy
+
+| Method | Path | Purpose |
+| --- | --- | --- |
+| `GET` | `/workflows/{name}/versions/{version}/nodes` | List executor nodes with declared, tenant, and effective gates |
+| `PUT` | `/workflows/{name}/versions/{version}/nodes` | Configure several nodes in one all-or-nothing write |
+| `PUT` | `/workflows/{name}/versions/{version}/nodes/{executorId}` | Configure one node |
+| `DELETE` | `/workflows/{name}/versions/{version}/nodes/{executorId}` | Drop the tenant's override for one node |
+
+The tenant is taken from the `X-Tenant-Id` header or the `tenant_id` claim. `version` accepts a concrete version or `latest`.
+
+A policy body requires `mode` (`autonomous` or `requireApproval`; `conditional` is code-only) and optionally `reason`, `assignees`, `requiredApprovers`, `expirySeconds`, `onExpiry`, `escalationAssignees`, `allowModification`, and `requireSegregationOfDuties`. Omitted fields inherit the declared gate.
+
+```bash
+curl -X PUT http://localhost:5000/workflows/order/versions/1.0.0/nodes/submit-payment \
+  -H 'X-Tenant-Id: acme' -H 'Content-Type: application/json' \
+  -d '{"mode":"requireApproval","assignees":["group:finance"],"requiredApprovers":2}'
+```
+
+Unknown workflow, version, or executor returns `404`. A raw node or an unsupported mode returns `400`. A policy that would weaken a locked gate returns `409`. See [Tenant executor configuration](#tenant-executor-configuration) for precedence and locking rules.
+
 ### Instances and diagnostics
 
 | Method | Path | Purpose |
@@ -672,7 +736,9 @@ The built-in request/response logging middleware samples bodies and applies an a
 
 ### Tenancy and authorization
 
-Instances and approvals carry a tenant identifier. The API and store implementations must enforce tenant isolation at the request boundary and persistence boundary. The in-memory reference stores model the contract but are not a replacement for a production identity and authorization system.
+Instances, approvals, and gate policies carry a tenant identifier. The API and store implementations must enforce tenant isolation at the request boundary and persistence boundary. The in-memory reference stores model the contract but are not a replacement for a production identity and authorization system.
+
+Because gate policies decide whether an executor runs without a human decision, the node configuration endpoints are privileged: authorize them for tenant administrators rather than for anyone who can start an instance. The host derives the tenant from `X-Tenant-Id` when no `tenant_id` claim is present, which is a development convenience — in production, bind the tenant to the authenticated principal so a caller cannot configure another tenant's workflows by setting a header.
 
 ### Secrets
 
@@ -862,6 +928,8 @@ Implement `IWorkflowMiddleware` for run-wide behavior or `IExecutorMiddleware` f
 
 Provide an `IGatePolicyStore` implementation when approval requirements depend on tenant, workflow, executor, amount, role, or environment. Keep policy evaluation deterministic and observable.
 
+`FindAsync` resolves one executor's gate for a tenant and must itself apply the scope precedence — per-instance override, then the tenant's policy, then the host-wide policy stored under a null tenant — returning `null` when no policy exists so the definition's gate stands. `ListAsync` returns exactly one scope's entries without merging, which is what lets the API show a tenant's own overrides separately from the host default. The locked-gate floor is enforced by the runtime on top of whatever the store returns, so a custom implementation cannot accidentally weaken a protected executor.
+
 ### Custom event sinks and buses
 
 The runner publishes through `IEventSink`. A sink may persist the event, relay it to an event bus, or do both. Preserve the per-instance sequence when forwarding to SSE or external consumers.
@@ -919,6 +987,14 @@ Check that `AddBackgroundServices()` is registered and that the dispatcher is ru
 
 Inspect the approval state, decision authorization, quorum, expiry, and instance status. A successful decision wakes the instance by moving it to a claimable state; the dispatcher must be running to execute the resumed run.
 
+### An executor pauses for approval although the definition left it autonomous
+
+A tenant or host-wide policy is gating it. Call `GET /workflows/{name}/versions/{version}/nodes` as that tenant and read `effectiveSource`: `tenant` means the tenant configured it, `host` means a host-wide policy applies. `DELETE` the node's override to restore the declared gate. Remember the policy is version-scoped — check the version the instance actually pinned, not the latest.
+
+### A tenant's configuration appears to be ignored
+
+Confirm the tenant used to configure the workflow is the tenant the instance runs under; the runner resolves gates for `instance.TenantId`, not for the caller who last edited the policy. Also check the instance's workflow version against the version the policy was written for, and whether a per-instance override outranks it.
+
 ### An outbound call is blocked
 
 Check the URL scheme, whether the target resolves to an internal address, and whether the hostname matches `WorkflowHost:Egress:AllowedHosts`. Keep `Egress:Enforce=true` unless this is a controlled local test.

src/Abacus.Run.Service/Infrastructure/SqlServerInfrastructureStores.cs

@@ -99,20 +99,69 @@ public async ValueTask DeleteAsync(string uri, CancellationToken cancellationTok
 
 public sealed class SqlServerGatePolicyStore(IDbContextFactory<AbacusDbContext> factory) : IGatePolicyStore
 {
-    public async ValueTask<ApprovalGate?> FindAsync(string workflowName, string workflowVersion, string executorId, string? instanceId, CancellationToken cancellationToken)
+    private const string HostScope = "*";
+
+    public async ValueTask<ApprovalGate?> FindAsync(string? tenantId, string workflowName, string workflowVersion, string executorId, string? instanceId, CancellationToken cancellationToken)
+    {
+        // Highest scope first, falling through to the next when nothing is stored at that level.
+        var keys = new List<string>(3);
+        if (instanceId is { Length: > 0 }) keys.Add($"gate-instance:{instanceId}:{executorId}");
+        if (tenantId is { Length: > 0 }) keys.Add(PolicyKey(tenantId, workflowName, workflowVersion, executorId));
+        keys.Add(PolicyKey(null, workflowName, workflowVersion, executorId));
+
+        await using AbacusDbContext db = await factory.CreateDbContextAsync(cancellationToken);
+        List<JsonRow> rows = await db.JsonRows.AsNoTracking()
+            .Where(item => item.Kind == "gate" && keys.Contains(item.Key))
+            .ToListAsync(cancellationToken);
+
+        foreach (string key in keys)
+        {
+            JsonRow? row = rows.Find(item => item.Key == key);
+            if (row is not null) return JsonSerializer.Deserialize<ApprovalGate>(row.Payload);
+        }
+
+        return null;
+    }
+
+    public async ValueTask<IReadOnlyDictionary<string, ApprovalGate>> ListAsync(string? tenantId, string workflowName, string workflowVersion, CancellationToken cancellationToken)
     {
+        string prefix = PolicyKey(tenantId, workflowName, workflowVersion, string.Empty);
+
         await using AbacusDbContext db = await factory.CreateDbContextAsync(cancellationToken);
-        string key = instanceId is null ? $"gate:{workflowName}:{workflowVersion}:{executorId}" : $"gate-instance:{instanceId}:{executorId}";
-        JsonRow? row = await db.JsonRows.AsNoTracking().SingleOrDefaultAsync(item => item.Kind == "gate" && item.Key == key, cancellationToken);
-        return row is null ? null : JsonSerializer.Deserialize<ApprovalGate>(row.Payload);
+        List<JsonRow> rows = await db.JsonRows.AsNoTracking()
+            .Where(item => item.Kind == "gate" && item.Key.StartsWith(prefix))
+            .ToListAsync(cancellationToken);
+
+        var gates = new Dictionary<string, ApprovalGate>(StringComparer.Ordinal);
+        foreach (JsonRow row in rows)
+        {
+            ApprovalGate? gate = JsonSerializer.Deserialize<ApprovalGate>(row.Payload);
+            if (gate is not null) gates[row.Key[prefix.Length..]] = gate;
+        }
+
+        return gates;
     }
 
-    public ValueTask SetAsync(string workflowName, string workflowVersion, string executorId, ApprovalGate gate, CancellationToken cancellationToken)
-        => SetCoreAsync($"gate:{workflowName}:{workflowVersion}:{executorId}", gate, cancellationToken);
+    public ValueTask SetAsync(string? tenantId, string workflowName, string workflowVersion, string executorId, ApprovalGate gate, CancellationToken cancellationToken)
+        => SetCoreAsync(PolicyKey(tenantId, workflowName, workflowVersion, executorId), gate, cancellationToken);
+
+    public async ValueTask<bool> RemoveAsync(string? tenantId, string workflowName, string workflowVersion, string executorId, CancellationToken cancellationToken)
+    {
+        string key = PolicyKey(tenantId, workflowName, workflowVersion, executorId);
+        await using AbacusDbContext db = await factory.CreateDbContextAsync(cancellationToken);
+        JsonRow? row = await db.JsonRows.SingleOrDefaultAsync(item => item.Kind == "gate" && item.Key == key, cancellationToken);
+        if (row is null) return false;
+        db.JsonRows.Remove(row);
+        await db.SaveChangesAsync(cancellationToken);
+        return true;
+    }
 
     public ValueTask SetInstanceOverrideAsync(string instanceId, string executorId, ApprovalGate gate, CancellationToken cancellationToken)
         => SetCoreAsync($"gate-instance:{instanceId}:{executorId}", gate, cancellationToken);
 
+    private static string PolicyKey(string? tenantId, string workflowName, string workflowVersion, string executorId)
+        => $"gate:{(string.IsNullOrEmpty(tenantId) ? HostScope : tenantId)}:{workflowName}:{workflowVersion}:{executorId}";
+
     private async ValueTask SetCoreAsync(string key, ApprovalGate gate, CancellationToken cancellationToken)
     {
         await using AbacusDbContext db = await factory.CreateDbContextAsync(cancellationToken);