From 1af11b55c4740c691f8859d5b9383c9ec2fcc75a Mon Sep 17 00:00:00 2001 From: Eitan Yarmush Date: Thu, 2 Jul 2026 19:07:41 +0000 Subject: [PATCH 1/5] design: expected final ateapi.proto for decoupled substrate resources Draft the target ateapi surface for issue #368 (ActorTemplate, WorkerPool, SandboxConfig, WorkerPoolGrant moved out of Kubernetes CRDs), following the draft API style guide (#351), plus design rationale and CRD migration notes. For review only: codegen and server implementation intentionally lag the proto. --- docs/design/ateapi-crd-migration.md | 67 +++ docs/design/ateapi-resource-api.md | 140 +++++ pkg/proto/ateapipb/ateapi.proto | 888 +++++++++++++++++++++++----- 3 files changed, 936 insertions(+), 159 deletions(-) create mode 100644 docs/design/ateapi-crd-migration.md create mode 100644 docs/design/ateapi-resource-api.md diff --git a/docs/design/ateapi-crd-migration.md b/docs/design/ateapi-crd-migration.md new file mode 100644 index 000000000..42445298d --- /dev/null +++ b/docs/design/ateapi-crd-migration.md @@ -0,0 +1,67 @@ +# Migrating substrate-plane resources from CRDs to ateapi (issue #368) + +Companion to [ateapi-resource-api.md](ateapi-resource-api.md), which defines +the target API. This doc covers the transition: what changes relative to the +Kubernetes CRDs and the POC proto (`poc-decouple-upstream`), and the +migration path. + +## Why the POC shape changes + +The POC copied the CRD Go types field-for-field to move fast. That imported +Kubernetes idioms that don't pull their weight in a native API: + +| # | Leak | Where | Problem | +|---|------|-------|---------| +| 1 | `deployment_atespace` | WorkerPoolSpec | It's a **Kubernetes namespace** (the projector uses it verbatim as the Deployment namespace), mislabeled with a substrate term. Renamed `kubernetes.namespace`. | +| 2 | `WorkerPoolPodTemplate` + Toleration/NodeAffinity/ResourceRequirements at top level | WorkerPool | corev1 mirrored into the public API without a boundary marking it k8s-specific. Quarantined into `KubernetesPlacement`. | +| 3 | `EnvVarSource`/`SecretKeySelector` | Container env | corev1's three-field env shape; invalid states representable (value + value_from). Now a `oneof`. | +| 4 | `LabelSelector.match_expressions` | template worker_selector | Set-based operators copied but only equality is implemented or meaningful today. Dropped. | +| 5 | `Condition` list + string `phase` | ActorTemplateStatus | KRM condition machinery; `observed_generation` has no generation to observe. Replaced by a state enum. | +| 6 | Stringly-typed enums (`on_pause: "Full"`) | SnapshotsConfig | CEL-validated strings instead of proto enums. Now `Scope` enum. | +| 7 | Update RPCs with unspecified FieldMask semantics | templates, pools, configs | The POC marked mask semantics TODO. Now defined (top-level paths), and templates lose Update entirely. | +| 8 | Grant identity `(atespace, name)` with free-form name | WorkerPoolGrant | The scheduler looks up grants by `(atespace, pool)`; the POC allows multiple grants for the same pair, making revocation ambiguous. The POC's own `GetWorkerPoolGrant` is documented "by atespace and worker pool" but keyed by name. Now: Create enforces at most one grant per (atespace, pool). | + +## Behavior deltas vs the POC + +1. **No UpdateActorTemplate** — POC has it; the target API drops it + (immutable spec). +2. **Grant uniqueness** — Create rejects a second grant for the same + (atespace, worker_pool) pair with `ALREADY_EXISTS`; the store gains a + `(atespace, pool)` index so the scheduler check stays a point lookup. +3. **Delete returns the resource** — POC returned `Empty` for + templates/pools/configs/grants and an empty message for actors. +4. **Required update_mask** — Update without a mask becomes + `INVALID_ARGUMENT`; POC treated masks as TODO/full-replace. +5. **Actor RPC shapes** — bare-resource returns, `ObjectRef` identity, and + the legacy `actor_template_namespace/name` create path is gone. The + package rename (`ateapi` → `ateapi.v1alpha1`) makes every message a new + proto type, so this is a clean break: field numbers are assigned fresh, + no `reserved` scar tissue, and migration is a coordinated client cutover + rather than in-place field evolution. (The POC has no external users.) +6. **Referential integrity on delete** — template↔actor, pool↔grant, + pool↔assigned-actor, config↔pool checks return `FAILED_PRECONDITION`; + the POC checks only atespace emptiness. +7. **Watch SYNCED event** — new; consumers can await cache completeness. + +## CRD validations that move into handlers + +Validation currently expressed as CEL/OpenAPI on the CRDs moves into +Create (and Update, where applicable) handlers: + +- DNS-1123 label names (atespace, name — per style guide §2.3) +- template spec immutability (`self == oldSelf` → no Update method) +- container images pinned by digest +- volume mount-path rules +- `snapshots_config.on_commit` ⊆ `on_pause` +- at most one `default: true` SandboxConfig + +## Migration path + +1. Land the target proto as `ateapi.v1alpha1` alongside the POC `ateapi` + package; generate both. +2. Port store keys/values: add the grant `(atespace, pool)` uniqueness + index; template records gain immutability enforcement. +3. Move CRD validation into handlers (table above); delete the CRDs and the + projector's CRD-watching path. +4. Cut clients (atectl, atelet caches, demos) over to `v1alpha1`; delete + the POC package. diff --git a/docs/design/ateapi-resource-api.md b/docs/design/ateapi-resource-api.md new file mode 100644 index 000000000..1305db5e9 --- /dev/null +++ b/docs/design/ateapi-resource-api.md @@ -0,0 +1,140 @@ +# ateapi resource API (issue #368) + +Proto: [`pkg/proto/ateapipb/ateapi.proto`](../../pkg/proto/ateapipb/ateapi.proto) + +This is the proposed API surface once ActorTemplate, WorkerPool, +SandboxConfig and WorkerPoolGrant move out of Kubernetes CRDs into the +substrate API ([#368](https://github.com/agent-substrate/substrate/issues/368)). +It builds on [decoupling-planes.md](https://github.com/agent-substrate/substrate/blob/poc-decouple-upstream/docs/design/decoupling-planes.md) +(on the POC branch), which settled the +resource model (global pools + per-atespace grants); this doc covers the API +shape. The transition from the CRDs/POC is covered separately in +[ateapi-crd-migration.md](ateapi-crd-migration.md). + +The API follows the draft +[API style guide](https://github.com/agent-substrate/substrate/pull/351); +deliberate divergences are listed at the [end](#style-guide-conformance). + +## Resource model + +| Resource | Scope | Service | Mutable? | Notes | +|---|---|---|---|---| +| Atespace | global | Control | no | isolation boundary | +| ActorTemplate | atespace | Control | no (immutable spec) | + Watch | +| Actor | atespace | Control | `worker_selector` only | + Suspend/Pause/Resume | +| WorkerPool | global | Admin | yes | + Watch; labels are the selector match target | +| SandboxConfig | global | Admin | yes | de-facto registry of sandbox classes | +| WorkerPoolGrant | atespace | Admin | no | at most one grant per (atespace, pool) | +| Worker | — | Admin | — | debug projection, ListWorkers only | + +All managed resources carry `ResourceMetadata meta = 1` (atespace, name, +uid, version, create_time, update_time, labels) and use the standard method +shapes from the style guide: Get/Create/Update return the bare resource, +Delete returns the deleted resource, List paginates with +`page_size`/`page_token`, identity travels as `ObjectRef{atespace, name}`, +and optimistic concurrency uses `meta.version` (int64, `ABORTED` on +mismatch, 0 skips). + +## Design decisions + +**D1. Versioned package `ateapi.v1alpha1`.** gRPC method paths embed the +package, so the package name is the API-versioning mechanism. + +**D2. Two services with distinct audiences.** `Control` is the tenant +surface (atespaces, templates, actors); `Admin` is the platform surface +(pools, sandbox configs, grants, debug). Watch RPCs live next to their +resource. The split gives a future authz story a natural boundary. + +**D3. ActorTemplate spec is immutable — no UpdateActorTemplate.** Golden +snapshots are only valid for the exact spec they were taken from, so +in-place mutation is meaningless. Template changes = create a new template +and migrate actors. Delete returns `FAILED_PRECONDITION` while any actor +references the template. + +**D4. At most one WorkerPoolGrant per (atespace, worker pool).** Grants are +fully conventional resources (caller-named, standard methods), but Create +enforces uniqueness on the pair — a duplicate grant for the same pool +returns `ALREADY_EXISTS` regardless of its name (same pattern as the +single-default SandboxConfig rule). This keeps revocation unambiguous and +lets the scheduler's `(atespace, pool)` check stay a point lookup via a +server-side index. Grants are immutable; future policy fields (quotas) +would add Update. + +**D5. One Kubernetes pocket: `WorkerPoolSpec.kubernetes`.** Everything +k8s-specific about materializing a pool — namespace, node_selector, +tolerations, node_affinity, resources, priority_class_name — lives in a +single `KubernetesPlacement` message. Everything outside it is +plane-neutral; a future non-k8s substrate adds a sibling placement message. + +**D6. Referential integrity on delete.** Deletes that would orphan +dependents return `FAILED_PRECONDITION`: atespace↔actors/templates, +template↔actors, pool↔grants and pool↔assigned actors, config↔pools. + +**D7. Status is a closed enum, not phase strings or conditions.** +`ActorTemplateStatus.state` ∈ {PENDING, SNAPSHOTTING, READY, FAILED} plus +`golden_actor_id`, `golden_snapshot`, `error_message`. Likewise +`SnapshotsConfig` scopes are a `Scope` enum (FULL/DATA). + +**D8. Equality-only `Selector{match_labels}`** for template and per-actor +worker selectors, matched against `WorkerPool.meta.labels`. The scheduler +implements equality matching only; set-based expressions can be added as a +new field if a consumer appears. + +**D9. Env vars are a `oneof {value | secret}`** with a plane-neutral +`SecretRef{name, key, optional}` — value-vs-secret is exclusive by +construction, and no k8s types leak into the container spec. + +**D10. `sandbox_class` is an open string** on templates and pools. +SandboxConfigs are the class registry; discoverability comes from +ListSandboxConfigs, not a proto enum. + +**D11. Update masks accept top-level paths only.** `update_mask` is +required (per the style guide); paths one level below the resource root +(`spec.replicas`, `spec.kubernetes`, `meta.labels`) are accepted, and +naming a message or map field replaces it wholesale. Unknown, immutable, or +deeper paths → `INVALID_ARGUMENT`. Servers may widen to deeper paths later +without breaking clients. + +**D12. Watch = snapshot, SYNCED, then incremental events.** Streams +(templates, pools) deliver an initial snapshot, a `SYNCED` marker, then +at-least-once CREATED/UPDATED/DELETED events carrying full resource state. +No resume tokens: consumers are in-cluster caches and re-syncing is cheap; +`meta.version` lets them discard stale events. + +**D13. Worker stays a debug projection.** No `ResourceMetadata`, no CRUD. +Making Worker a managed resource would promise lifecycle semantics the +system doesn't offer — workers are pool-managed. + +**D14. Actor is a managed resource like the rest.** `meta` carries its +identity; `actor_template` and `worker_pool` are `ObjectRef`s; lifecycle +verbs (Suspend/Pause/Resume) are custom methods returning the Actor. The +`ateom_pod_*` fields remain, marked output-only infrastructure debugging. + +**D15. Atespace and SessionIdentity are unchanged** beyond conforming to +`meta` and standard method shapes. Redesigning them is a non-goal of #368. + +## Style-guide conformance + +The surface follows PR #351 (identity model, `ResourceMetadata`, +`ObjectRef`, standard method shapes, required `update_mask`, `version` +concurrency). Four deliberate divergences, feedback filed on PR #351: + +1. **`labels` in `ResourceMetadata`** — the guide's meta has no labels + field, but scheduling requires pool labels as a selector match target. +2. **Immutable resources may omit Update** (D3, D4) — the guide implies all + five standard methods everywhere. +3. **Top-level-only mask paths** (D11) — the guide requires the mask but + leaves path granularity undefined. +4. **Empty `atespace` = list across all atespaces** — the guide reads as if + `atespace` were required on List; admin/debug flows need cross-atespace + listing. + +## Open questions + +- **Secrets plane**: `SecretRef` resolves against the worker's environment + (a k8s Secret today). Freeze that rule, or design secret distribution + before v1beta1? +- **Pagination defaults**: pick a server default page size (proposal: 500, + max 1000) and document it once, centrally. +- **Watch scope**: watches are global today (atelet caches everything). + Add an optional `atespace` filter now, or when a consumer needs it? diff --git a/pkg/proto/ateapipb/ateapi.proto b/pkg/proto/ateapipb/ateapi.proto index 249d13c7d..7d615f8de 100644 --- a/pkg/proto/ateapipb/ateapi.proto +++ b/pkg/proto/ateapipb/ateapi.proto @@ -12,64 +12,302 @@ // See the License for the specific language governing permissions and // limitations under the License. -// TODO: Here and everywhere, s/Session/Actor to align with the glossary. +// DESIGN DRAFT (issue #368) — expected final shape of the ateapi surface +// after decoupling substrate-plane resources from Kubernetes CRDs. +// For review only: generated code and server implementation intentionally +// lag this file until the shape is agreed. +// +// The API follows docs/api-style-guide.md (PR #351); deliberate +// divergences are listed in docs/design/ateapi-resource-api.md +// ("Style-guide conformance"). +// +// Because the package renames from `ateapi` to `ateapi.v1alpha1`, every +// message here is a new proto type: field numbers are assigned fresh and +// carry no wire-compatibility obligation to the previous proto. Migration +// is a coordinated client cutover, not in-place field evolution. +// +// Error model (canonical gRPC codes): +// - INVALID_ARGUMENT: malformed names, missing required fields, +// empty/absent update_mask, unknown mask paths, +// negative page_size, scope violations +// (e.g. atespace set on a global resource). +// - NOT_FOUND: resource does not exist. +// - ALREADY_EXISTS: Create with an existing (atespace, name). +// - ABORTED: version guard mismatch (client echoed a non-zero +// meta.version / request version that is stale). +// - FAILED_PRECONDITION: referential-integrity or state-machine +// violations (delete a referenced template/pool, +// delete a non-empty atespace, resume a +// non-suspended actor, second default +// SandboxConfig, ...). syntax = "proto3"; -package ateapi; +package ateapi.v1alpha1; + +import "google/protobuf/field_mask.proto"; +import "google/protobuf/timestamp.proto"; +// Go package layout (ateapipb vs ateapipb/v1alpha1) is decided at +// implementation time; out of scope for this design. option go_package = "github.com/agent-substrate/substrate/pkg/proto/ateapipb"; +// --------------------------------------------------------------------------- +// Services +// --------------------------------------------------------------------------- -// Control is the primary RPC interface for Agentic Substrate. +// Control is the user-plane API: everything a tenant needs to define +// templates and run actors inside an atespace. service Control { + // --- Atespace (global-scoped, immutable: no Update) --- + + // Create an Atespace. + rpc CreateAtespace(CreateAtespaceRequest) returns (Atespace) {} + + // Get an Atespace. + rpc GetAtespace(GetAtespaceRequest) returns (Atespace) {} + + // List Atespaces. + rpc ListAtespaces(ListAtespacesRequest) returns (ListAtespacesResponse) {} + + // Delete an empty Atespace. + // Returns FAILED_PRECONDITION if any actors or actor templates remain. + rpc DeleteAtespace(DeleteAtespaceRequest) returns (Atespace) {} + + // --- ActorTemplate (atespace-scoped, immutable spec: no Update) --- + + // Create an ActorTemplate. The spec is immutable after creation: golden + // snapshots are only valid for the exact spec they were taken from. To + // change a template, create a new one and migrate actors. + rpc CreateActorTemplate(CreateActorTemplateRequest) returns (ActorTemplate) {} + + // Get an ActorTemplate. + rpc GetActorTemplate(GetActorTemplateRequest) returns (ActorTemplate) {} + + // List ActorTemplates. + rpc ListActorTemplates(ListActorTemplatesRequest) returns (ListActorTemplatesResponse) {} + + // Delete an ActorTemplate. + // Returns FAILED_PRECONDITION while any actor references it. + rpc DeleteActorTemplate(DeleteActorTemplateRequest) returns (ActorTemplate) {} + + // Watch ActorTemplate changes after an initial snapshot of current state. + // Delivery is at-least-once; events carry full resource state. A SYNCED + // event marks the end of the initial snapshot. + rpc WatchActorTemplates(WatchActorTemplatesRequest) returns (stream WatchActorTemplatesResponse) {} + + // --- Actor (atespace-scoped) --- + + // Create a new Actor deriving from an ActorTemplate. + rpc CreateActor(CreateActorRequest) returns (Actor) {} + // Get an Actor. - rpc GetActor(GetActorRequest) returns (GetActorResponse) {} + rpc GetActor(GetActorRequest) returns (Actor) {} + + // List Actors. + rpc ListActors(ListActorsRequest) returns (ListActorsResponse) {} - // Create a new Actor deriving from a given ActorTemplate. - rpc CreateActor(CreateActorRequest) returns (CreateActorResponse) {} + // Update mutable Actor fields (currently only worker_selector). + // Changes take effect on the next ResumeActor call. + rpc UpdateActor(UpdateActorRequest) returns (Actor) {} - // Update mutable fields on an existing Actor. - rpc UpdateActor(UpdateActorRequest) returns (UpdateActorResponse) {} + // Delete an Actor. Returns FAILED_PRECONDITION unless the actor is + // suspended. + rpc DeleteActor(DeleteActorRequest) returns (Actor) {} - // Suspend a given actor to a new snapshot. - rpc SuspendActor(SuspendActorRequest) returns (SuspendActorResponse) {} + // Suspend an actor to a new snapshot. + rpc SuspendActor(SuspendActorRequest) returns (Actor) {} - // Pause a given actor and keep its snapshots on node VM. - rpc PauseActor(PauseActorRequest) returns (PauseActorResponse) {} + // Pause an actor, keeping its snapshots on the node VM. + rpc PauseActor(PauseActorRequest) returns (Actor) {} // Resume an actor from its latest snapshot. - rpc ResumeActor(ResumeActorRequest) returns (ResumeActorResponse) {} + rpc ResumeActor(ResumeActorRequest) returns (Actor) {} +} - // Delete an actor. Only suspended actors can be deleted. - rpc DeleteActor(DeleteActorRequest) returns (DeleteActorResponse) {} +// Admin is the platform-plane API: capacity (worker pools), sandbox +// configuration, and cross-atespace access control (grants). Intended for +// platform operators, not tenants. +service Admin { + // --- WorkerPool (global-scoped) --- - // List all workers currently reflected in redis. - rpc ListWorkers(ListWorkersRequest) returns (ListWorkersResponse) {} + // Create a WorkerPool. + rpc CreateWorkerPool(CreateWorkerPoolRequest) returns (WorkerPool) {} - // List all actors currently reflected in redis. - rpc ListActors(ListActorsRequest) returns (ListActorsResponse) {} + // Get a WorkerPool. + rpc GetWorkerPool(GetWorkerPoolRequest) returns (WorkerPool) {} - // Create a new Atespace. Substrate-native, stored in Redis. - rpc CreateAtespace(CreateAtespaceRequest) returns (CreateAtespaceResponse) {} + // List WorkerPools. + rpc ListWorkerPools(ListWorkerPoolsRequest) returns (ListWorkerPoolsResponse) {} - // Get an Atespace by name. - rpc GetAtespace(GetAtespaceRequest) returns (GetAtespaceResponse) {} + // Update a WorkerPool. + rpc UpdateWorkerPool(UpdateWorkerPoolRequest) returns (WorkerPool) {} - // List all Atespaces. - rpc ListAtespaces(ListAtespacesRequest) returns (ListAtespacesResponse) {} + // Delete a WorkerPool. + // Returns FAILED_PRECONDITION while any WorkerPoolGrant references it or + // any actor is currently assigned a worker from it. + rpc DeleteWorkerPool(DeleteWorkerPoolRequest) returns (WorkerPool) {} + + // Watch WorkerPool changes after an initial snapshot of current state. + // Same delivery semantics as WatchActorTemplates. + rpc WatchWorkerPools(WatchWorkerPoolsRequest) returns (stream WatchWorkerPoolsResponse) {} + + // --- SandboxConfig (global-scoped) --- + + // Create a SandboxConfig. At most one SandboxConfig may set + // spec.default=true; violating this returns FAILED_PRECONDITION. + rpc CreateSandboxConfig(CreateSandboxConfigRequest) returns (SandboxConfig) {} + + // Get a SandboxConfig. + rpc GetSandboxConfig(GetSandboxConfigRequest) returns (SandboxConfig) {} + + // List SandboxConfigs. + rpc ListSandboxConfigs(ListSandboxConfigsRequest) returns (ListSandboxConfigsResponse) {} + + // Update a SandboxConfig. + rpc UpdateSandboxConfig(UpdateSandboxConfigRequest) returns (SandboxConfig) {} + + // Delete a SandboxConfig. + // Returns FAILED_PRECONDITION while any WorkerPool references it. + rpc DeleteSandboxConfig(DeleteSandboxConfigRequest) returns (SandboxConfig) {} - // Delete an empty Atespace. Rejects (FailedPrecondition) if any actors remain. - rpc DeleteAtespace(DeleteAtespaceRequest) returns (DeleteAtespaceResponse) {} + // --- WorkerPoolGrant (atespace-scoped, immutable) --- + + // Grant an atespace access to a worker pool. At most one grant may + // exist per (atespace, worker_pool) pair; a second grant for the same + // pool returns ALREADY_EXISTS regardless of its name. + rpc CreateWorkerPoolGrant(CreateWorkerPoolGrantRequest) returns (WorkerPoolGrant) {} + + // Get a WorkerPoolGrant. + rpc GetWorkerPoolGrant(GetWorkerPoolGrantRequest) returns (WorkerPoolGrant) {} + + // List WorkerPoolGrants, optionally scoped to one atespace. + rpc ListWorkerPoolGrants(ListWorkerPoolGrantsRequest) returns (ListWorkerPoolGrantsResponse) {} + + // Revoke (delete) a WorkerPoolGrant. Existing assignments are not + // interrupted; the grant is checked at scheduling time. + rpc DeleteWorkerPoolGrant(DeleteWorkerPoolGrantRequest) returns (WorkerPoolGrant) {} + + // --- Debug / introspection --- + + // List all workers currently reflected in the store. Debug surface: + // exposes infrastructure detail and is not a managed resource API. + rpc ListWorkers(ListWorkersRequest) returns (ListWorkersResponse) {} // Debugging: drop all data from the ate database. rpc DebugClear(DebugClearRequest) returns (DebugClearResponse) {} } +// --------------------------------------------------------------------------- +// Shared types +// --------------------------------------------------------------------------- + +// ResourceMetadata holds the standard fields common to all managed +// resources. Always field 1 of the resource message, named `meta`. +// See docs/api-style-guide.md section 6. +message ResourceMetadata { + // The atespace this resource belongs to. Must be empty for global-scoped + // resource types (Atespace, WorkerPool, SandboxConfig) and set for + // atespace-scoped types (Actor, ActorTemplate, WorkerPoolGrant). + string atespace = 1; + + // The name of this resource, unique within its scope. Must be a valid + // RFC-1123 DNS label. Caller-specified on Create. + string name = 2; + + // Server-assigned UUID4, immutable for the life of the resource. + string uid = 3; + + // Monotonically increasing version: starts at 1 on creation and + // increments by 1 on every mutation, including system-internal ones. + // Output-only; echo a non-zero value back on Update/Delete to guard + // against concurrent modification (mismatch returns ABORTED). + int64 version = 4; + + // When the resource was created. Output-only. + google.protobuf.Timestamp create_time = 5; + + // When the resource was last updated. Output-only. + google.protobuf.Timestamp update_time = 6; + + // labels are user-defined key/value pairs used for organization and + // selection. WorkerPool labels are matched by template and actor + // worker_selectors at scheduling time. Mutable on resources that + // support Update. (Not yet in the draft style guide's ResourceMetadata.) + map labels = 7; +} + +// ObjectRef is a pointer to a resource: request identity and +// cross-resource references. For references to global-scoped resources, +// atespace must be empty. +message ObjectRef { + string atespace = 1; + string name = 2; +} + +// Selector matches worker pools by their meta.labels. +// Only equality-based matching is supported (all pairs must match). +// An empty selector matches every pool. +message Selector { + map match_labels = 1; +} + +// ResourceEventType classifies watch stream events. +enum ResourceEventType { + RESOURCE_EVENT_TYPE_UNSPECIFIED = 0; + RESOURCE_EVENT_TYPE_CREATED = 1; + RESOURCE_EVENT_TYPE_UPDATED = 2; + RESOURCE_EVENT_TYPE_DELETED = 3; + // SYNCED is sent exactly once, after the initial snapshot has been fully + // delivered. It carries no resource. Consumers can use it to know their + // local cache is complete before acting. + RESOURCE_EVENT_TYPE_SYNCED = 4; +} + +// --------------------------------------------------------------------------- +// Atespace +// --------------------------------------------------------------------------- + +// Atespace is the isolation boundary actors are created into. +// Global-scoped; immutable (no Update method). +message Atespace { + ResourceMetadata meta = 1; +} + +message CreateAtespaceRequest { + // atespace.meta.name is required; atespace.meta.atespace must be empty. + Atespace atespace = 1; +} + +message GetAtespaceRequest { + ObjectRef atespace = 1; +} + +message ListAtespacesRequest { + // Maximum number of results to return; see style guide 3.2 for paging + // semantics (applies to all List methods in this file). + int32 page_size = 1; + string page_token = 2; +} +message ListAtespacesResponse { + repeated Atespace atespaces = 1; + string next_page_token = 2; +} + +message DeleteAtespaceRequest { + ObjectRef atespace = 1; + // Optional freshness guard; 0 = skip check. + int64 version = 2; +} + +// --------------------------------------------------------------------------- +// Actor +// --------------------------------------------------------------------------- + enum SnapshotType { SNAPSHOT_TYPE_UNSPECIFIED = 0; SNAPSHOT_TYPE_LOCAL = 1; - SNAPSHOT_TYPE_EXTERNAL = 2; + SNAPSHOT_TYPE_EXTERNAL = 2; } message ExternalSnapshotInfo { @@ -78,7 +316,7 @@ message ExternalSnapshotInfo { message LocalSnapshotInfo { string snapshot_prefix = 1; - // Node VMs that have local snapshots for this actor, while it's PAUSED. + // Node VMs that have local snapshots for this actor while it is PAUSED. repeated string node_vms_with_local_snapshots = 2; } @@ -90,18 +328,15 @@ message SnapshotInfo { } } -// Selector matches worker pools by label. -// Only equality-based matching is supported. -message Selector { - map match_labels = 1; -} - +// Actor is a running (or suspended) instance of an ActorTemplate. +// Atespace-scoped. message Actor { - string actor_id = 1; - int64 version = 2; + ResourceMetadata meta = 1; - string actor_template_namespace = 3; - string actor_template_name = 4; + // The ActorTemplate this actor derives from. Set at creation; immutable. + // Must currently reside in the actor's own atespace; the ObjectRef shape + // leaves room for cross-atespace template sharing later. + ObjectRef actor_template = 2; enum Status { STATUS_UNSPECIFIED = 0; @@ -112,207 +347,542 @@ message Actor { STATUS_PAUSING = 5; STATUS_PAUSED = 6; } - Status status = 5; + // Output-only lifecycle state. + Status status = 3; - string ateom_pod_namespace = 6; - string ateom_pod_name = 7; - string ateom_pod_ip = 8; + // worker_selector is the per-actor placement constraint. The scheduler + // evaluates the AND of this selector and the template's worker_selector + // to find eligible pools. Set at CreateActor; mutable via UpdateActor. + // Changes take effect on the next ResumeActor call. + Selector worker_selector = 4; - // last_snapshot - reserved 9; - string in_progress_snapshot = 10; - string ateom_pod_uid = 11; - SnapshotInfo latest_snapshot_info = 12; + // The WorkerPool that owns the currently assigned worker. Output-only: + // set by the scheduler at assignment time, cleared when the worker is + // freed on suspend/pause. + ObjectRef worker_pool = 5; - // worker_selector is the per-actor placement constraint. The scheduler - // evaluates the AND of this selector and the template's workerSelector to - // find eligible pools. Set at CreateActor; may be updated at any time via - // UpdateActor. Changes take effect on the next ResumeActor call. - Selector worker_selector = 13; + // Output-only snapshot bookkeeping. + string in_progress_snapshot = 6; + SnapshotInfo latest_snapshot_info = 7; - // worker_pool_name is the name of the WorkerPool that owns the currently - // assigned worker, in the same namespace as ateom_pod_namespace (pool and - // pod namespaces always match). Set by the scheduler at assignment time - // and cleared when the worker is freed. Needed to release the worker on - // suspend/pause since eligibility is no longer a single fixed pool - // reference on the ActorTemplate. - string worker_pool_name = 14; + // Output-only infrastructure debugging fields describing the Kubernetes + // pod currently backing this actor. Not part of the portable resource + // model; may be moved to a debug-only surface in a future version. + string ateom_pod_namespace = 8; + string ateom_pod_name = 9; + string ateom_pod_ip = 10; + string ateom_pod_uid = 11; +} - // The atespace this actor belongs to. Part of the actor's - // resource identity; folded into the Redis key as actor::. - string atespace = 15; +message CreateActorRequest { + // The actor to create. Caller sets meta.atespace, meta.name (a valid + // DNS-1123 label), actor_template, and optionally worker_selector. + // All other fields are output-only and must be unset. + Actor actor = 1; } -// Atespace is the isolation boundary an Actor is created into. -message Atespace { - string name = 1; +message GetActorRequest { + ObjectRef actor = 1; } -// ActorRef identifies an actor: a name is only unique within its atespace. -message ActorRef { +// List actors with pagination. This operation provides "soft" guarantees: +// actors may be missed or duplicated if the system state changes during +// pagination. +message ListActorsRequest { + // The atespace to list actors from. Empty lists across all atespaces. string atespace = 1; - string name = 2; + int32 page_size = 2; + string page_token = 3; +} +message ListActorsResponse { + repeated Actor actors = 1; + string next_page_token = 2; } -message CreateAtespaceRequest { - string name = 1; +message UpdateActorRequest { + // actor.meta.atespace and actor.meta.name identify the resource. + // A non-zero actor.meta.version acts as a freshness guard. + Actor actor = 1; + + // Required. The only updatable path today is "worker_selector"; any + // other path returns INVALID_ARGUMENT. + google.protobuf.FieldMask update_mask = 2; } -message CreateAtespaceResponse { - Atespace atespace = 1; + +message DeleteActorRequest { + ObjectRef actor = 1; + // Optional freshness guard; 0 = skip check. + int64 version = 2; } -message GetAtespaceRequest { +message SuspendActorRequest { + ObjectRef actor = 1; +} + +message PauseActorRequest { + ObjectRef actor = 1; +} + +message ResumeActorRequest { + ObjectRef actor = 1; + + // If true, skip the golden snapshot and boot the workload from scratch. + bool boot = 2; +} + +// --------------------------------------------------------------------------- +// ActorTemplate +// --------------------------------------------------------------------------- + +// ActorTemplate defines the workload actors are stamped from. +// Atespace-scoped. The spec is immutable after creation (golden snapshots +// are only valid for the exact spec they were taken from), so there is no +// UpdateActorTemplate method: create a new template instead. +message ActorTemplate { + ResourceMetadata meta = 1; + ActorTemplateSpec spec = 2; + // Output-only. + ActorTemplateStatus status = 3; +} + +message ActorTemplateSpec { + // Image used to hold the sandbox while no actor is resumed into it. + string pause_image = 1; + + // The workload containers. Images must be pinned by digest. + repeated Container containers = 2; + + SnapshotsConfig snapshots_config = 3; + + // The sandbox class this workload requires. Matched against + // WorkerPoolSpec.sandbox_class at scheduling time. Open string: the set + // of classes is defined by the SandboxConfigs installed by the platform + // operator. + string sandbox_class = 4; + + // Restricts which worker pools actors from this template may be placed + // on. ANDed with the per-actor worker_selector. + Selector worker_selector = 5; + + repeated Volume volumes = 6; +} + +message ActorTemplateStatus { + enum State { + STATE_UNSPECIFIED = 0; + // Waiting for a golden snapshot to be scheduled. + STATE_PENDING = 1; + // The golden snapshot is being taken. + STATE_SNAPSHOTTING = 2; + // Golden snapshot exists; actors can be created from this template. + STATE_READY = 3; + // Golden snapshotting failed; see error_message. + STATE_FAILED = 4; + } + State state = 1; + + // The golden actor used to produce the golden snapshot. + string golden_actor_id = 2; + + // URI prefix of the golden snapshot, once taken. + string golden_snapshot = 3; + + // Human-readable failure detail when state is FAILED. + string error_message = 4; +} + +message Container { string name = 1; + // Must be pinned by digest (image@sha256:...) so that golden snapshots + // cannot drift from the running image. + string image = 2; + repeated string command = 3; + repeated EnvVar env = 4; + ContainerReadyz readyz = 5; + repeated VolumeMount volume_mounts = 6; } -message GetAtespaceResponse { - Atespace atespace = 1; + +message ContainerReadyz { + HTTPGetAction http_get = 1; } -message ListAtespacesRequest {} -message ListAtespacesResponse { - repeated Atespace atespaces = 1; +message HTTPGetAction { + string path = 1; + int32 port = 2; } -message DeleteAtespaceRequest { +message EnvVar { string name = 1; + oneof source { + // Literal value. + string value = 2; + // Value read from a named secret at actor start. + SecretRef secret = 3; + } } -message DeleteAtespaceResponse {} -message GetActorRequest { - ActorRef actor_ref = 1; +// SecretRef names a secret managed by the platform the workers run on +// (a Kubernetes Secret today). Deliberately plane-neutral: no k8s types. +message SecretRef { + // Name of the secret, resolved in the worker's environment. + string name = 1; + // Key within the secret. + string key = 2; + // If true, a missing secret or key is skipped instead of failing the + // actor start. + bool optional = 3; } -message GetActorResponse { - Actor actor = 1; +message Volume { + string name = 1; + VolumeSource volume_source = 2; } -// Request to create a new Actor. -message CreateActorRequest { - // The new actor's atespace and actor_id (actor_id must be a valid DNS-1123 label). - ActorRef actor_ref = 1; +// VolumeSource is an extension point; DurableDir is the only source today. +message VolumeSource { + DurableDirVolumeSource durable_dir = 1; +} - // The namespace of the ActorTemplate to derive from. - string actor_template_namespace = 2; +message DurableDirVolumeSource {} - // The name of the ActorTemplate to derive from. - string actor_template_name = 3; +message VolumeMount { + string name = 1; + string mount_path = 2; +} - // worker_selector sets the actor's placement constraint at creation time. - // If empty, the actor matches any pool admitted by the template's selector. - Selector worker_selector = 4; +message SnapshotsConfig { + enum Scope { + SCOPE_UNSPECIFIED = 0; + // Snapshot the full sandbox state (memory + filesystem). + SCOPE_FULL = 1; + // Snapshot durable data only. + SCOPE_DATA = 2; + } + + // URI prefix under which snapshots are stored. + string location = 1; + + // Scope taken on PauseActor / SuspendActor. + Scope on_pause = 2; + + // Scope taken on commit. Must be a subset of on_pause + // (INVALID_ARGUMENT otherwise). + Scope on_commit = 3; } -message CreateActorResponse { - Actor actor = 1; +message CreateActorTemplateRequest { + ActorTemplate actor_template = 1; } -// Request to update mutable fields on an existing Actor. -// May be called regardless of the actor's current status. -// Changes take effect on the next ResumeActor call. -message UpdateActorRequest { - ActorRef actor_ref = 1; +message GetActorTemplateRequest { + ObjectRef actor_template = 1; +} - // worker_selector replaces the actor's current placement constraint. - // Takes effect on the next ResumeActor call. - Selector worker_selector = 2; +message ListActorTemplatesRequest { + // The atespace to list templates from. Empty lists across all atespaces. + string atespace = 1; + int32 page_size = 2; + string page_token = 3; +} +message ListActorTemplatesResponse { + repeated ActorTemplate actor_templates = 1; + string next_page_token = 2; } -message UpdateActorResponse { - Actor actor = 1; +message DeleteActorTemplateRequest { + ObjectRef actor_template = 1; + // Optional freshness guard; 0 = skip check. + int64 version = 2; } -message SuspendActorRequest { - ActorRef actor_ref = 1; +message WatchActorTemplatesRequest {} +message WatchActorTemplatesResponse { + ResourceEventType type = 1; + // Unset when type is SYNCED. + ActorTemplate actor_template = 2; } -message SuspendActorResponse { - Actor actor = 1; +// --------------------------------------------------------------------------- +// WorkerPool +// --------------------------------------------------------------------------- + +// WorkerPool declares a pool of ready workers that actors are multiplexed +// onto. Global-scoped: pools are platform capacity, shared across +// atespaces via WorkerPoolGrants. meta.labels are the match target for +// template/actor worker_selectors. +message WorkerPool { + ResourceMetadata meta = 1; + WorkerPoolSpec spec = 2; + // Output-only. + WorkerPoolStatus status = 3; } -message PauseActorRequest { - ActorRef actor_ref = 1; +message WorkerPoolSpec { + // Desired number of ready workers. + int32 replicas = 1; + + // Image of the ateom management container run in each worker. + string ateom_image = 2; + + // The sandbox class workers in this pool provide. Matched against + // ActorTemplateSpec.sandbox_class at scheduling time. + string sandbox_class = 3; + + // The SandboxConfig applied to workers in this pool (global-scoped + // reference: atespace must be empty). + ObjectRef sandbox_config = 4; + + // How the pool is materialized on Kubernetes. This is the single, + // clearly-marked pocket of Kubernetes-specific configuration in the + // substrate API; everything outside it is plane-neutral. A future + // non-k8s substrate would add a sibling placement message. + KubernetesPlacement kubernetes = 5; } -message PauseActorResponse { - Actor actor = 1; +message WorkerPoolStatus { + // Number of workers currently backing the pool. + int32 replicas = 1; } -message ResumeActorRequest { - ActorRef actor_ref = 1; +// KubernetesPlacement configures how backing worker pods are placed on a +// Kubernetes cluster. Messages below mirror k8s.io/api/core/v1 verbatim +// and are quarantined to this pocket. +message KubernetesPlacement { + // Kubernetes namespace the backing Deployment/pods are created in. + // (The POC called this deployment_atespace; it is and always was a + // Kubernetes namespace, so it is named honestly here.) + string namespace = 1; - // If true, skip golden snapshot and boot the workload from scratch. - bool boot = 2; + map node_selector = 2; + repeated Toleration tolerations = 3; + NodeAffinity node_affinity = 4; + ResourceRequirements resources = 5; + string priority_class_name = 6; } -message ResumeActorResponse { - Actor actor = 1; +message Toleration { + string key = 1; + string operator = 2; + string value = 3; + string effect = 4; + optional int64 toleration_seconds = 5; } -message DeleteActorRequest { - ActorRef actor_ref = 1; +message NodeAffinity { + NodeSelector required_during_scheduling_ignored_during_execution = 1; + repeated PreferredSchedulingTerm preferred_during_scheduling_ignored_during_execution = 2; } -message DeleteActorResponse {} +message NodeSelector { + repeated NodeSelectorTerm node_selector_terms = 1; +} -message ListWorkersRequest {} +message PreferredSchedulingTerm { + int32 weight = 1; + NodeSelectorTerm preference = 2; +} -message ListWorkersResponse { - repeated Worker workers = 1; +message NodeSelectorTerm { + repeated NodeSelectorRequirement match_expressions = 1; + repeated NodeSelectorRequirement match_fields = 2; } -// Request to list actors with pagination. -// This operation provides "soft" guarantees: actors may be missed or duplicated -// if the system state changes during pagination. -message ListActorsRequest { - // The maximum number of actors to return. The server may enforce a hard limit - // (e.g., 1000) regardless of the requested size. +message NodeSelectorRequirement { + string key = 1; + string operator = 2; + repeated string values = 3; +} + +message ResourceRequirements { + map limits = 1; + map requests = 2; +} + +message CreateWorkerPoolRequest { + WorkerPool worker_pool = 1; +} + +message GetWorkerPoolRequest { + ObjectRef worker_pool = 1; +} + +message ListWorkerPoolsRequest { int32 page_size = 1; + string page_token = 2; +} +message ListWorkerPoolsResponse { + repeated WorkerPool worker_pools = 1; + string next_page_token = 2; +} + +message UpdateWorkerPoolRequest { + // worker_pool.meta.name identifies the resource; a non-zero + // worker_pool.meta.version acts as a freshness guard. + WorkerPool worker_pool = 1; + + // Required. Top-level field paths only (e.g. "spec.replicas", + // "spec.kubernetes", "meta.labels"); naming a message or map field + // replaces that field wholesale. Unknown or immutable paths return + // INVALID_ARGUMENT. + google.protobuf.FieldMask update_mask = 2; +} + +message DeleteWorkerPoolRequest { + ObjectRef worker_pool = 1; + // Optional freshness guard; 0 = skip check. + int64 version = 2; +} + +message WatchWorkerPoolsRequest {} +message WatchWorkerPoolsResponse { + ResourceEventType type = 1; + // Unset when type is SYNCED. + WorkerPool worker_pool = 2; +} + +// --------------------------------------------------------------------------- +// SandboxConfig +// --------------------------------------------------------------------------- - // An opaque pagination token obtained from a previous ListActorsResponse. - // Empty for the first request. +// SandboxConfig defines a sandbox class: the runtime assets and settings +// workers of that class are provisioned with. Global-scoped. The set of +// SandboxConfigs is the de-facto registry of valid sandbox_class values. +message SandboxConfig { + ResourceMetadata meta = 1; + SandboxConfigSpec spec = 2; +} + +message SandboxConfigSpec { + // The sandbox class this config defines. + string sandbox_class = 1; + + // If true, this config is applied to pools that do not name one + // explicitly. At most one SandboxConfig may be default. + bool default = 2; + + // Assets provisioned into the sandbox, keyed by asset group name. + map assets = 3; +} + +message SandboxAssetFiles { + // Files keyed by destination path. + map files = 1; +} + +message AssetFile { + string url = 1; + string sha256 = 2; +} + +message CreateSandboxConfigRequest { + SandboxConfig sandbox_config = 1; +} + +message GetSandboxConfigRequest { + ObjectRef sandbox_config = 1; +} + +message ListSandboxConfigsRequest { + int32 page_size = 1; string page_token = 2; +} +message ListSandboxConfigsResponse { + repeated SandboxConfig sandbox_configs = 1; + string next_page_token = 2; +} + +message UpdateSandboxConfigRequest { + SandboxConfig sandbox_config = 1; - // If set, list only actors in this atespace (scoped SCAN actor::*). - string atespace = 3; + // Required. Same path semantics as UpdateWorkerPoolRequest.update_mask + // (e.g. "spec.assets" replaces the whole assets map). + google.protobuf.FieldMask update_mask = 2; } -// Response for a ListActors operation. -message ListActorsResponse { - // The page of actors. This list may be empty even if there are more results. - repeated Actor actors = 1; +message DeleteSandboxConfigRequest { + ObjectRef sandbox_config = 1; + // Optional freshness guard; 0 = skip check. + int64 version = 2; +} + +// --------------------------------------------------------------------------- +// WorkerPoolGrant +// --------------------------------------------------------------------------- + +// WorkerPoolGrant allows one atespace to schedule actors onto one worker +// pool. At most one grant may exist per (atespace, worker_pool) pair: +// Create returns ALREADY_EXISTS for a duplicate pair regardless of the +// grant's name, so revocation is always unambiguous. +// +// Grants are immutable (no Update). Future per-grant policy fields (e.g. +// quotas) would make them mutable and add an Update method. +message WorkerPoolGrant { + ResourceMetadata meta = 1; + + // The worker pool being granted (global-scoped reference: atespace must + // be empty). Required on Create; immutable. + ObjectRef worker_pool = 2; +} + +message CreateWorkerPoolGrantRequest { + WorkerPoolGrant worker_pool_grant = 1; +} + +message GetWorkerPoolGrantRequest { + ObjectRef worker_pool_grant = 1; +} - // An opaque token to retrieve the next page of results. - // If empty, there are no more results. Clients must loop until this is empty. +message ListWorkerPoolGrantsRequest { + // The atespace to list grants from. Empty lists across all atespaces. + string atespace = 1; + int32 page_size = 2; + string page_token = 3; +} +message ListWorkerPoolGrantsResponse { + repeated WorkerPoolGrant worker_pool_grants = 1; string next_page_token = 2; } +message DeleteWorkerPoolGrantRequest { + ObjectRef worker_pool_grant = 1; + // Optional freshness guard; 0 = skip check. + int64 version = 2; +} + +// --------------------------------------------------------------------------- +// Worker (debug projection, not a managed resource) +// --------------------------------------------------------------------------- + +// Worker is a read-only projection of a worker pod's state in the store. +// It is not a managed resource (no ResourceMetadata, no CRUD): it exists +// for operator debugging via ListWorkers and intentionally exposes +// infrastructure detail. message Worker { string worker_namespace = 1; string worker_pool = 2; string worker_pod = 3; - Assignment assignment = 4; - - string ip = 5; - int64 version = 6; - string worker_pod_uid = 7; - string node_name = 8; + string actor_atespace = 4; + string actor_template = 5; + string actor_id = 6; + string ip = 7; + int64 version = 8; + string worker_pod_uid = 9; + string node_name = 10; } -message Assignment { - KubeNamespacedObjectRef actor_template = 1; - ActorRef actor = 2; -} - -message KubeNamespacedObjectRef { - string namespace = 1; - string name = 2; +message ListWorkersRequest {} +message ListWorkersResponse { + repeated Worker workers = 1; } message DebugClearRequest {} - message DebugClearResponse {} +// --------------------------------------------------------------------------- +// SessionIdentity (unchanged from the POC surface) +// --------------------------------------------------------------------------- + // SessionIdentity allows substrate workloads to exchange their // infrastructure-level credentials (k8s service account token, etc.) for a // substrate session-level credential. A given substrate session might migrate From abbcee6a6c01a1c5703b2516b5b74bbab05db19b Mon Sep 17 00:00:00 2001 From: Eitan Yarmush Date: Thu, 2 Jul 2026 19:16:42 +0000 Subject: [PATCH 2/5] docs: explain worker pool grants --- docs/design/ateapi-resource-api.md | 28 ++++++++++++++++++++-------- pkg/proto/ateapipb/ateapi.proto | 14 ++++++++++---- 2 files changed, 30 insertions(+), 12 deletions(-) diff --git a/docs/design/ateapi-resource-api.md b/docs/design/ateapi-resource-api.md index 1305db5e9..8d2be3b5c 100644 --- a/docs/design/ateapi-resource-api.md +++ b/docs/design/ateapi-resource-api.md @@ -51,14 +51,26 @@ in-place mutation is meaningless. Template changes = create a new template and migrate actors. Delete returns `FAILED_PRECONDITION` while any actor references the template. -**D4. At most one WorkerPoolGrant per (atespace, worker pool).** Grants are -fully conventional resources (caller-named, standard methods), but Create -enforces uniqueness on the pair — a duplicate grant for the same pool -returns `ALREADY_EXISTS` regardless of its name (same pattern as the -single-default SandboxConfig rule). This keeps revocation unambiguous and -lets the scheduler's `(atespace, pool)` check stay a point lookup via a -server-side index. Grants are immutable; future policy fields (quotas) -would add Update. +**D4. WorkerPoolGrant is a separate resource because pool capacity and +atespace access have different owners and lifecycles.** A WorkerPool is +global platform capacity; an ActorTemplate is tenant/atespace-scoped demand. +The scheduler needs an explicit admission edge between them: selectors answer +"does this pool match?", while grants answer "may this atespace use it?". +Putting grants on WorkerPool would make every pool update rewrite a shared +allow-list and would couple capacity changes to tenant access control. +Putting grants on ActorTemplate would duplicate the same permission across +templates and make revocation ambiguous. A separate resource gives admins a +small auditable object to create/delete, supports future per-grant policy +(quotas, priority, expiry), and keeps scheduling as a simple +`(atespace, worker_pool)` check. + +At most one WorkerPoolGrant may exist per `(atespace, worker_pool)`. Grants +are fully conventional resources (caller-named, standard methods), but Create +enforces uniqueness on the pair — a duplicate grant for the same pool returns +`ALREADY_EXISTS` regardless of its name (same pattern as the single-default +SandboxConfig rule). This keeps revocation unambiguous and lets the +scheduler's `(atespace, pool)` check stay a point lookup via a server-side +index. Grants are immutable; future policy fields would add Update. **D5. One Kubernetes pocket: `WorkerPoolSpec.kubernetes`.** Everything k8s-specific about materializing a pool — namespace, node_selector, diff --git a/pkg/proto/ateapipb/ateapi.proto b/pkg/proto/ateapipb/ateapi.proto index 7d615f8de..716ec4275 100644 --- a/pkg/proto/ateapipb/ateapi.proto +++ b/pkg/proto/ateapipb/ateapi.proto @@ -809,10 +809,16 @@ message DeleteSandboxConfigRequest { // WorkerPoolGrant // --------------------------------------------------------------------------- -// WorkerPoolGrant allows one atespace to schedule actors onto one worker -// pool. At most one grant may exist per (atespace, worker_pool) pair: -// Create returns ALREADY_EXISTS for a duplicate pair regardless of the -// grant's name, so revocation is always unambiguous. +// WorkerPoolGrant is the explicit admission edge between tenant demand and +// platform capacity: selectors decide whether a pool matches an actor, grants +// decide whether an atespace may use that pool. Keeping this as its own +// resource avoids embedding tenant allow-lists in global WorkerPools or +// duplicating access on every ActorTemplate, and gives admins a small object +// to audit and revoke. +// +// At most one grant may exist per (atespace, worker_pool) pair: Create +// returns ALREADY_EXISTS for a duplicate pair regardless of the grant's name, +// so revocation is always unambiguous. // // Grants are immutable (no Update). Future per-grant policy fields (e.g. // quotas) would make them mutable and add an Update method. From c8e1f0e6a05901128d43babd446064b80c826e0d Mon Sep 17 00:00:00 2001 From: Eitan Yarmush Date: Thu, 2 Jul 2026 19:17:25 +0000 Subject: [PATCH 3/5] docs: note scoped worker pool grant tradeoff --- docs/design/ateapi-resource-api.md | 5 +++++ pkg/proto/ateapipb/ateapi.proto | 2 ++ 2 files changed, 7 insertions(+) diff --git a/docs/design/ateapi-resource-api.md b/docs/design/ateapi-resource-api.md index 8d2be3b5c..507b1ac08 100644 --- a/docs/design/ateapi-resource-api.md +++ b/docs/design/ateapi-resource-api.md @@ -64,6 +64,11 @@ small auditable object to create/delete, supports future per-grant policy (quotas, priority, expiry), and keeps scheduling as a simple `(atespace, worker_pool)` check. +If WorkerPools were atespace-scoped, this resource would not be necessary: +pool ownership would already define who may schedule onto it. The grant exists +because this design keeps WorkerPools global, so access from an atespace to a +global pool must be represented explicitly. + At most one WorkerPoolGrant may exist per `(atespace, worker_pool)`. Grants are fully conventional resources (caller-named, standard methods), but Create enforces uniqueness on the pair — a duplicate grant for the same pool returns diff --git a/pkg/proto/ateapipb/ateapi.proto b/pkg/proto/ateapipb/ateapi.proto index 716ec4275..4c70101f6 100644 --- a/pkg/proto/ateapipb/ateapi.proto +++ b/pkg/proto/ateapipb/ateapi.proto @@ -815,6 +815,8 @@ message DeleteSandboxConfigRequest { // resource avoids embedding tenant allow-lists in global WorkerPools or // duplicating access on every ActorTemplate, and gives admins a small object // to audit and revoke. +// If WorkerPools become atespace-scoped, ownership would already define +// scheduling access and this resource would no longer be needed. // // At most one grant may exist per (atespace, worker_pool) pair: Create // returns ALREADY_EXISTS for a duplicate pair regardless of the grant's name, From 0c2317ab2a915cfd3331727e688331f5dde10a8f Mon Sep 17 00:00:00 2001 From: Eitan Yarmush Date: Tue, 7 Jul 2026 19:29:59 +0000 Subject: [PATCH 4/5] Document worker-centric ateapi design --- docs/design/ateapi-crd-migration.md | 210 +++++++++---- docs/design/ateapi-resource-api.md | 157 ++++----- docs/design/label-registry.md | 16 + pkg/proto/ateapipb/ateapi.proto | 472 +++++++--------------------- 4 files changed, 362 insertions(+), 493 deletions(-) create mode 100644 docs/design/label-registry.md diff --git a/docs/design/ateapi-crd-migration.md b/docs/design/ateapi-crd-migration.md index 42445298d..0014c43eb 100644 --- a/docs/design/ateapi-crd-migration.md +++ b/docs/design/ateapi-crd-migration.md @@ -1,67 +1,149 @@ # Migrating substrate-plane resources from CRDs to ateapi (issue #368) Companion to [ateapi-resource-api.md](ateapi-resource-api.md), which defines -the target API. This doc covers the transition: what changes relative to the -Kubernetes CRDs and the POC proto (`poc-decouple-upstream`), and the -migration path. - -## Why the POC shape changes - -The POC copied the CRD Go types field-for-field to move fast. That imported -Kubernetes idioms that don't pull their weight in a native API: - -| # | Leak | Where | Problem | -|---|------|-------|---------| -| 1 | `deployment_atespace` | WorkerPoolSpec | It's a **Kubernetes namespace** (the projector uses it verbatim as the Deployment namespace), mislabeled with a substrate term. Renamed `kubernetes.namespace`. | -| 2 | `WorkerPoolPodTemplate` + Toleration/NodeAffinity/ResourceRequirements at top level | WorkerPool | corev1 mirrored into the public API without a boundary marking it k8s-specific. Quarantined into `KubernetesPlacement`. | -| 3 | `EnvVarSource`/`SecretKeySelector` | Container env | corev1's three-field env shape; invalid states representable (value + value_from). Now a `oneof`. | -| 4 | `LabelSelector.match_expressions` | template worker_selector | Set-based operators copied but only equality is implemented or meaningful today. Dropped. | -| 5 | `Condition` list + string `phase` | ActorTemplateStatus | KRM condition machinery; `observed_generation` has no generation to observe. Replaced by a state enum. | -| 6 | Stringly-typed enums (`on_pause: "Full"`) | SnapshotsConfig | CEL-validated strings instead of proto enums. Now `Scope` enum. | -| 7 | Update RPCs with unspecified FieldMask semantics | templates, pools, configs | The POC marked mask semantics TODO. Now defined (top-level paths), and templates lose Update entirely. | -| 8 | Grant identity `(atespace, name)` with free-form name | WorkerPoolGrant | The scheduler looks up grants by `(atespace, pool)`; the POC allows multiple grants for the same pair, making revocation ambiguous. The POC's own `GetWorkerPoolGrant` is documented "by atespace and worker pool" but keyed by name. Now: Create enforces at most one grant per (atespace, pool). | - -## Behavior deltas vs the POC - -1. **No UpdateActorTemplate** — POC has it; the target API drops it - (immutable spec). -2. **Grant uniqueness** — Create rejects a second grant for the same - (atespace, worker_pool) pair with `ALREADY_EXISTS`; the store gains a - `(atespace, pool)` index so the scheduler check stays a point lookup. -3. **Delete returns the resource** — POC returned `Empty` for - templates/pools/configs/grants and an empty message for actors. -4. **Required update_mask** — Update without a mask becomes - `INVALID_ARGUMENT`; POC treated masks as TODO/full-replace. -5. **Actor RPC shapes** — bare-resource returns, `ObjectRef` identity, and - the legacy `actor_template_namespace/name` create path is gone. The - package rename (`ateapi` → `ateapi.v1alpha1`) makes every message a new - proto type, so this is a clean break: field numbers are assigned fresh, - no `reserved` scar tissue, and migration is a coordinated client cutover - rather than in-place field evolution. (The POC has no external users.) -6. **Referential integrity on delete** — template↔actor, pool↔grant, - pool↔assigned-actor, config↔pool checks return `FAILED_PRECONDITION`; - the POC checks only atespace emptiness. -7. **Watch SYNCED event** — new; consumers can await cache completeness. - -## CRD validations that move into handlers - -Validation currently expressed as CEL/OpenAPI on the CRDs moves into -Create (and Update, where applicable) handlers: - -- DNS-1123 label names (atespace, name — per style guide §2.3) -- template spec immutability (`self == oldSelf` → no Update method) -- container images pinned by digest -- volume mount-path rules -- `snapshots_config.on_commit` ⊆ `on_pause` -- at most one `default: true` SandboxConfig - -## Migration path - -1. Land the target proto as `ateapi.v1alpha1` alongside the POC `ateapi` - package; generate both. -2. Port store keys/values: add the grant `(atespace, pool)` uniqueness - index; template records gain immutability enforcement. -3. Move CRD validation into handlers (table above); delete the CRDs and the - projector's CRD-watching path. -4. Cut clients (atectl, atelet caches, demos) over to `v1alpha1`; delete - the POC package. +the target API. This doc covers how to move the current Kubernetes-backed +implementation to the Worker-centric model. + +## Target shape + +Substrate core should manage scheduling intent and observed worker capacity: + +- `ActorTemplate` and `Actor` are tenant/control-plane resources in ateapi. +- `Worker` is the substrate scheduling primitive and a managed Admin resource. +- Kubernetes may still have a WorkerPool CRD, Helm value, Deployment wrapper, + or autoscaler input, but that belongs to the Kubernetes worker provider. + +The split mirrors Kubernetes Nodes and cloud-provider NodePools: core +substrate schedules against Workers; providers decide how to create and manage +many Workers. + +## Current CRD responsibilities + +The existing Kubernetes CRDs combine several responsibilities that need to be +split apart: + +| Current concept | Current responsibility | Target owner | +|---|---|---| +| `ActorTemplate` CRD | workload template and scheduling constraints | ateapi `ActorTemplate` | +| `WorkerPool` CRD | desired worker capacity, Kubernetes placement, common labels | Kubernetes worker provider | +| Worker pod | actual schedulable capacity | ateapi `Worker` projection/resource | + +The key change is that substrate scheduling no longer follows +`ActorTemplate -> WorkerPool -> Pod`. It follows +`ActorTemplate/Actor -> Worker`. + +## Kubernetes provider flow + +The Kubernetes implementation can start with a simple pod watcher: + +1. Some Kubernetes mechanism creates worker Pods. This can be a Deployment, + Helm-managed workload, an existing WorkerPool CRD, or a future autoscaler. +2. Worker Pods opt into substrate by setting + `pod.ate.dev/is-worker: "true"` from + [label-registry.md](label-registry.md). +3. A worker sync goroutine watches those Pods. +4. For each matching Pod, the syncer creates or updates an ateapi `Worker`. +5. When a matching Pod disappears, the syncer deletes or marks the Worker not + ready. +6. The scheduler uses the existing worker cache, but the cache is populated + from `WatchWorkers` / `ListWorkers` rather than Kubernetes-specific pod + state. + +The syncer can initially run inside `ateapi` as a standalone goroutine. It +should be factored so it can later move into a separate worker-provider +process without changing the core API. + +## Worker data copied from Pods + +The first Kubernetes syncer should populate provider-neutral Worker fields: + +- `meta.name`: stable substrate Worker name. +- `meta.labels`: scheduling labels copied under a documented policy. +- `spec.provider`: `"kubernetes"`. +- `spec.provider_id`: Kubernetes Pod UID or namespace/name for correlation. +- `spec.address`: routable ateom endpoint. +- `spec.sandbox_class`: runtime class advertised by this Worker. +- `status.state`: ready/not-ready/draining. +- `status.slots_total` and `status.slots_available`: reported capacity. +- `status.last_heartbeat_time`: last observed provider update. + +Open label policy: copy all Pod labels, only substrate-owned labels, or a +configured allowlist. Because selectors depend on Worker labels, copied labels +become API contract and should be documented before broad use. + +## Kubernetes WorkerPool + +WorkerPool remains useful as a Kubernetes/provider abstraction, just not as a +portable substrate API resource. + +Provider-level WorkerPool can own Kubernetes-specific concerns: + +- desired replica count or autoscaling policy +- pod template +- node selectors, tolerations, affinity, resources, priority class +- common labels copied onto produced Workers +- future desired distributions of worker sizes, e.g. `4x2Gi`, `8x4Gi`, + `12x8Gi` + +Substrate core should not depend on that shape. If the Kubernetes provider +uses WorkerPool internally, it reconciles WorkerPool to Pods; the pod syncer +then reconciles Pods to ateapi Workers. + +## Behavior deltas + +1. **Scheduling uses Workers directly.** Selectors match + `Worker.meta.labels`, and sandbox compatibility matches + `ActorTemplate.spec.sandbox_class` to `Worker.spec.sandbox_class`. +2. **Worker is no longer debug-only.** It becomes the managed Admin resource + that worker providers create/update/delete. +3. **Kubernetes fields leave core ateapi.** Pod namespace/name/UID, node name, + tolerations, affinity, and resource requirements are provider details. + Correlation can live in `Worker.spec.provider_id`; placement stays in the + provider. +4. **Process snapshots, including goldens, are deferred.** P0 should work + without golden snapshots. Re-enabling them requires a compatibility model + across runtime version, architecture, kernel/hardware shape, and possibly N + goldens per template. +5. **Delete returns the resource.** Standard methods return the deleted + resource instead of `Empty`. +6. **Required update_mask.** Update without a mask returns + `INVALID_ARGUMENT`; accepted paths are defined by the API. +7. **Watch SYNCED event.** Watch streams deliver initial state, `SYNCED`, then + incremental events. + +## Validation migration + +Validation currently expressed as CEL/OpenAPI on CRDs moves into ateapi +handlers or provider controllers, depending on ownership: + +| Validation | Target owner | +|---|---| +| DNS-1123 names for ateapi resources | ateapi handlers | +| ActorTemplate spec immutability | no UpdateActorTemplate method | +| container images pinned by digest | ateapi handlers | +| volume mount-path rules | ateapi handlers | +| `snapshots_config.on_commit` subset of `on_pause` | ateapi handlers | +| Worker labels and selector syntax | ateapi handlers | +| Worker status/capacity invariants | ateapi handlers | +| Kubernetes namespace, pod template, tolerations, affinity, resources | Kubernetes provider | + +## Implementation migration path + +1. Update the draft proto to expose Worker CRUD/watch. +2. Update the store to key Workers by provider-neutral resource identity + instead of Kubernetes namespace/pool/pod. +3. Convert `cmd/ateapi/internal/workercache` to consume `ListWorkers` and + `WatchWorkers` over managed Worker records. +4. Add a Kubernetes worker sync goroutine that watches Pods labeled + `pod.ate.dev/is-worker=true` and writes Worker records through the same + store/API path as any future provider. +5. Change scheduling to select Workers directly using Worker labels, + `sandbox_class`, readiness, and capacity. +6. Remove Actor references to WorkerPool; record the assigned Worker instead. +7. Move Kubernetes WorkerPool reconciliation out of substrate core. It may + stay as a Kubernetes provider controller or be replaced by plain + Deployments for the first cut. +8. Cut clients and demos to create ActorTemplates/Actors through ateapi and to + inspect Workers through Admin APIs. +9. Delete the old CRD-watching/projector path from core once the Kubernetes + provider path owns pod-to-Worker reconciliation. diff --git a/docs/design/ateapi-resource-api.md b/docs/design/ateapi-resource-api.md index 507b1ac08..9415706d6 100644 --- a/docs/design/ateapi-resource-api.md +++ b/docs/design/ateapi-resource-api.md @@ -2,13 +2,13 @@ Proto: [`pkg/proto/ateapipb/ateapi.proto`](../../pkg/proto/ateapipb/ateapi.proto) -This is the proposed API surface once ActorTemplate, WorkerPool, -SandboxConfig and WorkerPoolGrant move out of Kubernetes CRDs into the -substrate API ([#368](https://github.com/agent-substrate/substrate/issues/368)). -It builds on [decoupling-planes.md](https://github.com/agent-substrate/substrate/blob/poc-decouple-upstream/docs/design/decoupling-planes.md) -(on the POC branch), which settled the -resource model (global pools + per-atespace grants); this doc covers the API -shape. The transition from the CRDs/POC is covered separately in +This is the target API direction for moving substrate scheduling resources out +of Kubernetes CRDs into the substrate API +([#368](https://github.com/agent-substrate/substrate/issues/368)). Substrate +core understands Workers, while WorkerPool is a provider-level capacity +abstraction, similar to Kubernetes understanding Nodes while cloud providers +manage NodePools. The transition from the Kubernetes-backed implementation is +covered separately in [ateapi-crd-migration.md](ateapi-crd-migration.md). The API follows the draft @@ -22,10 +22,7 @@ deliberate divergences are listed at the [end](#style-guide-conformance). | Atespace | global | Control | no | isolation boundary | | ActorTemplate | atespace | Control | no (immutable spec) | + Watch | | Actor | atespace | Control | `worker_selector` only | + Suspend/Pause/Resume | -| WorkerPool | global | Admin | yes | + Watch; labels are the selector match target | -| SandboxConfig | global | Admin | yes | de-facto registry of sandbox classes | -| WorkerPoolGrant | atespace | Admin | no | at most one grant per (atespace, pool) | -| Worker | — | Admin | — | debug projection, ListWorkers only | +| Worker | global | Admin | yes | provider-owned capacity; + Watch | All managed resources carry `ResourceMetadata meta = 1` (atespace, name, uid, version, create_time, update_time, labels) and use the standard method @@ -35,6 +32,36 @@ Delete returns the deleted resource, List paginates with and optimistic concurrency uses `meta.version` (int64, `ABORTED` on mismatch, 0 skips). +WorkerPool is not part of the core ateapi resource model in this direction. A +Kubernetes provider may still have a WorkerPool CRD or config object, but that +object belongs to the provider implementation. Its job is to create +Pods/processes that become Workers in substrate. + +## Revised direction + +1. **Worker is the substrate scheduling primitive.** The scheduler matches + Actor/ActorTemplate requirements against Worker labels, runtime class, and + readiness/capacity. It does not schedule against WorkerPool. +2. **WorkerPool is provider-level.** Kubernetes can keep a WorkerPool CRD, + Deployment wrapper, Helm value, or autoscaler input, but substrate core + only sees the Workers produced by that provider. WorkerPool may later grow + useful provider semantics such as desired distributions of worker sizes. +3. **Worker registration is one-directional.** Worker owners create, update, + heartbeat, and delete Workers. The scheduler already talks to ateom for + placement, so the worker-owner API does not need assignment watches. +4. **Kubernetes worker discovery starts with Pods.** A standalone sync + goroutine can watch Pods labeled `pod.ate.dev/is-worker=true`, copy the + approved metadata onto Worker records, and later move into a distinct + process. +5. **Owned label keys must be documented.** Scheduling labels become API + contract, so substrate-owned labels live in a registry: + [label-registry.md](label-registry.md). +6. **Process snapshots and golden snapshots are deferred for P0.** They are + valuable, but they imply compatibility constraints across gVisor/runtime + version, kernel/hardware shape, architecture, and possibly one golden per + template/runtime/hardware version. The first API should work without + depending on process snapshots. + ## Design decisions **D1. Versioned package `ateapi.v1alpha1`.** gRPC method paths embed the @@ -42,58 +69,35 @@ package, so the package name is the API-versioning mechanism. **D2. Two services with distinct audiences.** `Control` is the tenant surface (atespaces, templates, actors); `Admin` is the platform surface -(pools, sandbox configs, grants, debug). Watch RPCs live next to their -resource. The split gives a future authz story a natural boundary. - -**D3. ActorTemplate spec is immutable — no UpdateActorTemplate.** Golden -snapshots are only valid for the exact spec they were taken from, so -in-place mutation is meaningless. Template changes = create a new template -and migrate actors. Delete returns `FAILED_PRECONDITION` while any actor -references the template. - -**D4. WorkerPoolGrant is a separate resource because pool capacity and -atespace access have different owners and lifecycles.** A WorkerPool is -global platform capacity; an ActorTemplate is tenant/atespace-scoped demand. -The scheduler needs an explicit admission edge between them: selectors answer -"does this pool match?", while grants answer "may this atespace use it?". -Putting grants on WorkerPool would make every pool update rewrite a shared -allow-list and would couple capacity changes to tenant access control. -Putting grants on ActorTemplate would duplicate the same permission across -templates and make revocation ambiguous. A separate resource gives admins a -small auditable object to create/delete, supports future per-grant policy -(quotas, priority, expiry), and keeps scheduling as a simple -`(atespace, worker_pool)` check. - -If WorkerPools were atespace-scoped, this resource would not be necessary: -pool ownership would already define who may schedule onto it. The grant exists -because this design keeps WorkerPools global, so access from an atespace to a -global pool must be represented explicitly. - -At most one WorkerPoolGrant may exist per `(atespace, worker_pool)`. Grants -are fully conventional resources (caller-named, standard methods), but Create -enforces uniqueness on the pair — a duplicate grant for the same pool returns -`ALREADY_EXISTS` regardless of its name (same pattern as the single-default -SandboxConfig rule). This keeps revocation unambiguous and lets the -scheduler's `(atespace, pool)` check stay a point lookup via a server-side -index. Grants are immutable; future policy fields would add Update. - -**D5. One Kubernetes pocket: `WorkerPoolSpec.kubernetes`.** Everything -k8s-specific about materializing a pool — namespace, node_selector, -tolerations, node_affinity, resources, priority_class_name — lives in a -single `KubernetesPlacement` message. Everything outside it is -plane-neutral; a future non-k8s substrate adds a sibling placement message. +(workers and debug). Watch RPCs live next to their resource. The split gives +a future authz story a natural boundary. + +**D3. ActorTemplate spec is immutable — no UpdateActorTemplate.** Template +changes = create a new template and migrate actors. Delete returns +`FAILED_PRECONDITION` while any actor references the template. + +**D4. Worker is a managed Admin resource.** Worker owners create/update/delete +Workers through ateapi; substrate stores them, watches them, and schedules +Actors onto them. The existing internal worker cache remains useful, but its +input should become Worker CRUD/watch rather than Kubernetes-specific pod +state. + +**D5. WorkerPool policy is provider-owned.** Once WorkerPool is not a core +resource, tenant access to provider capacity is not modeled as a core ateapi +object. If a provider-level WorkerPool needs multi-tenant policy, that policy +belongs to the provider until substrate has a concrete cross-provider access +model. **D6. Referential integrity on delete.** Deletes that would orphan dependents return `FAILED_PRECONDITION`: atespace↔actors/templates, -template↔actors, pool↔grants and pool↔assigned actors, config↔pools. +template↔actors, worker↔assigned actors. -**D7. Status is a closed enum, not phase strings or conditions.** -`ActorTemplateStatus.state` ∈ {PENDING, SNAPSHOTTING, READY, FAILED} plus -`golden_actor_id`, `golden_snapshot`, `error_message`. Likewise -`SnapshotsConfig` scopes are a `Scope` enum (FULL/DATA). +**D7. Status is a closed enum, not phase strings or conditions.** Actors, +templates, and workers use closed state enums plus a human-readable error or +message field where needed. **D8. Equality-only `Selector{match_labels}`** for template and per-actor -worker selectors, matched against `WorkerPool.meta.labels`. The scheduler +worker selectors, matched against Worker labels. The scheduler implements equality matching only; set-based expressions can be added as a new field if a consumer appears. @@ -101,31 +105,33 @@ new field if a consumer appears. `SecretRef{name, key, optional}` — value-vs-secret is exclusive by construction, and no k8s types leak into the container spec. -**D10. `sandbox_class` is an open string** on templates and pools. -SandboxConfigs are the class registry; discoverability comes from -ListSandboxConfigs, not a proto enum. +**D10. `sandbox_class` is an open string** on templates and workers. It is a +scheduling compatibility label. Runtime setup remains provider/runtime +configuration. Workers may technically support multiple sandbox classes; P0 +keeps this scalar and can extend it to a repeated field once scheduling +semantics for multi-runtime workers are needed. **D11. Update masks accept top-level paths only.** `update_mask` is required (per the style guide); paths one level below the resource root -(`spec.replicas`, `spec.kubernetes`, `meta.labels`) are accepted, and +(`spec.address`, `status`, `meta.labels`) are accepted, and naming a message or map field replaces it wholesale. Unknown, immutable, or deeper paths → `INVALID_ARGUMENT`. Servers may widen to deeper paths later without breaking clients. **D12. Watch = snapshot, SYNCED, then incremental events.** Streams -(templates, pools) deliver an initial snapshot, a `SYNCED` marker, then +(templates, workers) deliver an initial snapshot, a `SYNCED` marker, then at-least-once CREATED/UPDATED/DELETED events carrying full resource state. No resume tokens: consumers are in-cluster caches and re-syncing is cheap; `meta.version` lets them discard stale events. -**D13. Worker stays a debug projection.** No `ResourceMetadata`, no CRUD. -Making Worker a managed resource would promise lifecycle semantics the -system doesn't offer — workers are pool-managed. +**D13. Worker identity is provider-neutral.** Core identity is +`ResourceMetadata.name`. Provider details such as Kubernetes namespace, Pod +name, Pod UID, VM ID, or runtime endpoint are fields on Worker spec/status for +debugging and connection, not part of the resource key. **D14. Actor is a managed resource like the rest.** `meta` carries its -identity; `actor_template` and `worker_pool` are `ObjectRef`s; lifecycle -verbs (Suspend/Pause/Resume) are custom methods returning the Actor. The -`ateom_pod_*` fields remain, marked output-only infrastructure debugging. +identity; `actor_template` and assigned `worker` are `ObjectRef`s; lifecycle +verbs (Suspend/Pause/Resume) are custom methods returning the Actor. **D15. Atespace and SessionIdentity are unchanged** beyond conforming to `meta` and standard method shapes. Redesigning them is a non-goal of #368. @@ -134,11 +140,11 @@ verbs (Suspend/Pause/Resume) are custom methods returning the Actor. The The surface follows PR #351 (identity model, `ResourceMetadata`, `ObjectRef`, standard method shapes, required `update_mask`, `version` -concurrency). Four deliberate divergences, feedback filed on PR #351: +concurrency). Four deliberate divergences remain in this draft: 1. **`labels` in `ResourceMetadata`** — the guide's meta has no labels - field, but scheduling requires pool labels as a selector match target. -2. **Immutable resources may omit Update** (D3, D4) — the guide implies all + field, but scheduling requires Worker labels as a selector match target. +2. **Immutable resources may omit Update** (D3) — the guide implies all five standard methods everywhere. 3. **Top-level-only mask paths** (D11) — the guide requires the mask but leaves path granularity undefined. @@ -148,6 +154,15 @@ concurrency). Four deliberate divergences, feedback filed on PR #351: ## Open questions +- **Label copy policy**: for Kubernetes-discovered Workers, copy all labels, + only `*.ate.dev/*`, or a configured allowlist? This is part of the API + contract once selectors depend on it. +- **Worker heartbeat semantics**: define whether Worker owners use Update, + an explicit Heartbeat RPC, or both. CRUD is enough on paper; Heartbeat may + be operationally clearer. +- **Golden snapshots**: process snapshots, including goldens, are deferred. + If re-enabled, define the compatibility key (runtime version, architecture, + kernel/hardware shape) and whether templates maintain N goldens. - **Secrets plane**: `SecretRef` resolves against the worker's environment (a k8s Secret today). Freeze that rule, or design secret distribution before v1beta1? diff --git a/docs/design/label-registry.md b/docs/design/label-registry.md new file mode 100644 index 000000000..5055ad3a8 --- /dev/null +++ b/docs/design/label-registry.md @@ -0,0 +1,16 @@ +# Agent Substrate label registry + +This document lists label keys Agent Substrate claims as API or integration +contract. Provider implementations may use other labels internally, but only +labels listed here should be treated as portable substrate behavior. + +## Pod labels + +### `pod.ate.dev/is-worker` + +Value: `"true"` + +Marks a Kubernetes Pod as an Agent Substrate Worker. A Kubernetes worker syncer +may watch Pods with this label and create/update/delete corresponding Worker +records in ateapi. + diff --git a/pkg/proto/ateapipb/ateapi.proto b/pkg/proto/ateapipb/ateapi.proto index 4c70101f6..34d7fd57d 100644 --- a/pkg/proto/ateapipb/ateapi.proto +++ b/pkg/proto/ateapipb/ateapi.proto @@ -36,10 +36,9 @@ // - ABORTED: version guard mismatch (client echoed a non-zero // meta.version / request version that is stale). // - FAILED_PRECONDITION: referential-integrity or state-machine -// violations (delete a referenced template/pool, +// violations (delete a referenced template/worker, // delete a non-empty atespace, resume a -// non-suspended actor, second default -// SandboxConfig, ...). +// non-suspended actor, ...). syntax = "proto3"; @@ -76,9 +75,8 @@ service Control { // --- ActorTemplate (atespace-scoped, immutable spec: no Update) --- - // Create an ActorTemplate. The spec is immutable after creation: golden - // snapshots are only valid for the exact spec they were taken from. To - // change a template, create a new one and migrate actors. + // Create an ActorTemplate. The spec is immutable after creation: to change + // a template, create a new one and migrate actors. rpc CreateActorTemplate(CreateActorTemplateRequest) returns (ActorTemplate) {} // Get an ActorTemplate. @@ -125,74 +123,32 @@ service Control { rpc ResumeActor(ResumeActorRequest) returns (Actor) {} } -// Admin is the platform-plane API: capacity (worker pools), sandbox -// configuration, and cross-atespace access control (grants). Intended for -// platform operators, not tenants. +// Admin is the platform-plane API: worker capacity and debug surfaces. +// Intended for platform operators and worker providers, not tenants. service Admin { - // --- WorkerPool (global-scoped) --- + // --- Worker (global-scoped) --- - // Create a WorkerPool. - rpc CreateWorkerPool(CreateWorkerPoolRequest) returns (WorkerPool) {} + // Create a Worker. Worker providers call this when capacity becomes + // available to substrate. + rpc CreateWorker(CreateWorkerRequest) returns (Worker) {} - // Get a WorkerPool. - rpc GetWorkerPool(GetWorkerPoolRequest) returns (WorkerPool) {} + // Get a Worker. + rpc GetWorker(GetWorkerRequest) returns (Worker) {} - // List WorkerPools. - rpc ListWorkerPools(ListWorkerPoolsRequest) returns (ListWorkerPoolsResponse) {} + // List Workers. + rpc ListWorkers(ListWorkersRequest) returns (ListWorkersResponse) {} - // Update a WorkerPool. - rpc UpdateWorkerPool(UpdateWorkerPoolRequest) returns (WorkerPool) {} + // Update mutable Worker fields. Worker providers may use this for + // heartbeat-style status updates. + rpc UpdateWorker(UpdateWorkerRequest) returns (Worker) {} - // Delete a WorkerPool. - // Returns FAILED_PRECONDITION while any WorkerPoolGrant references it or - // any actor is currently assigned a worker from it. - rpc DeleteWorkerPool(DeleteWorkerPoolRequest) returns (WorkerPool) {} + // Delete a Worker. Returns FAILED_PRECONDITION while any actor is assigned + // to it. + rpc DeleteWorker(DeleteWorkerRequest) returns (Worker) {} - // Watch WorkerPool changes after an initial snapshot of current state. + // Watch Worker changes after an initial snapshot of current state. // Same delivery semantics as WatchActorTemplates. - rpc WatchWorkerPools(WatchWorkerPoolsRequest) returns (stream WatchWorkerPoolsResponse) {} - - // --- SandboxConfig (global-scoped) --- - - // Create a SandboxConfig. At most one SandboxConfig may set - // spec.default=true; violating this returns FAILED_PRECONDITION. - rpc CreateSandboxConfig(CreateSandboxConfigRequest) returns (SandboxConfig) {} - - // Get a SandboxConfig. - rpc GetSandboxConfig(GetSandboxConfigRequest) returns (SandboxConfig) {} - - // List SandboxConfigs. - rpc ListSandboxConfigs(ListSandboxConfigsRequest) returns (ListSandboxConfigsResponse) {} - - // Update a SandboxConfig. - rpc UpdateSandboxConfig(UpdateSandboxConfigRequest) returns (SandboxConfig) {} - - // Delete a SandboxConfig. - // Returns FAILED_PRECONDITION while any WorkerPool references it. - rpc DeleteSandboxConfig(DeleteSandboxConfigRequest) returns (SandboxConfig) {} - - // --- WorkerPoolGrant (atespace-scoped, immutable) --- - - // Grant an atespace access to a worker pool. At most one grant may - // exist per (atespace, worker_pool) pair; a second grant for the same - // pool returns ALREADY_EXISTS regardless of its name. - rpc CreateWorkerPoolGrant(CreateWorkerPoolGrantRequest) returns (WorkerPoolGrant) {} - - // Get a WorkerPoolGrant. - rpc GetWorkerPoolGrant(GetWorkerPoolGrantRequest) returns (WorkerPoolGrant) {} - - // List WorkerPoolGrants, optionally scoped to one atespace. - rpc ListWorkerPoolGrants(ListWorkerPoolGrantsRequest) returns (ListWorkerPoolGrantsResponse) {} - - // Revoke (delete) a WorkerPoolGrant. Existing assignments are not - // interrupted; the grant is checked at scheduling time. - rpc DeleteWorkerPoolGrant(DeleteWorkerPoolGrantRequest) returns (WorkerPoolGrant) {} - - // --- Debug / introspection --- - - // List all workers currently reflected in the store. Debug surface: - // exposes infrastructure detail and is not a managed resource API. - rpc ListWorkers(ListWorkersRequest) returns (ListWorkersResponse) {} + rpc WatchWorkers(WatchWorkersRequest) returns (stream WatchWorkersResponse) {} // Debugging: drop all data from the ate database. rpc DebugClear(DebugClearRequest) returns (DebugClearResponse) {} @@ -207,8 +163,8 @@ service Admin { // See docs/api-style-guide.md section 6. message ResourceMetadata { // The atespace this resource belongs to. Must be empty for global-scoped - // resource types (Atespace, WorkerPool, SandboxConfig) and set for - // atespace-scoped types (Actor, ActorTemplate, WorkerPoolGrant). + // resource types (Atespace, Worker) and set for atespace-scoped types + // (Actor, ActorTemplate). string atespace = 1; // The name of this resource, unique within its scope. Must be a valid @@ -231,9 +187,9 @@ message ResourceMetadata { google.protobuf.Timestamp update_time = 6; // labels are user-defined key/value pairs used for organization and - // selection. WorkerPool labels are matched by template and actor - // worker_selectors at scheduling time. Mutable on resources that - // support Update. (Not yet in the draft style guide's ResourceMetadata.) + // selection. Worker labels are matched by template and actor + // worker_selectors at scheduling time. Mutable on resources that support + // Update. (Not yet in the draft style guide's ResourceMetadata.) map labels = 7; } @@ -245,9 +201,9 @@ message ObjectRef { string name = 2; } -// Selector matches worker pools by their meta.labels. +// Selector matches workers by their meta.labels. // Only equality-based matching is supported (all pairs must match). -// An empty selector matches every pool. +// An empty selector matches every worker. message Selector { map match_labels = 1; } @@ -352,26 +308,17 @@ message Actor { // worker_selector is the per-actor placement constraint. The scheduler // evaluates the AND of this selector and the template's worker_selector - // to find eligible pools. Set at CreateActor; mutable via UpdateActor. + // to find eligible workers. Set at CreateActor; mutable via UpdateActor. // Changes take effect on the next ResumeActor call. Selector worker_selector = 4; - // The WorkerPool that owns the currently assigned worker. Output-only: - // set by the scheduler at assignment time, cleared when the worker is - // freed on suspend/pause. - ObjectRef worker_pool = 5; + // The currently assigned Worker. Output-only: set by the scheduler at + // assignment time, cleared when the worker is freed on suspend/pause. + ObjectRef worker = 5; // Output-only snapshot bookkeeping. string in_progress_snapshot = 6; SnapshotInfo latest_snapshot_info = 7; - - // Output-only infrastructure debugging fields describing the Kubernetes - // pod currently backing this actor. Not part of the portable resource - // model; may be moved to a debug-only surface in a future version. - string ateom_pod_namespace = 8; - string ateom_pod_name = 9; - string ateom_pod_ip = 10; - string ateom_pod_uid = 11; } message CreateActorRequest { @@ -426,7 +373,8 @@ message PauseActorRequest { message ResumeActorRequest { ObjectRef actor = 1; - // If true, skip the golden snapshot and boot the workload from scratch. + // If true, boot the workload from scratch. Process snapshots, including + // golden snapshots, are deferred from the P0 API. bool boot = 2; } @@ -435,8 +383,7 @@ message ResumeActorRequest { // --------------------------------------------------------------------------- // ActorTemplate defines the workload actors are stamped from. -// Atespace-scoped. The spec is immutable after creation (golden snapshots -// are only valid for the exact spec they were taken from), so there is no +// Atespace-scoped. The spec is immutable after creation, so there is no // UpdateActorTemplate method: create a new template instead. message ActorTemplate { ResourceMetadata meta = 1; @@ -455,13 +402,12 @@ message ActorTemplateSpec { SnapshotsConfig snapshots_config = 3; // The sandbox class this workload requires. Matched against - // WorkerPoolSpec.sandbox_class at scheduling time. Open string: the set - // of classes is defined by the SandboxConfigs installed by the platform - // operator. + // WorkerSpec.sandbox_class at scheduling time. Open string: providers are + // responsible for creating workers that advertise accurate classes. string sandbox_class = 4; - // Restricts which worker pools actors from this template may be placed - // on. ANDed with the per-actor worker_selector. + // Restricts which workers actors from this template may be placed on. ANDed + // with the per-actor worker_selector. Selector worker_selector = 5; repeated Volume volumes = 6; @@ -469,32 +415,24 @@ message ActorTemplateSpec { message ActorTemplateStatus { enum State { - STATE_UNSPECIFIED = 0; - // Waiting for a golden snapshot to be scheduled. - STATE_PENDING = 1; - // The golden snapshot is being taken. - STATE_SNAPSHOTTING = 2; - // Golden snapshot exists; actors can be created from this template. - STATE_READY = 3; - // Golden snapshotting failed; see error_message. - STATE_FAILED = 4; + STATE_UNSPECIFIED = 0; + // Waiting for validation or required provider state. + STATE_PENDING = 1; + // Actors can be created from this template. + STATE_READY = 2; + // Template validation failed; see error_message. + STATE_FAILED = 3; } State state = 1; - // The golden actor used to produce the golden snapshot. - string golden_actor_id = 2; - - // URI prefix of the golden snapshot, once taken. - string golden_snapshot = 3; - // Human-readable failure detail when state is FAILED. - string error_message = 4; + string error_message = 2; } message Container { string name = 1; - // Must be pinned by digest (image@sha256:...) so that golden snapshots - // cannot drift from the running image. + // Must be pinned by digest (image@sha256:...) so actor starts are + // reproducible. string image = 2; repeated string command = 3; repeated EnvVar env = 4; @@ -553,8 +491,9 @@ message VolumeMount { message SnapshotsConfig { enum Scope { SCOPE_UNSPECIFIED = 0; - // Snapshot the full sandbox state (memory + filesystem). - SCOPE_FULL = 1; + // Deprecated for P0: process snapshots imply runtime/version/hardware + // compatibility constraints that are intentionally deferred. + SCOPE_FULL = 1 [deprecated = true]; // Snapshot durable data only. SCOPE_DATA = 2; } @@ -603,292 +542,109 @@ message WatchActorTemplatesResponse { } // --------------------------------------------------------------------------- -// WorkerPool +// Worker // --------------------------------------------------------------------------- -// WorkerPool declares a pool of ready workers that actors are multiplexed -// onto. Global-scoped: pools are platform capacity, shared across -// atespaces via WorkerPoolGrants. meta.labels are the match target for -// template/actor worker_selectors. -message WorkerPool { +// Worker is the substrate scheduling primitive. Worker providers create and +// maintain Workers; substrate schedules Actors onto Workers by matching +// Actor/ActorTemplate requirements against Worker labels, sandbox_class, and +// readiness/capacity. +message Worker { ResourceMetadata meta = 1; - WorkerPoolSpec spec = 2; - // Output-only. - WorkerPoolStatus status = 3; -} - -message WorkerPoolSpec { - // Desired number of ready workers. - int32 replicas = 1; - - // Image of the ateom management container run in each worker. - string ateom_image = 2; - - // The sandbox class workers in this pool provide. Matched against - // ActorTemplateSpec.sandbox_class at scheduling time. - string sandbox_class = 3; - - // The SandboxConfig applied to workers in this pool (global-scoped - // reference: atespace must be empty). - ObjectRef sandbox_config = 4; - - // How the pool is materialized on Kubernetes. This is the single, - // clearly-marked pocket of Kubernetes-specific configuration in the - // substrate API; everything outside it is plane-neutral. A future - // non-k8s substrate would add a sibling placement message. - KubernetesPlacement kubernetes = 5; -} - -message WorkerPoolStatus { - // Number of workers currently backing the pool. - int32 replicas = 1; + WorkerSpec spec = 2; + // Output/status from the worker provider. + WorkerStatus status = 3; } -// KubernetesPlacement configures how backing worker pods are placed on a -// Kubernetes cluster. Messages below mirror k8s.io/api/core/v1 verbatim -// and are quarantined to this pocket. -message KubernetesPlacement { - // Kubernetes namespace the backing Deployment/pods are created in. - // (The POC called this deployment_atespace; it is and always was a - // Kubernetes namespace, so it is named honestly here.) - string namespace = 1; +message WorkerSpec { + // Provider that owns this Worker, e.g. "kubernetes", "vm", or "local". + string provider = 1; - map node_selector = 2; - repeated Toleration tolerations = 3; - NodeAffinity node_affinity = 4; - ResourceRequirements resources = 5; - string priority_class_name = 6; -} + // Provider-local identity for debug/correlation. For Kubernetes this may be + // a Pod UID or namespace/name. Not part of the substrate resource key. + string provider_id = 2; -message Toleration { - string key = 1; - string operator = 2; - string value = 3; - string effect = 4; - optional int64 toleration_seconds = 5; -} + // Routable ateom endpoint for this Worker. + string address = 3; -message NodeAffinity { - NodeSelector required_during_scheduling_ignored_during_execution = 1; - repeated PreferredSchedulingTerm preferred_during_scheduling_ignored_during_execution = 2; + // Runtime compatibility class this Worker provides. Matched against + // ActorTemplateSpec.sandbox_class at scheduling time. This is scalar for P0; + // it can be extended to repeated sandbox classes if workers need to + // advertise multiple compatible runtimes. + string sandbox_class = 4; } -message NodeSelector { - repeated NodeSelectorTerm node_selector_terms = 1; -} +message WorkerStatus { + enum State { + STATE_UNSPECIFIED = 0; + // Worker can accept actor assignments. + STATE_READY = 1; + // Worker exists but should not receive new assignments. + STATE_NOT_READY = 2; + // Worker is intentionally draining and should not receive new assignments. + STATE_DRAINING = 3; + } -message PreferredSchedulingTerm { - int32 weight = 1; - NodeSelectorTerm preference = 2; -} + State state = 1; -message NodeSelectorTerm { - repeated NodeSelectorRequirement match_expressions = 1; - repeated NodeSelectorRequirement match_fields = 2; -} + // Capacity reported by the worker provider. + int32 slots_total = 2; + int32 slots_available = 3; -message NodeSelectorRequirement { - string key = 1; - string operator = 2; - repeated string values = 3; -} + // Human-readable status detail. + string message = 4; -message ResourceRequirements { - map limits = 1; - map requests = 2; + // Last successful provider heartbeat/update time. Output-only. + google.protobuf.Timestamp last_heartbeat_time = 5; } -message CreateWorkerPoolRequest { - WorkerPool worker_pool = 1; +message CreateWorkerRequest { + // worker.meta.name is required; worker.meta.atespace must be empty. + Worker worker = 1; } -message GetWorkerPoolRequest { - ObjectRef worker_pool = 1; +message GetWorkerRequest { + ObjectRef worker = 1; } -message ListWorkerPoolsRequest { +message ListWorkersRequest { int32 page_size = 1; string page_token = 2; } -message ListWorkerPoolsResponse { - repeated WorkerPool worker_pools = 1; +message ListWorkersResponse { + repeated Worker workers = 1; string next_page_token = 2; } -message UpdateWorkerPoolRequest { - // worker_pool.meta.name identifies the resource; a non-zero - // worker_pool.meta.version acts as a freshness guard. - WorkerPool worker_pool = 1; +message UpdateWorkerRequest { + // worker.meta.name identifies the resource; a non-zero + // worker.meta.version acts as a freshness guard. + Worker worker = 1; - // Required. Top-level field paths only (e.g. "spec.replicas", - // "spec.kubernetes", "meta.labels"); naming a message or map field - // replaces that field wholesale. Unknown or immutable paths return - // INVALID_ARGUMENT. + // Required. Top-level field paths only (e.g. "spec.address", + // "status", "meta.labels"); naming a message or map field replaces it + // wholesale. Unknown or immutable paths return INVALID_ARGUMENT. google.protobuf.FieldMask update_mask = 2; } -message DeleteWorkerPoolRequest { - ObjectRef worker_pool = 1; +message DeleteWorkerRequest { + ObjectRef worker = 1; // Optional freshness guard; 0 = skip check. int64 version = 2; } -message WatchWorkerPoolsRequest {} -message WatchWorkerPoolsResponse { +message WatchWorkersRequest {} +message WatchWorkersResponse { ResourceEventType type = 1; // Unset when type is SYNCED. - WorkerPool worker_pool = 2; -} - -// --------------------------------------------------------------------------- -// SandboxConfig -// --------------------------------------------------------------------------- - -// SandboxConfig defines a sandbox class: the runtime assets and settings -// workers of that class are provisioned with. Global-scoped. The set of -// SandboxConfigs is the de-facto registry of valid sandbox_class values. -message SandboxConfig { - ResourceMetadata meta = 1; - SandboxConfigSpec spec = 2; -} - -message SandboxConfigSpec { - // The sandbox class this config defines. - string sandbox_class = 1; - - // If true, this config is applied to pools that do not name one - // explicitly. At most one SandboxConfig may be default. - bool default = 2; - - // Assets provisioned into the sandbox, keyed by asset group name. - map assets = 3; -} - -message SandboxAssetFiles { - // Files keyed by destination path. - map files = 1; -} - -message AssetFile { - string url = 1; - string sha256 = 2; -} - -message CreateSandboxConfigRequest { - SandboxConfig sandbox_config = 1; -} - -message GetSandboxConfigRequest { - ObjectRef sandbox_config = 1; -} - -message ListSandboxConfigsRequest { - int32 page_size = 1; - string page_token = 2; -} -message ListSandboxConfigsResponse { - repeated SandboxConfig sandbox_configs = 1; - string next_page_token = 2; -} - -message UpdateSandboxConfigRequest { - SandboxConfig sandbox_config = 1; - - // Required. Same path semantics as UpdateWorkerPoolRequest.update_mask - // (e.g. "spec.assets" replaces the whole assets map). - google.protobuf.FieldMask update_mask = 2; -} - -message DeleteSandboxConfigRequest { - ObjectRef sandbox_config = 1; - // Optional freshness guard; 0 = skip check. - int64 version = 2; -} - -// --------------------------------------------------------------------------- -// WorkerPoolGrant -// --------------------------------------------------------------------------- - -// WorkerPoolGrant is the explicit admission edge between tenant demand and -// platform capacity: selectors decide whether a pool matches an actor, grants -// decide whether an atespace may use that pool. Keeping this as its own -// resource avoids embedding tenant allow-lists in global WorkerPools or -// duplicating access on every ActorTemplate, and gives admins a small object -// to audit and revoke. -// If WorkerPools become atespace-scoped, ownership would already define -// scheduling access and this resource would no longer be needed. -// -// At most one grant may exist per (atespace, worker_pool) pair: Create -// returns ALREADY_EXISTS for a duplicate pair regardless of the grant's name, -// so revocation is always unambiguous. -// -// Grants are immutable (no Update). Future per-grant policy fields (e.g. -// quotas) would make them mutable and add an Update method. -message WorkerPoolGrant { - ResourceMetadata meta = 1; - - // The worker pool being granted (global-scoped reference: atespace must - // be empty). Required on Create; immutable. - ObjectRef worker_pool = 2; -} - -message CreateWorkerPoolGrantRequest { - WorkerPoolGrant worker_pool_grant = 1; -} - -message GetWorkerPoolGrantRequest { - ObjectRef worker_pool_grant = 1; -} - -message ListWorkerPoolGrantsRequest { - // The atespace to list grants from. Empty lists across all atespaces. - string atespace = 1; - int32 page_size = 2; - string page_token = 3; -} -message ListWorkerPoolGrantsResponse { - repeated WorkerPoolGrant worker_pool_grants = 1; - string next_page_token = 2; -} - -message DeleteWorkerPoolGrantRequest { - ObjectRef worker_pool_grant = 1; - // Optional freshness guard; 0 = skip check. - int64 version = 2; -} - -// --------------------------------------------------------------------------- -// Worker (debug projection, not a managed resource) -// --------------------------------------------------------------------------- - -// Worker is a read-only projection of a worker pod's state in the store. -// It is not a managed resource (no ResourceMetadata, no CRUD): it exists -// for operator debugging via ListWorkers and intentionally exposes -// infrastructure detail. -message Worker { - string worker_namespace = 1; - string worker_pool = 2; - string worker_pod = 3; - - string actor_atespace = 4; - string actor_template = 5; - string actor_id = 6; - string ip = 7; - int64 version = 8; - string worker_pod_uid = 9; - string node_name = 10; -} - -message ListWorkersRequest {} -message ListWorkersResponse { - repeated Worker workers = 1; + Worker worker = 2; } message DebugClearRequest {} message DebugClearResponse {} // --------------------------------------------------------------------------- -// SessionIdentity (unchanged from the POC surface) +// SessionIdentity // --------------------------------------------------------------------------- // SessionIdentity allows substrate workloads to exchange their From eff3cc279443095b40fb6f3f2e2b1533b94f450e Mon Sep 17 00:00:00 2001 From: Eitan Yarmush Date: Tue, 7 Jul 2026 19:35:09 +0000 Subject: [PATCH 5/5] Make actor templates global in ateapi design --- docs/design/ateapi-crd-migration.md | 3 ++- docs/design/ateapi-resource-api.md | 10 ++++++---- pkg/proto/ateapipb/ateapi.proto | 18 ++++++++---------- 3 files changed, 16 insertions(+), 15 deletions(-) diff --git a/docs/design/ateapi-crd-migration.md b/docs/design/ateapi-crd-migration.md index 0014c43eb..f5597634f 100644 --- a/docs/design/ateapi-crd-migration.md +++ b/docs/design/ateapi-crd-migration.md @@ -8,7 +8,8 @@ implementation to the Worker-centric model. Substrate core should manage scheduling intent and observed worker capacity: -- `ActorTemplate` and `Actor` are tenant/control-plane resources in ateapi. +- `ActorTemplate` is a global control-plane resource in ateapi. +- `Actor` is an atespace-scoped tenant/control-plane resource in ateapi. - `Worker` is the substrate scheduling primitive and a managed Admin resource. - Kubernetes may still have a WorkerPool CRD, Helm value, Deployment wrapper, or autoscaler input, but that belongs to the Kubernetes worker provider. diff --git a/docs/design/ateapi-resource-api.md b/docs/design/ateapi-resource-api.md index 9415706d6..efa30b864 100644 --- a/docs/design/ateapi-resource-api.md +++ b/docs/design/ateapi-resource-api.md @@ -20,7 +20,7 @@ deliberate divergences are listed at the [end](#style-guide-conformance). | Resource | Scope | Service | Mutable? | Notes | |---|---|---|---|---| | Atespace | global | Control | no | isolation boundary | -| ActorTemplate | atespace | Control | no (immutable spec) | + Watch | +| ActorTemplate | global | Control | no (immutable spec) | + Watch | | Actor | atespace | Control | `worker_selector` only | + Suspend/Pause/Resume | | Worker | global | Admin | yes | provider-owned capacity; + Watch | @@ -73,7 +73,9 @@ surface (atespaces, templates, actors); `Admin` is the platform surface a future authz story a natural boundary. **D3. ActorTemplate spec is immutable — no UpdateActorTemplate.** Template -changes = create a new template and migrate actors. Delete returns +changes = create a new template and migrate actors. ActorTemplates start +global-scoped because it is easier to add atespace scoping later than to +relax it after clients rely on cross-atespace reuse. Delete returns `FAILED_PRECONDITION` while any actor references the template. **D4. Worker is a managed Admin resource.** Worker owners create/update/delete @@ -89,8 +91,8 @@ belongs to the provider until substrate has a concrete cross-provider access model. **D6. Referential integrity on delete.** Deletes that would orphan -dependents return `FAILED_PRECONDITION`: atespace↔actors/templates, -template↔actors, worker↔assigned actors. +dependents return `FAILED_PRECONDITION`: atespace↔actors, template↔actors, +worker↔assigned actors. **D7. Status is a closed enum, not phase strings or conditions.** Actors, templates, and workers use closed state enums plus a human-readable error or diff --git a/pkg/proto/ateapipb/ateapi.proto b/pkg/proto/ateapipb/ateapi.proto index 34d7fd57d..7ed6419d1 100644 --- a/pkg/proto/ateapipb/ateapi.proto +++ b/pkg/proto/ateapipb/ateapi.proto @@ -70,10 +70,10 @@ service Control { rpc ListAtespaces(ListAtespacesRequest) returns (ListAtespacesResponse) {} // Delete an empty Atespace. - // Returns FAILED_PRECONDITION if any actors or actor templates remain. + // Returns FAILED_PRECONDITION if any actors remain. rpc DeleteAtespace(DeleteAtespaceRequest) returns (Atespace) {} - // --- ActorTemplate (atespace-scoped, immutable spec: no Update) --- + // --- ActorTemplate (global-scoped, immutable spec: no Update) --- // Create an ActorTemplate. The spec is immutable after creation: to change // a template, create a new one and migrate actors. @@ -163,8 +163,8 @@ service Admin { // See docs/api-style-guide.md section 6. message ResourceMetadata { // The atespace this resource belongs to. Must be empty for global-scoped - // resource types (Atespace, Worker) and set for atespace-scoped types - // (Actor, ActorTemplate). + // resource types (Atespace, ActorTemplate, Worker) and set for + // atespace-scoped types (Actor). string atespace = 1; // The name of this resource, unique within its scope. Must be a valid @@ -290,8 +290,8 @@ message Actor { ResourceMetadata meta = 1; // The ActorTemplate this actor derives from. Set at creation; immutable. - // Must currently reside in the actor's own atespace; the ObjectRef shape - // leaves room for cross-atespace template sharing later. + // ActorTemplates are global resources, so actor_template.atespace must be + // empty. ObjectRef actor_template = 2; enum Status { @@ -518,10 +518,8 @@ message GetActorTemplateRequest { } message ListActorTemplatesRequest { - // The atespace to list templates from. Empty lists across all atespaces. - string atespace = 1; - int32 page_size = 2; - string page_token = 3; + int32 page_size = 1; + string page_token = 2; } message ListActorTemplatesResponse { repeated ActorTemplate actor_templates = 1;