diff --git a/docs/T4_ARCHITECTURE.html b/docs/T4_ARCHITECTURE.html index c9e0bd09..2473bb20 100644 --- a/docs/T4_ARCHITECTURE.html +++ b/docs/T4_ARCHITECTURE.html @@ -330,7 +330,7 @@

T4 Code local and managed architecture

  • Advanced installationExisting-cluster installation
  • @@ -527,9 +527,9 @@

    Product goal

    T4 Hub product flow T4 Code desktop and mobile clients connect over Tailscale to T4 Hub. - T4 Hub records durable commands and desired state, directs the T4 - Operator in k3s, launches pinned stock OMP runtime pods, and gives - them access to shared CephFS projects, worktrees, and services. + T4 Hub records durable commands and desired state, while the narrow T4 + Operator reconciles pinned stock OMP runtime pods that mount projects + through an administrator-selected RWX StorageClass. Product goal - k3s cluster and T4 Operator + Kubernetes and T4 Operator reconciles one managed runtime pod per OMP session @@ -638,9 +638,9 @@

    Product goal

    - Shared CephFS project + Shared RWX project - repositories, OMP worktrees, and project services + provider-neutral POSIX storage for repositories and worktrees @@ -683,22 +683,23 @@

    Product profiles

    Personal Hub - One Linux k3s node runs Hub, operator, PostgreSQL, and managed OMP - pods. + One Linux Kubernetes node runs Hub, the T4 operator, and managed OMP + pods. The optional T4 Stack chart provides a tested default path. - Size-one CephFS; pod restart on one failure domain, not hardware HA. + A conformant RWX StorageClass; one failure domain is explicitly + unprotected, not HA. HA Hub - At least three tested Linux failure domains run replicated managed - services. + At least three tested Linux failure domains run replicated T4 + services against separately owned platform dependencies. - Replicated CephFS; portable only after hard stale-writer fencing is - proven. + Replicated provider-managed RWX storage; portable only after hard + stale-writer fencing is proven. @@ -717,7 +718,7 @@

    Product profiles

    T4 Code -> T4 Local Service -> shared OMP adapter -> official OMP -> local files
    -T4 Code -> T4 Hub -> managed runtime -> shared OMP adapter -> official OMP -> CephFS
    +T4 Code -> T4 Hub -> managed runtime -> shared OMP adapter -> official OMP -> RWX project
                          \-> Workstation Runner -> shared OMP adapter -> official OMP -> local files

    T4 Hub integrates source control and CI/CD activity into durable managed @@ -768,15 +769,24 @@

    Product names

    T4 Node - A Linux machine enrolled into the k3s cluster to provide - control-plane, storage, or execution capacity. + A Linux machine enrolled into a Kubernetes cluster to provide T4 + control-plane or execution capacity. T4 Operator - Kubernetes controller that reconciles durable T4 session intent into - runtime pods and related resources. + Kubernetes controller that owns T4 custom resources and reconciles + session intent into runtime pods, Services, PVC claims, and bounded + infrastructure status. It does not operate platform dependencies. + + + + T4 Stack + + Optional Helm distribution that installs T4 components and supplies + tested defaults or references for independently owned database, + storage, ingress, object-storage, and observability services. @@ -791,9 +801,9 @@

    Product names

    Users interact with T4 Code, T4 Local Service, T4 Hub, and optionally a - Workstation Runner or T4 Node. Kubernetes, containerd, PostgreSQL - replication, storage placement, and telemetry topology remain - managed-profile implementation details. + Workstation Runner or T4 Node. Cluster administrators may install the + portable T4 charts against existing services or use the optional T4 Stack + profile. Provider lifecycle and topology remain outside the T4 operator.

    Architectural invariants

    The following invariants apply to managed Hub sessions:

    @@ -806,6 +816,12 @@

    Architectural invariants

    Kubernetes reconciles execution resources; Kubernetes objects are not the product event database. +
  • + The T4 operator owns only T4 custom resources and their runtime Pods, + Services, PVC claims, finalizers, and infrastructure conditions. It + references but never installs or reconciles database, storage-provider, + ingress, object-store, or observability operators. +
  • T4 creates every managed OMP session and therefore knows its legitimate owner from creation. @@ -835,16 +851,17 @@

    Architectural invariants

    Kubernetes resources or direct pod connections.
  • - Development, single-node, and HA installations use the same images, - migrations, protocol, and release chart. Profiles alter topology rather - than behavior. + Development, single-node, and HA installations use the same T4 images, + migrations, protocols, and portable charts. Optional stack profiles + select tested dependency defaults without changing T4 semantics or + transferring provider ownership to the T4 operator.
  • No deployment is described as HA until it has at least three suitable failure domains and passes automated recovery checks.
  • - Product progress is durable application state. Grafana telemetry + Product progress is durable application state. External telemetry supplements it but never replaces it.
  • @@ -862,7 +879,10 @@

    Architectural invariants

    A disconnected workstation session becomes unavailable; Hub never assigns it elsewhere. -
  • Local files are never continuously synchronized into CephFS.
  • +
  • + Local files are never continuously synchronized into managed RWX + storage. +
  • Local Service is a single-machine authority, not a PostgreSQL replacement or cluster control plane. @@ -885,9 +905,9 @@

    Managed system topology

    Desktop and mobile T4 Code clients connect through Tailscale to the T4 Hub control plane. The Hub persists state in PostgreSQL, coordinates - durable events and CI/CD, and uses the T4 Operator to manage OMP - session runtimes in a k3s cluster. Runtime pods share CephFS, while - MinIO stores durable objects and Grafana provides observability. + durable events and CI/CD, and uses the narrow T4 Operator to manage + OMP session runtimes. Runtime pods mount provider-neutral RWX storage; + optional S3-compatible object storage and telemetry remain external. Managed system topology HA control plane - k3s execution cluster + + Kubernetes application plane + - T4 Nodes, session runtimes, and shared services + T4 runtimes consuming referenced platform services @@ -1049,16 +1071,22 @@

    Managed system topology

    managed session pod - CephFS - shared projects + RWX storage + + selected StorageClass + - MinIO - durable objects + S3 API + + optional object store + - Grafana - LGTM stack + Telemetry + + optional external backend + @@ -1091,27 +1119,29 @@

    Managed system topology

    artifacts
    - T4 Hub is the durable authority; k3s runtimes are replaceable workers - sharing CephFS. + T4 Hub is the durable authority; Kubernetes runtimes are replaceable + workers sharing a conformant RWX project volume. Platform services keep + their own APIs, operators, RBAC, and upgrade lifecycles.

    This topology covers Personal Hub and HA Hub. T4 Local has no Kubernetes, - PostgreSQL, CephFS, MinIO, or Grafana dependency; its client connects to - native Local Service over loopback. + PostgreSQL, network filesystem, object-store, or telemetry dependency; its + client connects to native Local Service over loopback.

    T4 Hub placement

    - In a single-node installation, the one Linux machine is both T4 Hub and T4 - Node. In an HA installation, Hub API, database, operator, storage, and - telemetry replicas are distributed across the cluster. Clients use one - stable Tailnet service address and do not select a control-plane replica. + In a single-node installation, one Linux machine may host T4 and compact + dependency instances. In HA mode, T4 API and operator replicas are + distributed independently from provider-managed database, storage, and + telemetry replicas. Clients use one stable Tailnet service address and do + not select a control-plane replica.

    A Linux Hub machine may be a dedicated mini PC, home or office server, NAS-hosted VM, cloud VM, or Linux workstation. macOS and Windows run T4 Code and may run native T4 Local Service; macOS and Linux may additionally - register as Workstation Runners. Hub and k3s node implementations remain + register as Workstation Runners. Managed T4 node implementations remain Linux-only, so a native Mac never needs a managed Linux VM for T4 Local.

    Protocol boundaries

    @@ -1451,33 +1481,33 @@

    Shared project filesystem

    node loss, snapshots, and acceptable metadata performance.

    - Rook-managed CephFS is the selected live shared filesystem. T4 provisions - one CephFS subvolume or equivalent isolation boundary per project and - mounts it through the Ceph CSI driver as RWX storage. Ceph placement must - use explicit nodes and devices; the installer never consumes an unapproved - disk automatically. + Managed deployments consume an administrator-selected Kubernetes + StorageClass that provisions real RWX volumes. T4Workspace + records the requested class and size; the T4 operator creates the PVC and + mounts the bound claim into session pods. It does not install a CSI + driver, create backend-specific CRs, select disks, or change storage + topology.

    - A single-node profile runs one nonredundant Ceph monitor, manager, - metadata service, and size-one data/metadata pools. The GUI labels this - state as unprotected and requires the user to select suitable Ceph - storage. When two additional storage nodes join, T4 distributes monitors, - enables a standby metadata service, changes pool placement to the host - failure domain, raises replication to three, waits for backfill, and - reports HA only after Ceph returns healthy. + The optional T4 Stack profile may install a vetted default storage chart + for users who want a guided single-machine path. That provider retains its + own operator, CRDs, RBAC, status, upgrades, and recovery procedures. An + existing cluster may instead supply any implementation that passes the + same contract.

    - CephFS remains subject to a representative conformance suite covering real - OMP worktree concurrency, large repositories, package trees, pod and node - failure, expansion from one to three nodes, metadata-service failover, - pool backfill, snapshot, restore, and - git fsck integrity. This validates the selected - implementation rather than reopening the provider decision. + Storage conformance covers real OMP worktree concurrency, POSIX locking, + large repositories, package trees, pod and node failure, remount, + snapshot, restore, stale-writer fencing, metadata performance, and + git fsck integrity. A provider name or RWX declaration alone + is not evidence.

    - MinIO is object storage rather than the live POSIX filesystem. It stores - artifacts, uploads, transcript and log chunks, filesystem backups, - database backups, and observability objects. + Optional S3-compatible object storage is a separate contract for immutable + artifacts, uploads, large transcript or log chunks, and backup objects. T4 + receives an endpoint and credential reference; it does not expose + provider-specific object-store CRs. The initial compact profile may keep + bounded objects on RWX storage and omit S3 entirely.

    Technology stack

    @@ -1485,13 +1515,14 @@

    Technology stack

    official OMP release, Git, bounded protocol frames, and platform-native credential storage. T4 Local adds only a native background service, SQLite, loopback or optional Tailnet transport, and the local filesystem. - The following additional stack is selected for managed Hub profiles: + Managed Hub profiles add the following T4 components and provider-neutral + contracts:

    - + @@ -1512,7 +1543,7 @@

    Technology stack

    - + @@ -1539,18 +1570,18 @@

    Technology stack

    - + - + - + @@ -1564,28 +1595,11 @@

    Technology stack

    - - - - - - - - - - - - - - - - - - - - - - + + @@ -1597,11 +1611,11 @@

    Technology stack

    LayerSelected technologyT4 choice or external contract
    Clusterk3s using standard Kubernetes APIsKubernetes 1.30+; k3s is the optional T4 Stack default
    Container runtime
    Product databasePostgreSQL managed by CloudNativePG in HA mode + PostgreSQL protocol; an existing service or an optional stack + profile managed by a dedicated PostgreSQL operator +
    Shared filesystem - Rook-managed CephFS exposed through the Ceph CSI driver as RWX - project storage - Administrator-selected, conformance-tested RWX StorageClass
    Object storageMinIO Community EditionOptional S3-compatible endpoint and credential reference
    Source workflow
    DashboardsGrafana OSS
    Telemetry collectionGrafana Alloy
    MetricsGrafana Mimir
    LogsGrafana Loki
    TracesGrafana Tempo
    InstrumentationOpenTelemetryObservability + Prometheus-format metrics and OpenTelemetry export to an optional + administrator-selected backend +
    Secrets

    - The self-hosted software stack can operate without software license fees, - subject to the applicable open-source and source-available licenses. - Tailscale's hosted control plane, GitHub plans, hardware, bandwidth, store - accounts, and operational support may incur costs. AGPL components must - remain license-compliant, especially if modified or redistributed. + Provider choices must satisfy their own licenses and remain independently + replaceable. The optional T4 Stack must not bundle a copyleft or + source-available dependency by default without an explicit distribution + and network-use compliance decision. Hosted services, hardware, bandwidth, + store accounts, and operational support may still incur costs.

    Deployment profiles

    @@ -1613,7 +1627,8 @@

    T4 Local

    T4 Local Service runs natively on the same macOS or Linux workstation as OMP and the project. Flutter connects over loopback by default. No Linux - VM, k3s, PostgreSQL, CephFS, MinIO, or Grafana service is required. + VM, Kubernetes cluster, PostgreSQL service, network filesystem, object + store, or telemetry backend is required.

    Benefits

      @@ -1656,23 +1671,24 @@

      Workstation Runner

      disconnects.
    • - No automatic takeover, local-to-CephFS synchronization, or portable - uncommitted state. + No automatic takeover, local-to-managed-filesystem synchronization, or + portable uncommitted state.

    Personal Hub

    - One always-on Linux computer or VM runs the full managed stack and managed + One always-on Linux computer or VM runs the T4 components and managed runtimes. It may sit on a desk, on a LAN, or remotely; desktop and mobile - clients connect through a stable Tailscale service. CephFS, PostgreSQL, - and telemetry use deliberate size-one compact topology. + clients connect through a stable Tailscale service. The optional T4 Stack + chart supplies a tested compact path, while every database, storage, and + telemetry dependency keeps an independent ownership boundary.

    Benefits

    • Work continues while client devices are off.
    • - Same k3s, protocol, operator, runtime image, and storage APIs as larger - deployments. + Same Kubernetes APIs, protocol, operator, runtime image, and storage + contracts as larger deployments.
    • Can expand into the intended HA topology.
    @@ -1687,9 +1703,9 @@

    Personal Hub

    Highly available Hub

    - At least three suitable Linux machines run k3s with embedded-etcd quorum, - distributed Hub replicas, PostgreSQL replicas, replicated CephFS, and - schedulable runtime capacity. + At least three suitable Linux failure domains run Kubernetes quorum, + distributed T4 replicas, a highly available PostgreSQL service, replicated + provider-managed RWX storage, and schedulable runtime capacity.

    Benefits

      @@ -1713,15 +1729,16 @@

      Highly available Hub

      Two nodes may add capacity but are not labeled HA because they cannot preserve quorum after an arbitrary partition. No managed session moves - across nodes until hard Ceph writer fencing and owner-epoch recovery pass - the failure matrix. + across nodes until hard storage-writer fencing and owner-epoch recovery + pass the failure matrix.

      -

      Advanced installation

      +

      Existing-cluster installation

      - An explicit advanced path may support an existing k3s cluster, dedicated - control/storage/worker roles, external PostgreSQL or object storage, - custom storage classes, and specialized runner pools. Unsupported - combinations are rejected rather than accepted optimistically. + The portable charts support an existing conformant Kubernetes cluster, + PostgreSQL service, RWX StorageClass, optional S3-compatible endpoint, + ingress, and telemetry backend. The optional T4 Stack is a convenience + distribution of the same T4 artifacts, not a second architecture and not + authority for the referenced providers.

      Guided managed installation

      @@ -1743,8 +1760,9 @@

      First Hub

      The user installs T4 Code, chooses a local or remote Linux Hub, identifies the machine through Tailscale, runs the signed T4 Node installer, exchanges a one-time enrollment code, passes hardware and - network preflight, installs the signed k3s release bundle, bootstraps - platform services, runs an end-to-end health check, and pairs T4 Code. + network preflight, installs the signed T4 Stack or portable charts, + binds platform services, runs an end-to-end health check, and pairs T4 + Code. First Hub 7 Install signed release - k3s and pinned OCI bundle + + T4 charts and pinned OCI bundle + 8 - Bootstrap services + Install or reference services - PostgreSQL, CephFS, MinIO, Grafana + PostgreSQL, RWX storage, optional S3 and telemetry @@ -1889,21 +1909,23 @@

      First Hub

      - The guided flow hides Kubernetes and storage administration without - bypassing verification. + The optional T4 Stack gives ordinary users a guided default without + widening the T4 operator's ownership boundary.

      - Users do not run kubectl, edit Helm values, manage database - credentials, or place replicas manually. + Stack-profile users do not need to run kubectl or place + replicas manually. Existing-cluster administrators use Helm values and + Secret references to bind T4 to services they already operate.

      Cluster expansion

      - The GUI exposes Make highly available or - Add node. T4 enrolls additional Linux machines, expands - embedded-etcd quorum, creates PostgreSQL and storage replicas, expands or - migrates MinIO safely, adds Hub replicas, rebalances workloads, and runs - failure checks before reporting HA. + The GUI may expose Make highly available or + Add node for a tested T4 Stack profile. T4 may enroll nodes, + expand Kubernetes quorum, and add T4 replicas, but provider-specific + database and storage expansion remains the responsibility of their own + operators. The installer coordinates health gates without assuming that + the T4 operator owns those topology changes.

      The client endpoint, project identifiers, session history, worktree paths, @@ -1919,9 +1941,9 @@

      Configuration UX

      External backup: Not configured Recommendation: Add two Linux nodes for high availability.

      - Ordinary users choose among versioned, tested profiles. Arbitrary - independent replica counts and service placements remain an advanced - operator concern. + Ordinary users choose among versioned, tested T4 Stack profiles. + Existing-cluster administrators choose provider topology through the + providers' own APIs, not through T4 custom resources.

      CI/CD and product progress

      @@ -1945,28 +1967,27 @@

      CI/CD and product progress

      session.recovered

      These events are persisted and shown in T4 Code. The GUI does not infer - authoritative progress by scraping terminal text or querying Grafana. + authoritative progress by scraping terminal text or querying an + observability backend.

      Observability

      -

      The optional managed self-hosted Grafana LGTM stack is:

      -
        -
      • Grafana for dashboards and alerts;
      • -
      • Grafana Alloy for collection;
      • -
      • Grafana Mimir for metrics;
      • -
      • Grafana Loki for logs;
      • -
      • Grafana Tempo for traces;
      • -
      • OpenTelemetry for T4 instrumentation;
      • -
      • MinIO as object storage for Mimir, Loki, and Tempo.
      • -

      - No Prometheus server is required. Alloy may scrape Prometheus-format - metrics exposed by Kubernetes and its components. + T4 exposes bounded Prometheus-format metrics, structured logs, health + endpoints, and OpenTelemetry signals. An administrator may connect + Grafana, Prometheus, an LGTM deployment, or another compatible backend. + The T4 operator does not install or reconcile those systems. +

      +

      + The portable chart may create narrowly scoped integration resources such + as a ServiceMonitor or alert rules only when their CRDs + already exist and the administrator enables them. The optional T4 Stack + may offer a separately owned observability profile; it is not required for + product correctness.

      - Personal Hub uses compact, bounded-retention deployments; HA Hub may use - distributed deployments. Observability failure cannot prevent session - execution or alter durable product state. T4 Local requires no Grafana - services and uses bounded native diagnostics. + Observability failure cannot prevent session execution or alter durable + product state. T4 Local requires no external telemetry service and uses + bounded native diagnostics.

      Performance and resource discipline @@ -1998,13 +2019,13 @@

      T4 Local preserves native filesystem and toolchain performance. Managed - benchmarks emphasize CephFS metadata-heavy work: Git status, worktree + benchmarks emphasize RWX metadata-heavy work: Git status, worktree creation, package trees, file watching, dependency installation, search, - large builds, and concurrent OMP sessions. Personal Hub uses one replica - of each required service, size-one Ceph pools, conservative database - buffers, bounded telemetry retention, trace sampling, node-local - disposable caches, and suspended idle runtime pods. It adds no service - mesh, Kafka, NATS, or per-runtime Tailnet sidecars without measured need. + large builds, and concurrent OMP sessions. Personal Hub uses compact T4 + replicas, conservative database buffers, bounded telemetry retention, + node-local disposable caches, and suspended idle runtime pods. It adds no + service mesh, Kafka, NATS, or per-runtime Tailnet sidecars without + measured need.

      Security boundaries

        @@ -2123,18 +2144,18 @@

        Gate 1: contract freeze

        owns outbox, owner epoch, and lease state.
      • - Stale-writer fencing for CephFS. A replacement cannot start until the - old runtime is observed terminated or its Ceph access is revoked and - eviction is confirmed. + Provider-neutral stale-writer fencing. A replacement cannot start until + the old runtime is observed terminated or its write access is revoked + and eviction is confirmed.
      • - Immutable MinIO object upload, checksum, PostgreSQL manifest commit, - orphan collection, and database-filesystem-object backup generation - ordering. + Immutable object upload, checksum, PostgreSQL manifest commit, orphan + collection, and database-filesystem-object backup generation ordering + for every enabled S3-compatible provider.
      • - Stable project, session, command, cursor, runner, CephFS subvolume, - worktree, Tailnet service, OCI image, and compatibility identifiers. + Stable project, session, command, cursor, runner, PVC, worktree, Tailnet + service, OCI image, and compatibility identifiers.
      • Versioned Local, Personal Hub, HA Hub, and Workstation Runner schemas @@ -2195,18 +2216,24 @@

        Deliberate parallel work

        Operator - Session reconciliation, pod identity, owner epochs, Ceph mounts, and - bounded replacement. + T4 custom-resource reconciliation, pod identity, owner epochs, PVC + claims and mounts, finalizers, and bounded replacement. + + + No duplicate runtime across restart and lease-loss cases; no + provider-specific CRs or cluster-wide dependency ownership. - No duplicate runtime across restart and lease-loss cases. - Platform + Platform integration - k3s, Tailscale ingress, Rook/CephFS, PostgreSQL, MinIO, and compact - Grafana release profile. + Portable Kubernetes and Helm APIs plus references to ingress, + PostgreSQL, RWX storage, optional S3, and optional telemetry. + + + One reproducible existing-cluster install and one optional + single-node T4 Stack install. - One reproducible single-node installation and reboot. Flutter @@ -2230,11 +2257,10 @@

        Deliberate parallel work

        - The installer does not stabilize in parallel with changing profile - semantics. k3s datastore, Ceph pool, PostgreSQL, and MinIO topology - migrations are never run concurrently during expansion. Those are serial - evidence gates because concurrent migrations make failure attribution and - rollback ambiguous. + T4 chart semantics and provider contracts stabilize before the optional + installer. Provider topology migrations remain owned and reported by their + respective operators. A T4 Stack expansion coordinates these independent + operations serially so failure attribution and rollback stay explicit.

        Gate 2A: T4 Local slice

        @@ -2248,23 +2274,24 @@

        Gate 2A: T4 Local slice

        Crash immediately after dispatch and confirm the command becomes indeterminate unless OMP proves durable acceptance. Close the workstation service and - confirm no other machine claims the local session. This gate has no k3s, - PostgreSQL, CephFS, MinIO, or Grafana dependency. + confirm no other machine claims the local session. This gate has no + Kubernetes, PostgreSQL, network-filesystem, object-store, or telemetry + dependency.

        Gate 2B: Personal Hub substrate

        - Install the first real Linux node through the intended path: k3s with - embedded etcd from the beginning, containerd, a stable Tailscale - Kubernetes service identity, one CloudNativePG instance, explicitly - enrolled size-one Rook/CephFS pools, MinIO, and a compact nonblocking - Grafana LGTM profile. Use one versioned release chart; Docker, Docker - Desktop, k3d, and a separate Docker Compose architecture are not required. + Prove both supported installation forms: portable T4 charts against + existing PostgreSQL, ingress, and RWX services; and the optional T4 Stack + on one real Linux node. Both forms deploy the same Hub, operator, and + runtime artifacts. Docker Desktop, k3d, and a separate Docker Compose + architecture are not required.

        - Before running OMP, mount one CephFS project from multiple disposable test - pods, reboot the node, remount it, and verify content and filesystem - semantics. PostgreSQL, MinIO, Ceph, and telemetry health must be - distinguishable; Grafana impairment must never block product commands. + Before running OMP, mount one conformant RWX project from multiple + disposable pods, reboot the node, remount it, and verify filesystem + semantics. Database, filesystem, optional object-store, and telemetry + health remain distinguishable; external-service impairment cannot grant + the T4 operator provider authority or corrupt product commands.

        Gate 3: Personal Hub physical-client slice @@ -2276,14 +2303,16 @@

      • T4 Hub persists a project, session, and prompt in PostgreSQL.
      • - T4 Operator starts a real T4 Session Runtime with the correct CephFS - project mounted. + T4 Operator starts a real T4 Session Runtime with the correct + administrator-selected RWX project mounted.
      • The runtime launches pinned official OMP and reconciles its durable entry watermark or fails closed where stock OMP cannot provide one.
      • -
      • OMP creates or uses its worktree and changes a file on CephFS.
      • +
      • + OMP creates or uses its worktree and changes a file on the RWX volume. +
      • Durable progress and transcript events reach the existing Flutter GUI projection. @@ -2314,10 +2343,10 @@

        Gate 4: recovery and restore

        valid.

        - Create a coordinated PostgreSQL, CephFS, and MinIO backup generation on an - independent destination, wipe the test installation, and restore it onto a - clean host. Backup creation without a successful bare-host restore is not - evidence. + Create a coordinated PostgreSQL, shared-filesystem, and optional + object-store backup generation on an independent destination, wipe the + test installation, and restore it onto a clean host. Backup creation + without a successful bare-host restore is not evidence.

        Workstation Runner proof

        @@ -2330,20 +2359,20 @@

        Workstation Runner proof

        Gate 5: guided installer

        Turn the exact managed automation used by Gates 2B through 4 into the - signed T4 Node bootstrap and the Flutter setup flow. Installation, health + signed T4 Node bootstrap and Flutter setup flow. Installation, health checks, support collection, update, failed-update stop, rollback, and recovery must be repeatable from a clean supported Linux image without - requiring the user to understand Kubernetes, Ceph, PostgreSQL, or MinIO. + making an ordinary stack-profile user administer Kubernetes or its + independently owned providers.

        Gate 6: HA expansion

        - Add two validated Linux nodes and perform topology changes serially: - expand embedded-etcd quorum, add Ceph OSDs and monitors, raise CephFS - pools from size one to host-distributed size three and wait for backfill, - add CloudNativePG replicas, migrate or expand MinIO through a proven path, - add Hub and operator replicas, then validate the stable Tailnet endpoint. - After every store reports healthy, remove one node and prove continued - operation before applying the HA label. + Add two validated Linux nodes and perform changes serially: expand + Kubernetes quorum, request provider-owned database and storage expansion, + add Hub and T4 operator replicas, then validate the stable Tailnet + endpoint. Every provider must report its own healthy state, and the T4 + operator must remain unable to mutate provider topology. Remove one node + and prove continued operation before applying the HA label.

        Product breadth and CI/CD

        @@ -2370,13 +2399,13 @@

        First Linux test node

        and reboot persistence.
      • - A separately approved raw SSD or NVMe device addressed by stable - /dev/disk/by-id, with no filesystem signatures or data to - preserve, for Ceph. + Storage capacity and any provider-specific devices required by the + selected optional T4 Stack profile, explicitly approved and containing + no data to preserve.
      • - Measured CPU, memory, root-disk, inode, and Ceph capacity headroom for - one OMP session plus the compact platform; final profile floors come + Measured CPU, memory, root-disk, inode, and provider capacity headroom + for one OMP session plus the compact platform; final profile floors come from this evidence rather than guesses.
      • @@ -2423,8 +2452,8 @@

        Workspace

        • Concurrent real OMP worktrees preserve repository integrity.
        • - Managed pod and node loss preserve work only after Ceph fencing and - recovery proof. + Managed pod and node loss preserve work only after the selected storage + provider passes fencing and recovery proof.
        • Local and runner files remain on their named machine and are never @@ -2453,8 +2482,8 @@

          Managed control plane

          history.
        • - MinIO or Grafana impairment degrades explicit features without - corrupting product state. + Optional object-store or telemetry impairment degrades only explicit + dependent features without corrupting product state.

        Client

        @@ -2518,11 +2547,17 @@

        Out of scope

      • Automatic takeover of Local or Workstation Runner sessions.
      • - Continuous bidirectional synchronization between local files and CephFS. + Continuous bidirectional synchronization between local files and the + managed shared filesystem. +
      • +
      • + Installing or lifecycle-managing database, filesystem, object-store, + ingress, or observability providers from the T4 operator.
      • -
      • Multiple ordinary-user managed storage-provider choices.
      • Active-active cross-cluster filesystem failover.
      • -
      • Making Grafana the user-facing product event database.
      • +
      • + Making an observability backend the user-facing product event database. +
      • Retaining the private Lycaon OMP bridge after official-adapter parity is proven. @@ -2549,8 +2584,8 @@

        Open decisions requiring proof

        GPU runners.
      • - Retention defaults for Mimir, Loki, Tempo, transcripts, artifacts, and - backups. + Retention defaults for product events, transcripts, artifacts, backups, + and optional external telemetry.