feat(sandbox): sandbox controller in Go, studio talks to it over HTTP

[pedrofrxncx]

Implements packages/sandbox/sandbox-controller-spec.md: sandbox provisioning
moves out of the Studio API process into a separate Go service, so Studio's
ServiceAccount stops holding claim CRUD, pods get/list/watch,
pods/portforward, HTTPRoute CRUD and Service patch in agent-sandbox-system.

What's here

packages/sandbox/controller-go/ — its own Go module, net/http +
encoding/json, no framework. Sibling to daemon-go/, same shape.

package	what
protocol/	the wire contract, mirrored by provider/remote/protocol.ts
runtime/	Runtime interface, registry, probing, placement
agentsandbox/	the K8s runtime, on the upstream generated clientsets
store/	sandbox_runner_state over pgx, incl. the advisory lock

Studio keeps implementing SandboxProvider: RemoteSandboxProvider
(STUDIO_SANDBOX_PROVIDER=remote) speaks HTTP to the controller. No caller in
apps/api changed.

Control plane only

The controller answers where a sandbox's daemon is and what token opens
it; Studio's existing proxyDaemonRequest dials the pod. Nothing is relayed,
so streaming dispatch bodies, preview SSE and websocket upgrades are the same
traffic they were before the split, and the controller never reimplements any
of it.

With a preview gateway that address is in-cluster Service DNS. Without one it
is a controller-held port-forward returning http://127.0.0.1:<port> — correct
exactly when controller and Studio share a host, which is the local-dev case.

One definition of the daemon's boot contract

daemon-go/pkg/protocol is the daemon's first exported package;
internal/config now aliases those types instead of owning them, so a shape
change is a compile error on both ends. That is the concrete payoff of both
services being Go, and it retires the hand-written duplicate in
server/daemon-client.ts + shared/build-config-payload.ts for the config
path.

Deviation from the spec

The clone-credential callback names {connectionId, cloneUrl}, not a handle,
and Studio verifies the pair against a live sandbox row or a configured warm
pool before minting. A handle could not be threaded through
withFreshCloneUrl's five call sites — two are warm-pool paths with no handle
at all. The verification closes the same oracle the spec was guarding against:
you can only refresh credentials for repos a sandbox already has, which is what
the caller could read off the persisted clone URL anyway.

Testing

packages/sandbox/controller-e2e/ is the black-box suite: spawn the binary,
drive it over HTTP through the same client Studio uses, assert only on
responses. 13/13 against a k3s cluster running the agent-sandbox operator —
real sandbox provisioned through the contract, lifecycle streamed to ready,
daemon reached directly at the returned address (and rejecting a wrong token),
TTL patched both directions, deleted with a real drain.

CONTROLLER_E2E_URL=http://127.0.0.1:8787 bun test packages/sandbox/controller-e2e/

Skipped when the env var is unset, so bun test stays green without a cluster.
Placement gets ordinary Go table tests (runtime/runtime_test.go).

The Service-DNS path was verified separately, because a controller running
outside the cluster cannot resolve the name it correctly emits: with
previewUrlPattern set the controller returned
http://<sandbox>.agent-sandbox-system.svc.cluster.local:9000, and a fetch
from inside the Studio pod got 200 off the daemon's /health. That is the
path the spec called an unproven future hardening pass.

That exercise found a real bug. ensureServicePort was gated on the
preview gateway, but a ports-less Service routes nothing at all — not through
Envoy and not through kube-proxy — so Service DNS to the daemon needs the port
whether or not a gateway exists. The Go runtime gates it on previewUrlPattern
alone. The TypeScript runner could conflate the two because it only ever
reached the daemon by port-forward.

bun run check, bun run lint (0 errors), bun test packages/sandbox/,
go vet ./... and go test ./... in both Go modules all pass.

Not in this PR

• Rollout step 1 (kill Studio's production port-forward) is moot rather
  than done: the controller reaches the daemon over Service DNS itself, and
  Studio under remote has no port-forward to kill. Hardening runner.ts too
  would harden an implementation step 5 deletes.
• Tenant warm pools. Warm-pool mode — the sentinel bearer and its
  rotation to a per-claim token — is ported and exercised; per-org pools are
  not, so STUDIO_SANDBOX_TENANT_POOLS is ignored by the Go runtime today.
• The docker runtime. The registry, probing, /runtimes and placement
  all landed with agent-sandbox as the only member, so adding it is one
  implementation behind an interface that already has tests.
• Helm chart + published image, and the deletion of Studio's
  sandbox-rbac.yaml Role. That deletion is rollout step 5 and must land in
  the same PR that flips the flag, or this has made things worse (two
  implementations) rather than better.

Default stays agent-sandbox in-process; nothing in production changes until
STUDIO_SANDBOX_PROVIDER=remote is set, and that flip is a drained cutover,
not a rolling deploy (see the spec's State section).

---

Summary by cubic

Moves sandbox provisioning out of Studio into a standalone Go controller and adds a remote SandboxProvider that talks to it over HTTP, removing Studio’s Kubernetes/CRD and port‑forward privileges. Behavior change: Studio no longer provisions directly (old); it asks the controller for the daemon address and token and talks to the daemon itself (new). Side effects: both controller and Studio callback fail closed unless mTLS or a bearer token is configured; preview resurrection is restored.

• New packages/sandbox/controller-go: HTTP server with agent-sandbox runtime (placement and capacity probe, lifecycle phases stream, TTL renew, termination reason); fixes Service‑DNS port routing; port‑forward fallback for local dev now evicts dead forwards and drops stale entries on reprovision; Postgres store for sandbox_runner_state with WithLock queries running on the transaction to avoid pool deadlocks; CI vet/test/build job added; black‑box e2e in packages/sandbox/controller-e2e (opt‑in via CONTROLLER_E2E_URL), green on k3s.

• One boot contract: daemon config/health types exported at packages/sandbox/daemon-go/pkg/protocol, mirrored in packages/sandbox/server/provider/remote/protocol.ts; a Go↔TS contract test now covers envelopes and payload types.

• RemoteSandboxProvider (packages/sandbox/server/provider/remote): ensure/delete/status/capacity/lifecycle phases; preserves direct daemon proxying; adds opt‑in resurrection via GET /sandboxes/{handle}?resurrect=1; guards 401 retries with canRetryBody.

• API mounts controller routes at apps/api/src/api/routes/sandbox-controller.ts for clone‑credential re‑mint using timingSafeEqual; both controller and callback now refuse requests unless mTLS or a bearer is configured (dev opt‑out SANDBOX_CONTROLLER_INSECURE=1).

• Store reuse and types: KyselySandboxProviderStateStore moved to @decocms/sandbox/provider/kysely-state-store with a guarded DB shape; adds "remote" to SandboxProviderKind across settings, tools, and SDK; GitHub webhook tenant‑pool refresh behind a type predicate; handle slugification avoids ReDoS by replacing trailing‑quantifier regexes with a linear scan; connection create test stubs MCP tool fetch to avoid network timeouts.

• Rollout

  • Default stays unchanged until STUDIO_SANDBOX_PROVIDER=remote is set.
  • To enable: deploy the controller with Postgres access and Kubernetes RBAC; configure mTLS or a bearer and Studio’s controller URL/token.
  • Side effects: with a preview gateway, Studio reaches the daemon via Service DNS; in local dev, the controller holds the port‑forward and returns http://127.0.0.1:<port>.

^{Written for commit 08cae04. Summary will update on new commits.}