Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
24 changes: 24 additions & 0 deletions .changeset/zombie-sandbox-containers-fix.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
---
'@electric-ax/agents-runtime': patch
'@electric-ax/agents': patch
---

Fix Docker sandbox containers (`electric-sbx-*`) accumulating as zombies, and
stop creating containers for wakes that never use their sandbox:

- The boot sweep now reclaims RUNNING orphans whose owning process died
(owner-pid label + in-container adoption marker), instead of only exited
ephemeral leftovers — previously crash/quit leftovers were never cleaned up.
- Runtime shutdown flushes the debounced idle teardowns (stop persistent /
remove ephemeral) instead of letting the unref'd timers die with the
process, which leaked a running container on every quit.
- A failed post-start init no longer leaves a running, untracked container
behind, and a 409 from inside creation is no longer misread as a name
conflict (which "reattached" to a removed container).
- Sandbox creation is now lazy: the container is only created/started when a
wake actually uses its sandbox, so backlog bursts of trivial wakes (cron
ticks, bookkeeping) on runner reconnect no longer spin up containers.
Terminal reclaim and spawn-`inherit` still work for never-used sandboxes,
and concurrent container creations are capped to smooth real bursts.
- All sandbox containers carry `com.docker.compose.project=electric-sandboxes`
so Docker GUIs group them and they can be stopped/removed together.
11 changes: 9 additions & 2 deletions packages/agents-runtime/src/process-wake.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,7 @@ import { createSetupContext } from './setup-context'
import { createEntityLogPrefix, runtimeLog } from './log'
import { createRuntimeServerClient } from './runtime-server-client'
import { unrestrictedSandbox } from './sandbox/unrestricted'
import { ensureSandboxMaterialized } from './sandbox/lazy'
import { resolveSandboxIdentity } from './sandbox/identity'
import { appendPathToUrl } from './url'
import { manifestChildKey } from './manifest-helpers'
Expand Down Expand Up @@ -1302,7 +1303,7 @@ export async function processWake(
const requestedInherit =
opts?.sandbox === `inherit` ||
(typeof opts?.sandbox === `object` && opts.sandbox.inherit === true)
const sandbox = requestedInherit
const childSandbox = requestedInherit
? resolvedSandboxSelection
? {
profile: resolvedSandboxSelection.profile,
Expand All @@ -1316,14 +1317,20 @@ export async function processWake(
: opts?.sandbox === `inherit`
? undefined
: opts?.sandbox
// An inheriting child only ever ATTACHES by key — make sure the
// owner's (lazily-created) container/workspace actually exists before
// the child can wake, even if this wake never ran a tool itself.
if (requestedInherit && resolvedSandboxSelection && sandbox) {
await ensureSandboxMaterialized(sandbox)
}
return serverClient.spawnEntity({
type: childType,
id: childId,
args: spawnArgs,
parentUrl,
initialMessage: opts?.initialMessage,
tags: opts?.tags,
sandbox,
sandbox: childSandbox,
wake: wakeOpt,
})
},
Expand Down
2 changes: 2 additions & 0 deletions packages/agents-runtime/src/sandbox-docker.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,8 @@

export {
dockerSandbox,
reclaimDockerSandboxByKey,
shutdownAllDockerSandboxes,
sweepOrphanedDockerSandboxes,
__resetPersistentRegistryForTests,
} from './sandbox/docker'
Expand Down
2 changes: 2 additions & 0 deletions packages/agents-runtime/src/sandbox.ts
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ export type { RemoteProvider, RemoteSandboxOpts } from './sandbox/remote'
export type { RemoteSandboxClient } from './sandbox/remote/types'
export { isE2BAvailable } from './sandbox/remote/e2b'
export { chooseDefaultSandbox } from './sandbox/default'
export { ensureSandboxMaterialized, lazySandbox } from './sandbox/lazy'
export type { LazySandboxOpts } from './sandbox/lazy'
export { SandboxError } from './sandbox/types'
export type {
Sandbox,
Expand Down
Loading
Loading