AFT bridge memory bloat: callgraph warm build has no size/memory budget — RSS reaches 3–4 GiB on large directories, builds never complete and restart from zero after every respawn

[ICEY1W32]

Short description

After spawn, every bridge instance unconditionally schedules a callgraph warm build during configure, and that build has no file-count or memory budget: on a non-git directory of roughly 24k files (including nested repositories and dependency/cache trees), the build's working set drives single-instance RSS to 3.0–3.9 GiB, resident indefinitely. On small-RAM hosts the instance is killed by the system OOM daemon before the build can finish — and the respawned instance rebuilds from zero, creating a grow → killed → respawn → grow-from-zero death loop. Stats show the warm build was scheduled ~486 times for this root and completed 0 times; the store directory contains only never-promoted .sqlite.tmp intermediates. The bloat is independent of whether the agent performs any code operations: a purely idle session pays the full cost.

What happened?

Environment

Item	Value
AFT binary	v0.51.3 (~/.cache/aft/bin/v0.51.3/aft)
Plugin	@cortexkit/aft-opencode@latest
OpenCode	1.18.19
OS	Debian 13 (Trixie)
Config	Stock defaults (~/.config/cortexkit/aft.jsonc contains only $schema)

Steps to Reproduce

1. Enable the @cortexkit/aft-opencode plugin with a large non-git project root (tens of thousands of files)
2. Just use the session normally — no code edits needed; the warm build is scheduled automatically during configure
3. Watch any aft instance's RSS (e.g. ps -o rss,comm -C aft) alongside ~/.local/share/cortexkit/aft/logs/aft-plugin.log

Actual Behavior

Instance RSS grows from tens of MiB at spawn past 3 GiB and stays resident; on small-RAM hosts this enters a grow → killed-by-OOM-daemon → respawned-seconds-later → grow-from-zero loop roughly once per minute.

Evidence A: direct measurement of the bloat (system journal aligned second-by-second with the plugin log)

The system daemon log records the measured RSS at kill time:

2026-08-21T02:25:48+00:00 oom-daemon: sending SIGTERM to process … "aft": oom_score 759, VmRSS 3287 MiB
2026-08-21T02:26:52+00:00 oom-daemon: sending SIGTERM to process … "aft": oom_score 770, VmRSS 3662 MiB
2026-08-21T02:27:48+00:00 oom-daemon: sending SIGTERM to process … "aft": oom_score 758, VmRSS 3237 MiB

Plugin log at the same moments:

[02:25:49.055Z] INFO  [aft-plugin] Process exited: code=143, signal=null
[02:25:49.055Z] ERROR [aft-plugin] Binary crashed (restarts: 3).
[02:25:53.707Z] INFO  [aft-plugin] Spawning binary: …   ← respawned 5 s later
[02:26:52.919Z] INFO  [aft-plugin] Process exited: code=143, signal=null
[02:26:53.028Z] INFO  [aft-plugin] Spawning binary: …   ← respawned 0.1 s later
[02:27:48.397Z] INFO  [aft-plugin] Process exited: code=143, signal=null

Evidence B: the Sisyphean loop — builds never complete and restart from zero

$ grep -c "warm build scheduled" aft-plugin.log    # scheduled (at time of writing)
486
$ grep -cE "warm.*complete|cold_build.*files=" aft-plugin.log   # completed: just 1, and for a different project root
1

The callgraph store directory for this root contains only .sqlite.tmp intermediates from successive PIDs (221 MB) — a final database is never published — plus stale writer.lease locks left by dead processes. Every respawned instance repeats the full parse from scratch.

Corroborating: before being killed, /proc/<pid>/status showed VmData 4.28 GB vs RSS 676 MB, and pmap revealed a dozen-plus fully-resident anonymous arenas of 64–130 MB — allocator retention pins the peak into RSS.

Evidence C: ruled out

• Embedding model never loaded: semantic={files:0,chunks:0} throughout; the model cache directory does not exist; the backend is lazily initialized — fastembed is not the source of the bloat
• Watcher bookkeeping negligible: ingested:0
• No segfaults (every exit is an external SIGTERM)

Analysis

1. Root cause: an unconditionally-scheduled configure-time warm build with no budget. The schema only has semantic.max_files, guarding the default-off semantic path; the callgraph path has no max_files or memory cap — walk and parse scale are determined entirely by directory contents.
2. Large non-git roots are especially expensive. The walker aggregates .gitignore files across the full depth (1,609 patterns observed) and parses vendored trees at source level; nested repos and dependency caches all count.
3. Kills × rebuild-from-zero = the cost is paid infinitely often. Build progress lives only in the process heap and an unpromoted tmp database; death resets everything, so even idle sessions keep paying full price.
4. Allocator retention amplifies residency. glibc multi-arena behavior (MALLOC_ARENA_MAX unset) plus runtime thread-pool reservations keep the allocated peak pinned in RSS instead of returning it.

Impact

Even a session performing zero code operations balloons each bridge instance to 3–4 GiB and enters minute-scale kill–respawn loops: repeated re-indexing burns CPU, in-flight notifications can be lost (the plugin injects "outcome UNKNOWN" warnings), the plugin is effectively unusable on small-RAM hosts, and pressure worsens linearly with concurrent session count.

Suggested Fix

1. Budget the callgraph index: provide max_files / memory-cap knobs (mirroring the existing semantic.max_files), checked before the walk; degrade to lazy loading with an explicit notice when exceeded
2. Gate the warm build: do not schedule it unconditionally during configure; lazy-build on first callgraph/inspect query, or skip very large / non-source roots with a notice
3. Cross-instance build handoff: use the existing writer.lease so latecomers take over an in-progress tmp database — or back off and serve read-only stale data while a fresh lease is held — ending the rebuild-from-zero loop
4. Isolate the build: run the warm build in a short-lived child process (the kernel reclaims everything on exit), or trim/discard parse arenas after publishing
5. Built-in ignore tables for non-git roots: skip node_modules/.venv/target/go-build etc. even without a .gitignore
6. Report external termination accurately: detect OOM-daemon kills and report a precise cause with backoff instead of a generic Binary crashed; reset the counter properly after giving up

Diagnostics

Plugin version

No response

AFT binary version

No response

Platform

No response

Log output (optional)

[alfonso-aft]

Exceptional report — the journal-aligned kill timeline, the scheduled-vs-completed counts, and the arena forensics name the full defect stack, and we confirm all four layers: the callgraph warm build has no budget of any kind, build progress is not durable (death resets to zero), nothing learns from repeated build deaths (486 schedules is the proof), and allocator retention pins the peak into RSS.

Direction we're taking, in design review now:

1. Durable build progress — the structural killer of the grow→killed→restart-from-zero loop. A build that commits progress incrementally survives its process; the respawned instance resumes instead of re-parsing 24k files. We're prioritizing this over file-count caps: a budget knob would make the loop less likely, but resumable builds make it structurally impossible to pay the full cost twice.
2. A general build-death breaker — any per-root background build whose process dies repeatedly stops being rescheduled (durable marker, honest degraded status naming the reason) instead of retrying forever. Applies to callgraph and search index alike.
3. Root-type policy — a large non-git directory (nested repos, dependency caches) is a poor candidate for an eager code-graph build; for such roots the callgraph moves to on-demand while text/semantic search stay available. Code-specific indexes shouldn't tax non-code directories.
4. Allocator retention (your MALLOC_ARENA_MAX observation is correct) is tracked separately — it's tied to a broader single-instance memory-target effort rather than a per-build patch.

Crash hygiene from your evidence lands with the same wave: orphaned .sqlite.tmp intermediates from dead PIDs and stale writer leases are exactly the debris the loop mints.

We'll update here as the pieces land.