SIGSEGV: context.dispose() races the un-awaited post-generation takeCheckpoint() on hybrid models (qwen35, Metal)

[KureijiSan]

Summary

After LlamaChatSession.prompt() resolves, node-llama-cpp launches a fire-and-forget context-state checkpoint (void this.sequence.takeCheckpoint(), LlamaChat.js, generateResponse's finally). If the caller then disposes the context — the documented, normal pattern — llama_free() runs on a libuv worker while the checkpoint worker is still serializing the sequence state, and the process segfaults with no catchable JS error.

For hybrid architectures (e.g. qwen35 — Qwen3.5, SSM + full-attention mix), LlamaContextSequence.needsCheckpoints is unconditionally true, so every successful generation ends with this un-awaited checkpoint, and every prompt() → dispose() cycle rolls the dice.

Environment

• node-llama-cpp 3.19.0 (latest at time of writing), bundled llama.cpp b9842, Metal build (libllama.metal.b9842.dylib)
• macOS Darwin 25.5.0, Apple Silicon (M3 Max, arm64)
• Node.js v22.15.0
• Model: Qwen3.5-2B-Q4_K_S.gguf — GGUF metadata: general.architecture = qwen35, ssm block present, full_attention_interval = 4

Reproduction

import { getLlama, LlamaChatSession } from "node-llama-cpp";

const llama = await getLlama();
const model = await llama.loadModel({ modelPath: "Qwen3.5-2B-Q4_K_S.gguf" });

for (let i = 0; i < 10; i++) {
  const context = await model.createContext({ contextSize: 8192 });
  const session = new LlamaChatSession({ contextSequence: context.getSequence() });
  await session.prompt("Tell me a short story.");   // resolves…
  await context.dispose();                          // …while the checkpoint worker is still running
  // -> intermittent SIGSEGV here (probability grows with context state size)
}

With a per-request create/dispose lifecycle, we hit 6 crashes in ~1 hour of interactive chat.

Crash evidence (6 macOS .ips reports, two signatures, same root cause)

Signature A (5/6 crashes) — thread pool worker, EXC_BAD_ACCESS (SIGSEGV), KERN_INVALID_ADDRESS at 0x250:

llama_context::synchronize()                (libllama.metal.b9842.dylib)
llama_state_seq_get_data_ext
AddonContextSequenceCheckpointInitWorker::Execute()   (llama-addon.node)
Napi::AsyncWorker::OnExecute

0x250 = null-pointer + member offset: llama_state_seq_get_data_ext(ctx, …) is called with ctx == nullptr (nulled by AddonContext::disposeMemory() on the concurrent unload worker); ctx->synchronize() reads this->sched (inlined ggml_backend_sched_synchronize path) at offset 0x250.

Signature B (1/6 crashes) — same worker, further along:

_platform_memmove
llama_context::state_seq_get_data(int, unsigned char*, unsigned long, unsigned int)
AddonContextSequenceCheckpointInitWorker::Execute()

KERN_INVALID_ADDRESS at 0x13bf70000 — the free won the race slightly later: the context was llama_free()d while the worker was copying KV/SSM state out of it.

At crash time the main thread is idle in kevent (no active evaluation): the generation has completed; only the checkpoint worker and the unload worker are running.

Root cause (code-level)

1. dist/evaluator/LlamaChat/LlamaChat.js — generateResponse ends with:

   finally {
       await generateResponseState.dispose();
       if (!hadError && this.sequence.needsCheckpoints)
           void this.sequence.takeCheckpoint();   // fire-and-forget
   }

   The checkpoint is intentionally not awaited, so prompt() resolves with native work still queued/in flight, and callers have no handle to await it.

2. dist/evaluator/LlamaContext/LlamaContext.js —

   get needsCheckpoints() {
       if (this.model.fileInsights.isHybrid || this.model.fileInsights.isRecurrent)
           return true;                       // qwen35 => always true
       else if (this.model.fileInsights.swaSize != null && !this._context._swaFullCache)
           return true;
       return false;
   }

   (GgufInsights.isHybrid lists qwen35/qwen35moe.) Note swaFullCache: true does not disable checkpoints for hybrid models — there is no public API to opt out.

3. llama/addon/AddonContext.cpp —

   • AddonContextSequenceCheckpointInitWorker::Execute() calls llama_memory_seq_pos_min/max, llama_state_seq_get_size_ext and llama_state_seq_get_data_ext directly on context->ctx (raw pointer), holding only a Napi Ref() on the JS wrapper — nothing that keeps ctx itself alive.
   • AddonContext::Dispose queues AddonContextUnloadContextWorker, which calls disposeMemory() → sets ctx = nullptr then llama_free(old). disposeMutex protects only the flags; no synchronization with in-flight checkpoint workers. Both workers run concurrently on the libuv pool.

4. JS-level locking doesn't cover it either: takeCheckpoint() runs under withLock([context, "context"]), but LlamaContext.dispose() (_disposeAggregator) never acquires that lock before calling the native this._ctx.dispose().

Suggested fixes (any one breaks the race; a+b is defense in depth)

a. Track and await pending checkpoints on dispose: keep the promise of the post-generation takeCheckpoint() on the sequence/context and await it in LlamaContext.dispose() (or simply await it in generateResponse's finally — it's already behind needsCheckpoints).
b. Native refcount: have AddonContext count in-flight workers that use ctx (checkpoint init, state get/set) and make disposeMemory() wait for the count to reach zero before llama_free() — the same pattern used for evaluation workers, if one exists.
c. Make LlamaContext.dispose() acquire the ["context"] lock that takeCheckpoint() holds.

Workarounds for users (until fixed)

• Don't dispose the context right after prompt() — reuse a long-lived context per session (this removes the race entirely; also matches the intended LlamaChatSession lifecycle).
• For hybrid models there is currently no supported way to disable checkpoints; a public checkpoints: false context/sequence option would be a useful mitigation knob.

Related upstream context (llama.cpp, not this bug)

Hybrid/recurrent (GDN/DeltaNet/SSM) models can't do plain KV prefix caching — see ggml-org/llama.cpp#19794, #21831, #22384 (checkpoint restore bugs for DeltaNet/Mamba). This report is about a node-llama-cpp lifecycle race, orthogonal to those.

[giladgd]

Thanks for doing research on this!
I managed to consistently replicate this issue, and opened PR #632 to fix it.
I've added a test to ensure this won't regress in the future.

I'll release a new version with the fix soon.

[github-actions]

🎉 This issue has been resolved in version 3.19.1 🎉

The release is available on:

• v3.19.1
• npm package (@latest dist-tag)
• GitHub release
• v3.19.1

Your semantic-release bot 📦🚀