Skip to content

fix(core): unbind state-saver on agent close to prevent OOM leak#2322

Open
Buktal wants to merge 1 commit into
agentscope-ai:mainfrom
Buktal:fix/graceful-shutdown-state-savers-leak
Open

fix(core): unbind state-saver on agent close to prevent OOM leak#2322
Buktal wants to merge 1 commit into
agentscope-ai:mainfrom
Buktal:fix/graceful-shutdown-state-savers-leak

Conversation

@Buktal

@Buktal Buktal commented Jul 20, 2026

Copy link
Copy Markdown
Contributor

AgentScope-Java Version

2.0.0

Description

Fixes #2321.

Background

GracefulShutdownManager.stateSavers is a process-wide singleton ConcurrentHashMap that was effectively write-only: bindStateSaver() puts an entry keyed by agentId, but there was no removal path anywhere in production code (only stateSavers.clear() inside resetForTesting()). Since agentId = UUID.randomUUID() is a fresh key per instance, and the registered saver lambda captured the enclosing agent this (via the instance field stateStore), every agent built with a stateStore permanently pinned its entire object graph (model/HttpClient, toolkit, stateCache conversation history, middlewares) from the GracefulShutdownManager.INSTANCE GC root - even after agent.close() was correctly called.

In "create a fresh agent per call" deployments (e.g. wiring a new agent per request against Spring @RefreshScope / Nacos dynamic config), this produced monotonic heap growth ending in OutOfMemoryError (observed in production: ~2.74 GB retained by the singleton, 373 leaked HttpClient SelectorManager threads, recurring OOM within days).

The sibling map activeRequestsById already has a symmetric registerRequest / unregisterRequest lifecycle (the latter invoked from doFinally); stateSavers was missing that unregister half.

Changes

  1. GracefulShutdownManager.unbindStateSaver(Agent) - new public method, symmetric with bindStateSaver, mirroring the registerRequest/unregisterRequest pair. No-op on null / unregistered agent.

  2. ReActAgent.close() - now calls shutdownManager.unbindStateSaver(this) so the entry's lifetime is bound to the agent instance. HarnessAgent.close() already delegates to ReActAgent.close() via delegate.close(), so wrapped agents are covered automatically. No try/finally is needed since unbindStateSaver cannot throw.

  3. ReActAgent constructor - the saver lambda now captures a final AgentStateStore stateSaverStore local instead of the enclosing this. Defense in depth: even if an entry is somehow left behind (e.g. a caller forgets close()), it no longer pins the entire agent object graph - only a single store reference. (Item 1 is the root fix; item 3 alone is insufficient since the entry would still accumulate.)

Tests

Added 3 unit tests in GracefulShutdownTest:

  • unbindStateSaver removes a previously bound saver - end-to-end: bind -> registerRequest -> saveOnInterruptObserved invokes the saver; unbind -> registerRequest -> saveOnInterruptObserved is a no-op (verifies the saver is actually removed from the map, since ActiveRequestContext.saveState short-circuits on a null saver).
  • unbindStateSaver with null agent is no-op
  • unbindStateSaver for an unregistered agent is no-op

How to test

mvn -pl agentscope-core test

Result: Tests run: 2218, Failures: 0, Errors: 0, Skipped: 8 - all green, including the 3 new tests.

Note: agentscope-harness shows 7 JUnit Failed to close extension context errors on this Windows machine, but these are a pre-existing JUnit @TempDir cleanup issue on Windows (java.nio.file.DirectoryNotEmptyException in WindowsFileSystemProvider.implDelete) - verified by stashing this PR's changes and reproducing the same failures on clean main. They are unrelated to this change and do not occur on Linux CI.

Checklist

  • Code has been formatted with mvn spotless:apply
  • All tests are passing (mvn test) - core suite green (2218 tests); harness failures are pre-existing Windows @TempDir issues (see note above)
  • Javadoc comments are complete and follow project conventions
  • Related documentation has been updated (N/A - internal lifecycle fix; the new method carries full Javadoc)
  • Code is ready for review

GracefulShutdownManager.stateSavers was write-only (bindStateSaver puts,
no removal path), so every agent built with a stateStore pinned its
entire object graph forever via the manager's process-wide singleton.
With agentId = UUID per instance and a saver lambda capturing enclosing
this, short-lived agents created per call caused monotonic heap growth
ending in OOM.

- Add GracefulShutdownManager.unbindStateSaver(Agent), symmetric with
  bindStateSaver, mirroring registerRequest/unregisterRequest.
- Call it from ReActAgent.close() (HarnessAgent delegates here via
  delegate.close()).
- Capture the store in a final local so the saver lambda does not
  capture enclosing this, shrinking any leaked entry.

Closes agentscope-ai#2321
@codecov

codecov Bot commented Jul 20, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 71.42857% with 2 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
...e/src/main/java/io/agentscope/core/ReActAgent.java 33.33% 2 Missing ⚠️

📢 Thoughts on this report? Let us know!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: GracefulShutdownManager.stateSavers never unregisters agents, causing OOM when an agent is created per call

1 participant