Reduce shutdown latency by parallelizing session destruction by AkCodes23 · Pull Request #398 · github/copilot-sdk

AkCodes23 · 2026-02-07T05:55:24Z

This PR optimizes the CopilotClient.stop() method by destroying all active sessions concurrently instead of sequentially.

Previously, sessions were torn down one at a time, causing shutdown latency to scale linearly with the number of sessions—further amplified by per-session retry logic and exponential backoff. This change removes that bottleneck by running all session destruction attempts in parallel.

Behavior Changes

All active sessions are destroyed concurrently using Promise.all

Per-session retry and exponential backoff logic is preserved

Errors from individual session destructions are collected and returned

this.sessions is cleared only after all destruction attempts complete

No external behavior or public API semantics are changed.

Performance Impact

Mock benchmark (100 ms teardown delay per session):

Sessions Before After
10 ~1000 ms ~100 ms

Shutdown time now scales with the slowest session, rather than the total number of sessions.

Testing

Verified with existing unit tests

Verified with client end-to-end tests

Parallelization does not alter observable cleanup behavior

Notes

This change introduces unbounded parallelism. If session counts grow significantly in the future, a concurrency limit may be considered, but current usage patterns make full parallelization the most efficient approach.

Optimized the `stop()` method in `CopilotClient` to destroy all active sessions in parallel using `Promise.all`. Previously, sessions were destroyed sequentially, leading to a linear increase in shutdown time as the number of sessions grew. With this change, the total time for session destruction is approximately the duration of the slowest individual destruction. Measurements: - 10 sessions with 100ms destruction delay: - Before: ~1000ms - After: ~100ms - Performance gain: ~10x for 10 sessions. The individual retry logic and exponential backoff for each session are preserved. All failures are still collected and reported. Co-authored-by: AkCodes23 <135016848+AkCodes23@users.noreply.github.com>

…18781846015088 ⚡ Bolt: Parallelize session destruction in Node.js SDK

Copilot

Pull request overview

Optimizes Node.js SDK client shutdown by parallelizing CopilotClient.stop() session teardown so shutdown time scales with the slowest session destruction rather than the total number of sessions.

Changes:

Parallelize session destruction with per-session retry/backoff preserved using Promise.all.
Aggregate per-session destruction failures into the returned Error[] after all attempts complete.
Add an internal note documenting the rationale and cross-SDK direction for parallel session cleanup.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

File	Description
nodejs/src/client.ts	Updates `stop()` to destroy all active sessions concurrently and collect errors after completion.
.jules/bolt.md	Adds internal documentation/learning note advocating parallel session cleanup across SDKs.

Copilot · 2026-02-07T05:58:36Z

nodejs/src/client.ts

+        // Destroy all active sessions in parallel with retry logic
+        const sessionPromises = Array.from(this.sessions.values()).map(async (session) => {
            const sessionId = session.sessionId;
            let lastError: Error | null = null;


Using Array.from(this.sessions.values()) snapshots the sessions at the start of stop(). If a session is created/resumed while stop() is running, it won’t be destroyed but will still be removed by this.sessions.clear(), which can skip session.destroy() cleanup (and differs from the previous live Map iteration behavior). Consider blocking session creation while stopping (e.g., a stopping flag/state checked in createSession/resumeSession) or draining the map as you schedule destructions so sessions added during shutdown are either rejected or deterministically cleaned up.

@copilot open a new pull request to apply changes based on this feedback

AkCodes23 · 2026-02-07T06:13:20Z

@copilot open a new pull request to apply changes based on the comments in this thread

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 1 comment.

Copilot · 2026-02-10T18:28:05Z

nodejs/src/client.ts

+        // Destroy all active sessions in parallel with retry logic
+        const sessionPromises = Array.from(this.sessions.values()).map(async (session) => {
            const sessionId = session.sessionId;
            let lastError: Error | null = null;



There’s no direct unit test coverage for the updated shutdown behavior (parallel destruction + error aggregation). Adding a focused test for stop() (e.g., multiple mock sessions with controlled destroy() delays/failures) would help ensure it still waits for all destructions, returns all per-session errors, and only clears this.sessions after completion.

AkCodes23 · 2026-02-11T06:31:41Z

@copilot open a new pull request to apply changes based on the comments in this thread

Copilot

Pull request overview

Copilot reviewed 2 out of 2 changed files in this pull request and generated 3 comments.

Copilot · 2026-02-19T02:18:20Z

nodejs/src/client.ts

@@ -321,12 +321,18 @@ export class CopilotClient {
            }

            if (lastError) {
-                errors.push(
-                    new Error(
-                        `Failed to destroy session ${sessionId} after 3 attempts: ${lastError.message}`
-                    )
+                return new Error(
+                    `Failed to destroy session ${sessionId} after 3 attempts: ${lastError.message}`
                );
            }
+            return null;
+        });
+
+        const results = await Promise.all(sessionPromises);
+        for (const result of results) {
+            if (result instanceof Error) {
+                errors.push(result);
+            }
        }
        this.sessions.clear();


stop() snapshots this.sessions with Array.from(this.sessions.values()) and only calls this.sessions.clear() after all destroys finish. If a session is created/resumed while stop() is running, it won't be included in sessionPromises (so session.destroy() won't run) but it will still be removed by this.sessions.clear(), skipping per-session cleanup and changing behavior vs the previous live Map iteration. Consider preventing session creation/resume during shutdown (e.g., set a "stopping" state checked by createSession/resumeSession) or draining/deleting sessions from the map as you schedule destruction so any sessions added mid-stop are either rejected or deterministically cleaned up.

Copilot · 2026-02-19T02:18:21Z

.jules/bolt.md

+## 2025-05-15 - [Sequential session destruction in SDKs]
+**Learning:** All Copilot SDKs (Node.js, Python, Go, .NET) were initially implementing session destruction sequentially during client shutdown. This leads to a linear increase in shutdown time as the number of active sessions grows, especially when individual destructions involve retries and backoff.
+**Action:** Parallelize session cleanup using language-specific concurrency primitives (e.g., `Promise.all` in Node.js, `asyncio.gather` in Python, `Task.WhenAll` in .NET, or WaitGroups/Channels in Go) to ensure shutdown time remains constant and minimal.


There doesn’t appear to be any repo integration or references to the new .jules/ directory (no mentions in docs/CI/config). If this is intended as internal notes, it may be better to keep it out of the shipped repo (e.g., add to .gitignore) or move the content into an existing documentation location (like docs/ or CONTRIBUTING.md) where contributors will find it.

Copilot · 2026-02-19T02:18:21Z

nodejs/src/client.ts

@@ -321,12 +321,18 @@ export class CopilotClient {
            }

            if (lastError) {
-                errors.push(
-                    new Error(
-                        `Failed to destroy session ${sessionId} after 3 attempts: ${lastError.message}`
-                    )
+                return new Error(
+                    `Failed to destroy session ${sessionId} after 3 attempts: ${lastError.message}`
                );
            }
+            return null;
+        });
+
+        const results = await Promise.all(sessionPromises);
+        for (const result of results) {
+            if (result instanceof Error) {
+                errors.push(result);
+            }
        }
        this.sessions.clear();


The new parallel shutdown behavior introduces a race where sessions could be created/resumed during stop() and then cleared without being destroyed. Please add test coverage that exercises shutdown with multiple sessions and validates the shutdown behavior under concurrent session creation/resume (or validates that creation/resume is rejected once shutdown begins).

SteveSandersonMS · 2026-02-23T11:41:08Z

Thanks for the contribution, @AkCodes23. Hope you don't mind but I'm going to close it because this is not a direction we're planning to go in. The SDK isn't oriented around having huge numbers of concurrent sessions within a single app instance, and it really would have to be a huge number before parallelizing session shutdown would make any useful difference. And it also introduces new potential race issues that we can avoid by not doing that.

google-labs-jules bot and others added 3 commits February 6, 2026 17:52

Merge pull request #1 from AkCodes23/bolt-parallel-stop-nodejs-138360…

e436065

…18781846015088 ⚡ Bolt: Parallelize session destruction in Node.js SDK

Merge branch 'github:main' into main

52c48f1

AkCodes23 requested a review from a team as a code owner February 7, 2026 05:55

Copilot AI review requested due to automatic review settings February 7, 2026 05:55

Copilot started reviewing on behalf of AkCodes23 February 7, 2026 05:55 View session

Copilot AI reviewed Feb 7, 2026

View reviewed changes

AkCodes23 requested a review from Copilot February 10, 2026 18:24

Copilot started reviewing on behalf of AkCodes23 February 10, 2026 18:25 View session

Copilot AI reviewed Feb 10, 2026

View reviewed changes

AkCodes23 requested a review from Copilot February 19, 2026 02:14

Copilot started reviewing on behalf of AkCodes23 February 19, 2026 02:15 View session

Copilot AI reviewed Feb 19, 2026

View reviewed changes

SteveSandersonMS closed this Feb 23, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Comments

Reduce shutdown latency by parallelizing session destruction#398

Reduce shutdown latency by parallelizing session destruction#398
AkCodes23 wants to merge 3 commits intogithub:mainfrom
AkCodes23:main

AkCodes23 commented Feb 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 7, 2026

Uh oh!

AkCodes23 Feb 7, 2026

Uh oh!

AkCodes23 commented Feb 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 10, 2026

Uh oh!

AkCodes23 commented Feb 11, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Feb 19, 2026

Uh oh!

Copilot AI Feb 19, 2026

Uh oh!

Copilot AI Feb 19, 2026

Uh oh!

SteveSandersonMS commented Feb 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Comments

Conversation

AkCodes23 commented Feb 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 7, 2026

Choose a reason for hiding this comment

Uh oh!

AkCodes23 Feb 7, 2026

Choose a reason for hiding this comment

Uh oh!

AkCodes23 commented Feb 7, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

AkCodes23 commented Feb 11, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

SteveSandersonMS commented Feb 23, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants