fix(openai): recover from stale pooled Responses WebSocket connections#6523
Open
longcw wants to merge 2 commits into
Open
fix(openai): recover from stale pooled Responses WebSocket connections#6523longcw wants to merge 2 commits into
longcw wants to merge 2 commits into
Conversation
A pooled Responses socket closed while idle isn't reflected in ws.closed until a read observes it, so it fails on the next send and burns a full outer LLM retry per stale socket. Add a ws heartbeat to keep idle connections warm and detect dead peers, and reconnect in place when a reused connection fails on send instead of raising a retryable error; a fresh connection failing to send is still raised.
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
chenghao-mou
approved these changes
Jul 23, 2026
chenghao-mou
left a comment
chenghao-mou
left a comment
Member
There was a problem hiding this comment.
lgtm, one small question about cancellation.
| ws = await self._pool.get(timeout=self._timeout) | ||
| reused = self._pool.last_connection_reused | ||
| try: | ||
| await ws.send_str(data) |
Member
There was a problem hiding this comment.
Should we remove the socket when cancelled as well?
except Exception in _acquire_and_send didn't catch CancelledError, so a cancellation during send_str left the acquired socket orphaned in the pool.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #6513.
Problem
The Responses WebSocket transport pools and reuses connections. A pooled socket closed while idle (by OpenAI or a NAT/proxy) is not reflected in
ws.closeduntil a read observes it, so it fails on the nextsend_str()— burning a full outer LLM retry (with backoff) per stale socket, adding seconds of latency to a voice turn and eventually failing once enough accumulate.Fix
When a reused connection fails on send, discard it and try the next instead of raising a retryable error. A freshly created connection failing to send is still raised, and a connection returns to the pool only on clean completion. Also sets a WebSocket heartbeat so idle sockets stay warm and dead peers are detected.
Alternatives considered
#6514 / #6515 add
validate_cb=lambda ws: not ws.closed, but that is ineffective — aiohttp keepsws.closedFalse on an idle drop, so the check hands a dead socket straight through (verified against aiohttp 3.14). Opened as a new PR since the mechanism differs: react on send vs. validate on acquire.