fix(inference): treat a dropped TTS gateway session as a failed attempt - #2144
Open
toubatbrian wants to merge 1 commit into
Open
fix(inference): treat a dropped TTS gateway session as a failed attempt#2144toubatbrian wants to merge 1 commit into
toubatbrian wants to merge 1 commit into
Conversation
🦋 Changeset detectedLatest commit: ef4db13 The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
toubatbrian
force-pushed
the
brian/fix-tts-pool-session-leak
branch
from
July 28, 2026 04:03
0ab7edb to
7577b26
Compare
toubatbrian
changed the title
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
This comment was marked as resolved.
This comment was marked as resolved.
Sorry, something went wrong.
This was referenced Jul 28, 2026
toubatbrian
force-pushed
the
brian/fix-tts-pool-session-leak
branch
from
July 28, 2026 23:57
7577b26 to
93686d6
Compare
`session.closed` part-way through a reply was treated as successful completion. The rest of the reply's text was then discarded with no error and no retry, and the gateway websocket went back into the ConnectionPool mid-synthesis, so the next reply read the previous reply's outstanding audio as its own. Flush the audio the dropped session produced, mark that segment's last frame final, and reject with a retryable APIStatusError so the socket is evicted and the retry finishes the reply. Gate socket reuse on having observed `done`. The cancelled attempt's input task reads with its abort signal so it stops waiting instead of consuming the text the retry needs, and re-checks `closing` after each read: cleanup closes the sentence tokenizer synchronously, so a chunk that lands during the drop would otherwise be pushed into a closed stream and fail the request with a plain, unretryable `Error`. Python has no session.closed branch: the drop surfaces as a timeout or closed socket, the exception leaves _run, and the pool evicts the connection. Resolving it was the divergence. Co-authored-by: Cursor <cursoragent@cursor.com>
toubatbrian
force-pushed
the
brian/fix-tts-pool-session-leak
branch
from
July 29, 2026 00:25
93686d6 to
ef4db13
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
When the inference gateway ends a TTS session with
session.closedpart-way through a reply, the JS client treats it as a successful completion. One mistake, two consequences.The rest of the reply is thrown away.
resourceCleanup()setsclosing, which kills the input task, so the remaining text is discarded with no error, no warning and no retry. The framerun()holds back so it can mark it final is dropped too, so the segment just stops.And the socket goes back into the
ConnectionPoolmid-synthesis, so the next reply can pick it up and read the previous reply's outstanding audio as its own.Fix
Flush the audio the dropped session did produce, mark that segment's last frame final, and reject with a retryable
APIStatusErrorso the socket is evicted and the retry finishes the reply. Socket reuse is additionally gated on having observeddone.END_OF_STREAMis deliberately not emitted — it is the stream terminator thatttsNodebreaks on, and the stream is not done. The cancelled attempt's input task now reads with its abort signal, so it stops waiting instead of consuming and dropping the chunk of text the retry needs.Testing
tts_session_closed.test.tsfails onmainwithexpected 6400 to be 8000(100ms of synthesized audio never delivered) andexpected '' to contain 'He climbed the stairs and lit the lamp.'(the rest of the reply never resynthesized).tts_pool_reuse.test.tsfails onmainwithexpected Set{ 1000 } to deeply equal Set{ 2000 }— reply 2 speaking only reply 1's audio. Green here;inference+voicepass at 808 tests.Python parity
inference/tts.pyhas nosession.closedbranch at all: the dropped session surfaces as a read timeout or a closed socket, the exception leaves_run, and the pool's context manager evicts the connection. Failing the attempt is Python's behaviour. Resolving it was the divergence.Scope — read this before merging
This was opened while root-causing a live report of a reply speaking the previous reply's audio after a barge-in, and originally attributed that incident to
session.closed. That attribution was wrong and the description has been corrected.The real cause of that incident is a separate defect one step earlier: the gateway emits multiple
doneevents persession.flushon some provider paths, and the client returns on the first one. A direct probe against the gateway records 6dones under a single session id for one flush, the first covering 19.6% of the audio, and switching only the TTS provider to one that emits a singledonemakes the symptom disappear with no code change. Python has that defect identically.So this PR does not fix that report, and it should not be merged as if it did. It stands on its own: treating a dropped session as success truncates the reply and drops the socket back in the pool, and both are wrong regardless of what caused the incident that surfaced them.