fix(voice): stop a confirmed interruption from resuming the interrupted speech - #2122
fix(voice): stop a confirmed interruption from resuming the interrupted speech#2122toubatbrian wants to merge 2 commits into
Conversation
…umable A verdict of "this overlap really was a barge-in" left the speech parked by interruptByAudioActivity, which the false-interruption timer is free to resume once its timeout elapses. That timer is for overlaps nobody has ruled on yet, so end the pause outright once the model has ruled. Also stops the agent ~700ms sooner, since committing the interruption no longer waits on the final transcript. Co-authored-by: Cursor <cursoragent@cursor.com>
🦋 Changeset detectedLatest commit: 3e10112 The changes in this PR will be included in the next version bump. This PR includes changesets to release 39 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
cancelSpeechPause() opened the sink's pause gate without first calling clearBuffer(), so frames of the speech it had just interrupted — parked at that gate for the duration of the pause — were released and the interrupted speech continued from exactly where it stopped. Runtime evidence from a barge-in run: the sink gate opened with clearBufferAlreadyFired=false and one parked frame then reached the wire. This path needs no false-interruption timer, which is why affected sessions show the audio resuming with no agent_false_interruption event. Co-authored-by: Cursor <cursoragent@cursor.com>
toubatbrian
changed the title
|
Do not merge this without #2124. #2124 fixes a pre-existing bug where Measured with a real
That is a reply that stops dead after ~40ms of 400ms, which the session still commits to chat context as fully spoken and not interrupted. This PR needs no changes. #2124 resolves it, and the two were verified together: all scenarios deliver 20/20 with both applied, and this PR's own behaviour is preserved — its parked frames are still dropped, identically with and without #2124. Either merge order is fine as long as both land together. |
|
The first commit — Verified there against a real The second commit — the #2126 also asks reviewers to confirm the product judgement this change makes: committing the verdict removes the STT final-transcript safety net, so a false positive from the interruption model now permanently ends the speech. Python's Closing in favour of #2126. |
|
Redirect: half of this PR now lives in #2134 (stack 5/5, based on #2133). The other half is not being shipped. Kept — Dropped — having The cost of dropping it is measured in #2134: on the path where no STT final transcript arrives, the stack is identical to |
1 participant
Problem
When the adaptive model rules an overlap a genuine barge-in, the agent pauses — and then puts the
interrupted speech back on the wire, continuing from exactly where it stopped. Reported from a live
session: "when I interrupt after asking for a long story, it does pause, but then it continues the
audio from where it's been cut off."
Two independent defects produce that.
1. The sink gate is opened before the interruption reaches the sink.
ParticipantAudioOutput.captureFrameparks frames at a pause gate and races them againstinterruptedFuture, whichclearBuffer()resolves:cancelSpeechPauseinterrupts the paused speech and then callsaudio.resume()without callingclearBuffer(). That resume is only meant to admit the next speech, but it also releases everyframe parked during the pause. Whether you hear the interrupted speech continue depends on which of
clearBuffer()(async, via the playout task) andresume()(sync, at the end ofcancelSpeechPause) wins — so it reproduces intermittently and needs no false-interruption timer,which is why affected sessions show no
agent_false_interruptionevent.2. A confirmed interruption was left resumable.
onInterruptiononly parked the speech throughinterruptByAudioActivity, leaving thefalse-interruption timer free to resume it after its timeout. That timer exists for overlaps nobody
has ruled on yet, so a confirmed verdict now ends the pause outright.
Runtime evidence
Instrumented barge-in run, consecutive events:
_output.ts:pauseinterruptedAlready: false, 1 pending segmentcancelSpeechPause:ungateSink_output.ts:resumeclearBufferAlreadyFired: falsecaptureFramepast the gateOver that run 14 frames were parked: 13 dropped (clearBuffer won) and 1 released (resume won).
After the fix, across three runs every resume that actually opened a closed gate saw
clearBufferAlreadyFired: true, with zero violations and zero released frames.Changes
cancelSpeechPausecallsclearBuffer()beforeresume()when it interrupted the paused speech.onInterruptioncommits the interruption instead of leaving it parked. Side benefit: the agentstops roughly 700ms sooner, since it no longer waits for the final transcript.
Test plan
clearBuffer→resumeordering; fails onmain(
expected [ 'resume' ] to deeply equal [ 'clearBuffer', 'resume' ]).agents/src/voice+agents/src/inference/interruption: 557 passed.a frame at the gate post-fix, so they exercise the ordering invariant but not the release path.