Skip to content

fix(voice): skip committing late STT finals after a turn flush (#6504)#6516

Open
SHAI-sambhav-suri wants to merge 1 commit into
livekit:mainfrom
SHAI-sambhav-suri:fix/eot-late-stt-final-6504
Open

fix(voice): skip committing late STT finals after a turn flush (#6504)#6516
SHAI-sambhav-suri wants to merge 1 commit into
livekit:mainfrom
SHAI-sambhav-suri:fix/eot-late-stt-final-6504

Conversation

@SHAI-sambhav-suri

Copy link
Copy Markdown

A late STT final transcript could arrive after VAD had already committed and flushed the turn. With no prediction future left in flight, the eou bounce logged "skipping eot prediction" but then fell through and committed a second turn with a null end_of_turn_probability, splitting one utterance into two conversation items where the second bypassed turn detection.

Genuinely short-circuit that case: when a streaming detector has been flushed and there is no prediction in flight, skip the commit instead of shipping a null-probability turn. The transcript is left intact so the trailing words fold into the next turn rather than being lost.

Fixes #6504

…it#6504)

A late STT final transcript could arrive after VAD had already committed
and flushed the turn. With no prediction future left in flight, the eou
bounce logged "skipping eot prediction" but then fell through and committed
a second turn with a null end_of_turn_probability, splitting one utterance
into two conversation items where the second bypassed turn detection.

Genuinely short-circuit that case: when a streaming detector has been
flushed and there is no prediction in flight, skip the commit instead of
shipping a null-probability turn. The transcript is left intact so the
trailing words fold into the next turn rather than being lost.

Fixes livekit#6504
@SHAI-sambhav-suri
SHAI-sambhav-suri requested a review from a team as a code owner July 22, 2026 20:32

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Open in Devin Review

@SHAI-sambhav-suri

SHAI-sambhav-suri commented Jul 23, 2026

Copy link
Copy Markdown
Author

What this fixes

Fixes #6504. An utterance with a mid-phrase pause ("No." … ~1s … "Date needed
now.") was split into two conversation items, and the second committed with
end_of_turn_probability: null — reaching the LLM having bypassed turn detection.

Why it happens

VAD detects the silence and commits the turn, which flushes the streaming turn
detector (_turn_detector_flushed = True, prediction future cleared). STT then
finalizes the trailing words a beat later, re-entering
_run_eou_detection(trigger="stt"). With no prediction future left, the eou
bounce logs "skipping eot prediction" — but doesn't actually skip: it falls
through with end_of_turn_probability = None and commits a second, bypassed turn.

The change

Make the skip real. In _bounce_eou_task, short-circuit before committing when a
late STT final arrives after the detector was flushed and no prediction is in flight:

if (
    trigger == "stt"
    and self._turn_detector_flushed
    and self._turn_detector_prediction_fut is None
    and isinstance(turn_detector, _StreamingTurnDetectorStream)
):
    self._on_missing_eot_prediction()
    return

No null-probability commit, and the log is now truthful. We return before
_audio_transcript is cleared, so the trailing words fold into the next turn
instead of being lost. The guard is scoped to this exact case, so the VAD path,
manual mode, and text detectors are unaffected.

Tests
Extended test_late_stt_final_after_flush_short_circuits to assert the contract
that was missing: on_end_of_turn is not called and the transcript is preserved
(plus no inference started / no on_eot_prediction). It fails on the old behavior
(commits twice) and passes with the fix. Full --audio_eot suite and
voice-related --unit tests pass; ruff and mypy clean.

@Propconnect-ai

Copy link
Copy Markdown

Reporter of #6504 here — this matches what I observed.

The four-condition guard covers the exact path from my logs: STT-triggered, detector already flushed, no prediction in flight, streaming detector. Returning before the commit means no phantom eot prediction event and no null-probability turn, so both halves of the issue are addressed rather than just the commit.

Preserving the transcript so it folds into the next turn is the right call. In my repro the trailing fragment became its own conversation item ("No. Date" then "needed now."), and the agent responded to the first half before the second arrived. Merging forward fixes the behaviour, not just the log.

One question on coverage: the guard requires _turn_detector_prediction_fut is None. I saw two variants in my sessions —

  1. stt transcript arrived after a turn flush, skipping eot prediction followed by an eot prediction with null fields
  2. the transcript arrives after turn has been committed. consider raising min_delay... warning, also followed by a null prediction

Both ended with end_of_turn_probability: null and speech_start_time: null, but I don't know from the logs alone whether a prediction future was in flight in either case. If a late final can arrive while one is still pending, that path would fall through this guard. Worth confirming, or explicitly out of scope.

Happy to re-run my repro against this branch if that's useful.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Turn commits with end_of_turn_probability: null when STT transcript arrives after turn flush, splitting one utterance into two turns

2 participants