fix(voice): deadline-based user_away_timeout ignores transcript-less noise (#6030)#6499
fix(voice): deadline-based user_away_timeout ignores transcript-less noise (#6030)#6499dorukdumlu wants to merge 5 commits into
Conversation
…noise (livekit#6030) Keep an absolute away deadline so speaking↔listening flips without a transcript (telephony background noise) no longer re-arm the full timeout. Agent activity and final STT transcripts still refresh the deadline.
| self._user_away_timer: asyncio.TimerHandle | None = None | ||
| # absolute deadline for "away"; preserved across transcript-less user | ||
| # speaking↔listening flips so telephony noise cannot defer the timeout | ||
| self._user_away_deadline: float | None = None |
There was a problem hiding this comment.
I think we can add an option to only cancel the user away timer when agent speaking or user input transcribed. right now it's cancelled when agent or user speaking that may too sensitive to the noises.
Address review on livekit#6499: drop redundant away-recovery timer re-arm (can fire during agent speech), and add user_away_on=conversation so VAD speaking no longer pauses the away countdown. Default stays presence with deadline preservation.
|
@longcw thanks — agreed the cancel policy was the missing piece (deadline alone only fixes flip-storms, not continuous noise stuck in Pushed:
Happy to rename the option or flip the default if you'd rather. |
Presence mode preserved the absolute deadline across speaking→listening, which correctly ignores transcript-less noise but could fire away immediately after a long real utterance (stale remaining=0). Track finals during the speech segment and reset the deadline on EOS when one was seen.
|
Follow-up for the Devin finding on stale
|
…aking In conversation mode, agent return-to-listening previously cancelled the away timer whenever the user was still VAD-speaking (noise), then never re-armed. Treat conversation-mode user speaking as idle for arming so away still fires on noisy telephony lines.
|
Follow-up for Devin's conversation-mode stall: When the agent returns to Added |
Summary
_user_away_deadlineforuser_away_timeoutso transcript-less userspeaking ↔ listeningflips re-arm with remaining time (telephony noise can no longer defer "away" indefinitely) — user_away_timeout getting triggered for SIP participant on telephony calls even when user is not speaking and taking background noise and conversations as user state speaking. #6030.user_away_on:"presence"(default, backcompat) vs"conversation"(ignore user VAD speaking; only pause while the agent is active / refresh on final transcript). Preferconversationfor noisy SIP +turn_detection="stt"._set_user_away_timer(reset=True)that could arm the timer while the agent is speaking (Devin review).Test plan
test_noise_user_state_flips_do_not_defer_away_deadlinetest_final_transcript_and_agent_activity_refresh_away_deadlinetest_away_recovery_transcript_does_not_arm_during_agent_speechtest_conversation_user_away_ignores_user_speakingtest_conversation_user_away_still_cancels_on_agent_speakingtest_final_transcript_resets_away_timer_when_not_speaking