Skip to content

fix(voice): deadline-based user_away_timeout ignores transcript-less noise (#6030)#6499

Open
dorukdumlu wants to merge 5 commits into
livekit:mainfrom
dorukdumlu:fix/user-away-deadline
Open

fix(voice): deadline-based user_away_timeout ignores transcript-less noise (#6030)#6499
dorukdumlu wants to merge 5 commits into
livekit:mainfrom
dorukdumlu:fix/user-away-deadline

Conversation

@dorukdumlu

@dorukdumlu dorukdumlu commented Jul 21, 2026

Copy link
Copy Markdown

Summary

Test plan

  • test_noise_user_state_flips_do_not_defer_away_deadline
  • test_final_transcript_and_agent_activity_refresh_away_deadline
  • test_away_recovery_transcript_does_not_arm_during_agent_speech
  • test_conversation_user_away_ignores_user_speaking
  • test_conversation_user_away_still_cancels_on_agent_speaking
  • Existing test_final_transcript_resets_away_timer_when_not_speaking

…noise (livekit#6030)

Keep an absolute away deadline so speaking↔listening flips without a transcript (telephony background noise) no longer re-arm the full timeout. Agent activity and final STT transcripts still refresh the deadline.
devin-ai-integration[bot]

This comment was marked as resolved.

self._user_away_timer: asyncio.TimerHandle | None = None
# absolute deadline for "away"; preserved across transcript-less user
# speaking↔listening flips so telephony noise cannot defer the timeout
self._user_away_deadline: float | None = None

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we can add an option to only cancel the user away timer when agent speaking or user input transcribed. right now it's cancelled when agent or user speaking that may too sensitive to the noises.

Address review on livekit#6499: drop redundant away-recovery timer re-arm (can fire during agent speech), and add user_away_on=conversation so VAD speaking no longer pauses the away countdown. Default stays presence with deadline preservation.
@dorukdumlu

Copy link
Copy Markdown
Author

@longcw thanks — agreed the cancel policy was the missing piece (deadline alone only fixes flip-storms, not continuous noise stuck in speaking).

Pushed:

  1. user_away_on"presence" (default) keeps current cancel-on-user-speaking + deadline preservation; "conversation" cancels only while the agent is active and ignores user VAD speaking. Telephony/user_away_timeout getting triggered for SIP participant on telephony calls even when user is not speaking and taking background noise and conversations as user state speaking. #6030 should use user_away_on="conversation".
  2. Devin bug — removed the redundant away-recovery reset=True that could arm the timer mid-agent-speech.

Happy to rename the option or flip the default if you'd rather.

devin-ai-integration[bot]

This comment was marked as resolved.

Presence mode preserved the absolute deadline across speaking→listening, which correctly ignores transcript-less noise but could fire away immediately after a long real utterance (stale remaining=0). Track finals during the speech segment and reset the deadline on EOS when one was seen.
@dorukdumlu

Copy link
Copy Markdown
Author

Follow-up for the Devin finding on stale remaining=0 after long genuine speech in presence mode:

conversation mode unchanged.

devin-ai-integration[bot]

This comment was marked as resolved.

…aking

In conversation mode, agent return-to-listening previously cancelled the away timer whenever the user was still VAD-speaking (noise), then never re-armed. Treat conversation-mode user speaking as idle for arming so away still fires on noisy telephony lines.
@dorukdumlu

Copy link
Copy Markdown
Author

Follow-up for Devin's conversation-mode stall:

When the agent returns to listening while the user is stuck in VAD speaking (noise), _update_agent_state now re-arms the away timer in user_away_on=""conversation"" — presence mode unchanged (still requires user listening).

Added test_conversation_rearms_away_after_agent_while_user_speaking + presence regression guard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants