fix(stt): reconnect on send-side socket drops in remaining plugins#6474
fix(stt): reconnect on send-side socket drops in remaining plugins#6474flasun wants to merge 3 commits into
Conversation
Follow-up to livekit#6429 and livekit#6467: wrap the send_task loop in the streaming STT plugins those PRs didn't cover, so a mid-send socket drop raises a retryable APIConnectionError (or triggers the plugin's internal reconnect) instead of a raw ConnectionResetError that permanently kills the session with no failover. The recv path in each of these plugins already turns an unexpected close into a retryable error; this makes the send path symmetric. Guard conditions (closing_ws / self._session.closed) mirror each plugin's own recv handling. Plugins: xai, fireworksai, simplismart, smallestai, telnyx, slng, gradium. Adds a regression test in tests/test_plugin_gradium_stt.py.
slng's _run keys its same-endpoint immediate-retry logic on isinstance(exc, APIStatusError); any other error triggers a permanent failover to a lower-priority endpoint. recv_task already raises APIStatusError on an unexpected close, so the send guard must too — otherwise a transient send-side blip permanently downgrades the provider for the rest of the session. Addresses the review finding on slng.
|
Good catch from the automated review on `slng` — thanks. Unlike the other six plugins (which propagate to the base `_main_task`, where any `APIError` is retried on the same connection), `slng` has custom per-endpoint failover that only grants same-endpoint immediate retries for `APIStatusError`; any other error permanently bumps `_active_endpoint_index`. Since `recv_task` already raises `APIStatusError` on an unexpected close, a send-side drop raising `APIConnectionError` would have permanently downgraded to a lower-priority endpoint instead of retrying the primary. Fixed in c32c19d: the `slng` send guard now raises `APIStatusError` (matching `recv_task`), so both paths get identical same-endpoint-retry-then-failover semantics. Verified the other six do not have this type-keyed failover logic, so their `APIConnectionError` remains correct. |
…vekit#6473) Generalizes the gradium regression test into a shared harness (tests/_stt_send_drop.py) plus per-plugin cases, so the send-side drop contract is asserted across all seven plugins: - inject ClientConnectionResetError / ClientOSError / ConnectionError on the send path and assert each surfaces as a retryable APIError (APIConnectionError; APIStatusError for slng) that _main_task turns into a bounded reconnect, never a raw error; - assert the in-flight audio send is attempted and surfaced (no silent swallow / tight replay); - assert a drop while the session is closing does not surface as a retryable error. Notes uncovered while running it against each plugin: - fireworks/telnyx guard quiet-return on their own closing_ws flag (not session.closed), so the session-closed case there is the normal end-of-input path; - xai gates send on a server-ready recv message (the harness feeds one); - slng reconnects inside _run via multi-endpoint failover, so it runs in drop_on_bytes mode (every reconnection handshake succeeds, only audio drops) and asserts APIStatusError.
fix(stt): reconnect on send-side socket drops in remaining plugins
What
Follow-up to #6429 (deepgram) and #6467 (assemblyai/baseten/speechmatics/elevenlabs/gladia/openai/inference), which fixed a class of bug where a send-side WebSocket drop crashed the STT session instead of reconnecting. This applies the same fix to the streaming STT plugins those PRs didn't cover.
Closes #6473.
Why
SpeechStream._main_taskonly reconnects onAPIError; any other exception kills the stream. When the peer resets the socket mid-stream,ws.send_bytes()/send_str()raises a rawaiohttp.ClientConnectionResetError(aConnectionResetError), which isn't anAPIError. In these plugins thesend_taskdoesn't wrap the send loop, so that raw exception escapes and permanently terminates the stream — dead air, no failover. The recv path in each of these plugins already turns an unexpected close into the retryableAPIStatusError(or an internal reconnect), so the fix just makes the send path symmetric.Changes
Wrap each plugin's
send_taskloop inexcept (aiohttp.ClientError, ConnectionError), mirroring the plugin's existing recv idiom:APIStatusError→ send raises the retryableAPIConnectionError):xai,fireworksai,smallestai,telnyx,slng,gradium. The guard (closing_ws/self._session.closed) matches each plugin's own recv guard, so a graceful close or a session teardown returns quietly rather than raising._reconnect_eventand returns):simplismart— the send path now triggers the same internal reconnect instead of crashing.fireworksaiandgradiumadditionally importAPIConnectionError.Test
tests/test_plugin_gradium_stt.py::test_send_side_socket_drop_raises_api_connection_errordrivesSpeechStream._runwith a fake WebSocket whose first audiosend_strdrops the socket, and asserts_runraisesAPIConnectionError(retryable) rather than the rawClientConnectionResetError. Verified it fails on the unpatched code and passes with the fix.gradiumis used as the representative case since it already has a plugin test module.Notes