fix(google): merge timeout into caller-set http_options instead of replacing#5974
Open
xlyoung wants to merge 1 commit into
Open
fix(google): merge timeout into caller-set http_options instead of replacing#5974xlyoung wants to merge 1 commit into
xlyoung wants to merge 1 commit into
Conversation
When a caller passes http_options to google.LLM (e.g. to set custom headers for Vertex AI Priority PayGo), the previous code used those options directly via `or`, which meant conn_options.timeout was silently dropped. This disabled timeout-based fallback in llm.FallbackAdapter. Now we merge the timeout field into the caller's http_options when they haven't set one explicitly, preserving both custom headers and framework-enforced timeouts. Fixes livekit#5972
|
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
In
LLMStream._run, the request HTTP options are resolved as:When a caller passes
http_optionstogoogle.LLM(...)(e.g. to set custom headers for Vertex AI Priority PayGo), theorshort-circuits and uses the caller's options directly — but the caller typically doesn't settimeout. This meansconn_options.timeoutis silently dropped, which disables timeout-based fallback forllm.FallbackAdapter.Fixes #5972
Fix
Merge the timeout field into the caller's
http_optionswhen they haven't set one explicitly:This preserves both custom headers and framework-enforced timeouts.