Skip to content

fix(google): merge timeout into caller-set http_options instead of replacing#5974

Open
xlyoung wants to merge 1 commit into
livekit:mainfrom
xlyoung:fix/google-plugin-http-options-timeout-merge
Open

fix(google): merge timeout into caller-set http_options instead of replacing#5974
xlyoung wants to merge 1 commit into
livekit:mainfrom
xlyoung:fix/google-plugin-http-options-timeout-merge

Conversation

@xlyoung
Copy link
Copy Markdown

@xlyoung xlyoung commented Jun 5, 2026

Problem

In LLMStream._run, the request HTTP options are resolved as:

http_options = self._llm._opts.http_options or types.HttpOptions(
    timeout=int(self._conn_options.timeout * 1000)
)

When a caller passes http_options to google.LLM(...) (e.g. to set custom headers for Vertex AI Priority PayGo), the or short-circuits and uses the caller's options directly — but the caller typically doesn't set timeout. This means conn_options.timeout is silently dropped, which disables timeout-based fallback for llm.FallbackAdapter.

Fixes #5972

Fix

Merge the timeout field into the caller's http_options when they haven't set one explicitly:

http_options = self._llm._opts.http_options or types.HttpOptions()
if http_options.timeout is None and self._conn_options.timeout:
    http_options = http_options.model_copy(
        update={"timeout": int(self._conn_options.timeout * 1000)}
    )

This preserves both custom headers and framework-enforced timeouts.

When a caller passes http_options to google.LLM (e.g. to set custom
headers for Vertex AI Priority PayGo), the previous code used those
options directly via `or`, which meant conn_options.timeout was
silently dropped. This disabled timeout-based fallback in
llm.FallbackAdapter.

Now we merge the timeout field into the caller's http_options when
they haven't set one explicitly, preserving both custom headers and
framework-enforced timeouts.

Fixes livekit#5972
@CLAassistant
Copy link
Copy Markdown

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no potential bugs to report.

View in Devin Review to see 2 additional findings.

Open in Devin Review

Copy link
Copy Markdown
Contributor

@longcw longcw left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

livekit-plugins-google: caller-set http_options silently drops conn_options.timeout (disables FallbackAdapter timeout)

3 participants