-
Notifications
You must be signed in to change notification settings - Fork 3k
Description
Problem
CachedContent.create() calls to Vertex AI can take 30-40 seconds, blocking the user's request for the entire duration. There is currently no way to set a timeout or other HTTP options on the cache creation call — ADK constructs the CreateCachedContentConfig internally without exposing the http_options field.
Proposed Solution
Add a create_http_options: Optional[types.HttpOptions] field to ContextCacheConfig that gets passed through to the CreateCachedContentConfig when creating a cache. This allows users to control timeout and other HTTP settings on cache creation calls.
When the cache creation call exceeds the configured timeout, it fails gracefully and the request proceeds without caching — the same behavior as if caching were disabled for that turn.
Usage Example
from google.genai import types
from google.adk.agents import ContextCacheConfig
# 10-second timeout on cache creation
cache_config = ContextCacheConfig(
cache_intervals=3,
ttl_seconds=3600,
min_tokens=1024,
create_http_options=types.HttpOptions(timeout=10000), # milliseconds
)Why not use client-level timeout?
The genai.Client accepts http_options at construction time (and ADK 1.26 supports bring-your-own-client), but a client-level timeout applies to all API calls — generate_content, caches.create, caches.delete, etc. Cache creation is uniquely slow (~38s) while other calls are fast, so a per-operation timeout is needed.
Design
- Zero behavior change when unset:
create_http_options=None(default) preserves existing behavior exactly - Graceful degradation: Timeout → cache creation fails → request proceeds uncached (existing error handling)
- No global state: Pure passthrough, no background tasks or in-memory registries
- Leverages existing SDK support:
CreateCachedContentConfigalready acceptshttp_options