feat(models): wire maxOutputTokens for native Gemini and Vertex#2147
feat(models): wire maxOutputTokens for native Gemini and Vertex#2147anthonytwh wants to merge 2 commits into
Conversation
There was a problem hiding this comment.
Pull request overview
This PR wires maxOutputTokens end-to-end for both the native Gemini and Gemini Vertex AI providers, carrying a model-level default from the Kubernetes ModelConfig CR through the Go translator into the Python ADK runtime, and applying it per-request without overriding any explicitly set per-request value.
Changes:
- Adds
maxOutputTokensto the Gemini model config API/CRD and propagates it through the Go ADK config serialization. - Introduces a shared Python
_GeminiGenerationConfigMixinto apply model-levelmax_output_tokensonly when the request doesn’t already specify it. - Adds Go translator tests and Python unit tests to verify correct precedence and no-op behavior when uncapped.
Reviewed changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.
Show a summary per file
| File | Description |
|---|---|
| python/packages/kagent-adk/tests/unittests/models/test_gemini.py | Adds unit tests validating model-level max_output_tokens behavior and precedence vs per-request config. |
| python/packages/kagent-adk/src/kagent/adk/types.py | Extends Gemini/Gemini Vertex AI model config schemas and passes max_output_tokens into runtime wrappers. |
| python/packages/kagent-adk/src/kagent/adk/models/_gemini.py | Adds generation-config mixin and a Vertex AI wrapper to apply model-level defaults per request. |
| python/packages/kagent-adk/src/kagent/adk/models/init.py | Exports the new Gemini Vertex AI wrapper. |
| helm/kagent-crds/templates/kagent.dev_modelconfigs.yaml | Updates Helm CRDs to expose spec.gemini.maxOutputTokens. |
| go/core/internal/controller/translator/agent/adk_api_translator.go | Copies MaxOutputTokens from ModelConfig into serialized ADK model config (Gemini + Vertex AI). |
| go/core/internal/controller/translator/agent/adk_api_translator_test.go | Adds translator tests verifying MaxOutputTokens is set for Gemini and Vertex AI. |
| go/api/v1alpha2/modelconfig_types.go | Adds MaxOutputTokens to the Gemini API type. |
| go/api/config/crd/bases/kagent.dev_modelconfigs.yaml | Updates generated CRD bases to include spec.gemini.maxOutputTokens. |
| go/api/adk/types.go | Adds max_output_tokens to serialized ADK Gemini / GeminiVertexAI model structs. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| // GeminiConfig contains Gemini (AI Studio, API-key) specific configuration options | ||
| type GeminiConfig struct { | ||
| // Maximum output tokens to generate for a single response | ||
| // +optional | ||
| MaxOutputTokens int `json:"maxOutputTokens,omitempty"` | ||
| } |
| description: Gemini-specific configuration | ||
| properties: | ||
| maxOutputTokens: | ||
| description: Maximum output tokens to generate for a single response | ||
| type: integer | ||
| type: object |
| description: Gemini-specific configuration | ||
| properties: | ||
| maxOutputTokens: | ||
| description: Maximum output tokens to generate for a single response | ||
| type: integer | ||
| type: object |
| class GeminiVertexAI(BaseLLM): | ||
| max_output_tokens: int | None = None | ||
| type: Literal["gemini_vertex_ai"] | ||
|
|
| class Gemini(BaseLLM): | ||
| max_output_tokens: int | None = None | ||
| type: Literal["gemini"] | ||
|
|
|
Thanks @copilot — addressed all five comments in ad27cd0. They were the same point (
Go translator tests + Python model tests pass. |
The native Gemini and GeminiVertexAI providers previously exposed no way to cap a response's output tokens via ModelConfig: GeminiConfig was an empty struct and GeminiVertexAIConfig.MaxOutputTokens was declared but never read by the translator or applied at runtime. Both providers subclass the ADK native Gemini model, which takes generation config from the per-request LlmRequest.config rather than the model definition, so nothing plumbed the ModelConfig-level setting through. Wire it end to end: - add MaxOutputTokens to GeminiConfig (v1alpha2) and regenerate the CRDs - carry it on the adk.Gemini / adk.GeminiVertexAI serialization structs - copy Spec.Gemini/Spec.GeminiVertexAI.MaxOutputTokens in the translator - parse it on the Gemini/GeminiVertexAI pydantic models and build the wrappers with it - apply it in a shared _GeminiGenerationConfigMixin that injects max_output_tokens into GenerateContentConfig without overriding a value the agent already set on the request A per-request value always wins over the model-level default. Adds Go translator tests and Python unit tests for the mixin. Signed-off-by: anthonytwh <anthony.tam@tigera.io>
Address review feedback: the translator treats maxOutputTokens <= 0 as "unset", so an invalid (0/negative) value was silently ignored rather than rejected. Enforce a positive value at validation time: - add +kubebuilder:validation:Minimum=1 to GeminiConfig and GeminiVertexAIConfig.MaxOutputTokens and regenerate the CRDs (Kubernetes now rejects invalid values on apply) - add pydantic ge=1 to the Gemini / GeminiVertexAI max_output_tokens fields so an invalid ModelConfig fails fast at parse time - add unit tests for the parse-time validation Signed-off-by: anthonytwh <anthony.tam@tigera.io>
ad27cd0 to
75d90bf
Compare
|
Thanks @copilot — all five comments make sense and are the same point (reject 0/negative
Added parse-time validation tests as well. Kubernetes now rejects invalid values on apply, and the pydantic models reject them on parse. |
What
Wire
maxOutputTokensend to end for the native Gemini and Gemini Vertex AI model providers.Previously neither provider could cap a response's output tokens via
ModelConfig:GeminiConfigwas an empty struct andGeminiVertexAIConfig.MaxOutputTokenswas declared but never read by the translator or applied at runtime. Both providers subclass the ADK native Gemini model, which takes generation config from the per-requestLlmRequest.configrather than the model definition, so nothing plumbed theModelConfig-level setting through to the API.Changes
MaxOutputTokenstoGeminiConfig(v1alpha2) and regenerate the CRDs (bothgo/api/config/crd/basesandhelm/kagent-crds).adk.Gemini/adk.GeminiVertexAIserialization structs.Spec.Gemini/Spec.GeminiVertexAI.MaxOutputTokensin the translator._GeminiGenerationConfigMixinthat injectsmax_output_tokensintoGenerateContentConfigwithout overriding a value the agent already set on the request.A per-request value always wins over the model-level default.
Tests
Test_AdkApiTranslator_GeminiMaxOutputTokens(native Gemini + Vertex AI) — pass; full translator package green.