Skip to content

feat(models): wire maxOutputTokens for native Gemini and Vertex#2147

Open
anthonytwh wants to merge 2 commits into
kagent-dev:mainfrom
anthonytwh:wire-gemini-generation-config
Open

feat(models): wire maxOutputTokens for native Gemini and Vertex#2147
anthonytwh wants to merge 2 commits into
kagent-dev:mainfrom
anthonytwh:wire-gemini-generation-config

Conversation

@anthonytwh

Copy link
Copy Markdown

What

Wire maxOutputTokens end to end for the native Gemini and Gemini Vertex AI model providers.

Previously neither provider could cap a response's output tokens via ModelConfig: GeminiConfig was an empty struct and GeminiVertexAIConfig.MaxOutputTokens was declared but never read by the translator or applied at runtime. Both providers subclass the ADK native Gemini model, which takes generation config from the per-request LlmRequest.config rather than the model definition, so nothing plumbed the ModelConfig-level setting through to the API.

Changes

  • Add MaxOutputTokens to GeminiConfig (v1alpha2) and regenerate the CRDs (both go/api/config/crd/bases and helm/kagent-crds).
  • Carry it on the adk.Gemini / adk.GeminiVertexAI serialization structs.
  • Copy Spec.Gemini / Spec.GeminiVertexAI.MaxOutputTokens in the translator.
  • Parse it on the Gemini / GeminiVertexAI pydantic models and build the wrappers with it.
  • Apply it in a shared _GeminiGenerationConfigMixin that injects max_output_tokens into GenerateContentConfig without overriding a value the agent already set on the request.

A per-request value always wins over the model-level default.

Tests

  • Go translator tests: Test_AdkApiTranslator_GeminiMaxOutputTokens (native Gemini + Vertex AI) — pass; full translator package green.
  • Python unit tests for the mixin: applies the cap when unset, does not override a per-request value, and is a no-op when uncapped — 3 passed.

Copilot AI review requested due to automatic review settings July 3, 2026 21:33
@github-actions github-actions Bot added the enhancement New feature or request label Jul 3, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR wires maxOutputTokens end-to-end for both the native Gemini and Gemini Vertex AI providers, carrying a model-level default from the Kubernetes ModelConfig CR through the Go translator into the Python ADK runtime, and applying it per-request without overriding any explicitly set per-request value.

Changes:

  • Adds maxOutputTokens to the Gemini model config API/CRD and propagates it through the Go ADK config serialization.
  • Introduces a shared Python _GeminiGenerationConfigMixin to apply model-level max_output_tokens only when the request doesn’t already specify it.
  • Adds Go translator tests and Python unit tests to verify correct precedence and no-op behavior when uncapped.

Reviewed changes

Copilot reviewed 10 out of 10 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
python/packages/kagent-adk/tests/unittests/models/test_gemini.py Adds unit tests validating model-level max_output_tokens behavior and precedence vs per-request config.
python/packages/kagent-adk/src/kagent/adk/types.py Extends Gemini/Gemini Vertex AI model config schemas and passes max_output_tokens into runtime wrappers.
python/packages/kagent-adk/src/kagent/adk/models/_gemini.py Adds generation-config mixin and a Vertex AI wrapper to apply model-level defaults per request.
python/packages/kagent-adk/src/kagent/adk/models/init.py Exports the new Gemini Vertex AI wrapper.
helm/kagent-crds/templates/kagent.dev_modelconfigs.yaml Updates Helm CRDs to expose spec.gemini.maxOutputTokens.
go/core/internal/controller/translator/agent/adk_api_translator.go Copies MaxOutputTokens from ModelConfig into serialized ADK model config (Gemini + Vertex AI).
go/core/internal/controller/translator/agent/adk_api_translator_test.go Adds translator tests verifying MaxOutputTokens is set for Gemini and Vertex AI.
go/api/v1alpha2/modelconfig_types.go Adds MaxOutputTokens to the Gemini API type.
go/api/config/crd/bases/kagent.dev_modelconfigs.yaml Updates generated CRD bases to include spec.gemini.maxOutputTokens.
go/api/adk/types.go Adds max_output_tokens to serialized ADK Gemini / GeminiVertexAI model structs.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +243 to +248
// GeminiConfig contains Gemini (AI Studio, API-key) specific configuration options
type GeminiConfig struct {
// Maximum output tokens to generate for a single response
// +optional
MaxOutputTokens int `json:"maxOutputTokens,omitempty"`
}
Comment on lines 536 to 541
description: Gemini-specific configuration
properties:
maxOutputTokens:
description: Maximum output tokens to generate for a single response
type: integer
type: object
Comment on lines 536 to 541
description: Gemini-specific configuration
properties:
maxOutputTokens:
description: Maximum output tokens to generate for a single response
type: integer
type: object
Comment on lines 296 to 299
class GeminiVertexAI(BaseLLM):
max_output_tokens: int | None = None
type: Literal["gemini_vertex_ai"]

Comment on lines 310 to 313
class Gemini(BaseLLM):
max_output_tokens: int | None = None
type: Literal["gemini"]

@anthonytwh

Copy link
Copy Markdown
Author

Thanks @copilot — addressed all five comments in ad27cd0. They were the same point (maxOutputTokens accepted 0/negative even though the translator treats <= 0 as "unset"), so I enforced a positive value at every layer:

  • +kubebuilder:validation:Minimum=1 on GeminiConfig and GeminiVertexAIConfig.MaxOutputTokens, CRDs regenerated (minimum: 1 now in both the bases and Helm copies) — Kubernetes rejects invalid values on apply.
  • pydantic ge=1 on the Gemini / GeminiVertexAI max_output_tokens fields so an invalid ModelConfig fails fast at parse time.
  • Added unit tests for the parse-time validation.

Go translator tests + Python model tests pass.

The native Gemini and GeminiVertexAI providers previously exposed no way to
cap a response's output tokens via ModelConfig: GeminiConfig was an empty
struct and GeminiVertexAIConfig.MaxOutputTokens was declared but never read
by the translator or applied at runtime. Both providers subclass the ADK
native Gemini model, which takes generation config from the per-request
LlmRequest.config rather than the model definition, so nothing plumbed the
ModelConfig-level setting through.

Wire it end to end:
- add MaxOutputTokens to GeminiConfig (v1alpha2) and regenerate the CRDs
- carry it on the adk.Gemini / adk.GeminiVertexAI serialization structs
- copy Spec.Gemini/Spec.GeminiVertexAI.MaxOutputTokens in the translator
- parse it on the Gemini/GeminiVertexAI pydantic models and build the
  wrappers with it
- apply it in a shared _GeminiGenerationConfigMixin that injects
  max_output_tokens into GenerateContentConfig without overriding a value
  the agent already set on the request

A per-request value always wins over the model-level default. Adds Go
translator tests and Python unit tests for the mixin.

Signed-off-by: anthonytwh <anthony.tam@tigera.io>
Address review feedback: the translator treats maxOutputTokens <= 0 as
"unset", so an invalid (0/negative) value was silently ignored rather
than rejected. Enforce a positive value at validation time:

- add +kubebuilder:validation:Minimum=1 to GeminiConfig and
  GeminiVertexAIConfig.MaxOutputTokens and regenerate the CRDs
  (Kubernetes now rejects invalid values on apply)
- add pydantic ge=1 to the Gemini / GeminiVertexAI max_output_tokens
  fields so an invalid ModelConfig fails fast at parse time
- add unit tests for the parse-time validation

Signed-off-by: anthonytwh <anthony.tam@tigera.io>
@anthonytwh anthonytwh force-pushed the wire-gemini-generation-config branch from ad27cd0 to 75d90bf Compare July 7, 2026 16:58
@anthonytwh anthonytwh requested a review from a team as a code owner July 7, 2026 16:58
@anthonytwh

Copy link
Copy Markdown
Author

Thanks @copilot — all five comments make sense and are the same point (reject 0/negative maxOutputTokens since the translator treats <= 0 as "unset"). They're already addressed in commit 75d90bfd ("validate: reject non-positive Gemini maxOutputTokens"), which landed after the review was posted:

  • Go: +kubebuilder:validation:Minimum=1 on GeminiConfig and GeminiVertexAIConfig.MaxOutputTokens (modelconfig_types.go:77,248)
  • CRDs regenerated with minimum: 1 in both api/config/crd/bases and helm/kagent-crds (lines 540 & 554)
  • Python: Field(default=None, ge=1) on Gemini and GeminiVertexAI.max_output_tokens (types.py:297,311) — fails fast at parse time

Added parse-time validation tests as well. Kubernetes now rejects invalid values on apply, and the pydantic models reject them on parse.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants