Skip to content

feat(models): add maxCompletionTokens for OpenAI#2175

Open
anthonytwh wants to merge 2 commits into
kagent-dev:mainfrom
anthonytwh:wire-openai-max-completion-tokens
Open

feat(models): add maxCompletionTokens for OpenAI#2175
anthonytwh wants to merge 2 commits into
kagent-dev:mainfrom
anthonytwh:wire-openai-max-completion-tokens

Conversation

@anthonytwh

Copy link
Copy Markdown

What

Add a maxCompletionTokens field to the OpenAI ModelConfig that maps to OpenAI's max_completion_tokens request parameter.

Why

OpenAI's max_tokens parameter is deprecated for the Chat Completions API and is rejected outright by reasoning models (GPT-5 / o-series):

400 Bad Request: Unsupported parameter: 'max_tokens' is not supported with this model.
Use 'max_completion_tokens' instead.

Today OpenAIConfig only exposes maxTokens (→ max_tokens), so there is no way to cap output tokens for GPT-5 / o-series models — setting maxTokens on such a model makes every request fail. (Per OpenAI's SDK docstrings, max_tokens "is now deprecated in favor of max_completion_tokens, and is not compatible with o-series models"; max_completion_tokens is "an upper bound … including visible output tokens and reasoning tokens".)

Approach — additive, non-breaking

maxTokens is left intact and unchanged:

  • Backward compatible — existing ModelConfigs behave identically.
  • OpenAI-compatible endpointsBaseOpenAI also serves baseUrl-overridden OpenAI-compatible servers, some of which still expect max_tokens. Keeping both lets operators choose the right one per model.

The two fields are independent; set maxCompletionTokens for reasoning models, maxTokens for legacy/compatible endpoints.

Changes

  • OpenAIConfig.MaxCompletionTokens (v1alpha2), +kubebuilder:validation:Minimum=1; CRDs regenerated (bases + Helm).
  • adk.OpenAI serialization struct carries max_completion_tokens.
  • Controller translator copies Spec.OpenAI.MaxCompletionTokens.
  • Python ADK OpenAI client emits max_completion_tokens; OpenAI config model gains the field (ge=1).
  • Go ADK OpenAI model (applyOpenAIConfig + CreateLLM) emits max_completion_tokens via the openai-go SDK.

Tests

  • Go translator test: Test_AdkApiTranslator_OpenAIMaxCompletionTokens — verifies the field flows through and max_tokens stays unset.
  • Go ADK apply test: TestApplyOpenAIConfig/config_with_max_completion_tokens.
  • Python client test: test_generate_content_async_with_max_completion_tokens — asserts max_completion_tokens is forwarded and max_tokens is absent.

All pass; go build ./api/... ./adk/... ./core/internal/controller/... is green and CRD generation is clean.

Note

Scope is the native OpenAI provider (the one that hits api.openai.com). Azure OpenAI reasoning deployments have the same underlying need; happy to extend AzureOpenAIConfig in a follow-up if maintainers prefer.

OpenAI's `max_tokens` request parameter is deprecated for the Chat
Completions API and is rejected outright by reasoning models (GPT-5 /
o-series):

  400 Bad Request: Unsupported parameter: 'max_tokens' is not supported
  with this model. Use 'max_completion_tokens' instead.

kagent's OpenAIConfig only exposed maxTokens (-> max_tokens), so there was
no way to cap output for those models. Add a sibling maxCompletionTokens
field that maps to the OpenAI `max_completion_tokens` parameter.

maxTokens is left intact and unchanged for backward compatibility and for
OpenAI-compatible endpoints (baseUrl) that still expect max_tokens; the two
are independent, so existing configs are unaffected.

Wired end to end:
- add MaxCompletionTokens to OpenAIConfig (v1alpha2, +kubebuilder Minimum=1)
  and regenerate the CRDs
- carry it on the adk.OpenAI serialization struct
- copy Spec.OpenAI.MaxCompletionTokens in the controller translator
- emit max_completion_tokens in the Python ADK OpenAI client and the Go ADK
  OpenAI model client

Adds Go translator + Go ADK tests and a Python client test.

Signed-off-by: anthonytwh <anthony.tam@tigera.io>
Copilot AI review requested due to automatic review settings July 7, 2026 17:28
@anthonytwh anthonytwh requested review from a team and supreme-gg-gg as code owners July 7, 2026 17:28
@github-actions github-actions Bot added the enhancement New feature or request label Jul 7, 2026

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds support for OpenAI’s max_completion_tokens parameter end-to-end (CRD/API → controller translation → ADK serialization → Python/Go clients), enabling token caps on reasoning models that reject the deprecated max_tokens.

Changes:

  • Introduces maxCompletionTokens in the Kubernetes OpenAIConfig (v1alpha2) with validation and regenerated CRDs/Helm templates.
  • Wires the new field through the controller translator and ADK model serialization for Go.
  • Adds Python and Go client support (plus tests) to emit max_completion_tokens on OpenAI chat completion requests.

Reviewed changes

Copilot reviewed 13 out of 13 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
python/packages/kagent-adk/tests/unittests/models/test_openai.py Adds a unit test asserting max_completion_tokens is forwarded and max_tokens is absent.
python/packages/kagent-adk/src/kagent/adk/types.py Extends the Python OpenAI config model and config→LLM factory to include max_completion_tokens.
python/packages/kagent-adk/src/kagent/adk/models/_openai.py Adds request serialization of max_completion_tokens in the OpenAI client wrapper.
helm/kagent-crds/templates/kagent.dev_modelconfigs.yaml Updates Helm-rendered CRD schema/docs for maxCompletionTokens and clarifies maxTokens deprecation.
go/core/internal/controller/translator/agent/adk_api_translator.go Copies Spec.OpenAI.MaxCompletionTokens into the ADK OpenAI model config.
go/core/internal/controller/translator/agent/adk_api_translator_test.go Adds coverage verifying maxCompletionTokens propagates and maxTokens remains unset when not configured.
go/api/v1alpha2/modelconfig_types.go Adds MaxCompletionTokens to the CRD API type with Minimum=1 validation and updated field docs.
go/api/config/crd/bases/kagent.dev_modelconfigs.yaml Regenerates base CRD schema/docs to include maxCompletionTokens.
go/api/adk/types.go Extends the ADK OpenAI serialization struct to include max_completion_tokens.
go/adk/pkg/models/openai.go Extends the Go ADK OpenAI config struct to carry MaxCompletionTokens.
go/adk/pkg/models/openai_adk.go Applies MaxCompletionTokens to openai-go ChatCompletionNewParams.
go/adk/pkg/models/openai_adk_test.go Adds a unit test for applyOpenAIConfig setting MaxCompletionTokens.
go/adk/pkg/agent/agent.go Threads MaxCompletionTokens from the ADK model into the Go OpenAI config used by the runtime.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +465 to +468
if self.max_tokens:
kwargs["max_tokens"] = self.max_tokens
if self.max_completion_tokens:
kwargs["max_completion_tokens"] = self.max_completion_tokens
Comment thread go/adk/pkg/models/openai_adk.go Outdated
Comment on lines +174 to +179
if cfg.MaxTokens != nil {
params.MaxTokens = openai.Int(int64(*cfg.MaxTokens))
}
if cfg.MaxCompletionTokens != nil {
params.MaxCompletionTokens = openai.Int(int64(*cfg.MaxCompletionTokens))
}
…xclusive

Addresses review feedback: if a ModelConfig set both maxTokens and
maxCompletionTokens, the client sent both request parameters. They are
mutually exclusive on the OpenAI API — reasoning models (GPT-5 / o-series)
reject max_tokens, and some OpenAI-compatible endpoints only accept
max_tokens — so sending both risks a hard 400 and makes migration brittle.

Send exactly one at request-build time, with max_completion_tokens (the
modern superset parameter) taking precedence when both are configured, in
both the Python and Go ADK OpenAI clients. Single-field configs are
unaffected. Adds precedence tests for both runtimes.

Signed-off-by: anthonytwh <anthony.tam@tigera.io>
@anthonytwh

Copy link
Copy Markdown
Author

Good catch @copilot — fixed in 0b813ae. maxTokens and maxCompletionTokens are now mutually exclusive at request-build time in both the Python (_openai.py) and Go (openai_adk.go) clients: exactly one parameter is sent, with max_completion_tokens taking precedence when both are configured.

I went with modern-parameter precedence rather than the base_url-based selection, because base_url being overridden doesn't reliably indicate a non-OpenAI backend — it's commonly a proxy/gateway that fronts the real OpenAI API (which still needs max_completion_tokens for reasoning models). Precedence is deterministic and never sends both. Single-field configs — including compat endpoints that only set maxTokens — are unchanged. Added precedence tests for both runtimes.

If you'd prefer an even stronger guard, I can also add a CRD-level CEL validation rejecting configs that set both fields — happy to include that if maintainers want it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants