Skip to content

Feat: strip unsupported sampling params for models flagged shouldSkipTemperature#73

Closed
cosminacho wants to merge 8 commits into
mainfrom
fix/uipath-chat-thinking-drops-temperature
Closed

Feat: strip unsupported sampling params for models flagged shouldSkipTemperature#73
cosminacho wants to merge 8 commits into
mainfrom
fix/uipath-chat-thinking-drops-temperature

Conversation

@cosminacho
Copy link
Copy Markdown
Collaborator

@cosminacho cosminacho commented Apr 23, 2026

Problem

Claude Opus 4.7 (and any future model that sets modelDetails.shouldSkipTemperature: true in discovery) rejects temperature / top_k / top_p with 400 Bad Request. Callers must not be forced to know which models are reasoning-only — the clients should strip these params transparently.

Solution

Single mechanism on UiPathBaseLLMClient / UiPathBaseChatModel:

  • UiPathBaseLLMClient.model_details: dict | None — optional ctor field carrying the discovery modelDetails dict. If omitted, resolved_model_details lazy-fetches via get_model_info().
  • UiPathBaseLLMClient._should_skip_sampling_params — reads modelDetails.shouldSkipTemperature, with a name-based fallback (is_claude_opus_4_or_above) for models not in discovery.
  • UiPathBaseChatModel._strip_unsupported_sampling_params_at_init (model_validator(mode="after")) — nulls temperature / top_k / top_p AND discards them from __pydantic_fields_set__, so UiPathChat(model="...", temperature=0.6) doesn't leak the value to payload builders that key on model_fields_set.
  • UiPathBaseChatModel._strip_sampling_from_kwargs — called inside _generate / _agenerate / _stream / _astream to pop the same keys from invocation-time kwargs. Handles llm.invoke("msg", temperature=0.7) and .bind(temperature=0.7).

Factory wiring (this branch's final commit):

  • get_chat_model / get_embedding_model extract modelDetails from the model_info they already fetch and pass it through as model_details (via setdefault, so caller override wins). Skips the redundant lazy-fetch inside resolved_model_details.

All five per-client sampling-param overrides removed.

UiPathChat._default_params still drops temperature when thinking is set (Anthropic extended thinking requires temperature=1, its default).

UiPathChatBedrockConverse._converse_params defensively filters None temperature / topP from inferenceConfig so boto3 doesn't serialize them as explicit nulls.

Core additions (uipath.llm_client.utils.model_family):

  • is_claude_opus_4_or_above(model_name) — name-based fallback for models missing from discovery
  • CLAUDE_OPUS_4_UNSUPPORTED_SAMPLING_PARAMS = frozenset({"temperature", "top_k", "top_p"})

Which packages

  • Core → 1.9.10 (model_family.py helpers)
  • Langchain → 1.9.9 (base-client refactor + factory piping)

Test plan

  • ruff check / ruff format --check / pyright
  • pytest tests — 1645 passed, 769 skipped, 9 xpassed
  • TestSamplingParamStripping in tests/langchain/clients/bedrock/test_unit.py covers ctor-time stripping, invocation-time stripping, and supported-model passthrough
  • New factory tests in tests/langchain/features/test_factory_function.py assert modelDetails propagates from discovery to client ctor and that caller overrides win

🤖 Generated with Claude Code

@cosminacho cosminacho changed the title Fix: drop temperature from UiPathChat payload when thinking is set Fix: strip unsupported sampling params for claude-opus-4+ models Apr 23, 2026
cosminacho and others added 3 commits April 23, 2026 14:48
Anthropic's extended thinking API requires temperature=1 (its default)
and rejects any other explicit value alongside a thinking config.
_default_params now silently removes temperature when thinking is
present so callers don't have to manually avoid the combination.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…thropicBedrock

Claude Opus 4+ (e.g. anthropic.claude-opus-4-7) are reasoning models
that reject temperature, top_k, and top_p with a 400 Bad Request.

Override _get_request_payload in UiPathChatAnthropicBedrock to pop
those params when the model name matches the claude-opus-4 pattern.
Params are untouched for all other models.

Also add anthropic.claude-opus-4-7 to the bedrock integration test
matrix (UiPathChatAnthropicBedrock, non-thinking configs only, as
thinking cassettes are not yet recorded for this model).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… Anthropic clients

Move detection logic and constant to core (model_family.py) so a single
definition covers all LangChain clients.

  - is_claude_opus_4_or_above()           — core model_family.py
  - CLAUDE_OPUS_4_UNSUPPORTED_SAMPLING_PARAMS  — core model_family.py
  - Re-exported from uipath_langchain_client.utils

Per-client strip strategy:
  - UiPathChatAnthropic          → _get_request_payload override
  - UiPathChatAnthropicBedrock   → _get_request_payload override (import from core)
  - UiPathChatBedrockConverse     → _converse_params override (strips temperature/topP
                                    from inferenceConfig)
  - UiPathChatBedrock (INVOKE)   → model_validator (nulls self.temperature,
                                    filters model_kwargs)
  - UiPathChat (normalized)      → _default_params (strips + existing thinking check)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cosminacho cosminacho force-pushed the fix/uipath-chat-thinking-drops-temperature branch from 427d8ed to 3e6aefa Compare April 23, 2026 11:51
@cosminacho cosminacho changed the title Fix: strip unsupported sampling params for claude-opus-4+ models Feat: strip unsupported sampling params for claude-opus-4+ across all Anthropic clients Apr 23, 2026
…matrix

Both models match is_claude_opus_4_or_above — sampling params are stripped
before the API call. Integration tests (live, 78 passed) confirm the fix
works correctly for all three Opus 4 variants.

Unit test extended to assert the pattern also matches the new model IDs.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Live testing confirmed only anthropic.claude-opus-4-7 rejects
temperature/top_k/top_p. claude-opus-4-5 and claude-opus-4-6 accept
sampling parameters normally.

Narrow is_claude_opus_4_or_above regex from claude-opus-4 to
claude-opus-4-7 and update docstring, unit test assertions, and
conftest comments accordingly.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
cosminacho and others added 2 commits April 23, 2026 15:35
…ampling params

Add _should_skip_sampling_params cached_property to UiPathBaseLLMClient.
It reads shouldSkipTemperature from get_model_info() (authoritative) and
falls back to the is_claude_opus_4_or_above() name heuristic when the
model is not found in discovery.

All five Anthropic clients now use self._should_skip_sampling_params
instead of calling is_claude_opus_4_or_above directly, so the logic
automatically adapts when the backend sets shouldSkipTemperature on
new models without requiring a code change.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the five per-client overrides with a single mechanism in
UiPathBaseLLMClient / UiPathBaseChatModel:

  - model_details: dict | None — ctor field; if None, resolved lazily
    from get_model_info(). Lets callers skip the discovery call.
  - resolved_model_details / _should_skip_sampling_params — cached
    properties; the latter reads modelDetails.shouldSkipTemperature with
    a name-based fallback (is_claude_opus_4_or_above).

Stripping happens at both sites:
  1. model_validator(mode="after") — nulls self.temperature/top_k/top_p
     AND discards them from __pydantic_fields_set__ so payload builders
     that filter on model_fields_set (UiPathChat) treat them as unset.
     Handles UiPathChat(model=..., temperature=0.6).
  2. _generate/_agenerate/_stream/_astream — pop the same keys from
     invocation kwargs. Handles llm.invoke("msg", temperature=0.7) and
     llm.bind(temperature=0.7).

Per-client overrides removed:
  - UiPathChatAnthropic._get_request_payload
  - UiPathChatAnthropicBedrock._get_request_payload
  - UiPathChatBedrock setup_uipath_client sampling-param logic
  - UiPathChat._default_params sampling-param logic (thinking rule kept)

UiPathChatBedrockConverse._converse_params kept but simplified to always
filter None temperature/topP from inferenceConfig defensively (so boto3
doesn't serialize self.temperature=None as explicit null to the wire).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
get_chat_model and get_embedding_model now extract modelDetails from the
model_info they already fetch and pipe it into the constructed client's
model_details kwarg (via setdefault so caller overrides still win).

This means UiPathBaseLLMClient.resolved_model_details no longer has to
lazy-fetch discovery on first access for factory-built clients, and the
shouldSkipTemperature flag is immediately available to the
model_validator that strips sampling params at init.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@cosminacho cosminacho changed the title Feat: strip unsupported sampling params for claude-opus-4+ across all Anthropic clients Feat: strip unsupported sampling params for models flagged shouldSkipTemperature Apr 23, 2026
@cosminacho cosminacho closed this Apr 23, 2026
@cosminacho cosminacho deleted the fix/uipath-chat-thinking-drops-temperature branch April 29, 2026 18:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant