Feat: strip unsupported sampling params for models flagged shouldSkipTemperature#73
Closed
cosminacho wants to merge 8 commits into
Closed
Feat: strip unsupported sampling params for models flagged shouldSkipTemperature#73cosminacho wants to merge 8 commits into
cosminacho wants to merge 8 commits into
Conversation
Anthropic's extended thinking API requires temperature=1 (its default) and rejects any other explicit value alongside a thinking config. _default_params now silently removes temperature when thinking is present so callers don't have to manually avoid the combination. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…thropicBedrock Claude Opus 4+ (e.g. anthropic.claude-opus-4-7) are reasoning models that reject temperature, top_k, and top_p with a 400 Bad Request. Override _get_request_payload in UiPathChatAnthropicBedrock to pop those params when the model name matches the claude-opus-4 pattern. Params are untouched for all other models. Also add anthropic.claude-opus-4-7 to the bedrock integration test matrix (UiPathChatAnthropicBedrock, non-thinking configs only, as thinking cassettes are not yet recorded for this model). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
… Anthropic clients
Move detection logic and constant to core (model_family.py) so a single
definition covers all LangChain clients.
- is_claude_opus_4_or_above() — core model_family.py
- CLAUDE_OPUS_4_UNSUPPORTED_SAMPLING_PARAMS — core model_family.py
- Re-exported from uipath_langchain_client.utils
Per-client strip strategy:
- UiPathChatAnthropic → _get_request_payload override
- UiPathChatAnthropicBedrock → _get_request_payload override (import from core)
- UiPathChatBedrockConverse → _converse_params override (strips temperature/topP
from inferenceConfig)
- UiPathChatBedrock (INVOKE) → model_validator (nulls self.temperature,
filters model_kwargs)
- UiPathChat (normalized) → _default_params (strips + existing thinking check)
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
427d8ed to
3e6aefa
Compare
…matrix Both models match is_claude_opus_4_or_above — sampling params are stripped before the API call. Integration tests (live, 78 passed) confirm the fix works correctly for all three Opus 4 variants. Unit test extended to assert the pattern also matches the new model IDs. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Live testing confirmed only anthropic.claude-opus-4-7 rejects temperature/top_k/top_p. claude-opus-4-5 and claude-opus-4-6 accept sampling parameters normally. Narrow is_claude_opus_4_or_above regex from claude-opus-4 to claude-opus-4-7 and update docstring, unit test assertions, and conftest comments accordingly. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…ampling params Add _should_skip_sampling_params cached_property to UiPathBaseLLMClient. It reads shouldSkipTemperature from get_model_info() (authoritative) and falls back to the is_claude_opus_4_or_above() name heuristic when the model is not found in discovery. All five Anthropic clients now use self._should_skip_sampling_params instead of calling is_claude_opus_4_or_above directly, so the logic automatically adapts when the backend sets shouldSkipTemperature on new models without requiring a code change. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Replace the five per-client overrides with a single mechanism in
UiPathBaseLLMClient / UiPathBaseChatModel:
- model_details: dict | None — ctor field; if None, resolved lazily
from get_model_info(). Lets callers skip the discovery call.
- resolved_model_details / _should_skip_sampling_params — cached
properties; the latter reads modelDetails.shouldSkipTemperature with
a name-based fallback (is_claude_opus_4_or_above).
Stripping happens at both sites:
1. model_validator(mode="after") — nulls self.temperature/top_k/top_p
AND discards them from __pydantic_fields_set__ so payload builders
that filter on model_fields_set (UiPathChat) treat them as unset.
Handles UiPathChat(model=..., temperature=0.6).
2. _generate/_agenerate/_stream/_astream — pop the same keys from
invocation kwargs. Handles llm.invoke("msg", temperature=0.7) and
llm.bind(temperature=0.7).
Per-client overrides removed:
- UiPathChatAnthropic._get_request_payload
- UiPathChatAnthropicBedrock._get_request_payload
- UiPathChatBedrock setup_uipath_client sampling-param logic
- UiPathChat._default_params sampling-param logic (thinking rule kept)
UiPathChatBedrockConverse._converse_params kept but simplified to always
filter None temperature/topP from inferenceConfig defensively (so boto3
doesn't serialize self.temperature=None as explicit null to the wire).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
get_chat_model and get_embedding_model now extract modelDetails from the model_info they already fetch and pipe it into the constructed client's model_details kwarg (via setdefault so caller overrides still win). This means UiPathBaseLLMClient.resolved_model_details no longer has to lazy-fetch discovery on first access for factory-built clients, and the shouldSkipTemperature flag is immediately available to the model_validator that strips sampling params at init. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Claude Opus 4.7 (and any future model that sets
modelDetails.shouldSkipTemperature: truein discovery) rejectstemperature/top_k/top_pwith400 Bad Request. Callers must not be forced to know which models are reasoning-only — the clients should strip these params transparently.Solution
Single mechanism on
UiPathBaseLLMClient/UiPathBaseChatModel:UiPathBaseLLMClient.model_details: dict | None— optional ctor field carrying the discoverymodelDetailsdict. If omitted,resolved_model_detailslazy-fetches viaget_model_info().UiPathBaseLLMClient._should_skip_sampling_params— readsmodelDetails.shouldSkipTemperature, with a name-based fallback (is_claude_opus_4_or_above) for models not in discovery.UiPathBaseChatModel._strip_unsupported_sampling_params_at_init(model_validator(mode="after")) — nullstemperature/top_k/top_pAND discards them from__pydantic_fields_set__, soUiPathChat(model="...", temperature=0.6)doesn't leak the value to payload builders that key onmodel_fields_set.UiPathBaseChatModel._strip_sampling_from_kwargs— called inside_generate/_agenerate/_stream/_astreamto pop the same keys from invocation-time kwargs. Handlesllm.invoke("msg", temperature=0.7)and.bind(temperature=0.7).Factory wiring (this branch's final commit):
get_chat_model/get_embedding_modelextractmodelDetailsfrom themodel_infothey already fetch and pass it through asmodel_details(viasetdefault, so caller override wins). Skips the redundant lazy-fetch insideresolved_model_details.All five per-client sampling-param overrides removed.
UiPathChat._default_paramsstill dropstemperaturewhenthinkingis set (Anthropic extended thinking requirestemperature=1, its default).UiPathChatBedrockConverse._converse_paramsdefensively filtersNonetemperature/topPfrominferenceConfigso boto3 doesn't serialize them as explicit nulls.Core additions (
uipath.llm_client.utils.model_family):is_claude_opus_4_or_above(model_name)— name-based fallback for models missing from discoveryCLAUDE_OPUS_4_UNSUPPORTED_SAMPLING_PARAMS = frozenset({"temperature", "top_k", "top_p"})Which packages
model_family.pyhelpers)Test plan
ruff check/ruff format --check/pyrightpytest tests— 1645 passed, 769 skipped, 9 xpassedTestSamplingParamStrippingintests/langchain/clients/bedrock/test_unit.pycovers ctor-time stripping, invocation-time stripping, and supported-model passthroughtests/langchain/features/test_factory_function.pyassertmodelDetailspropagates from discovery to client ctor and that caller overrides win🤖 Generated with Claude Code