[fix]: support custom model base URL and legacy chatcompletions endpoints#1871
Open
[fix]: support custom model base URL and legacy chatcompletions endpoints#1871
Conversation
🦋 Changeset detectedLatest commit: 04dcf91 The changes in this PR will be included in the next version bump. This PR includes changesets to release 5 packages
Not sure what this means? Click here to learn what changesets are. Click here if you're a maintainer who wants to add another changeset to this PR |
shrey150
changed the title
Thread modelBaseURL from x-model-base-url header through to V3 options, enabling providers like ZhipuAI, Ollama, and other OpenAI-compatible endpoints. Uses Chat Completions API (not Responses API) when a custom baseURL is set, and adds robust response coercion for models without native structured output support.
Adds "chatcompletions" as a generic provider that uses the Chat Completions API (/chat/completions) instead of the Responses API, for endpoints like ZhipuAI and Ollama. Also simplifies response coercion for models without native structured output support.
Try structured output (schema:) first for all models. Only fall back to no-schema + response coercion when the call fails and the model matches a known fallback pattern. This avoids degrading DeepSeek/Kimi which already work with schema:.
- Skip schema attempt for chatcompletions/ models (provider: openai.chat) since they can't do structured output — avoids a wasted LLM call per extract - Unify .chat() handling in getAISDKLanguageModel so chatcompletions/ works regardless of whether clientOptions are provided - Guard second schema.parse() with safeParse + descriptive error message
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
7 issues found across 26 files
Confidence score: 3/5
- There is concrete regression risk in
packages/core/lib/v3/llm/aisdk.ts: a raw upstream error is rethrown instead of a sanitized typed error, which can leak unsanitized messages and break expected error handling paths. packages/core/lib/v3/llm/aisdk.tsalso has a likely runtime-failure path (Object.entriesonnoSchemaResponse.objectwithout guarding non-object output), plus broad string-to-JSON parsing that may corrupt valid string fields.- Test reliability concerns are present in
packages/server-v3/test/integration/v3/extract.test.tsandpackages/server-v4/test/integration/v4/extract.test.ts, where startup failures can bypass cleanup and leak servers/Chrome processes; this is risky but mostly scoped to test/integration behavior. - Pay close attention to
packages/core/lib/v3/llm/aisdk.ts,packages/server-v3/test/integration/v3/extract.test.ts,packages/server-v4/test/integration/v4/extract.test.ts,packages/core/tests/unit/llm-provider.test.ts- error sanitization/runtime guards and cleanup/test coverage gaps are the main risk drivers.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="packages/core/lib/v3/llm/aisdk.ts">
<violation number="1" location="packages/core/lib/v3/llm/aisdk.ts:248">
P1: Custom agent: **Exception and error message sanitization**
Rule 1 violation: this rethrows a raw upstream error instead of throwing a sanitized typed error class. Wrap it in `CreateChatCompletionResponseError` (or another typed sanitized SDK error) before raising it.</violation>
<violation number="2" location="packages/core/lib/v3/llm/aisdk.ts:264">
P1: Guard `noSchemaResponse.object` before calling `Object.entries` to avoid runtime crashes on non-object model output.</violation>
<violation number="3" location="packages/core/lib/v3/llm/aisdk.ts:268">
P2: Only parse string fields that look like JSON containers; parsing all strings can corrupt valid string values.</violation>
</file>
<file name="packages/core/tests/unit/llm-provider.test.ts">
<violation number="1" location="packages/core/tests/unit/llm-provider.test.ts:68">
P2: The custom `baseURL` test does not validate `baseURL` behavior; it only repeats the same assertions as the no-options case, so regressions in forwarding `clientOptions.baseURL` would go undetected.
(Based on your team's feedback about adding focused unit tests for new behavior.) [FEEDBACK_USED]</violation>
</file>
<file name="packages/server-v3/test/integration/v3/extract.test.ts">
<violation number="1" location="packages/server-v3/test/integration/v3/extract.test.ts:500">
P2: Resource acquisition happens before the `try/finally`, so startup failures can bypass cleanup and leak test processes/servers.</violation>
</file>
<file name="packages/server-v4/test/integration/v4/extract.test.ts">
<violation number="1" location="packages/server-v4/test/integration/v4/extract.test.ts:142">
P2: Add failure-path cleanup in `startLocalChromeWithCdp`; rejected startup currently leaks spawned Chrome/temp-dir resources.</violation>
<violation number="2" location="packages/server-v4/test/integration/v4/extract.test.ts:500">
P2: Move resource startup into the guarded cleanup flow; a startup failure currently skips `finally` and leaks the fake chat server.</violation>
</file>
Architecture diagram
sequenceDiagram
participant SDK as Stagehand SDK
participant API as API Client (V3)
participant Srv as Stagehand Server (v3/v4)
participant Core as LLMProvider (Core)
participant AISDK as AI SDK Logic
participant LLM as OpenAI-Compatible API
Note over SDK,LLM: NEW: Request Flow with Custom Base URL and chatcompletions Provider
SDK->>API: init(modelBaseURL, modelName: "chatcompletions/...")
API->>Srv: POST /sessions/start
Note right of API: NEW: Includes x-model-base-url header
Srv->>Srv: CHANGED: Extract modelBaseURL from header/body
Srv-->>API: 201 Created (Session ID)
API-->>SDK: Session Initialized
Note over SDK,LLM: Runtime Execution (e.g., extract())
SDK->>Srv: POST /sessions/:id/extract
Srv->>Core: getAISDKLanguageModel(provider, model, options)
alt NEW: Provider is "chatcompletions"
Core->>Core: Map to createOpenAI() with custom baseURL
Core-->>Srv: Return OpenAI instance in .chat mode
end
Srv->>AISDK: generateObject(model, schema, messages)
alt NEW: Model is "chatcompletions" or Structured Output Fails
AISDK->>LLM: NEW: Fetch via /chat/completions (no-schema)
LLM-->>AISDK: Raw JSON String
AISDK->>AISDK: NEW: Fix stringified values (JSON.parse)
AISDK->>AISDK: NEW: Coerce missing arrays to []
alt Validation Success
AISDK-->>Srv: Validated Object
else Validation Failure
AISDK-->>Srv: Throw CreateChatCompletionResponseError
end
else Standard Provider
AISDK->>LLM: Standard Structured Output Call
LLM-->>AISDK: JSON Object
AISDK-->>Srv: Validated Object
end
Srv-->>SDK: SSE / JSON Response (Extracted Data)
Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.
| expect((model as { provider?: string }).provider).toBe("openai.chat"); | ||
| }); | ||
|
|
||
| it("uses the OpenAI chat provider with custom baseURL", () => { |
Contributor
There was a problem hiding this comment.
P2: The custom baseURL test does not validate baseURL behavior; it only repeats the same assertions as the no-options case, so regressions in forwarding clientOptions.baseURL would go undetected.
(Based on your team's feedback about adding focused unit tests for new behavior.)
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At packages/core/tests/unit/llm-provider.test.ts, line 68:
<comment>The custom `baseURL` test does not validate `baseURL` behavior; it only repeats the same assertions as the no-options case, so regressions in forwarding `clientOptions.baseURL` would go undetected.
(Based on your team's feedback about adding focused unit tests for new behavior.) </comment>
<file context>
@@ -57,6 +57,24 @@ describe("getAISDKLanguageModel", () => {
+ expect((model as { provider?: string }).provider).toBe("openai.chat");
+ });
+
+ it("uses the OpenAI chat provider with custom baseURL", () => {
+ const model = getAISDKLanguageModel("chatcompletions", "glm-4-flash", {
+ baseURL: "https://open.bigmodel.cn/api/paas/v4",
</file context>
…PI specs, and stainless config
shrey150
force-pushed
the
pr1804-analysis
branch
from
fc637ee to
e7623d8
Compare
Contributor
✱ Stainless preview buildsThis PR will update the Edit this comment to update it. It will appear in the SDK's changelogs.
|
shrey150
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
why
Generated Stagehand API / Stainless SDK users can already configure custom model settings locally, but custom OpenAI-compatible base URLs were not fully carried through the Stagehand API path. That makes it hard to use customer-provided endpoints, especially providers that only expose legacy
/chat/completions.This branch repackages Chris Read's
model-base-urlwork on top of currentmainand includes the follow-up fixes I found while validating it.what changed
chatcompletions/<model>so OpenAI-compatible requests can explicitly use the Chat Completions pathmodel_base_url/x-model-base-urlthrough the generated Stagehand API + SDK path so API/SDK users can target custom OpenAI-compatible endpointschatcompletionsmodel construction with custombaseURLchatcompletionssession start andx-model-base-urlextract requestsmainwithout restoring deleted oldserver-v4/sessions/*runtime/test files; keep the still-relevant shared API/header/OpenAPI/Stainless propagation that survives on currentmaintest plan
pnpm --filter @browserbasehq/stagehand build:esmpnpm --filter @browserbasehq/stagehand typecheckpnpm --filter @browserbasehq/stagehand test:core -- packages/core/dist/esm/tests/unit/llm-provider.test.jspnpm --filter @browserbasehq/stagehand-server-v3 buildpnpm --filter @browserbasehq/stagehand-server-v3 typecheckpnpm --filter @browserbasehq/stagehand-server-v4 buildpnpm --filter @browserbasehq/stagehand-server-v4 typecheckOPENAI_API_KEY=test-key CHROME_PATH='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome' STAGEHAND_SERVER_TARGET=local STAGEHAND_BASE_URL='http://127.0.0.1:3157' pnpm --filter @browserbasehq/stagehand-server-v3 test:server -- packages/server-v3/dist/tests/integration/v3/start.test.js -- --test-name-pattern='accept chatcompletions-prefixed model names'OPENAI_API_KEY=test-key CHROME_PATH='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome' STAGEHAND_SERVER_TARGET=local STAGEHAND_BASE_URL='http://127.0.0.1:3158' pnpm --filter @browserbasehq/stagehand-server-v3 test:server -- packages/server-v3/dist/tests/integration/v3/extract.test.js -- --test-name-pattern='use x-model-base-url for chatcompletions extract requests'