chore(evals): converge harness SDK pins onto the workspace catalog - #2743
chore(evals): converge harness SDK pins onto the workspace catalog#2743miguelg719 wants to merge 1 commit into
Conversation
PRs #2719/#2720 added @anthropic-ai/claude-agent-sdk (0.3.224) and @openai/codex-sdk (0.147.0) to the pnpm catalog for the integration examples, forking the versions evals pinned directly (^0.2.141 and 0.125.0). Move both evals deps to catalog: so the monorepo tracks a single version per SDK. Compat verified against the installed types: every query() option the claude_code harness passes (canUseTool, systemPrompt preset+append, settingSources, mcpServers, createSdkMcpServer/tool) exists in 0.3.224, and every codex option (startThread config, runStreamed outputSchema/ signal, constructor config bag incl. show_raw_agent_reasoning) exists in 0.147.0 with unchanged ThreadEvent kinds. Evals unit suite passes (420 tests).
|
|
Connected smokes now verified with real credentials:
TypeScript unit CI failure is unrelated and repo-wide: |
There was a problem hiding this comment.
No issues found across 2 files
Confidence score: 5/5
- Automated review surfaced no issues in the provided summaries.
- No files require special attention.
Architecture diagram
sequenceDiagram
participant EV as @browserbasehq/stagehand-evals
participant WS as Workspace Catalog
participant CL as claude-agent-sdk
participant CX as codex-sdk
participant PKG as Package Manager (turbo/pnpm)
Note over EV,PKG: Dependency Resolution Flow
EV->>WS: Declare @anthropic-ai/claude-agent-sdk: "catalog:"
EV->>WS: Declare @openai/codex-sdk: "catalog:"
WS-->>PKG: Resolve to claude-agent-sdk 0.3.224
WS-->>PKG: Resolve to codex-sdk 0.147.0
PKG->>CL: Pin to 0.3.224 (shared with integration examples)
PKG->>CX: Pin to 0.147.0 (shared with integration examples)
Note over EV,PKG: Build & Type Verification
EV->>PKG: turbo build (evals)
PKG->>CL: Compile-time imports
PKG->>CX: Compile-time imports
CL-->>PKG: Installed .d.ts (0.3.224)
CX-->>PKG: Installed .d.ts (0.147.0)
PKG->>PKG: Verify structural types (dynamic import)
Note over EV: Runtime harness adapter flow (unchanged behavior)
EV->>CL: query() with options (canUseTool, systemPrompt, mcpServers, pathToClaudeCodeExecutable)
CL-->>EV: Streamed events (unchanged)
EV->>CX: startThread({sandboxMode, approvalPolicy, networkAccessEnabled, webSearchMode, skipGitRepoCheck})
CX-->>EV: ThreadEvent kinds (unchanged)
Note over EV: Downstream dependency graph
CL->>PKG: Type-level import (constructor config, createSdkMcpServer)
CX->>PKG: Type-level import (runStreamed with outputSchema, signal)
What
Moves both external-harness SDK deps in
packages/evalsonto the workspace catalog:@anthropic-ai/claude-agent-sdk:^0.2.141→catalog:(0.3.224)@openai/codex-sdk:0.125.0→catalog:(0.147.0)Why
#2719 and #2720 added both SDKs to the catalog for the integration examples, forking the versions evals pinned directly — two copies of each SDK in the monorepo. This converges on one version per SDK, as a standalone pre-step to the broader harness-adapter consolidation.
Compat verification
Evals types both SDKs structurally (dynamic import), so
tsccannot catch drift — verified against the installed.d.tsinstead:query()option evals passes exists —canUseTool(updatedInputnow optional),systemPromptpreset+append,settingSources,mcpServers,pathToClaudeCodeExecutable,createSdkMcpServer/tool. Publish history shows 0.3.142 directly succeeds 0.2.141 (version-scheme continuation).startThreadoptions (sandboxMode,approvalPolicy,networkAccessEnabled,webSearchMode,skipGitRepoCheck),runStreamed({outputSchema, signal}), constructorconfigbag (arbitrary--configoverrides, soshow_raw_agent_reasoningstill flows) all present;ThreadEventkinds unchanged.Testing
turbo run build typecheck --filter @browserbasehq/stagehand-evals✅evals run b:webvoyager --harness claude_code|codex --tool stagehand_code -l 1 -e browserbase): pending — blocked on Browserbase credentials in the runner env; both attempts failed atStagehand initbefore any SDK code executed. Draft until the smoke passes.Summary by cubic
Converges
packages/evalsharness SDK dependencies to the workspace catalog to remove duplicate SDK pins and align with the integration examples. Old behavior: evals pinned@anthropic-ai/claude-agent-sdk^0.2.141 and@openai/codex-sdk0.125.0; new behavior: both usecatalog:(0.3.224 and 0.147.0). Side effect: lockfile updates only; no runtime changes expected.claude-agent-sdk@0.3.224retains allquery()options used by evals;codex-sdk@0.147.0retainsstartThread,runStreamed, and constructor config behaviors with unchangedThreadEventkinds.Written for commit ae1ccc8. Summary will update on new commits.