feat(evals): add LangSmith gating helpers + config (no behavior change)

[miguelg719]

why

First step of an opt-in OTEL trace transport that fans eval traces out to Braintrust and LangSmith. This PR adds only the gating/config surface — zero behavior change.

what changed

• framework/langsmith.ts (new): hasLangSmithApiKey, langSmithTracingEnabled (key + LANGSMITH_TRACING=true), lazy loadLangSmith, assertLangSmithReady, resolveTraceTransport() (default native)
• langsmith dependency; LANGSMITH_API_KEY rows in doctor + welcome status (incl. EVAL_TRACE_PRIMARY default fix); .env.example block; CONTRACT.md (invariants for the stack)

test plan

• Full unit suite green (358/358), incl. tests/tui/doctor.test.ts
• New module is imported by nothing → no hot-path change

Stack 1/5 → next: #2352

---

Summary by cubic

Adds LangSmith gating and config for an opt-in trace transport that can send eval traces to Braintrust and LangSmith. No runtime behavior changes; default transport is native, and experiments still require Braintrust.

• New Features
  • New framework/langsmith.ts helpers: hasLangSmithApiKey, langSmithTracingEnabled (reads process.env at call time), resolveTraceTransport (native | otel, default native), lazy loadLangSmith, assertLangSmithReady.
  • doctor and welcome status surface LANGSMITH_API_KEY; doctor warns when no experiment backend is set, when the selected EVAL_TRACE_PRIMARY key is missing, when EVAL_TRACE_PRIMARY is unrecognized, and when EVAL_TRACE_PRIMARY=langsmith but LANGSMITH_TRACING is not "true".
  • .env.example adds LANGSMITH_API_KEY, LANGSMITH_TRACING, EVAL_TRACE_TRANSPORT, EVAL_TRACE_PRIMARY (default braintrust).
  • Added langsmith dependency.

^{Written for commit 36d7164. Summary will update on new commits.}

[changeset-bot]

⚠️ No Changeset found

Latest commit: 36d7164

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

[cubic-dev-ai]

All reported issues were addressed across 7 files

Confidence score: 5/5

• Safe to merge after the addressed issues were fixed.

Architecture diagram

sequenceDiagram
    participant Env as Environment
    participant Welcome as welcomeStatus.ts
    participant Doctor as doctor.ts
    participant LangSmith as framework/langsmith.ts
    participant Braintrust as framework/braintrust.ts

    Note over Env,Braintrust: NEW: LangSmith gating + config (no runtime behavior change)

    Welcome->>Env: snapshotEnv(): read LANGSMITH_API_KEY
    Env-->>Welcome: value
    Welcome->>Welcome: NEW: providerEntry("LANGSMITH_API_KEY") -> EnvSnapshot.langsmith
    Welcome->>Braintrust: (unchanged) snapshot BRAINTRUST_API_KEY

    Doctor->>Env: buildReport(): read EVAL_TRACE_PRIMARY (default "braintrust")
    Doctor->>Env: read LANGSMITH_API_KEY, BRAINTRUST_API_KEY
    Env-->>Doctor: values
    Doctor->>Doctor: computeVerdict(keys, config, discovery, tracePrimary)
    alt CHANGED: No experiment backend (both missing)
        Doctor->>Doctor: Warn: “No experiment backend configured”
    else CHANGED: Primary backend missing
        Doctor->>Doctor: Warn: “EVAL_TRACE_PRIMARY=… but required key missing”
    else Both present
        Doctor->>Doctor: OK / warn only on BB partial
    end

    Note over LangSmith: NEW: gating helpers (not yet imported in hot path)

    LangSmith->>Env: hasLangSmithApiKey(): Boolean(LANGSMITH_API_KEY)
    LangSmith->>Env: langSmithTracingEnabled: key && LANGSMITH_TRACING==="true"
    LangSmith->>Env: resolveTraceTransport(): defaults to "native" unless "otel"
    LangSmith->>LangSmith: loadLangSmith(): lazy dynamic import (memoized)
    LangSmith->>LangSmith: assertLangSmithReady(): throws on missing key or tracing flag

    Note over Welcome,Doctor: No hot-path change: new module (langsmith.ts) is not referenced by any caller yet
    Note over Env: NEW: .env.example entries: LANGSMITH_API_KEY, LANGSMITH_TRACING, EVAL_TRACE_TRANSPORT, EVAL_TRACE_PRIMARY

Loading

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[cubic-dev-ai]

All reported issues were addressed across 5 files (changes from recent commits).

<sub>Reply with feedback, questions, or to request a fix.

Re-trigger cubic</sub>

[miguelg719]

Superseded by the Stagehand 4.0 retarget stack (#2757–#2762). Main moved to the 4.0 monorepo restructure (packages/core → packages/sdk-ts; evals package restructured), so this 3.x-era diff no longer applies. The OTEL/LangSmith tracing work — plus native-span capture via 4.0's global tracer and claude_code/codex agent spans — is reimplemented additively on 4.0 in the new stack.