diff --git a/packages/bcode-browser/skills/browser-execute/SKILL.md b/packages/bcode-browser/skills/browser-execute/SKILL.md index 44d8202d0..27c8b4d5c 100644 --- a/packages/bcode-browser/skills/browser-execute/SKILL.md +++ b/packages/bcode-browser/skills/browser-execute/SKILL.md @@ -4,14 +4,15 @@ description: Use ONLY when calling the `browser_execute` tool or driving a real --- The `browser_execute` tool evaluates JavaScript against a connected browser `session` via the Chrome DevTools Protocol. -The snippet runs in-process; `session` is bound to a long-lived CDP `Session` that persists. Connect once, then drive many snippets. +The snippet runs in-process; `session` is bound to a long-lived CDP `Session` that persists. There is no helper namespace, just `session`, `console`, and standard JS globals. Workspace: `/.bcode/agent-workspace/`. Read/write your reusable scripts here. Skills: `{{SKILLS_DIR}}/`. Read-only browser execute reference docs. ## Connecting -Always call `session.connect(...)` once at the start of your work. There are three connection methods: +In Browser Use Cloud API V4, `browser_execute` automatically connects and attaches the existing page once when the fresh run first uses this tool; do not call `session.connect()` or `session.use()` before driving it. +Otherwise, call `session.connect(...)` once at the start of your work. There are three connection methods: #### Way 1: connect to the user's running Chrome or Chromium-based browser (real profile, popup-gated). Choose when the task involves the user's logged-in sites, current browser state, cookies, saved data, etc. @@ -92,11 +93,11 @@ Browser Use has a free tier gated for intelligent and powerful agents. Unlimited #### Way 4: user-preconfigured endpoint Not a method you choose — a way for the user to hand you a pre-set endpoint. -If `BU_CDP_WS` (or its alias `BU_CDP_URL`) is set in the environment, `session.connect()` with no args connects to that endpoint directly. Explicit `{ wsUrl }` / `{ profileDir }` calls ignore the env var. +When `V4_RUN_ID` and `BU_CDP_WS` (or its alias `BU_CDP_URL`) are both set, `browser_execute` connects to that endpoint and attaches its existing non-internal page once before the first snippet. Go straight to driving it. Other environments keep the explicit connection flow, and explicit `{ wsUrl }` / `{ profileDir }` calls still connect to the requested endpoint instead. If that fixed endpoint closes or repeatedly fails its WebSocket upgrade, reconnecting to the same URL cannot recover it; the endpoint owner must replace it. ## Attaching to a target -After `connect()`, attach to a page target before driving the browser: +After connecting manually, attach to a page target before driving the browser. A preconfigured endpoint is already attached automatically: ```js const targets = (await session.Target.getTargets({})).targetInfos @@ -105,7 +106,9 @@ const page = targets.find(t => t.type === "page" && !t.url.startsWith("chrome:// await session.use(page.targetId) ``` -If a target-scoped command throws `CdpError` code `-32001` (`Session with given id not found`), the browser connection is still usable but the target session is stale. List targets again, `session.use(...)` the intended page, and retry the rejected command once. Repeated `session.connect()` calls do not replace a stale target session. +If a target-scoped command throws `CdpError` code `-32001` (`Session with given id not found`), the browser connection is still usable but the target session is stale. List targets again, `session.use(...)` the intended page, and retry the rejected command once. Calling `session.connect()` without arguments is a no-op while connected; it does not replace a stale target session. + +Every explicit reconnect or browser switch retires the previous socket and clears its active target attachment. Re-list targets, call `session.use(...)`, and rediscover DOM nodes and Runtime objects before continuing. ## Driving a page Domain methods follow `session..(params)` and return Promises. @@ -197,7 +200,7 @@ console.log(JSON.stringify(titles)) ## Guardrails - Top-level `import` statements inside the snippet body are not allowed. Use `await import(...)` instead. - No CPU-bound infinite loops without `await` — they ignore the timeout. Insert `await new Promise(r => setTimeout(r, 0))` to yield. -- `browser_execute` defaults to 60s (max 600s). For longer work, set the tool's top-level `timeout`; inner CDP timeouts do not extend it. Keep batches small and log progress — timeout errors return recent logs, and a timeout resets the CDP session (reconnect in the next snippet). +- `browser_execute` defaults to 60s (max 600s). For longer work, set the tool's top-level `timeout`; inner CDP timeouts do not extend it. Keep batches small and log progress — timeout errors return recent logs, and a timeout resets the CDP session. Reconnect deliberately after a timeout so a run that switched browsers cannot silently return to its original browser. ## Console - `console.log`, `console.error`, `console.warn`, `console.info`, `console.debug` are all captured and streamed to the user. Treat them as your stdout. Other `console.*` methods write to bcode's stderr without being captured into the tool result. diff --git a/packages/bcode-browser/src/browser-execute.ts b/packages/bcode-browser/src/browser-execute.ts index ddf756ef9..1e341cc5d 100644 --- a/packages/bcode-browser/src/browser-execute.ts +++ b/packages/bcode-browser/src/browser-execute.ts @@ -11,12 +11,15 @@ // `{log, error, warn, info}` API as the real console. // standard JS globals. // -// Nothing is auto-loaded. To reuse code from a previous snippet the agent -// writes plain `await import("/abs/path/foo.ts?t=" + Date.now())` against a -// `.ts` file it owns under `/.bcode/agent-workspace/`. Same -// mechanism for a 5-line wrapper and a 500-line scrape script. The Level-2 -// wrapper supplies `ctx.workspaceDir` so `.ts` files written under it can be -// addressed by absolute path; this resolver creates the dir on first use. +// When BU_CDP_WS or BU_CDP_URL binds the process to a provisioned browser, +// the tool connects and attaches its existing page before running a snippet. +// Local sessions keep explicit connection behavior. To reuse code from a +// previous snippet the agent writes plain +// `await import("/abs/path/foo.ts?t=" + Date.now())` against a `.ts` file it +// owns under `/.bcode/agent-workspace/`. Same mechanism for a +// 5-line wrapper and a 500-line scrape script. The Level-2 wrapper supplies +// `ctx.workspaceDir` so `.ts` files written under it can be addressed by +// absolute path; this resolver creates the dir on first use. // // Output capture: a per-call `console` object (`{log, error, warn, info}`) // is bound into the snippet's lexical scope as the second AsyncFunction @@ -53,6 +56,8 @@ const DEFAULT_TIMEOUT_MS = 60 * 1000 const MAX_TIMEOUT_MS = 10 * 60 * 1000 const MAX_TIMEOUT_OUTPUT_BYTES = 8 * 1024 const TIMEOUT_OUTPUT_TRUNCATED = "[partial console output truncated; showing final bytes]\n" +const v4Connections = new Map>() +const v4Bootstrapped = new Set() // Tail-cap the captured output for the timeout error: last 8 KiB, snapped // forward to a UTF-8 sequence start so multibyte characters survive the cut. @@ -86,9 +91,7 @@ export type Parameters = Schema.Schema.Type export interface ExecuteContext { // Identifies the per-opencode-session CDP Session to bind into the snippet. - // The same Session is reused across calls — the agent calls - // `session.connect(...)` in one snippet and subsequent snippets find the - // already-connected Session. + // Provisioned endpoints auto-connect and attach; local sessions connect explicitly. readonly sessionID: string // Per-project workspace dir: /.bcode/agent-workspace/. Created // on first call. The agent reads/writes/edits .ts files here via the @@ -159,9 +162,8 @@ const serialize = (v: unknown): string => { } // Snippet executor. The CDP Session is resolved per-call from `SessionStore` -// keyed on `ctx.sessionID`. The agent connects with `await session.connect(...)` -// in one snippet (Way 1 / Way 2 / Way 3 in skills/browser-execute/SKILL.md); the Session persists -// for follow-up snippets in the same opencode session. +// keyed on `ctx.sessionID`. Provisioned endpoints auto-connect and attach +// before the snippet; local sessions connect explicitly. // // `dataDir` is opencode's XDG_DATA_HOME for bcode (~/.local/share/bcode/ on // Linux/Mac). Compiled-mode skills are extracted to `/skills/` once @@ -189,6 +191,11 @@ export const make = Effect.fn("BrowserExecute.make")(function* (dataDir: string) catch: (err) => new Error(`syntax error in browser_execute snippet: ${err}`), }) + yield* Effect.tryPromise({ + try: () => ensureCloudConnected(ctx.sessionID, session), + catch: (err) => (err instanceof Error ? err : new Error(String(err))), + }) + const tee = (...a: unknown[]) => { if (!captured.active) return captured.output += a.map((x) => (typeof x === "string" ? x : serialize(x))).join(" ") + "\n" @@ -277,4 +284,32 @@ export const make = Effect.fn("BrowserExecute.make")(function* (dataDir: string) return { parameters, execute, skillsDir } }) +async function ensureCloudConnected(sessionID: string, session: ReturnType) { + if (!process.env.V4_RUN_ID || (!process.env.BU_CDP_WS && !process.env.BU_CDP_URL)) return + if (v4Bootstrapped.has(sessionID)) return + + const existing = v4Connections.get(sessionID) + if (existing) return existing + + const connecting = (async () => { + if (!session.isConnected()) await session.connect() + if (session.getActiveSession()) return + const page = (await session.domains.Target.getTargets({})).targetInfos.find( + (target) => target.type === "page" && !target.url.startsWith("chrome://"), + ) + if (page) await session.use(page.targetId) + })() + v4Connections.set(sessionID, connecting) + try { + await connecting + } finally { + // One automatic attempt per logical BrowserCode session. A later disconnect + // (including after timeout replacement) must be surfaced: + // BU_CDP_WS is the browser selected at run start, not necessarily a newer + // browser the agent explicitly switched to during this run. + v4Bootstrapped.add(sessionID) + if (v4Connections.get(sessionID) === connecting) v4Connections.delete(sessionID) + } +} + export * as BrowserExecute from "./browser-execute" diff --git a/packages/bcode-browser/src/cdp/session.ts b/packages/bcode-browser/src/cdp/session.ts index b19713c5e..5cfa693f6 100644 --- a/packages/bcode-browser/src/cdp/session.ts +++ b/packages/bcode-browser/src/cdp/session.ts @@ -89,6 +89,7 @@ export class Session implements Transport { } const envWsUrl = process.env.BU_CDP_WS ?? process.env.BU_CDP_URL; if (envWsUrl) { + if (this.isConnected()) return; await this.openWs(envWsUrl, timeoutMs); return; } @@ -122,6 +123,14 @@ export class Session implements Transport { if (this.invalidatedError) return Promise.reject(this.invalidatedError); return new Promise((res, rej) => { const ws = new WebSocket(wsUrl); + const previousWs = this.ws; + this.ws = ws; + this.activeSessionId = undefined; + if (previousWs) { + for (const [, p] of this.pending) p.reject(new Error('CDP connection replaced')); + this.pending.clear(); + try { previousWs.close(); } catch { /* ignore */ } + } let done = false; const finish = (err?: Error) => { if (done) return; @@ -131,15 +140,28 @@ export class Session implements Transport { else res(); }; const timer = setTimeout(() => finish(new Error(`timed out after ${timeoutMs}ms`)), timeoutMs); - ws.addEventListener('open', () => finish(this.invalidatedError)); + ws.addEventListener('open', () => { + if (this.ws !== ws) { + finish(new Error('CDP connection superseded')); + return; + } + finish(this.invalidatedError); + }); ws.addEventListener('error', (e) => finish(new Error(`WS error: ${(e as any)?.message ?? 'connect failed (likely 403, permission not granted, or port closed)'}`))); - ws.addEventListener('message', (e) => this.onMessage(String(e.data))); + ws.addEventListener('message', (e) => { + if (this.ws === ws) this.onMessage(String(e.data)); + }); ws.addEventListener('close', () => { + if (this.ws !== ws) { + finish(new Error('CDP connection superseded')); + return; + } + this.ws = undefined; + this.activeSessionId = undefined; for (const [, p] of this.pending) p.reject(this.invalidatedError ?? new Error('CDP socket closed')); this.pending.clear(); finish(this.invalidatedError ?? new Error('WS closed before open (likely 403 or port closed)')); }); - this.ws = ws; }); } @@ -483,4 +505,3 @@ async function tryReadDevToolsActivePort( return undefined; } } - diff --git a/packages/bcode-browser/test/browser-auto-connect.test.ts b/packages/bcode-browser/test/browser-auto-connect.test.ts new file mode 100644 index 000000000..f798e3664 --- /dev/null +++ b/packages/bcode-browser/test/browser-auto-connect.test.ts @@ -0,0 +1,398 @@ +import { afterAll, expect, test } from "bun:test"; +import fs from "fs/promises"; +import os from "os"; +import path from "path"; +import { Effect } from "effect"; +import { BrowserExecute } from "../src/browser-execute"; +import { SessionStore } from "../src/session-store"; + +let connections = 0; +let closedConnections = 0; +let attachedCalls = 0; +let pageCallsWithSession = 0; +let latestSocket: { close(): void } | undefined; +const server = Bun.serve({ + port: 0, + fetch(req, srv) { + return srv.upgrade(req) + ? undefined + : new Response("upgrade required", { status: 426 }); + }, + websocket: { + open(ws) { + connections++; + latestSocket = ws; + }, + message(ws, message) { + const request: unknown = JSON.parse(String(message)); + if ( + !request || + typeof request !== "object" || + !("id" in request) || + typeof request.id !== "number" || + !("method" in request) || + typeof request.method !== "string" + ) + return; + const result = (() => { + if (request.method === "Target.getTargets") + return { + targetInfos: [ + { + targetId: "page-1", + type: "page", + title: "", + url: "about:blank", + attached: false, + canAccessOpener: false, + }, + ], + }; + if (request.method === "Target.attachToTarget") { + attachedCalls++; + return { sessionId: "page-session-1" }; + } + if (request.method.startsWith("Page.") && "sessionId" in request) + pageCallsWithSession++; + return {}; + })(); + ws.send(JSON.stringify({ id: request.id, result })); + }, + close() { + closedConnections++; + }, + }, +}); + +const failingServer = Bun.serve({ + port: 0, + fetch() { + return new Response("unavailable", { status: 503 }); + }, +}); + +afterAll(() => { + server.stop(true); + failingServer.stop(true); +}); + +const wsUrl = `ws://127.0.0.1:${server.port}/`; + +const withEnv = async ( + vars: Record, + fn: () => Promise, +): Promise => { + const previous = Object.fromEntries( + Object.keys(vars).map((key) => [key, process.env[key]]), + ); + Object.entries(vars).forEach(([key, value]) => { + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + }); + try { + return await fn(); + } finally { + Object.entries(previous).forEach(([key, value]) => { + if (value === undefined) delete process.env[key]; + else process.env[key] = value; + }); + } +}; + +const withBrowserExecute = async ( + name: string, + fn: ( + impl: Effect.Success>, + sessionID: string, + workspaceDir: string, + ) => Promise, +) => { + const dataDir = await fs.mkdtemp( + path.join(os.tmpdir(), `bcode-auto-data-${name}-`), + ); + const workspaceDir = await fs.mkdtemp( + path.join(os.tmpdir(), `bcode-auto-ws-${name}-`), + ); + const sessionID = `auto-connect-${name}-${Math.random().toString(36).slice(2)}`; + try { + const impl = await Effect.runPromise( + Effect.scoped(BrowserExecute.make(dataDir)), + ); + await fn(impl, sessionID, workspaceDir); + } finally { + await SessionStore.evict(sessionID); + await Promise.all( + [dataDir, workspaceDir].map((dir) => + fs.rm(dir, { recursive: true, force: true }), + ), + ); + } +}; + +test("V4 endpoint auto-connects, attaches, and reuses one connection", async () => { + connections = 0; + attachedCalls = 0; + pageCallsWithSession = 0; + await withEnv( + { V4_RUN_ID: "run-reuse", BU_CDP_WS: wsUrl, BU_CDP_URL: undefined }, + () => + withBrowserExecute("reuse", async (impl, sessionID, workspaceDir) => { + const run = (code: string) => + Effect.runPromise( + impl.execute( + { description: "List browser targets", code }, + { sessionID, workspaceDir }, + ), + ); + + expect( + JSON.parse( + ( + await run( + "return await session.Page.navigate({ url: 'https://sap.com' })", + ) + ).result, + ), + ).toEqual({}); + expect( + JSON.parse( + ( + await run( + "await session.connect(); return (await session.Target.getTargets({})).targetInfos.length", + ) + ).result, + ), + ).toBe(1); + expect(connections).toBe(1); + expect(attachedCalls).toBe(1); + expect(pageCallsWithSession).toBe(1); + }), + ); +}); + +test("parallel first calls share one connection attempt", async () => { + connections = 0; + attachedCalls = 0; + await withEnv( + { V4_RUN_ID: "run-race", BU_CDP_WS: wsUrl, BU_CDP_URL: undefined }, + () => + withBrowserExecute("race", async (impl, sessionID, workspaceDir) => { + const run = () => + Effect.runPromise( + impl.execute( + { + description: "Read target count", + code: "return (await session.Target.getTargets({})).targetInfos.length", + }, + { sessionID, workspaceDir }, + ), + ); + + expect( + (await Promise.all([run(), run()])).map((result) => + JSON.parse(result.result), + ), + ).toEqual([1, 1]); + expect(connections).toBe(1); + expect(attachedCalls).toBe(1); + }), + ); +}); + +test("a dropped socket is surfaced instead of reconnecting the run-start browser", async () => { + connections = 0; + closedConnections = 0; + attachedCalls = 0; + await withEnv( + { V4_RUN_ID: "run-dropped", BU_CDP_WS: wsUrl, BU_CDP_URL: undefined }, + () => + withBrowserExecute("dropped", async (impl, sessionID, workspaceDir) => { + const run = () => + Effect.runPromise( + impl.execute( + { + description: "Navigate existing page", + code: "return await session.Page.navigate({ url: 'https://sap.com' })", + }, + { sessionID, workspaceDir }, + ), + ); + + await run(); + latestSocket?.close(); + await new Promise((resolve) => setTimeout(resolve, 10)); + await expect(run()).rejects.toThrow( + "Not connected. Call session.connect(...) first.", + ); + + expect(connections).toBe(1); + expect(closedConnections).toBe(1); + expect(attachedCalls).toBe(1); + }), + ); +}); + +test("an explicit browser switch retires the old socket and target attachment", async () => { + connections = 0; + closedConnections = 0; + attachedCalls = 0; + pageCallsWithSession = 0; + await withEnv( + { V4_RUN_ID: "run-switch", BU_CDP_WS: wsUrl, BU_CDP_URL: undefined }, + () => + withBrowserExecute("switch", async (impl, sessionID, workspaceDir) => { + const run = (code: string) => + Effect.runPromise( + impl.execute( + { description: "Switch provisioned browsers", code }, + { sessionID, workspaceDir }, + ), + ); + + await run( + "return await session.Page.navigate({ url: 'https://sap.com' })", + ); + const switched = await run(` + await session.connect({ wsUrl: ${JSON.stringify(wsUrl)} }) + await session.Page.navigate({ url: "https://example.com" }) + return { activeSession: session.getActiveSession() ?? null } + `); + expect(JSON.parse(switched.result)).toEqual({ activeSession: null }); + + await new Promise((resolve) => setTimeout(resolve, 10)); + expect(connections).toBe(2); + expect(closedConnections).toBe(1); + expect(attachedCalls).toBe(1); + expect(pageCallsWithSession).toBe(1); + + await run(` + const page = (await session.Target.getTargets({})).targetInfos[0] + await session.use(page.targetId) + return await session.Page.navigate({ url: "https://example.com" }) + `); + expect(attachedCalls).toBe(2); + expect(pageCallsWithSession).toBe(2); + }), + ); +}); + +test("a timeout replacement does not auto-attach the run-start browser again", async () => { + connections = 0; + await withEnv( + { V4_RUN_ID: "run-timeout", BU_CDP_WS: wsUrl, BU_CDP_URL: undefined }, + () => + withBrowserExecute("timeout", async (impl, sessionID, workspaceDir) => { + await expect( + Effect.runPromise( + impl.execute( + { + description: "Time out after initial V4 bootstrap", + code: "await new Promise(resolve => setTimeout(resolve, 100))", + timeout: 10, + }, + { sessionID, workspaceDir }, + ), + ), + ).rejects.toThrow("browser_execute timed out"); + + await expect( + Effect.runPromise( + impl.execute( + { + description: "Do not silently return to the run-start browser", + code: "return await session.Page.navigate({ url: 'https://sap.com' })", + }, + { sessionID, workspaceDir }, + ), + ), + ).rejects.toThrow("Not connected. Call session.connect(...) first."); + expect(connections).toBe(1); + }), + ); +}); + +test("sessions without a provisioned endpoint still require connect", async () => { + await withEnv( + { V4_RUN_ID: undefined, BU_CDP_WS: undefined, BU_CDP_URL: undefined }, + () => + withBrowserExecute("local", async (impl, sessionID, workspaceDir) => { + await expect( + Effect.runPromise( + impl.execute( + { + description: "Call CDP without connect", + code: "return await session.Target.getTargets({})", + }, + { sessionID, workspaceDir }, + ), + ), + ).rejects.toThrow("Not connected. Call session.connect(...) first."); + }), + ); +}); + +test("eval endpoints remain explicit without V4_RUN_ID", async () => { + connections = 0; + await withEnv( + { V4_RUN_ID: undefined, BU_CDP_WS: wsUrl, BU_CDP_URL: undefined }, + () => + withBrowserExecute("eval", async (impl, sessionID, workspaceDir) => { + await expect( + Effect.runPromise( + impl.execute( + { + description: "Call CDP without explicit eval connect", + code: "return await session.Target.getTargets({})", + }, + { sessionID, workspaceDir }, + ), + ), + ).rejects.toThrow("Not connected. Call session.connect(...) first."); + expect(connections).toBe(0); + }), + ); +}); + +test("a failed V4 bootstrap does not block an explicit replacement browser", async () => { + await withEnv( + { + V4_RUN_ID: "run-replacement", + BU_CDP_WS: `ws://127.0.0.1:${failingServer.port}/`, + BU_CDP_URL: undefined, + }, + () => + withBrowserExecute("failure", async (impl, sessionID, workspaceDir) => { + const failure = Effect.runPromise( + impl.execute( + { + description: "Do not run snippet", + code: 'throw new Error("snippet should not run")', + }, + { sessionID, workspaceDir }, + ), + ); + + await expect(failure).rejects.toThrow(/WS error|WS closed before open/); + await expect(failure).rejects.not.toThrow( + "browser_execute snippet threw", + ); + await expect(failure).rejects.not.toThrow("snippet should not run"); + + const recovered = await Effect.runPromise( + impl.execute( + { + description: "Connect a replacement browser", + code: ` + await session.connect({ wsUrl: ${JSON.stringify(wsUrl)} }) + const page = (await session.Target.getTargets({})).targetInfos[0] + await session.use(page.targetId) + return await session.Page.navigate({ url: "https://sap.com" }) + `, + }, + { sessionID, workspaceDir }, + ), + ); + expect(JSON.parse(recovered.result)).toEqual({}); + }), + ); +});