From eb21776dc46c66f7de986b9c2d0ca20277908e2f Mon Sep 17 00:00:00 2001 From: factory-daniel Date: Tue, 25 Aug 2026 20:18:03 +0000 Subject: [PATCH 1/2] docs(droid-control): update agent-browser skill Sync the droid-control browser driver guidance with agent-browser 0.35.0 while preserving the plugin-specific non-invocable wrapper. Co-authored-by: factory-droid[bot] <138933559+factory-droid[bot]@users.noreply.github.com> --- .../skills/agent-browser/SKILL.md | 601 ++++++++++++------ 1 file changed, 404 insertions(+), 197 deletions(-) diff --git a/plugins/droid-control/skills/agent-browser/SKILL.md b/plugins/droid-control/skills/agent-browser/SKILL.md index d50fd49..c45a1c1 100644 --- a/plugins/droid-control/skills/agent-browser/SKILL.md +++ b/plugins/droid-control/skills/agent-browser/SKILL.md @@ -8,318 +8,525 @@ user-invocable: false The orchestrator routed you here. Use these mechanics to execute your plan. -Control web pages and Electron desktop apps via the `agent-browser` CLI. Uses Playwright under the hood with a headless Chromium instance managed by a background daemon. +Fast browser automation CLI for AI agents. Chrome/Chromium via CDP, no Playwright or Puppeteer dependency. Accessibility-tree snapshots with compact `@eN` refs let agents interact with pages in ~200-400 tokens instead of parsing raw HTML. -## When to use +Most normal web tasks (navigate, read, click, fill, extract, screenshot) are covered here. Load a specialized skill when the task falls outside browser web pages — see [When to load another skill](#when-to-load-another-skill). -- Automating web app flows (login, form fill, data extraction, visual QA) -- Driving Electron apps (VS Code, Slack, Discord, Figma, Notion, Spotify) -- Visual verification -- screenshots and annotated element overlays -- DOM-level assertions where terminal snapshots are irrelevant +## The core loop -If the target is a terminal TUI, use **tuistory** or **true-input** instead. +```bash +agent-browser open # 1. Open a page +agent-browser snapshot -i # 2. See what's on it (interactive elements only) +agent-browser click @e3 # 3. Act on refs from the snapshot +agent-browser snapshot -i # 4. Re-snapshot after any page change +``` + +Refs (`@e1`, `@e2`, ...) are assigned fresh on every snapshot. They become **stale the moment the page changes** — after clicks that navigate, form submits, dynamic re-renders, dialog opens. Always re-snapshot before your next ref interaction. + +## Always use your own session + +Before your first command, set a named session for the whole task: + +```bash +export AGENT_BROWSER_SESSION="$(agent-browser session id --scope worktree --prefix task)" +``` + +The default (unnamed) session is a single shared browser: it is shared with every other agent on the machine and it persists across conversations, so working in it can hijack another agent's page mid-task or navigate away from something the human left open. Every example below assumes a named session is active. See [Run multiple browsers in parallel](#run-multiple-browsers-in-parallel) and `references/session-management.md`. -## Prerequisites +## Quickstart ```bash -agent-browser install # one-time: downloads bundled Chromium +# agent-browser is bundled with the Factory CLI -- no npm install required. + +# If doctor reports that Chrome or Linux browser libraries are missing: +agent-browser install +agent-browser install --with-deps + +# Take a screenshot of a page +agent-browser open https://example.com +agent-browser screenshot home.png +agent-browser close + +# Search, click a result, and capture it +agent-browser open https://duckduckgo.com +agent-browser snapshot -i # find the search box ref +agent-browser fill @e1 "agent-browser cli" +agent-browser press Enter +agent-browser wait --load networkidle +agent-browser snapshot -i # refs now reflect results +agent-browser click @e5 # click a result +agent-browser screenshot result.png ``` -For Electron apps, the target app must be launched with `--remote-debugging-port=`. +The browser stays running across commands so these feel like a single session. By default, an inactive daemon saves configured restore state, closes its headless browser, and exits after one hour; the next command starts it again. Without `--restore` or another restore key, shutdown discards transient browser state and open tabs. Dashboard mouse, keyboard, and touch input count as activity. Headed browsers, Safari and iOS WebDriver sessions, and user-attached browsers are exempt from the default; provider-owned cloud browsers are not. Use `--idle-timeout