Drive an Android emulator or device from the command line - accessibility snapshots with element refs, taps, drags, typing, and screenshots. Built for AI coding agents, useful for humans too.
An open-source project from SquadUp Labs.
$ agent-android snapshot -i
snapshot gen=2 app=host.exp.exponent screen=1080x2400
@e4 TextField val="Search Credentials" (307,181 687x100)
@e9 ScrollView "Vaccination Records, testing, Expires, 156 days" (0,449 1080x1888)
@e11 View "Add Credential" (828,2085 168x168)
$ agent-android tap @e4
tapped @e4
$ agent-android type "vaccine" --submit
typed 7 chars + return
adb already ships every primitive you need: uiautomator dump reads the UI, input tap injects touches, screencap grabs the screen. What it does not give you is a workflow. An agent running raw adb gets a 200 KB XML blob per screen with no element refs, no see-act-verify loop, and no protection against acting on a stale view.
agent-android is the missing front-end, and nothing more:
- Pure adb underneath. No on-device agent, no instrumentation APK, nothing installed on the emulator, nothing to build with Gradle. If
adbcan reach the device, agent-android works. - One dependency. The CLI's only package dependency is apple/swift-argument-parser. Everything else is Foundation and the adb you already have.
- Real accessibility tree. Snapshots come from
uiautomator, the same source Android's own accessibility services read, so they work on React Native / Expo, Jetpack Compose, classic Views, and system apps alike. - Real input events. Gestures go through
input/am/pm, the same channels the OS uses - the emulator can sit in the background while an agent works.
agent-android CLI (Swift binary)
|
| adb (exec-out uiautomator dump | input | screencap | am | pm)
v
+---------------------------------------------------------+
| Android emulator / device |
| uiautomator dump -> accessibility XML -> ref'd tree |
| input tap/swipe/text/keyevent -> gestures & typing |
| exec-out screencap -p -> screenshots |
| am / pm -> launch, permissions |
+---------------------------------------------------------+
There is no runner and no server. Every command is one or more adb calls. The one piece of state agent-android keeps is host-side: each snapshot writes the element refs (@e1 -> bounds) and a generation counter to ~/.agent-android/run/<serial>/state.json, so the next tap @e3 knows where @e3 is and whether it is stale.
Because iOS's simctl cannot inject touches or read the UI, agent-ios has to build and run an in-simulator XCUITest server. Android needs none of that - adb covers the whole surface - so agent-android is simpler by construction.
adb(Android platform-tools) on your PATH, orANDROID_HOMEset (verified on platform-tools 37)- A booted emulator or a device with USB debugging on (verified on Android 15 / API 35)
- Swift 5.10+ toolchain to build (macOS or Linux)
git clone <this repo> && cd agent-android
swift build
.build/debug/agent-android doctor # environment checks
.build/debug/agent-android snapshot -i # prints refs for what's on screen
.build/debug/agent-android tap @e3 # tap an element by ref
.build/debug/agent-android screenshot # prints a PNG pathTo put agent-android on your PATH, build a release binary and link it into a PATH directory:
swift build -c release
ln -sf "$(pwd)/.build/release/agent-android" ~/.local/bin/agent-android # if ~/.local/bin is on PATHThe binary is self-contained - there is no runtime project to find, so you can move or symlink it anywhere.
- Usage - full command reference, the ref workflow, exit codes, examples.
- Configuring your app - point agent-android at any Android app (Compose, Views, React Native, Expo, Flutter), not just the examples here.
- Integrations - wire agent-android into Claude Code, OpenAI Codex CLI, Cursor, and other agents.
- Internals - the adb command mapping and the host-side state files: the source of truth for what the CLI actually runs.
- Contributing · Releasing · Changelog · Security
agent-android is a plain CLI, so any agent that can run shell commands can drive the emulator with it. See docs/integrations.md for copy-paste setups for Claude Code (a skill), OpenAI Codex CLI (AGENTS.md), Cursor (.cursor/rules), and a note on MCP. The canonical agent workflow lives in docs/skill/SKILL.md.
agent-android snapshot(orsnapshot -ifor only interactive, on-screen elements) prints the accessibility tree. Interactive or labeled elements get refs:@e1,@e2, ...- Act on refs:
tap @e3,drag @e3 @e7,type "hi" --ref @e5. - Refs belong to one snapshot generation. After the UI changes, take a new snapshot - stale refs exit with code 3 and a "take a new snapshot" message.
screenshotany time you want visual confirmation.
Snapshot lines carry the element type, label in quotes, id= (the resource-id / React Native testID), val= (an editable field's text), the bounds in pixels (x,y wxh), and flags [disabled] [checked] [selected] [focused] [offscreen]. Offscreen elements (closed drawers, scrolled-away rows) keep their refs but refuse to tap with a helpful error.
| Command | What it does |
|---|---|
agent-android devices |
List emulators and devices, booted first |
agent-android apps |
User-installed packages (+ versionName) on the target device |
agent-android status |
Device, screen, session, and last-snapshot state |
agent-android doctor |
Environment checks with fixes |
agent-android attach [package] |
Set the session app; no arg auto-detects the foreground user app |
agent-android launch <package> [--relaunch] [--no-attach] |
Launch an app (implies attach) |
agent-android terminate <package> |
Force-stop an app |
agent-android open <url> |
Deep link via am start (Expo dev and app schemes work) |
agent-android screenshot [-o path] |
PNG of the screen; prints the path |
agent-android snapshot [-i] [--json] [--app pkg] |
Accessibility tree with refs |
agent-android tap @e3 | 200 400 [--double] [--long] [--duration S] |
Tap / double tap / long press |
agent-android drag @e3 @e7 | x1 y1 x2 y2 [--duration S] [--velocity slow|fast|N] |
Drag between targets |
agent-android swipe up|down|left|right [--from @ref|x,y] [--distance N] |
Swipe (defaults to screen center) |
agent-android type "text" [--ref @e5] [--submit] |
Type into the focused field; --ref taps first; --submit presses Enter |
agent-android key back|home|recents|return|delete|... [--times N] |
Key / device button presses |
agent-android alert [accept|dismiss|"Label"] |
Inspect or answer a system dialog |
agent-android permissions grant|revoke|reset <service> [package] |
Pre-answer permission dialogs |
agent-android install <path.apk> / uninstall <package> |
App management |
Global: --serial <SERIAL> on every command (defaults to the single connected device, or AGENT_ANDROID_SERIAL).
Exit codes: 0 ok, 1 generic failure, 2 device/adb unreachable, 3 stale/unknown ref, 4 no session app.
uiautomator could not capture the UI. The dump fails while the app never goes idle - a spinner, a Lottie animation, an infinite progress bar. agent-android retries a few times; if it still fails, the screen is genuinely never settling. Take a screenshot and tap by coordinate instead, or wait for the animation to end.
Typing does nothing / keyboard=visible never shows. type injects text with input text whether or not a soft keyboard is up, so it usually still works. But React Native text inputs on an emulator need the AVD's hw.keyboard=yes (Pixel profiles default it to no), and keyboard-visibility detection needs settings put secure show_ime_with_hard_keyboard 1. agent-android doctor checks both.
Snapshot only shows one app. uiautomator dump only ever contains the focused window. You cannot snapshot a background app; bring it to the foreground first (agent-android launch <package>). This is also why --app errors if the package you name is not on screen.
A permission dialog is blocking everything. Snapshots warn with [!] a system dialog is on screen. Use agent-android alert to list its buttons, agent-android alert accept / dismiss / an exact label to answer it, or agent-android permissions grant ... beforehand to avoid it entirely.
Only ASCII types. input text is limited to printable ASCII; agent-android rejects anything else with a clear message. For emoji or non-Latin text, set the field through your app's debug tooling or a deep link, or install an ADB-keyboard IME.
A window comes back nearly empty. Some screens set FLAG_SECURE (password managers, payment sheets); uiautomator cannot read them and often screencap renders them black. agent-android flags a near-empty dump. Drive those by coordinate from whatever you can see.
Coordinates are physical pixels. Unlike iOS points, every coordinate agent-android prints or accepts is a physical pixel - the same space adb shell input tap and wm size use. No density scaling anywhere.
- Appium / uiautomator2: a WebDriver-protocol server plus client libraries, meant to be driven by a test framework - sessions, capabilities, an on-device instrumentation server. agent-android is a single binary for an agent that runs shell commands; no server, no session lifecycle.
- Maestro: a test-authoring product with YAML flows for humans writing tests. agent-android is not a test framework - it is a low-level, imperative CLI, one command per agent step.
- droidrun / mobile-mcp and similar: agent-first, but shaped as a framework or an MCP server. agent-android is a plain CLI any agent can call, and it ships no daemon.
- raw adb: what agent-android wraps. adb has the primitives; agent-android adds the refs, the generation/staleness contract, the readable tree, and the see-act-verify loop.
Contributions are welcome. See CONTRIBUTING.md for how to build, test, and submit changes. We use Conventional Commits and Semantic Versioning; releases are described in RELEASING.md. Please follow the Code of Conduct, and report security issues per SECURITY.md.
MIT - see LICENSE. Copyright (c) 2026 SquadUp Labs.