Skip to content

Commit e7c8d68

Browse files
d-biehld-biehl
committed
docs: add "Working with AI Agents" reference page
Describe how RobotCode lets AI coding agents work through the project's own robotcode CLI instead of guessing: - the chat plugin (a skill) and what it teaches the agent — discover, libdoc, REPL, results, analyze on the resolved project - the bundled VS Code plugin and its enableChatPlugins toggle, plus the extension shipping the robotcode CLI on the integrated terminal PATH - installing the plugin in other agents via the Open Plugins marketplace - recommended project-context setup (AGENTS.md / CLAUDE.md): environment, library init steps, the system under test, conventions - the CLI's AI-agent detection and its override env vars/flags - troubleshooting
1 parent b93cbf6 commit e7c8d68

1 file changed

Lines changed: 151 additions & 0 deletions

File tree

‎docs/03_reference/ai-agents.md‎

Lines changed: 151 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,151 @@
1+
# Working with AI Agents
2+
3+
AI coding agents are good at writing code and bad at guessing how *your* Robot Framework project is actually wired. Which files become suites, which tags a test really ends up with, where a keyword comes from, what a finished run contained — none of that is reliably visible by reading `.robot` files. It is decided at runtime by `robot.toml`, profiles, variables, the installed library versions, and pre-run modifiers.
4+
5+
**RobotCode** closes that gap by teaching the agent to work through the project's own [`robotcode`](cli.md) CLI instead of guessing. The agent discovers tests, runs suites, inspects results, looks up keywords, and explores live in the REPL — all through the same resolved view of the project that the rest of **RobotCode** uses. The result is an agent that behaves less like a generic code model and more like a Robot Framework engineer who knows your setup.
6+
7+
There are two pieces, and they work independently:
8+
9+
- **Chat plugins** (this page's main subject) — a RobotCode plugin that packages instructions for the agent. Today it ships a single *skill* that tells the agent *how* to use `robotcode`: which command answers which question, and which habits to avoid (don't grep for tests, don't load `output.xml` into the chat). The plugin is built to grow — more skills, agents, and other capabilities will ship in the same package.
10+
- **AI-agent detection** ([below](#ai-agent-detection)) — the CLI noticing it runs inside an agent session and quietly adjusting its terminal output for clean capture.
11+
12+
**Who this is for:**
13+
14+
- **Developers using GitHub Copilot Chat in VS Code** — the plugin ships with the extension and is on by default.
15+
- **Users of other agents** — Claude Code, Codex, GitHub Copilot CLI, and other [Open Plugins](https://open-plugins.com/)–compatible tools install the same plugin from a marketplace.
16+
- **Anyone scripting agents against Robot Framework projects** — the CLI's agent-aware output makes capture predictable.
17+
18+
## What you can ask
19+
20+
Once the plugin is active, everyday Robot Framework requests are handled through the CLI with the right options, against your project's real configuration. For example:
21+
22+
- *"run the smoke suite with the `ci` profile"* — runs it through the selected profile and reports pass/fail counts
23+
- *"rerun just the tests that failed last time"*
24+
- *"why did `Login Works` fail in last night's run?"* — inspects the existing results, no re-run
25+
- *"what tests and tags exist?"* — resolves the real set with `discover` (paths, profiles, variables, pre-run modifiers), not a file scan
26+
- *"what arguments does our `Create Order` keyword take?"* — looks it up against the installed libraries and local resources
27+
- *"is there already a keyword for waiting until the spinner is gone?"*
28+
- *"try the new login flow against the real app, with a visible browser"* — drives it live in the REPL
29+
- *"set up a `prod` profile with `BASE_URL=https://prod.example.com`"*
30+
- *"lint only the files I changed today"*
31+
32+
## Prerequisites
33+
34+
The chat plugin guides the agent in using `robotcode`; it does not bundle or replace the CLI. The [`robotcode` CLI tools](cli.md#installation) must be installed in the **project's** Python environment — not an isolated runner like `uvx` or `pipx`, which cannot see the project's libraries, resources, and local modules. If the CLI is missing, the agent is instructed to walk you through installing it (choosing scope and extras) before running anything.
35+
36+
## In VS Code (GitHub Copilot Chat)
37+
38+
The VS Code extension bundles the RobotCode chat plugin for GitHub Copilot Chat. It is **enabled by default** — no installation step, no marketplace. The agent picks it up automatically and starts using `robotcode` whenever a request is about Robot Framework.
39+
40+
You also don't need to install the `robotcode` CLI separately for this. The extension ships a bundled CLI — every subcommand included — and adds it to the integrated terminal's `PATH`; because Copilot Chat's agent runs its commands in that terminal, `robotcode` is available there out of the box. The bundle is only the *tool*, though: it runs against the terminal's active Python interpreter, so Robot Framework and your project's libraries still have to be installed in that environment for the CLI to inspect and run your project correctly.
41+
42+
Toggle it with the setting:
43+
44+
```json
45+
"robotcode.ai.enableChatPlugins": true
46+
```
47+
48+
Set it to `false` to turn the bundled plugin off — for example if you prefer to install the plugin from the marketplace instead (see the [duplication note](#avoiding-duplicates) below).
49+
50+
## Other agents — the marketplace
51+
52+
The same plugin is published as an [Open Plugins](https://open-plugins.com/) marketplace, [`robotcodedev/robotframework-agent-plugins`](https://github.com/robotcodedev/robotframework-agent-plugins), so agents outside VS Code can install it once and reuse it. Supported agents include Claude Code, GitHub Copilot (CLI and Chat), and Codex; check [open-plugins.com/supported-agents](https://open-plugins.com/supported-agents) for the current list.
53+
54+
Each agent uses its own commands to register a marketplace and install a plugin. For example:
55+
56+
```sh
57+
# Claude Code
58+
claude plugin marketplace add robotcodedev/robotframework-agent-plugins
59+
claude plugin install robotcode@robotframework-agent-plugins
60+
61+
# GitHub Copilot CLI
62+
copilot plugin marketplace add robotcodedev/robotframework-agent-plugins
63+
copilot plugin install robotcode@robotframework-agent-plugins
64+
```
65+
66+
See the [marketplace README](https://github.com/robotcodedev/robotframework-agent-plugins#install-per-agent) for the exact commands per agent, and for agents that load skill folders directly without a marketplace.
67+
68+
### Adding the marketplace from VS Code
69+
70+
To use the marketplace version in VS Code's Copilot Chat (instead of, or alongside, the bundled plugin), the extension provides two Command Palette commands that edit the user-level `chat.plugins.marketplaces` setting for you:
71+
72+
- **RobotCode: Add Chat Plugins Marketplace**
73+
- **RobotCode: Remove Chat Plugins Marketplace**
74+
75+
Only the applicable one is shown at a time — *Add* until the marketplace is registered, *Remove* afterwards. (Removing also clears the entry if it was added by hand as a GitHub URL rather than the `owner/repo` shorthand.)
76+
77+
The `chat.plugins.marketplaces` setting is owned by GitHub Copilot Chat, not by RobotCode, so these commands require Copilot Chat (or another agent that provides that setting) to be installed — without it, updating it fails with an error.
78+
79+
### Avoiding duplicates
80+
81+
The bundled plugin and the marketplace plugin are the *same* plugin. If you install it from the marketplace, turn the bundled copy off so the agent doesn't load two identical plugins:
82+
83+
```json
84+
"robotcode.ai.enableChatPlugins": false
85+
```
86+
87+
The *Add Chat Plugins Marketplace* command offers to do this for you when the bundled plugin is still enabled.
88+
89+
## How it works
90+
91+
The plugin currently contains a single *skill* — a set of instructions the agent loads when a request looks Robot Framework–related — and is designed to gain more skills and agents over time. The skill calls no libraries or models of its own; it teaches the agent which `robotcode` command answers which question, and how to read the project the way Robot Framework does. The core habits it installs:
92+
93+
- **Inventory via [`discover`](discovering-tests.md), never by grepping files.** Which tests, tasks, suites, and tags exist is resolved at runtime — Robot's parsing rules, `robot.toml` paths, profiles, variables, and pre-run modifiers that add, remove, rename, or retag tests. A static file scan gets it wrong; `discover` runs the real resolution with the installed Robot Framework.
94+
- **Keyword and library lookup via [`libdoc`](cli.md#libdoc), before generic knowledge.** `libdoc` reflects the *installed* library versions, the project's import arguments, the Python path, and local `.resource` files — things external documentation can't see.
95+
- **Live exploration via the [REPL](repl.md).** Uncertain locators, keyword sequencing, or "does this work against the running app?" get verified interactively instead of guessed, optionally with a visible browser to watch.
96+
- **Result inspection via [`results`](analyzing-results.md), not raw `output.xml`.** Finished runs are queried for bounded summaries, listings, traces, and diffs — including CI artifacts and a colleague's run — rather than loading a potentially huge XML file into the chat.
97+
- **Static checks via [`analyze code`](analyzing-code.md)** before a run, and **runs via [`robot`](cli.md#robot)** honoring the active profile.
98+
99+
Throughout, the agent respects the project's [`robot.toml`](config.md) and profiles, so it operates on the same resolved configuration as the editor and the CLI.
100+
101+
## Give the agent project context
102+
103+
The skill teaches the agent *how* to drive `robotcode`, and `robotcode` resolves everything in your configuration at runtime — paths, profiles, tags, suites, installed library versions — so you never have to repeat that for the agent.
104+
105+
What it **can't** infer is the surrounding setup and intent. Capturing those once in your agent's instructions file — `AGENTS.md` (an emerging cross-agent convention), `CLAUDE.md` for Claude Code, `.github/copilot-instructions.md` for GitHub Copilot, or whatever your agent reads — saves it from rediscovering them on every task and keeps it from guessing wrong. Worth recording:
106+
107+
- **Environment & package manager** — which tool manages dependencies (`uv`, `hatch`, `poetry`, `pip`, …) and where the project's virtual environment lives, so the agent runs `robotcode` from the right interpreter (see [Prerequisites](#prerequisites)).
108+
- **Installed libraries and their setup steps** — especially any one-off initialization needed after an install or update. For example, Browser Library needs `rfbrowser init` to download its Playwright browsers before its keywords work; an agent that doesn't know this runs into confusing failures.
109+
- **The system under test** — what the application is, how to reach it (base URLs per environment, which profile points where), and any test data or accounts. Keep secrets out — describe *how* credentials are provided (env vars, a vault), not the values themselves.
110+
- **Project conventions** — naming, tags, suite layout, and where new keywords belong, so generated tests match what you already have.
111+
112+
Keep it short and factual; this is standing context the agent reads on every request, not a place to duplicate `robot.toml`.
113+
114+
## AI-agent detection
115+
116+
Independently of the chat plugins, the `robotcode` CLI detects when it is running inside an AI-agent session and adjusts its presentation defaults so captured output stays clean:
117+
118+
- ANSI **colors** and the **pager** are disabled, so escape sequences and paging controls don't leak into the agent's captured stdout.
119+
- [`robotcode repl`](repl.md) falls back to the **plain input backend**, so completion popups and prompt redraws don't interfere with stdin/stdout.
120+
121+
Detection is based on environment markers set by popular tools — Claude Code, Cursor, GitHub Copilot (CLI and VS Code agent flow), Codex, OpenCode, Gemini CLI, and others — plus the generic `AI_AGENT` and `AGENT` conventions. A marker counts as active when it is present with any value other than empty or `0`.
122+
123+
You rarely need to touch this, but every default can be overridden:
124+
125+
| Override | Effect |
126+
| --- | --- |
127+
| `ROBOTCODE_FORCE_AI_AGENT=1` | Force detection **on** (e.g. an agent whose marker isn't recognized yet). Wins over everything. |
128+
| `ROBOTCODE_NO_AI_AGENT=1` | Force detection **off**. Wins over tool markers, loses to `ROBOTCODE_FORCE_AI_AGENT`. |
129+
| `--color` / `--no-color`, `NO_COLOR`, `FORCE_COLOR` | Decide coloring explicitly, regardless of detection. |
130+
| `--pager` / `--no-pager` | Decide paging explicitly. |
131+
| `--plain` / `--backend`, `ROBOTCODE_REPL_PLAIN`, `ROBOTCODE_REPL_BACKEND` | Decide the REPL backend explicitly. |
132+
133+
Explicit flags and environment variables always win over auto-detection, so you can opt back into colored, paged, or full-featured output inside an agent session when you want it.
134+
135+
## Troubleshooting
136+
137+
**The agent answers from generic knowledge instead of using `robotcode`.** A skill is loaded by *relevance* — the agent matches your request against the skill's description and only pulls it in when the request reads as Robot Framework work. A terse prompt in a mixed-language repo ("run it", "fix this test") may not trigger it. Name the domain — mention Robot Framework, a suite, a profile, a keyword, or `robotcode` itself — or invoke the skill explicitly by name (it's called **`robotcode`**). In VS Code, also confirm `robotcode.ai.enableChatPlugins` is `true` — see [In VS Code](#in-vs-code-github-copilot-chat).
138+
139+
**The agent's answers don't match your project** — libraries or keywords it should see are reported as missing, or argument lists look wrong. It is most likely driving a `robotcode` from the wrong environment rather than the project's (see [Prerequisites](#prerequisites)). Have the agent run `robotcode discover info` to show which interpreter and versions it is using, and compare that against the project's environment.
140+
141+
**The agent writes a `.robot` file when you only wanted it to *do* something** — try a keyword, check a locator, watch a flow against the running app. This is the skill's most common misfire. Tell it not to write a test and to use the REPL instead ("don't write a test, just run it live"); it can save the session as a test afterwards if you ask. See [Interactive Robot Framework REPL](repl.md).
142+
143+
**A marketplace install behaves differently from the bundled VS Code plugin.** The two copies are versioned independently: the bundled one ships with the extension, the marketplace one updates through your agent's `plugin marketplace update`. If behavior diverges, update the marketplace copy — and make sure you aren't running both at once (see [Avoiding duplicates](#avoiding-duplicates)).
144+
145+
## See also
146+
147+
- [Command Line Interface](cli.md) — every `robotcode` command the agent drives.
148+
- [Discovering Tests, Tasks and Suites](discovering-tests.md) — how project inventory is resolved.
149+
- [Analyzing Run Results](analyzing-results.md) — inspecting finished runs.
150+
- [Interactive Robot Framework REPL](repl.md) — the live keyword shell.
151+
- [`robotframework-agent-plugins`](https://github.com/robotcodedev/robotframework-agent-plugins) — the marketplace and plugin source.

0 commit comments

Comments
 (0)