feat: add TUI agent harness with MCP server#548
Open
aidandaly24 wants to merge 7 commits intomainfrom
Open
Conversation
Headless terminal harness using node-pty + @xterm/headless that spawns real CLI processes in a PTY and reads screen state programmatically. Core components: - TuiSession: spawn, sendKeys, sendSpecialKey, readScreen, waitFor, close - SettlingMonitor: text-content comparison to filter cursor blink - Screen reader: viewport/scrollback reading, numbered output - Key map: named keys to escape sequence mapping - Session manager: global registry with process-exit cleanup - Availability check: graceful skip when node-pty is missing waitFor() throws WaitForTimeoutError on timeout (not silent return). launch() races settle vs process exit, throws LaunchError on crash. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
7 MCP tools exposed via stdio transport for AI agents to drive the TUI:
tui_launch, tui_send_keys, tui_read_screen, tui_wait_for,
tui_screenshot, tui_close, tui_list_sessions.
Session map with max 10 concurrent sessions. tui_wait_for catches
WaitForTimeoutError and returns {found: false} (not an MCP error).
tui_launch defaults to AgentCore CLI when no command specified.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- devDependencies: @xterm/headless, @modelcontextprotocol/sdk - optionalDependencies: node-pty (native addon, graceful skip) - esbuild: second entry point for mcp-harness bundle - vitest: new 'tui' project with fileParallelism: false - .mcp.json: MCP server discovery for Claude Code - package.json: bin entry + test:tui script Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
AGENTS.md: TUI harness section with MCP tool reference, complete 27-step create wizard example (verified against real TUI), screen identification markers table, screenshot format, error recovery patterns, navigation patterns, and known limitations. TESTING.md: TUI integration test guide with TuiSession API reference, ScreenState type, special keys list, waitFor vs settling guidance, WaitForTimeoutError output example, and LaunchError handling. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Move harness code from two separate directories (src/test-utils/tui-harness/ and src/mcp-harness/) into a single src/tui-harness/ directory with lib/ and mcp/ subdirectories. Also cleans up dead code in tools.ts, derives SpecialKey type from SPECIAL_KEY_VALUES array (single source of truth), and fixes cross-boundary import of createMinimalProjectDir. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
15 tasks
Contributor
Coverage Report
|
The harness is dev-only tooling for AI agents and integration tests. It should not ship to end users who install the CLI. - Gate MCP harness esbuild behind BUILD_HARNESS=1 env var - Remove agent-tui-harness bin entry from package.json - Add !dist/mcp-harness to files array (npm publish exclusion) - Remove node-pty from optionalDependencies (stays in devDependencies) - Add build:harness script, update test:tui to use it - Update AGENTS.md to reference npm run build:harness Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
jesseturner21
previously approved these changes
Mar 17, 2026
Contributor
jesseturner21
left a comment
jesseturner21
left a comment
There was a problem hiding this comment.
Just one nit comment about the agents.md file
Reduces AGENTS.md context overhead for agents that don't need TUI harness details. Leaves a one-line pointer to the full guide in docs/. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a TUI agent harness that enables AI agents (Claude Code, Kiro) to programmatically interact with the AgentCore CLI's Terminal UI through headless pseudo-terminals.
Architecture:
src/tui-harness/lib/) —TuiSessionclass wrappingnode-pty+@xterm/headlessfor headless terminal emulation. ProvidessendKeys,sendSpecialKey,readScreen,waitFor, andcloseoperations. Includes settling detection (waits for terminal output to stabilize), session management with signal-handler cleanup, and screen reading utilities.src/tui-harness/mcp/) — Exposes 7 tools (tui_launch,tui_send_keys,tui_read_screen,tui_wait_for,tui_screenshot,tui_close,tui_list_sessions) over stdio transport using@modelcontextprotocol/sdk. Built as a standalone binary atdist/mcp-harness/index.mjs.AGENTS.mdupdated with complete harness usage guide including screen identification markers, a 27-step create wizard example captured from ground truth, navigation patterns, and error recovery guidance.Key design decisions:
@xterm/headlessmarked asexternalin esbuild (CJS-only package, bundling mangles default export)SpecialKeytype derived fromSPECIAL_KEY_VALUESconst array (single source of truth)WaitForTimeoutErrorreturns{found: false}from MCP layer, notisError— lets agents decide next actiontui_launch({})defaults tonode dist/cli/index.mjsfor zero-config AgentCore CLI launchingRelated Issue
Closes #
Documentation PR
N/A — documentation is inline in AGENTS.md and docs/TESTING.md
Type of Change
Testing
How have you tested the change?
npm run test:unitandnpm run test:integnpm run typechecknpm run lintsrc/assets/, I rannpm run test:update-snapshotsand committed the updated snapshotsAdditional verification:
Checklist
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the
terms of your choice.