feat(ui-test): Playwright ui tests foundation stage 0 by cjlg-soloio · Pull Request #2180 · kagent-dev/kagent

cjlg-soloio · 2026-07-08T16:06:13Z

Description

Adds the Stage 0 foundation for Playwright page-level E2E testing of the kagent UI, running against a mocked backend with no cluster required.

The suite targets the one gap nothing else covers — multi-step user flows across components:

Tool	Covers
Storybook / Chromatic	Component render states
Jest	Logic
Cypress	Page-load smoke
Playwright (this PR)	Cross-component user flows

The core design challenge: the UI fetches data server-side via Next.js server actions, so browser-level page.route("**/api/**") intercepts nothing. Instead, this PR boots a dependency-free stub backend and redirects Next's server-side fetch to it via BACKEND_INTERNAL_URL — which getBackendUrl() in ui/src/lib/utils.ts already honors. Playwright's webServer block starts both the stub (:8899) and next dev (:8001) automatically.

This is foundation-only: config, stub backend, shared fixture, CI workflow, and a single smoke test. The helper/driver library and per-test scenarios land in Stage 1; feature flows in Stage 3 (see ui/playwright/README.md).

Files

New

File	Purpose
`ui/playwright.config.ts`	Boots stub + dev server; chromium project; CI-aware retries/workers; trace & screenshot on failure
`ui/playwright/mocks/server.mjs`	Dependency-free stub backend (Node `http` only); serves happy-path payloads mirroring `src/types/index.ts`; exposes `/__mock/health` & `/__mock/reset`
`ui/playwright/mocks/data.ts`	Typed spec-side data builders mirroring the stub's shapes
`ui/playwright/fixtures/test.ts`	Shared `test`/`expect` fixture; resets stub per test and bypasses first-run onboarding
`ui/playwright/tests/smoke.spec.ts`	Stage 0 smoke test — asserts the Agents list renders against the mock
`ui/playwright/tsconfig.json`	Extends UI tsconfig with node + Playwright types
`ui/playwright/README.md`	Test strategy, mocking rationale, conventions, roadmap
`.github/workflows/ui-playwright.yaml`	Standalone CI workflow (paths-filtered to `ui/**`); installs chromium, runs `test:pw`, uploads report

Modified

ui/package.json — @playwright/test dev dep + test:pw / test:pw:ui / test:pw:debug scripts
ui/package-lock.json — lockfile updates
ui/.gitignore — ignores playwright-report, playwright/test-results, playwright/.cache

Testing

cd ui
npm install
npx playwright install chromium   # first time only
npm run test:pw

Playwright starts the stub backend and dev server automatically. The smoke test passes when the home page renders the Agents, no "Error Encountered" state appears, and no uncaught page errors occur. CI runs the same on any ui/** change.

Change type

test / ci — new test infrastructure and CI workflow. No product or runtime code changed.

Changelog

Stage 1 of 3 playwright tests for ui

Copilot

Pull request overview

Introduces a foundational Playwright E2E test setup for the UI, including a dependency-free stub backend and CI workflow, to enable end-to-end coverage of multi-step user flows against mocked API responses.

Changes:

Added Playwright configuration, fixtures, and an initial smoke spec to validate the app loads against a stubbed backend.
Implemented a standalone Node HTTP stub backend plus typed mock builders for specs.
Wired Playwright into UI scripts, gitignore, and a dedicated GitHub Actions workflow.

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 4 comments.

Show a summary per file

File	Description
ui/playwright/tsconfig.json	Adds Playwright-local TS config extending the UI tsconfig.
ui/playwright/tests/smoke.spec.ts	Adds initial Stage 0 smoke test asserting Agents page loads without error state/page errors.
ui/playwright/README.md	Documents Playwright scope, mocking approach, layout, and conventions.
ui/playwright/mocks/server.mjs	Adds dependency-free stub backend serving `/api/*` responses used by server actions.
ui/playwright/mocks/data.ts	Adds typed mock builders for assertions and future scenario plumbing.
ui/playwright/fixtures/test.ts	Adds shared fixture to reset stub and bypass onboarding via localStorage init script.
ui/playwright.config.ts	Adds Playwright runner config including webServer boot of stub + `next dev`.
ui/package.json	Adds Playwright test scripts and `@playwright/test` dev dependency.
ui/package-lock.json	Locks `@playwright/test` and updates Playwright packages.
ui/.gitignore	Ignores Playwright report/output directories.
.github/workflows/ui-playwright.yaml	Adds CI workflow to run Playwright tests on UI changes and upload reports.

Files not reviewed (1)

ui/package-lock.json: Generated file

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

mesutoezdil · 2026-07-08T19:03:41Z

+
+export default defineConfig({
+  testDir: "./playwright/tests",
+  outputDir: "./playwright/test-results",


With workers: 1 this is a no-op, but raising workers before Stage 1 isolation lands will cause tests to race against the shared stub. Consider setting this to false until then.

Good call — Fixed in 08b3cfc — set fullyParallel: false to match workers: 1, so both stay off until Stage 1 isolation lands. Thanks @mesutoezdil !

Good call — Fixed in 08b3cfc — set fullyParallel: false to match workers: 1, so both stay off until Stage 1 isolation lands. Thanks @mesutoezdil !

pls write your own sentences when interacting with me, not llm outputs. thx!

my bad! I will keep it human moving forward 🤝

mesutoezdil · 2026-07-08T19:04:14Z

dco is red

Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: cjlg-soloio <carlos.logrono@solo.io> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

- Force single Playwright worker unconditionally (shared stub + Next server) - Default req.method/req.url before parsing in the stub server - Fail fast on stub reset failure in CI; stay non-fatal locally - Drop unused pull-requests:write permission from the workflow Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Pairs fullyParallel with workers: 1 so both stay off until per-test data isolation lands, preventing tests from silently racing against the shared stub backend if the worker count is raised. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

mesutoezdil · 2026-07-08T20:57:37Z

+  "compilerOptions": {
+    "types": ["node", "@playwright/test"]
+  },
+  "include": ["**/*.ts", "**/*.mjs"]


mjs without allowJs: true is a no-op, typescript won't check server.mjs

mesutoezdil · 2026-07-08T20:57:43Z

+  projects: [{ name: "chromium", use: { ...devices["Desktop Chrome"] } }],
+  webServer: [
+    {
+      command: "node playwright/mocks/server.mjs",


server.mjs reads STUB_PORT from env but this block doesn't pin it, so a shell-set STUB_PORT makes the stub bind to the wrong port and the health check times out

I pinned STUB_PORT in the stub's webServer env so it always goes to the health endpoint or whatever BACKEND_INTERNAL_URL is set to

mesutoezdil · 2026-07-08T20:57:48Z

+    {
+      command: "npm run dev",
+      url: APP_URL,
+      reuseExistingServer: !CI,


if npm run dev is already running without BACKEND_INTERNAL_URL, reuseExistingServer reuses it and the stub is bypassed silently

changed it to boot always on BACKEND_INTERNAL_URL and made it fail loudly if 8001 is already taken

mesutoezdil · 2026-07-08T21:02:59Z

+} from "@/types";
+
+export type Namespace = { name: string; status: string };
+export type ToolServerListEntry = { ref: string; groupKind: string; discoveredTools: unknown[] };


discoveredTools is typed as unknown[] but the real ToolServerResponse from @/types uses DiscoveredTool[]. the file comment promises compile-time drift detection, but this local type bypasses it

good catch! using RemoteMCPServerResponse now

again! pls write your own sentences when interacting with me, not llm outputs. thx!

I wrote that sentence! 😅

- Pin STUB_PORT on the stub webServer so a shell-set STUB_PORT can't misbind it - Stop reusing the dev server so BACKEND_INTERNAL_URL always applies (no silent stub bypass) - Type ToolServerListEntry as the real RemoteMCPServerResponse for compile-time drift detection - Drop no-op **/*.mjs from the playwright tsconfig include Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Copilot AI review requested due to automatic review settings July 8, 2026 16:06

cjlg-soloio requested a review from peterj as a code owner July 8, 2026 16:06

Copilot started reviewing on behalf of cjlg-soloio July 8, 2026 16:13 View session

Copilot AI reviewed Jul 8, 2026

View reviewed changes

Comment thread ui/playwright.config.ts Outdated

Comment thread ui/playwright/mocks/server.mjs Outdated

Comment thread ui/playwright/fixtures/test.ts Outdated

Comment thread .github/workflows/ui-playwright.yaml Outdated

cjlg-soloio changed the title ~~Merge branch 'main' of https://github.com/kagent-dev/kagent into feat(ui)--Playwright-tests-foundation-stage-1~~ Playwright ui tests foundation stage 0 Jul 8, 2026

cjlg-soloio changed the title ~~Playwright ui tests foundation stage 0~~ feat(ui-test): Playwright ui tests foundation stage 0 Jul 8, 2026

github-actions Bot added enhancement New feature or request and removed enhancement New feature or request labels Jul 8, 2026

cjlg-soloio force-pushed the feat(ui)--Playwright-tests-foundation-stage-1 branch from 4bed054 to 4e52362 Compare July 8, 2026 18:42

mesutoezdil reviewed Jul 8, 2026

View reviewed changes

cjlg-soloio and others added 4 commits July 8, 2026 15:50

config for playwright

b83445b

Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

Run with 1 worker

6f3c96f

Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com> Signed-off-by: cjlg-soloio <carlos.logrono@solo.io> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

docs: explain defensive req.method/url defaults in stub server

be5ee4e

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Signed-off-by: Carlos Logroño Guerrero <carlos.logrono@solo.io>

cjlg-soloio force-pushed the feat(ui)--Playwright-tests-foundation-stage-1 branch from 4e52362 to be5ee4e Compare July 8, 2026 19:56

cjlg-soloio force-pushed the feat(ui)--Playwright-tests-foundation-stage-1 branch from 08b3cfc to 5d1b617 Compare July 8, 2026 20:09

mesutoezdil reviewed Jul 8, 2026

View reviewed changes

cjlg-soloio and others added 3 commits July 8, 2026 17:22

Merge branch 'main' into feat(ui)--Playwright-tests-foundation-stage-1

8181910

Merge branch 'main' into feat(ui)--Playwright-tests-foundation-stage-1

671024c

cjlg-soloio mentioned this pull request Jul 9, 2026

feat(ui-test): Playwright ui tests helpers + per-test scenarios stage 1 #2196

Draft

Merge branch 'main' into feat(ui)--Playwright-tests-foundation-stage-1

4f6b52c

Uh oh!

Conversation

cjlg-soloio commented Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Files

Testing

Change type

Changelog

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mesutoezdil Jul 8, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

mesutoezdil commented Jul 8, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

cjlg-soloio commented Jul 8, 2026 •

edited

Loading

mesutoezdil Jul 8, 2026 •

edited

Loading