docs: add playwright a11y tests for the composition app

.github/workflows/cps-shared-ui-checkers.yml

@@ -155,6 +155,24 @@ jobs:
           job-summary: true
           custom-info: ${{ steps.artifact-links.outputs.msg }}
 
+  pa11y:
+    runs-on: ubuntu-latest
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v4
+
+      - name: Setup Node.js
+        uses: actions/setup-node@v4
+        with:
+          node-version: 22.x
+          cache: npm
+
+      - name: Install dependencies
+        run: npm ci
+
+      - name: Run pa11y accessibility checks
+        run: npm run test:pa11y:ci
+
   generate-api-data:
     runs-on: ubuntu-latest
     steps:

.pa11yci

@@ -3,8 +3,19 @@
     "standard": "WCAG2AA",
     "timeout": 30000,
     "wait": 1000,
-    "runners": ["axe"],
-    "hideElements": ".cps-loader",
+    "runners": [
+      "axe"
+    ],
+    "rules": [
+      "target-size"
+    ],
+    "chromeLaunchConfig": {
+      "args": [
+        "--no-sandbox",
+        "--disable-setuid-sandbox"
+      ]
+    },
+    "hideElements": ".cps-loader-overlay-content-text",
     "reporter": "cli",
     "level": "error"
   },

README.md

@@ -7,6 +7,10 @@ This repository consists of two projects:
 - `cps-ui-kit` - shared components library itself
 - `composition` - application for previewing compositions of components consumed from the library
 
+#### Accessibility
+
+This library's components and the composition app are tested for WCAG 2.2 AA compliance both with automated tooling ([axe-core](https://github.com/dequelabs/axe-core), via a Playwright test suite and pa11y-ci) and manual accessibility review. See [Run accessibility tests](#run-accessibility-tests) for how to run the automated checks yourself.
+
 #### Available components
 
 - Autocomplete
@@ -104,13 +108,13 @@ Versioning and the changelog are driven by [Conventional Commits](https://www.co
 
 **Type** determines the version bump and changelog section:
 
-| Type                                                          | Bump      | Changelog                  |
-| ------------------------------------------------------------- | --------- | -------------------------- |
-| `feat`                                                        | **minor** | Features                   |
-| `fix`                                                         | **patch** | Bug Fixes                  |
-| `perf`                                                        | **patch** | Performance Improvements   |
-| `feat!` / `fix!` / `BREAKING CHANGE:` footer                  | **major** | Breaking section           |
-| `docs`, `style`, `refactor`, `test`, `build`, `ci`, `chore`   | none      | hidden                     |
+| Type                                                        | Bump      | Changelog                |
+| ----------------------------------------------------------- | --------- | ------------------------ |
+| `feat`                                                      | **minor** | Features                 |
+| `fix`                                                       | **patch** | Bug Fixes                |
+| `perf`                                                      | **patch** | Performance Improvements |
+| `feat!` / `fix!` / `BREAKING CHANGE:` footer                | **major** | Breaking section         |
+| `docs`, `style`, `refactor`, `test`, `build`, `ci`, `chore` | none      | hidden                   |
 
 **Scope** _(optional)_ is the component the change targets, written as the component name **without the `cps-` prefix**. It only affects how the entry is grouped/labelled in the changelog (this is a single package, so the scope does not change versioning). Use the component name from the list above:
 
@@ -140,9 +144,21 @@ The current released version is tracked in [`.release-please-manifest.json`](.re
 
 #### Run accessibility tests
 
-The project uses [pa11y-ci](https://github.com/pa11y/pa11y-ci) to test all components for WCAG 2.0 AA compliance.
+Accessibility is covered by two complementary tools, run separately and by different CI jobs:
+
+**Playwright + axe-core** — scans individual `cps-ui-kit` components (in isolation) and the full `composition` app (pages, shell, interactive states) against WCAG 2.0/2.1/2.2 A/AA + best-practice rules.
+
+```bash
+npm run test:playwright:accessibility              # everything (cps-ui-kit + composition)
+npm run test:playwright:cps-ui-kit:accessibility    # cps-ui-kit components only
+npm run test:playwright:composition:accessibility   # composition app only
+```
+
+These run as part of the `playwright` CI job. See [playwright/README.md](playwright/README.md) for full details, including how component/page entries are structured.
+
+**pa11y-ci** — scans all 33 composition demo pages (one per component) against WCAG 2.0 AA, using axe-core as its underlying test engine. This is a separate, independent check from the Playwright one above and runs as its own `pa11y` CI job.
 
-To run accessibility tests:
+To run it manually:
 
 1. Start the development server:
 
@@ -152,27 +168,25 @@ To run accessibility tests:
 
 2. In a separate terminal, run the accessibility tests:
    ```bash
-   npm run test:a11y
+   npm run test:pa11y
    ```
 
-Alternatively, use the combined script that starts the server and runs tests:
+Alternatively, use the combined script that starts the server and shows a colorful summary with statistics:
 
 ```bash
-npm run test:a11y:local
+npm run test:pa11y:local
 ```
 
-For a colorful summary with statistics:
+`npm run test:pa11y:summary` produces that same summary, but assumes the server is already running.
 
-```bash
-npm run test:a11y:summary
-```
+`npm run test:pa11y:ci` is the CI equivalent of `test:pa11y:local` — it starts the server and runs the plain `test:pa11y` reporter (full per-URL violation output) instead of the summary. This is what the `pa11y` CI job runs.
 
-This will display:
+The summary variants display:
 
 - Total URLs tested with pass/fail ratio
 - Total accessibility errors found
 - Accessibility standard (WCAG 2.0 AA)
 - Test engine (axe-core via pa11y-ci)
 - Top 10 components with the most issues
 
-The tests will check all 33 components for accessibility issues and report any violations found.
+Both `test:pa11y` and `test:pa11y:ci` fail (non-zero exit code) if any accessibility error is found on any page.

package.json

@@ -15,9 +15,10 @@
     "lint": "eslint \"**/*.ts\"",
     "typecheck": "tsc --noEmit",
     "generate-json-api": "node ./api-generator/api-generator.js",
-    "test:a11y": "pa11y-ci --threshold 1000",
-    "test:a11y:summary": "pa11y-ci --threshold 1000 --reporter=json > .pa11y-temp.json 2>/dev/null && jq -C '{\"Total URLs tested\": .total, \"Passed\": \"\\(.passes)/\\(.total)\", \"Total errors found\": .errors, \"Standard\": \"WCAG 2.0 AA\", \"Test engine\": \"axe-core via pa11y-ci\", \"Top 10 components with errors\": (.results | to_entries | map({component: (.key | split(\"/\") | .[-2]), errors: .value | length}) | sort_by(-.errors) | .[0:10])}' .pa11y-temp.json && rm -f .pa11y-temp.json",
-    "test:a11y:local": "npm run start & sleep 10 && npm run test:a11y:summary && kill %1",
+    "test:pa11y": "pa11y-ci",
+    "test:pa11y:ci": "npm run start & SERVER_PID=$!; for i in $(seq 1 60); do curl -sf http://localhost:4200 > /dev/null && break; sleep 1; done; npm run test:pa11y; TEST_EXIT=$?; kill $SERVER_PID; exit $TEST_EXIT",
+    "test:pa11y:summary": "pa11y-ci --reporter=json > .pa11y-temp.json 2>/dev/null && jq -C '{\"Total URLs tested\": .total, \"Passed\": \"\\(.passes)/\\(.total)\", \"Total errors found\": .errors, \"Standard\": \"WCAG 2.0 AA\", \"Test engine\": \"axe-core via pa11y-ci\", \"Top 10 components with errors\": (.results | to_entries | map({component: (.key | split(\"/\") | .[-2]), errors: .value | length}) | map(select(.errors > 0)) | sort_by(-.errors) | .[0:10])}' .pa11y-temp.json && rm -f .pa11y-temp.json",
+    "test:pa11y:local": "npm run start & SERVER_PID=$!; for i in $(seq 1 60); do curl -sf http://localhost:4200 > /dev/null && break; sleep 1; done; npm run test:pa11y:summary; TEST_EXIT=$?; kill $SERVER_PID; exit $TEST_EXIT",
     "test:playwright": "playwright test",
     "test:playwright:headed": "playwright test --headed",
     "test:playwright:interactive": "playwright test --ui",
@@ -27,8 +28,11 @@
     "test:playwright:webkit": "playwright test --project=webkit",
     "test:playwright:webkit:headed": "playwright test --project=webkit --headed",
     "test:playwright:accessibility": "playwright test --project=accessibility",
-    "test:playwright:accessibility:matrix": "node playwright/a11y-matrix-generator.js",
-    "test:playwright:accessibility:matrix:from-cache": "node playwright/a11y-matrix-generator.js --from-file playwright-report/a11y-report.json",
+    "test:playwright:composition:all": "playwright test playwright/composition --project=chromium --project=webkit --project=accessibility",
+    "test:playwright:composition:accessibility": "playwright test playwright/composition/accessibility-composition.spec.ts --project=accessibility",
+    "test:playwright:cps-ui-kit:all": "playwright test playwright/cps-ui-kit --project=chromium --project=webkit --project=accessibility",
+    "test:playwright:cps-ui-kit:accessibility": "playwright test playwright/cps-ui-kit/accessibility-cps-ui-kit.spec.ts --project=accessibility",
+    "test:playwright:cps-ui-kit:components": "playwright test playwright/cps-ui-kit/components --project=chromium --project=webkit",
     "format": "prettier --write \"**/*.{ts,html}\""
   },
   "private": true,

playwright.config.ts

@@ -45,17 +45,17 @@ export default defineConfig({
   projects: [
     {
       name: 'chromium',
-      testIgnore: /cps-accessibility/,
+      testIgnore: /accessibility/,
       use: { ...devices['Desktop Chrome'] }
     },
     {
       name: 'webkit',
-      testIgnore: /cps-accessibility/,
+      testIgnore: /accessibility/,
       use: { ...devices['Desktop Safari'] }
     },
     {
       name: 'accessibility',
-      testMatch: /cps-accessibility/,
+      testMatch: /accessibility/,
       use: { ...devices['Desktop Chrome'] }
     }
   ],

playwright/README.md

@@ -1,36 +1,60 @@
 # Playwright E2E Tests
 
-End-to-end tests for the CPS UI Kit components, using [Playwright](https://playwright.dev/).
+End-to-end and accessibility tests for the CPS UI Kit components and the composition app, using [Playwright](https://playwright.dev/).
 
 ## Structure
 
 ```
 playwright/
-├── cps-xxx.spec.ts            # Components tests
-├── fixtures/                  # Test fixture files (e.g. expected xlsx exports)
-└── tsconfig.json              # TypeScript config for editor support
+├── cps-ui-kit/
+│   ├── accessibility-cps-ui-kit.spec.ts   # axe-core scans, per cps-ui-kit component
+│   └── components/                        # Functional/behavioral component tests
+│       ├── cps-autocomplete.spec.ts
+│       ├── cps-scheduler.spec.ts
+│       └── cps-table.spec.ts
+├── composition/
+│   └── accessibility-composition.spec.ts  # axe-core scans across composition app routes
+├── fixtures/
+│   ├── axe-helpers.ts                     # Shared axe-core Playwright fixture + assertion helpers
+│   ├── composition-components.ts          # Component/page registry used to drive scans
+│   └── table_6_fixture.xlsx               # Test data (e.g. expected xlsx exports)
+└── tsconfig.json                          # TypeScript config for editor support
 ```
 
 Tests run against the composition app (`ng serve`) which showcases every component.
 
+Accessibility coverage is split from functional coverage by filename: any spec file with `accessibility` in its name is routed to the dedicated `accessibility` project (see [Browser projects](#browser-projects)); everything else runs under `chromium`/`webkit`. See the root [README's "Run accessibility tests" section](../README.md#run-accessibility-tests) for how this relates to the separate `pa11y-ci` check.
+
 ## Running Tests
 
 ```bash
-# Headless (all browsers)
+# Headless (all projects: chromium, webkit, accessibility)
 npm run test:playwright
 
-# Single browser
+# Single browser (functional tests only, accessibility spec files excluded)
 npm run test:playwright:chromium
 npm run test:playwright:webkit
 
 # Headed (see the browser)
 npm run test:playwright:headed
+npm run test:playwright:chromium:headed
+npm run test:playwright:webkit:headed
 
 # Interactive UI mode
 npm run test:playwright:interactive
 
 # View last HTML report
 npm run test:playwright:report
+
+# Accessibility scans only (Desktop Chrome, both cps-ui-kit and composition)
+npm run test:playwright:accessibility
+
+# Scoped slices
+npm run test:playwright:cps-ui-kit:all             # cps-ui-kit: components + accessibility, chromium+webkit+accessibility
+npm run test:playwright:cps-ui-kit:accessibility   # cps-ui-kit accessibility scans only
+npm run test:playwright:cps-ui-kit:components       # cps-ui-kit functional component tests only, chromium+webkit
+npm run test:playwright:composition:all             # composition: accessibility, chromium+webkit+accessibility
+npm run test:playwright:composition:accessibility   # composition accessibility scans only
 ```
 
 Playwright auto-starts the dev server (`npm run start`) if it isn't already running. If you already have `ng serve` running on port 4200, it will reuse that.
@@ -53,9 +77,9 @@ All config lives in [`playwright.config.ts`](../playwright.config.ts) at the pro
 | Scope                                    | Value |
 | ---------------------------------------- | ----- |
 | Test (`timeout`)                         | 30 s  |
-| Action (`actionTimeout`)                 | 30 s  |
+| Action (`actionTimeout`)                 | 5 s   |
 | Assertion (`expect.timeout`)             | 5 s   |
-| Navigation (`navigationTimeout`)         | 30 s  |
+| Navigation (`navigationTimeout`)         | 5 s   |
 | Dev server startup (`webServer.timeout`) | 120 s |
 
 ### Failure artifacts
@@ -72,17 +96,25 @@ These are written to `test-results/` and the HTML report goes to `playwright-rep
 
 The GitHub Actions workflow uploads artifacts and posts a PR comment summarising results:
 
-| Artifact / action          | Condition | Retention |
-| -------------------------- | --------- | --------- |
-| HTML report                | Always    | 10 days   |
-| Test results (screenshots, videos, traces) | On failure | 10 days |
-| PR comment (via `daun/playwright-report-summary`) | Always | — |
+| Artifact / action                                 | Condition  | Retention |
+| ------------------------------------------------- | ---------- | --------- |
+| HTML report                                       | Always     | 10 days   |
+| Test results (screenshots, videos, traces)        | On failure | 10 days   |
+| PR comment (via `daun/playwright-report-summary`) | Always     | —         |
 
 The PR comment includes direct download links to the uploaded artifacts.
 
 ### Browser projects
 
-Two browser projects are configured: **Chromium** and **WebKit**. Running `npm run test:playwright` executes against both. Use `--project=chromium` (or the npm script shortcuts) to target one.
+Three projects are configured in `playwright.config.ts`:
+
+| Project         | Browser        | Runs                                                |
+| --------------- | -------------- | --------------------------------------------------- |
+| `chromium`      | Desktop Chrome | Any spec file _without_ `accessibility` in its name |
+| `webkit`        | Desktop Safari | Any spec file _without_ `accessibility` in its name |
+| `accessibility` | Desktop Chrome | Any spec file _with_ `accessibility` in its name    |
+
+Running `npm run test:playwright` executes all three. Accessibility (axe-core) scans only run once, on Chrome — there's no need for cross-browser variation there since axe evaluates rendered DOM/ARIA state, not browser-specific rendering quirks. Functional/behavioral tests run on both `chromium` and `webkit`. Use `--project=<name>` (or the npm script shortcuts above) to target one.
 
 ### Web server
 
@@ -111,9 +143,11 @@ Install the [Playwright Test for VS Code](https://marketplace.visualstudio.com/i
 
 ## CI
 
-Playwright tests run as a separate job in the GitHub Actions workflow (`.github/workflows/cps-shared-ui-checkers.yml`). On failure, the HTML report and test artifacts (screenshots, videos) are uploaded as workflow artifacts.
+Playwright tests run as a separate job in the GitHub Actions workflow (`.github/workflows/cps-shared-ui-checkers.yml`). On failure, the HTML report and test artifacts (screenshots, videos) are uploaded as workflow artifacts. This is a separate job from the `pa11y` one, which runs `pa11y-ci` independently — see the root [README](../README.md#run-accessibility-tests) for how the two relate.
 
 ## Writing Tests
 
-- Test files go in `playwright/` and must end with `.spec.ts`
+- Functional/behavioral test files go in `playwright/cps-ui-kit/components/` or alongside the relevant app area, and must end with `.spec.ts`
+- Accessibility scan files must have `accessibility` in their filename so they're routed to the `accessibility` project — see `accessibility-cps-ui-kit.spec.ts` and `accessibility-composition.spec.ts` for the existing pattern
 - Put test data files in `playwright/fixtures/`
+- Use the `makeAxeBuilder` fixture from `playwright/fixtures/axe-helpers.ts` for any new axe-core scan, so the WCAG tag set (`wcag2a`, `wcag2aa`, `wcag21a`, `wcag21aa`, `wcag22a`, `wcag22aa`, `best-practice`) stays consistent across the suite