feat(trace): add action screenshot and aria snapshot events by pavelfeldman · Pull Request #42167 · microsoft/playwright

Pavel Feldman (pavelfeldman) · 2026-08-07T17:49:12Z

No description provided.

Yury Semikhatsky (yury-s) · 2026-08-07T20:19:21Z

-    const actionTitle = renderTitleForCall(metadata);
-    const utility = await page.mainFrame().utilityContext();
+    const actionTitle = renderTitleForCall(progress.metadata);
+    const utility = await progress.race(page.mainFrame().utilityContext());


what's the point in replacing one progress.race at the callsite with multiple inside the method?

Our linter demands that every awaited call needs to either race with progress or accept progress as the first argument. Here we pass it as the first arguments => must race.

Yury Semikhatsky (yury-s) · 2026-08-07T20:20:33Z

  traceSha1s: Set<string>,
  recording: boolean;
-  callIds: Set<string>;
+  callProgresses: Map<string, Progress | undefined>;


I don't see any usages of the values, why do we store them in the map?

No reason, I also wanted to make it a set. Done.

github-actions · 2026-08-07T23:27:36Z

🔴 The failure is caused by this PR

Hi, I'm the Playwright bot and I took a look at the CI failures here.

test-step.spec.ts:1264 › should report api step failure fails on ubuntu-latest-node24 and ubuntu-latest-node26 because the progress/timeout refactor in this PR now leaks a TimeoutNegativeWarning onto stdout, which breaks the test's byte-for-byte output assertion. Not a flake: it fails only on this PR's two SHAs and has never failed on any other.

Details

Caused by this PR

[playwright-test] › test-step.spec.ts:1264 › should report api step failure (node24 + node26) — the test runs page.click('input', { timeout: 1 }) and asserts the reporter output exactly. On this PR the received output gains three leading lines:
```
(node:30027) TimeoutNegativeWarning: -4.664999999999964 is a negative number.
Timeout duration was set to 1.
(Use `node --trace-warnings ...` to show where the warning was created)
```
The cause is the ProgressController.run change in packages/playwright-core/src/server/progress.ts. The deadline is now computed earlier via timeoutToDeadline(options?.timeout), then the progress object is built, and only afterwards the timer is armed with setTimeout(..., deadline - monotonicTime()). With a 1ms timeout the remaining delay is already negative by the time the timer is set, so Node ≥24 prints TimeoutNegativeWarning to stdout and pollutes the captured output. Older Node runtimes don't emit the warning, which is why only node24/node26 fail.

Fix: clamp the delay, e.g. setTimeout(..., Math.max(0, deadline - monotonicTime())).

Verification

All 4 failures in the results DB are on this PR only: SHAs 8fd27ff1 and 4c352624 (PR feat(trace): add action screenshot and aria snapshot events #42167), each on both ubuntu-latest-node24 and ubuntu-latest-node26, project playwright-test — hard failures, not retry flakes.
The same (playwright-test, test-step.spec.ts, should report api step failure) passed 4473 of 4477 runs across 646 SHAs; the only 4 failures are the ones above. No pre-existing history of this test failing.

The flaky entries in the reports (video/screencast, heap, page-event-request, ui-mode, locator-query, etc.) all passed on retry and don't touch the progress/timeout path, so there's nothing to triage there.

_{Triaged by the Playwright bot.}

_{Triaged by the Playwright bot - agent run}

github-actions · 2026-08-08T00:41:44Z

🟡 Probably not this PR, but one failure I can't fully clear

Hi, I'm the Playwright bot and I took a look at the CI failures here.

The latest run is clean on tests 1 (0 failures — the earlier test-step.spec.ts:1264 regression is gone in the newer commit). The MCP run has 3 failures, none of which I can pin on this PR: two are pre-existing flakes on the same Windows-Firefox bot, and one is a macOS-Chrome network race with too thin a history to certify. This PR touches the shared input/instrumentation path, but a real regression there would fail across browsers and OSes — not only one already-flaky combo.

Details

Overall: No failure is convincingly caused by this PR. All three are on a single MCP run (31230535587), each on one bot only, and every one of these tests passes on the other ~13 bots in the same run.

Pre-existing flake / infra

[firefox] › mcp/cli-core.spec.ts:140 › uncheck (mcp-windows-latest-firefox) — established flake on this exact bot. Same (firefox, cli-core.spec.ts, uncheck) has failed on 4 unrelated SHAs: push 30d5b2ec (no PR) and PRs test(websocket): cover setExtraHTTPHeaders and locale on the handshake request #42037, feat(mcp): default codegen language from PW_LANG_NAME #42150, feat(test-runner): annotate serial suites for custom sharding #42164. Signature is a mid-action snapshot race — Expected "checkbox [active]" vs Received "checkbox [checked]". Independent of this change.
[firefox] › mcp/cli-core.spec.ts:70 › dblclick (mcp-windows-latest-firefox) — failed in the same run and bot as uncheck above, i.e. one bad Windows-Firefox episode taking both input tests down together (toContain on an undefined response). The same test has also flaked on unrelated PRs feat(firefox): roll to r1538 #41938 (msedge) and test(mcp): diagnose startup stalls (do not merge) #41947 (chrome). Passes on Firefox everywhere else in this run.

Uncertain

[chrome] › mcp/network.spec.ts:210 › browser_network_requests lists a failed request once (mcp-macos-latest-chrome) — toHaveLength expected 1, received 0: the failed request wasn't captured in time, a classic network-timing race. For this exact (chrome, …) identity it's the only failure in the DB (1 of 1912 runs), so I can't call it a pre-existing flake by the strict bar. It has flaked once before on a sibling project (firefox, PR fix(aria): keep icon-only clickable elements in ai snapshots #42034), it passed on every other bot in this run, and this PR's changes don't touch MCP network capture — so most likely infra noise, but I couldn't prove it. A rerun of that job would settle it.

Why not the PR

The input-path edit in packages/playwright-core/src/server/input.ts (onBeforeInputAction now takes progress directly) is shared, browser-agnostic server code. If it broke click/dblclick/uncheck it would fail on Chromium and WebKit too, not just one Windows-Firefox bot — and it doesn't.

_{Triaged by the Playwright bot - agent run}

github-actions · 2026-08-08T19:30:16Z

Test results for "tests 1"

7 flaky

⚠️ [chromium-library] › library/chromium/oopif.spec.ts:150 › should take screenshot `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@frozen-time-library-chromium-linux`
⚠️ [chromium-library] › library/video.spec.ts:736 › screencast › should work with video+trace `@chromium-ubuntu-22.04-arm-node20`
⚠️ [chromium-library] › library/chromium/chromium.spec.ts:371 › should produce network events, routing, and annotations for Service Worker `@chromium-ubuntu-22.04-node20`
⚠️ [firefox-library] › library/heap.spec.ts:223 › should not leak workers `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-emulate-media.spec.ts:144 › should keep reduced motion and color emulation after reload `@firefox-ubuntu-22.04-node20`
⚠️ [firefox-page] › page/page-goto.spec.ts:90 › should work with Cross-Origin-Opener-Policy `@firefox-ubuntu-22.04-node20`

51035 passed, 1190 skipped

Merge workflow run.

github-actions · 2026-08-08T19:33:48Z

Test results for "MCP"

8063 passed, 1284 skipped

Merge workflow run.

This comment has been minimized.

Sign in to view

Yury Semikhatsky (yury-s) reviewed Aug 7, 2026

View reviewed changes

Yury Semikhatsky (yury-s) approved these changes Aug 7, 2026

View reviewed changes

Pavel Feldman (pavelfeldman) force-pushed the trace-aria-screenshot-events branch from 513262b to eb99642 Compare August 7, 2026 22:39

This comment has been minimized.

Sign in to view

feat(trace): add action screenshot and aria snapshot events

26dee18

Pavel Feldman (pavelfeldman) force-pushed the trace-aria-screenshot-events branch from d6c0188 to 26dee18 Compare August 8, 2026 18:48

Pavel Feldman (pavelfeldman) merged commit b4e7c87 into microsoft:main Aug 9, 2026
46 checks passed

Uh oh!

Conversation

Pavel Feldman (pavelfeldman) commented Aug 7, 2026

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

Uh oh!

Uh oh!

Yury Semikhatsky (yury-s) Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

Pavel Feldman (pavelfeldman) Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

Yury Semikhatsky (yury-s) Aug 7, 2026

Choose a reason for hiding this comment

Uh oh!

Pavel Feldman (pavelfeldman) Aug 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

github-actions Bot commented Aug 7, 2026

🔴 The failure is caused by this PR

Uh oh!

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

This comment has been minimized.

github-actions Bot commented Aug 8, 2026

🟡 Probably not this PR, but one failure I can't fully clear

Uh oh!

github-actions Bot commented Aug 8, 2026

Test results for "tests 1"

Uh oh!

github-actions Bot commented Aug 8, 2026

Test results for "MCP"

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Pavel Feldman (pavelfeldman) Aug 7, 2026 •

edited

Loading