fix(playwright): pin merge-multiple=false on results download#30470
Conversation
Single-shard PR runs (spec-only mode, one chromium lane) have been failing playwright-summary with: Shard chromium-01 did not upload a usable Playwright results artifact. Shard chromium-01 did not upload its execution status. The chromium-01 shard actually uploaded playwright-results-json-chromium-01 successfully (verified against the run's artifacts API — archive contains results.json + ci-status.json at the root). The download step then reports `success` too. But render_playwright_summary.cjs expects the on-disk layout to be: results/playwright-results-json-<shardId>/results.json That layout is what `pattern: playwright-results-json-*` produces when multiple artifacts match. When only one artifact matches, some download-artifact@v7 configurations flatten the contents directly into `path:` (results/results.json), which breaks the readdirSync-based shard discovery and makes the renderer report the shard as absent. Multi-shard runs never hit this because there's always >1 artifact matched — that's why we only see the failure on impact-selected / spec-only PRs. Pinning merge-multiple=false explicitly locks the per-artifact subdirectory layout regardless of match count, so single-shard runs render the same way multi-shard runs do. Seen on https://github.com/open-metadata/OpenMetadata/actions/runs/30088248354/job/89474166397?pr=30437 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
❌ PR checklist incompleteThis PR cannot be merged until the following are addressed on its linked issue:
The fields live on the linked issue in the Shipping project (open the issue → right sidebar → Projects). After you set them, re-run this check (or push a commit) — issue/project changes do not re-trigger it automatically. Maintainers can bypass this check by adding the |
The explicit merge-multiple: false pin added in the previous commit
matches what should be the documented default for pattern-based
downloads on actions/download-artifact@v7 — so if single-shard runs
STILL report the same "Shard chromium-01 did not upload a usable
Playwright results artifact" error, the flatten hypothesis was wrong
and the real bug is elsewhere.
Add a diagnostic step that runs unconditionally on non-skipped
playwright-ci-postgresql outcomes and dumps:
- `ls -laR results/` — the actual on-disk layout the renderer sees
- the shardId values from plan-playwright.matrix (so we can diff
"expected" vs "present")
If the pin works, this step just adds a few log lines. If it doesn't,
the log tells us precisely whether files landed at
results/playwright-results-json-<shardId>/results.json (expected)
or somewhere else (e.g., results/results.json flat) so we can pick
the right follow-up without more speculation. Comment notes it's
removable once single-shard runs are consistently green.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
Pushed 9c79d87391 — added a diagnostic Reason: after re-reading the actions/download-artifact release notes, the Comment on the step notes it's removable once single-shard runs are consistently green. Also — to answer @chirag Madlani's question about mixing upload-artifact@v6 with download-artifact@v7: no compatibility issue there. Both are on the same post-v4 storage backend (v4 introduced the immutable-artifacts model); download@v7 handles v6-uploaded zips identically. The v6→v7 upload change was purely additive (an |
✅ Playwright Results — workflow succeededValidated commit ✅ 536 passed · ❌ 0 failed · 🟡 1 flaky · ⏭️ 5 skipped · 🧰 0 lifecycle flaky PerformanceBlocking targets: ✅ met · Optimization targets: 🟡 in progress Shard-job maxima below are not the full workflow wall time; the linked run includes build, fixture, planning, and reporting. 🕒 Full workflow signal wall (to summary) 49m 54s ⏱️ Max setup 2m 56s · max shard execution 18m 22s · max shard-job elapsed before upload 22m 3s · reporting 4s 🌐 202.70 requests/attempt · 2.87 app boots/UI scenario · 17.70% common-shard skew Optimization targets still in progress:
🟡 1 flaky test(s) (passed on retry)
How to debug locally# Download playwright-test-results-<shard> artifact and unzip
npx playwright show-trace path/to/trace.zip # view trace |
Code Review ✅ ApprovedPins merge-multiple=false on the Playwright results download step to preserve per-shard subdirectory structures during single-shard runs. No issues found. OptionsDisplay: compact → Showing less information. Comment with these commands to change the behavior for this request:
Was this helpful? React with 👍 / 👎 | Gitar |
Summary
Follow-up to #30468. Single-shard PR runs (spec-only mode, one chromium lane) fail `playwright-summary` with:
```
Shard chromium-01 did not upload a usable Playwright results artifact.
Shard chromium-01 did not upload its execution status.
```
Seen on PR #30437 run 30088248354.
Root cause
Verified against the run's artifacts API — `playwright-results-json-chromium-01` was uploaded correctly, archive contents:
```
results.json (441110 bytes)
ci-status.json (470 bytes)
```
The download step reports `success`, but `render_playwright_summary.cjs` expects the on-disk layout:
```
results/playwright-results-json-/results.json
```
That layout is what `pattern: playwright-results-json-*` produces when multiple artifacts match. When only one artifact matches (single-shard runs), some `download-artifact@v7` configurations flatten the contents directly into `path:` (`results/results.json`) — breaking the `readdirSync`-based shard discovery in the renderer.
Multi-shard runs never hit this because there's always >1 artifact matched — which is why we only see this on impact-selected / spec-only PRs like #30437.
Change
One-line addition to the `Download all results JSON` step:
```yaml
merge-multiple: false
```
Locks the per-artifact subdirectory layout regardless of match count, so single-shard runs render exactly the same way multi-shard runs do. Also added an inline comment explaining the failure mode + linking the reference run.
Diff scope
```
.github/workflows/playwright-postgresql-e2e.yml | 11 +++++++++++
1 file changed, 11 insertions(+)
```
Pure workflow config. No script changes.
Not in scope (deferred)
A more resilient fix would harden `render_playwright_summary.cjs` to handle either layout (loose files at `results/` root vs. per-shard subdirs). That's Option B from the earlier analysis — can be a follow-up if this pin turns out not to help on single-shard runs.
Test plan
🤖 Generated with Claude Code
Greptile Summary
Adds an explicit
merge-multiple: falsesetting to preserve per-artifact Playwright result directories and logs the downloaded layout for diagnosing single-shard summary failures.Confidence Score: 5/5
The PR appears safe to merge because no blocking failures eligible for this follow-up review remain.
No blocking failure remains.
Important Files Changed
Reviews (2): Last reviewed commit: "add diagnostic ls -R for downloaded resu..." | Re-trigger Greptile