diff --git a/CLAUDE.md b/CLAUDE.md index af76f8da..8e9ef734 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -24,12 +24,12 @@ Run from repo root unless noted. | `pnpm build` | Build all packages (`pnpm -r build`). | | `pnpm test` | Run vitest suite once. | | `pnpm test:watch` | Run vitest in watch mode. | -| `pnpm test:coverage` | Run vitest with v8 coverage. The thresholds in `vitest.config.ts` are the floor — drops fail CI. | +| `pnpm test:coverage` | Run vitest with v8 coverage. The thresholds in `vitest.config.ts` are aspirational, not a gate: that file states CI does not run this and the suite is currently below all four. CI runs `test`/`lint`/`test:ui`. | | `pnpm lint` | Lint all packages in parallel. Includes `eslint-plugin-security` for a subset of CodeQL findings; deeper taint-flow checks surface on the PR's CodeQL scan. | | `pnpm demo:wdio` / `pnpm demo:nightwatch` / `pnpm demo:selenium` | Run the per-framework example projects. Useful for manual verification of UI or runtime changes. | | `pnpm dev` | Run all packages in parallel dev mode. | -`selenium-devtools` exposes per-runner variants of its example via `pnpm --filter @wdio/selenium-devtools example:mocha` / `:jest` / `:cucumber` / `:jasmine` / `:vitest`. +`selenium-devtools` exposes per-runner variants of its example via `pnpm --filter @wdio/selenium-devtools example:mocha` / `:mocha:allure` / `:jest` / `:cucumber`. --- @@ -102,7 +102,7 @@ No `any` crosses a package boundary. When a framework API forces a loosely-typed - Neither is added to a bundler's `external` config. Vite's `external` callback receives both the bare package name *and* the resolved absolute path (e.g. `/Users/.../packages/core/src/index.ts`); a check for only one form silently externalizes the other. - The same callback receives bare relative imports (`./utils.js`, `../constants.js`). A check that allows only `./` will externalize `../`-style imports from subfolders and the dist crashes with `ERR_MODULE_NOT_FOUND` at install time. - `packages/service/vite.config.ts` is the canonical pattern for getting both right. -- After any change to a bundler config or build script, `grep -E "@wdio/devtools-(core|shared)|/packages/(core|shared)/" packages//dist/*.js` should return nothing. That's how you catch the absolute-path leak. +- After any change to a bundler config or build script, `grep -nE "(from|require\()\s*['\"](@wdio/devtools-(core|shared)|.*/packages/(core|shared)/)" packages//dist/*.js` should return nothing. That's how you catch the absolute-path leak. Match on the `from`/`require(` prefix, not the bare package name: `LIBRARY_NAME = "@wdio/devtools-core"` (written into the trace's `context-options`) and `Symbol.for("@wdio/devtools-core/assert-patched")` are inlined string *values* that legitimately survive bundling, so a bare-name grep always reports a false leak. Bundlers in use: **vite** for `app`, `service`, `script`; **tsup** for `backend`, `nightwatch-devtools`, `selenium-devtools`. @@ -159,7 +159,7 @@ Unused exports, unused imports, commented-out blocks, and `_unused` parameters g ## Testing -The repo uses **vitest** at the root. The current state: 1315 tests across 112 files; thresholds at `vitest.config.ts` enforce a floor of 85/77/86/85 (statements/branches/functions/lines). Coverage is ratcheted upward as gaps close, never downward. +The repo uses **vitest** at the root. The current state: 1776 tests across 139 files; thresholds at `vitest.config.ts` enforce a floor of 85/77/86/85 (statements/branches/functions/lines). Coverage is ratcheted upward as gaps close, never downward. ### What gets tested @@ -241,8 +241,37 @@ Documented divergences from the conventions above. They exist today as debt to b - **Nightwatch: `retain-on-failure` works; the other retry-aware policies degrade.** Its `--retries` re-runs the testcase *internally* without re-firing the plugin's per-test hooks, and the per-testcase results carry no attempt/retry field (retries live only in undocumented, version-varying Nightwatch internals — `suiteRetries.testRetriesCount` / `reporter.testResults.retryTest`), so the ledger sees only the final attempt for the `describe/it` and exports-object interfaces. Cucumber scenarios expose per-scenario hooks, so the feed captures their attempts. Not cleanly fixable without depending on those internals. - WDIO `specFileRetries` spawns a fresh worker per retry, so cross-process attempts aren't in the (process-scoped) ledger. - Run identity across worker sockets is env-propagated. `core/run-id.ts` `resolveRunId()` publishes `DEVTOOLS_RUN_ID` (`RUNNER_ENV.RUN_ID`) and every worker socket carries it as `?runId=` (`WORKER_WS_QUERY`), so the backend keeps accumulated run state when the *next spec's* worker connects and wipes it only for a genuinely new run. Without it every connect read as a new run: Preserve & Rerun 409'd for every spec except the last one that ran, and a dashboard opened mid-run replayed only the current spec. The WDIO service stamps it in the launcher's `onPrepare`, before workers fork, so all workers of one run agree; single-process adapters self-stamp on first use. **Gap: multi-process parallel runs in Selenium/Nightwatch** (jest/vitest workers, nightwatch `test_workers`) load the plugin per worker with no launcher-side hook to stamp first, so each worker generates its own id and still reads as a new run — the pre-fix behaviour, not a regression. Deriving the fallback from `process.ppid` would group those siblings, but would also make two sequential single-process runs share an id and inherit each other's state against a standalone dashboard, so the per-process fallback stands. -- **Chrome 150 headless drops trusted input once capture traffic is running** — a click dispatches but never navigates, so a live-mode example fails on its second test with a missing-element error that looks like a devtools bug. It is a browser regression (fixed in 151), not fixable client-side; `browser.reloadSession()` between tests works around it. Pin the example's `browserVersion` to 149 when reproducing live-mode behaviour on a 150 machine. +- **Chrome discards all WebDriver-synthesized input to a tab after a breached credential is submitted.** The first time a test types a `(username, password)` pair that Chrome's password-leak check finds in a breach corpus into an `` and submits a form whose destination no longer shows that login form, Chrome queries `passwordsleakcheck-pa.googleapis.com` and ~0.3-0.9 s later stops delivering **all** synthesized input — mouse *and* keyboard — to that tab. chromedriver returns HTTP 200 for every subsequent Element Click / Send Keys; nothing reaches the page. Untrusted JS (`element.click()`) still works and direct CDP `Input.dispatchMouseEvent`/`dispatchKeyEvent` are equally dead, so this is Chrome, not chromedriver and not our capture. `tomsmith` / `SuperSecretPassword!` — the-internet's demo credential — triggers it; changing only the *username* does not, nor does a random password. + - **Workaround: add `--host-resolver-rules=MAP passwordsleakcheck-pa.googleapis.com 127.0.0.1` to the browser args.** Both examples do. Verified 3/3 on the WDIO mocha example and on the Nightwatch example, where it also fixes the **within-one-test** logout click that a session reset never could. `--guest` also works (3/3); `--incognito` works at the raw-WebDriver level but WebdriverIO rejects it at session creation; disabling the password manager via `prefs` does **not** (6/6 still fail). + - **Not a version regression, not headless-specific, not the site, not "the Nth navigation".** Measured identically on Chrome 149.0.7827.155 / 150.0.7871.124 / 151.0.7922.77 / 152.0.7977.30 with matched chromedrivers (5/5 each), headless and headed, and on a purely local two-page static form. It fires **once per browser profile** on a wall clock — a liveness probe that never navigates again goes dead 904 ms after the submit — so the historical ~25% intermittency was the race between the next input command and that round trip. Do **not** pin `browserVersion` to 149; every part of the earlier "Chrome 150 regression, fixed in 151" attribution is contradicted. + - Minimal reproduction (own HTTP server, raw `fetch` to chromedriver, no repo, no client library, no framework) is in the session scratchpad as `minimal-repro.mjs`; it is what an upstream chromedriver bug report needs. If a session is already stuck, navigating away and back or opening a new tab restores input (4/4 each); `refresh()`, ESC, JS focus/blur and a 10 s wait do not (0/4 each). +- **Live mode has no per-action DOM snapshot, so its replay is only as fresh as the last drain.** Per-action snapshots cost two injected scripts plus a screenshot and stay trace-only; all three adapters instead drain the collector after a command that could have moved the page. Service: `#drainAfterLiveCommand`. Selenium: `commandPostActions.ts` `warrantsLiveDrain` + `SessionCapturer.drainAfterLiveCommand`, the same deny-list shape over its own command vocabulary plus `mapAssertCommand` (a node:assert row never reaches the browser) — the predicate is *not* in core because the vocabularies are per-framework and only two `includes` calls would be shared. Without it Selenium drained only at navigation, and that hook is deferred behind an injection and a 500 ms settle: measured on the login example, **2 mutation entries and 2 anchors for a 16-row run**, with the page test 1 spent most of its life on never anchored, so all 11 of its rows replayed the page the test *ended* on (2 → 24 entries, 2 → 3 anchors, 0 → 21 field-state mutations after the fix). Selenium's drain is serialized on a tail because the driver patcher does not await `onCommand`, and the app scans the mutation stream in order and stops at the first entry past a row's window — an overtaken batch strands every row after it. +- **A live client receives commands in ARRIVAL order, and one consumer assumed timeline order.** Nightwatch withholds native asserts until their outcome is known and flushes them in one batch at test-end (BDD fires `afterEach` once per *module*, so a whole module's asserts arrive after every driver row). The display list already sorts and `utils/elapsed.ts` already treats capture order as untrusted, but `app/src/components/browser/mutation-at-command.ts` bounded a row's DOM by `commands[idx + 1]` — the *array* neighbour — so a row was bounded by a time before it ran: measured, the run's last `waitForElementVisible('#username')` took its bound from an assert that had run 7.6 s earlier and replayed `/secure`. It now orders by `(startTime ?? timestamp, sequence ?? 0, array index)` — the key `buildActionEvents` uses, with the index last so a chronologically ordered array (every trace) resolves to its own successor. Measured: live **5/21 → 1/21** rows on the wrong document, trace **0/21 with 0/21 selections differing**. Live-mode anchoring itself is not the gap: `processTracePayload` sends mutations upstream unconditionally, and a live run streams one anchor per document visited. + - Residual: a submit click whose end, destination birth and next command's start land in the **same millisecond** still shows its pre-navigation page; and the app deliberately leaves the **last** row unbounded, rendering the newest DOM. +- **The Nightwatch filmstrip was never losing frames across `browser.end()`.** Instrumented: 155 poll ticks, 129 frames appended, the 26 skipped only in the null-session gap, and the login-page image present once per session in the export. It works because Nightwatch mutates `sessionId` in place on one `browser` object and the screenshot probe reads it fresh. An observed 17→6 drop was `thinScreencastFrames`' byte-identical dedup meeting a different failure profile — 10 of the 17 were a **blinking text caret** captured while a `waitForElementVisible` sat 5 s on a focused form. Don't read a low polling-mode filmstrip count as frame loss without a sha1 histogram of the emitted events. What *was* real: `#emitTestArtifacts` read `recorder?.frames ?? filmstripFrames`, so once a recorder existed it dropped every frame from before a session change. - Eager per-test trace slice (Nightwatch + Selenium) can drop an action snapshot whose fire-and-forget capture hasn't resolved by `afterEach` / scenario end — the slice is written from whatever snapshots exist at flush time. The WDIO service is immune because it awaits each snapshot inline before flushing. +- **A command row is stamped at COMPLETION, and the DOM anchor carries the document's own birth time.** These two together are what make the replay line up; both adapters got them wrong in the same way and the fix is symmetric. (a) `selenium-devtools/src/driverPatcher.ts` and `nightwatch-devtools/src/helpers/browserProxy.ts` both ran their capture at completion but stamped `timestamp` with the *invocation* clock, keeping the invocation time as `startTime` only after this fix. The page-side mutation stream is on real time, so an invocation-stamped row ended before its own effect landed and replayed the page from before it — the `#username` fill rendered an empty field, the `#password` fill rendered only the username, and a navigation row rendered the page it had just left. Rows also now span their real duration instead of a synthetic 1 ms. (b) `collector.captureCurrentDom` (the only producer of a mutation with a `url`) stamps `performance.timeOrigin`, not the drain clock. A drain is forced from Node whenever a collector might be fresh, which is always after the navigation — a round trip at best, a whole page load at worst — so drain-stamping put the anchor after several later actions (measured: 9/15 Selenium and 8/15 Nightwatch rows on the wrong DOM). With both in place a navigation row ends after its destination document was born, so the anchor needs no repositioning at all. + - `core/trace-mutations.ts` `reattributeDomAnchors` remains as a narrow backstop for the one case the stamps can't cover: an anchor born *after* the last logged command, i.e. a click whose navigation commits once the click has already returned. It snaps such an anchor to the newest logged command, but **only when no logged command completed after it** — if one did, that command's row already resolves the anchor and pulling it earlier mis-credits it to a preceding action and steals the new page's DOM from rows still on the old one (measured: a 206 ms pull moved `/login` onto two rows that were on `/add_remove_elements`). Anchors are only pulled earlier, never past the newest timestamp already in the stream, or replay would apply the outgoing document's refs to the incoming tree. + - Residual, accepted: Nightwatch's `click` resolves *before* its navigation commits (measured 5 ms), so a submit-click row can still show its pre-navigation page. Selenium is immune — its click waits for page load. Not worth another heuristic; every heuristic tried here regressed a different row. +- **Document-start injection is what removes the whole race class; everything else is reconstruction.** `