chore: adopt openadapt-capture 0.5.4 and run the adapter tests in CI

.github/workflows/ci.yml

@@ -239,8 +239,18 @@ jobs:
           cache: pip
           cache-dependency-path: pyproject.toml
 
+      # The `capture` extra rides along so the desktop on-ramp adapter tests
+      # (tests/test_capture_adapter.py) run for REAL in this job instead of
+      # silently skipping: openadapt-capture >=0.5.4 imports clean headless
+      # (no screenshot at import), so no display is needed.
       - name: Install
-        run: pip install -e .[dev]
+        run: pip install -e '.[dev,capture]'
+
+      # Guard the invariant the adapter tests rely on: the capture extra must
+      # import headless. If this ever regresses upstream, fail here with a
+      # clear message rather than deep inside test collection.
+      - name: Verify openadapt-capture imports headless
+        run: python -c "import openadapt_capture"
 
       # Cache the Playwright browser binaries (chromium) so the lightweight
       # browser-backed unit tests don't re-download ~150MB every run. Keyed on

claims.yaml

@@ -266,7 +266,8 @@ claims:
         proves: >-
           The capture->recording bridge over a REAL openadapt-capture session
           (its own event-processing pipeline + frame extraction) feeding the
-          UNMODIFIED compiler.
+          UNMODIFIED compiler (runs on default CI: the `test` job installs the
+          `capture` extra; openadapt-capture >=0.5.4 imports clean headless).
       - path: docs/desktop/RECORDING.md
         kind: doc
         proves: >-
@@ -278,8 +279,9 @@ claims:
         Offline capture carries NO structural (UIA AutomationId) locator, so
         replay uses the visual ladder (template/ocr/geometry). The live capture
         needs a display (openadapt-capture is the optional `capture` extra); the
-        CI proof covers convert -> compile -> replay on the recording format,
-        not a live capture session.
+        CI proof covers a real on-disk capture session -> convert -> compile ->
+        replay, not a live recorder run (the live recorder has its own
+        integration suite in the openadapt-capture repo).
       - >-
         `--secret` is refused on the pixel/desktop substrate (no field geometry
         to redact), and `rdp` recording must be captured in the same pixel space

docs/VERIFICATION.md

@@ -147,12 +147,12 @@
 | Backing evidence | Kind | Gating / CI stage | Strength | Proves |
 |---|---|---|---|---|
 | `tests/test_desktop_record.py` | test | ci (required PR gate (test)) | supported | The live-capture orchestration + CLI wiring (record --backend windows|rdp -> openadapt-capture -> convert_capture), and that a desktop-shaped recording COMPILES into a bundle and REPLAYS to completion through the desktop backend path, resolving each click to its recorded target (runs on default CI). |
-| `tests/test_capture_adapter.py` | test | ci (required PR gate (test)) | supported | The capture->recording bridge over a REAL openadapt-capture session (its own event-processing pipeline + frame extraction) feeding the UNMODIFIED compiler. |
+| `tests/test_capture_adapter.py` | test | ci (required PR gate (test)) | supported | The capture->recording bridge over a REAL openadapt-capture session (its own event-processing pipeline + frame extraction) feeding the UNMODIFIED compiler (runs on default CI: the `test` job installs the `capture` extra; openadapt-capture >=0.5.4 imports clean headless). |
 | `docs/desktop/RECORDING.md` | doc | artifact (doc/benchmark) | roadmap | The precise REAL-vs-deferred map (visual-ladder-only structural rung, secret redaction, rdp coordinate space) and the reuse of openadapt-capture + the capture adapter. |
 
 **Caveats (honest limits):**
 
-- Offline capture carries NO structural (UIA AutomationId) locator, so replay uses the visual ladder (template/ocr/geometry). The live capture needs a display (openadapt-capture is the optional `capture` extra); the CI proof covers convert -> compile -> replay on the recording format, not a live capture session.
+- Offline capture carries NO structural (UIA AutomationId) locator, so replay uses the visual ladder (template/ocr/geometry). The live capture needs a display (openadapt-capture is the optional `capture` extra); the CI proof covers a real on-disk capture session -> convert -> compile -> replay, not a live recorder run (the live recorder has its own integration suite in the openadapt-capture repo).
 - `--secret` is refused on the pixel/desktop substrate (no field geometry to redact), and `rdp` recording must be captured in the same pixel space the backend replays in (no cross-machine coordinate remap yet).
 
 ### `citrix-pixel-validating` — validating — opt-in / infra-gated or field test

docs/desktop/RECORDING.md

@@ -31,7 +31,9 @@ is a thin orchestration over two components that already exist and are tested:
    (`meta.json` + `events.jsonl` + `frames/{i:04d}_before.png` / `_after.png`),
    running openadapt-capture's own event-processing pipeline (raw streams →
    merged clicks / typed text). This adapter is unit-tested end to end against a
-   real capture session in `tests/test_capture_adapter.py`.
+   real capture session in `tests/test_capture_adapter.py`, and those tests run
+   on default CI: the fast unit `test` job installs the `capture` extra
+   (openadapt-capture >=0.5.4 imports clean headless, so no display is needed).
 
 The genuinely new piece (`openadapt_flow/desktop_record.py`,
 `record_desktop_capture`) is the **live orchestration**: start a capture

docs/verification.json

@@ -381,7 +381,7 @@
       ],
       "strongest_evidence": "supported",
       "caveats": [
-        "Offline capture carries NO structural (UIA AutomationId) locator, so replay uses the visual ladder (template/ocr/geometry). The live capture needs a display (openadapt-capture is the optional `capture` extra); the CI proof covers convert -> compile -> replay on the recording format, not a live capture session.",
+        "Offline capture carries NO structural (UIA AutomationId) locator, so replay uses the visual ladder (template/ocr/geometry). The live capture needs a display (openadapt-capture is the optional `capture` extra); the CI proof covers a real on-disk capture session -> convert -> compile -> replay, not a live recorder run (the live recorder has its own integration suite in the openadapt-capture repo).",
         "`--secret` is refused on the pixel/desktop substrate (no field geometry to redact), and `rdp` recording must be captured in the same pixel space the backend replays in (no cross-machine coordinate remap yet)."
       ],
       "evidence": [
@@ -405,7 +405,7 @@
           "node": null,
           "node_found": null,
           "junit_status": null,
-          "proves": "The capture->recording bridge over a REAL openadapt-capture session (its own event-processing pipeline + frame extraction) feeding the UNMODIFIED compiler."
+          "proves": "The capture->recording bridge over a REAL openadapt-capture session (its own event-processing pipeline + frame extraction) feeding the UNMODIFIED compiler (runs on default CI: the `test` job installs the `capture` extra; openadapt-capture >=0.5.4 imports clean headless)."
         },
         {
           "path": "docs/desktop/RECORDING.md",

openadapt_flow/adapters/capture.py

@@ -57,12 +57,13 @@
 Coordinate spaces: capture mouse coordinates are in *logical points* (pynput);
 video frames are *physical pixels*. openadapt-flow requires event coordinates in
 the same pixel space as the frames, so points are scaled by
-``CaptureSession.pixel_ratio`` (physical / logical). NOTE: capture 0.5.x only
-persists ``pixel_ratio`` when the recorder wrote it into the recording
-``config`` JSON; absent that it defaults to 1.0 and coordinates pass through
-unscaled. On a HiDPI capture whose ``pixel_ratio`` was not persisted, click
-coordinates would be under-scaled — an honest limitation of the upstream
-metadata, not something this adapter can recover from pixels alone.
+``CaptureSession.pixel_ratio`` (physical / logical). NOTE: sessions recorded
+with capture >=0.5.4 persist ``pixel_ratio`` on the recording model itself, so
+scaling is always correct for them. Older 0.5.x sessions carry it only when
+the recorder wrote it into the recording ``config`` JSON; absent that it
+defaults to 1.0 and coordinates pass through unscaled — on such a legacy HiDPI
+session click coordinates would be under-scaled, an honest limitation of the
+old metadata that this adapter cannot recover from pixels alone.
 
 Frame selection: openadapt-capture records **action-gated** video (frames are
 encoded around user actions, not continuously). For an event at wall-clock time

pyproject.toml

@@ -108,10 +108,11 @@ service-mlx = [
 ]
 # Cross-platform desktop recording on-ramp: convert an openadapt-capture
 # session into a flow recording (openadapt_flow.adapters.capture). Optional and
-# imported lazily — the flow core never pulls it onto the replay hot path. Note:
-# openadapt-capture screenshots the display when imported, so this on-ramp needs
-# a real desktop session (not headless CI).
-capture = ["openadapt-capture>=0.5.1"]
+# imported lazily — the flow core never pulls it onto the replay hot path.
+# Floor 0.5.4: imports clean headless (no screenshot at import, fixed in
+# 0.5.3/0.5.4) and persists pixel_ratio on the recording model, so the adapter
+# tests run for real in headless CI (the `test` job installs this extra).
+capture = ["openadapt-capture>=0.5.4"]
 # Canonical action-vocabulary interop (openadapt_flow.interop.types). Boundary
 # only (emit / benchmark round-trip); never on the replay hot path. The shim
 # imports openadapt-types lazily, so the core stays importable without it.

tests/test_capture_adapter.py

@@ -10,9 +10,10 @@
 pass against a schema that no longer exists. The converted recording is fed to
 the UNMODIFIED compiler — the format bridge, proven end to end.
 
-openadapt-capture screenshots the display at import time, so the whole module
-is skipped when the package cannot be imported (headless CI / no display); it
-runs for real on a developer desktop with the ``capture`` extra installed.
+openadapt-capture >=0.5.4 imports clean headless (the historical
+screenshot-at-import side effect was removed in 0.5.3/0.5.4), so this module
+runs for real in headless CI — the `test` job installs the ``capture`` extra.
+It is skipped only when that optional extra is not installed.
 """
 
 from __future__ import annotations
@@ -23,24 +24,19 @@
 import numpy as np
 import pytest
 
-# openadapt-capture is an optional extra AND screenshots the display when
-# imported (a module-level side effect in its recorder). Skip the whole module
-# if it is absent or cannot be imported in this environment.
+# openadapt-capture is an optional extra: skip only when it is not installed.
+# Since 0.5.4 the import is headless-clean (no screenshot at import), so when
+# the extra IS installed — as in CI's `test` job — an import failure is a real
+# regression and must fail loudly here instead of silently skipping.
 if importlib.util.find_spec("openadapt_capture") is None:
     pytest.skip(
         "openadapt-capture not installed (capture extra)", allow_module_level=True
     )
-try:  # importing the package screenshots the display; skip if that fails
-    import openadapt_capture  # noqa: F401
-    from openadapt_capture.db import create_db
-    from openadapt_capture.db.models import ActionEvent, Recording
-    from openadapt_capture.video import VideoWriter
-except Exception as exc:  # pragma: no cover - environment dependent
-    pytest.skip(
-        f"openadapt-capture cannot be imported here ({exc})",
-        allow_module_level=True,
-    )
 
+import openadapt_capture  # noqa: F401
+from openadapt_capture.db import create_db
+from openadapt_capture.db.models import ActionEvent, Recording
+from openadapt_capture.video import VideoWriter
 from PIL import Image, ImageDraw
 
 from openadapt_flow.adapters.capture import convert_capture
@@ -137,7 +133,8 @@ def write_recording_db(path: Path, action_rows: list[dict]) -> None:
             video_start_time=VIDEO_T0,
             double_click_interval_seconds=0.5,
             double_click_distance_pixels=5.0,
-            # capture persists pixel_ratio only in the config JSON.
+            # config-JSON pixel_ratio: the legacy (pre-0.5.4) persistence path,
+            # which CaptureSession.pixel_ratio still honors as a fallback.
             config={"pixel_ratio": PIXEL_RATIO},
         )
         session.add(recording)