fix: remove SciPy from core runtime

.github/workflows/canvas-nodom-ladder.yml

@@ -31,7 +31,7 @@ jobs:
         run: |
           python -m pip install --upgrade pip
           pip install -e ".[ocr,cv]" || pip install -e .
-          pip install opencv-python-headless rapidocr-onnxruntime pillow imagehash playwright
+          pip install opencv-python-headless rapidocr-onnxruntime pillow numpy playwright
           python -m playwright install --with-deps chromium
 
       - name: Build the no-DOM canvas fixture image

.github/workflows/citrix-workspace-standin.yml

@@ -63,7 +63,7 @@ jobs:
         run: |
           python -m pip install --upgrade pip
           pip install -e ".[ocr,cv]" || pip install -e .
-          pip install opencv-python-headless rapidocr-onnxruntime pillow imagehash playwright
+          pip install opencv-python-headless rapidocr-onnxruntime pillow numpy playwright
           python -m playwright install --with-deps chromium
 
       - name: Build the no-DOM canvas fixture image (Part-1)

.github/workflows/docker-rdp-vision-ladder.yml

@@ -58,7 +58,7 @@ jobs:
       - name: Install flow with the vision + rdp stack
         run: |
           python -m pip install --upgrade pip
-          # Core owns cv2/RapidOCR/Pillow/imagehash; the rdp extra installs the
+          # Core owns cv2/RapidOCR/Pillow/NumPy hashing; the rdp extra installs the
           # product transport dependency even though this Linux fixture injects
           # the swappable DockerX11RdpTransport into the same backend.
           pip install -e ".[rdp]"

docs/ECOSYSTEM_INTEGRATION.md

@@ -367,7 +367,7 @@ tech debt. Explicit "keep it home-grown" calls:
    canonical schema. Those are flow's differentiators and don't belong in a passive shared
    type.
 2. **The replay hot path stays dependency-minimal.** `pydantic + opencv + rapidocr +
-   pillow + imagehash`. Grounding, capture, privacy, and types-interop are all **edges**
+   pillow + numpy`. Grounding, capture, privacy, and types-interop are all **edges**
    (compile-time or opt-in), never core. The property that "the whole loop runs in CI with
    no OS permissions and zero model calls" is the product; protect it.
 3. **Postcondition + identity verification stays home-grown.** Nothing in the ecosystem
@@ -406,6 +406,6 @@ The governing test for any future integration proposal:
 | Package | Import weight | On flow's hot path? |
 |---|---|---|
 | openadapt-types | pydantic only (light) | No — boundary/optional |
-| openadapt-capture | pynput + mss + PyAV + sounddevice (heavy, native, needs perms to record; import stays headless-safe) | No — compile-time on-ramp, optional extra |
+| openadapt-capture | Native input/screen APIs plus separately provisioned external FFmpeg for video | No — compile-time recorder on-ramp, optional extra |
 | openadapt-verifier | stdlib-only (light) | N/A — not integrated |
-| flow core | pydantic, numpy, opencv-headless, pillow, imagehash, rapidocr, playwright, httpx | This IS the hot path; keep it here |
+| flow core | pydantic, numpy, opencv-headless, pillow, rapidocr, playwright, httpx | This IS the hot path; keep it here |

openadapt_flow/adapters/capture.py

@@ -16,7 +16,7 @@
                               #     mouse_y, mouse_dx, mouse_dy,
                               #     mouse_button_name, mouse_pressed, key_name,
                               #     key_char, canonical_key_name, ...)
-      oa_recording-*.mp4      # action-gated screen video
+      captured media         # external-FFmpeg video or supported frame store
 
 The adapter is a thin bridge over openadapt-capture's **public API** — it does
 *no* raw SQL and knows nothing about capture's schema. It calls
@@ -54,8 +54,8 @@
 key), unmapped named keys, and any unknown input action type all raise instead
 of being ignored.
 
-Coordinate spaces: capture mouse coordinates are in *logical points* (pynput);
-video frames are *physical pixels*. openadapt-flow requires event coordinates in
+Coordinate spaces: capture mouse coordinates are in *logical points*;
+captured frames are *physical pixels*. openadapt-flow requires event coordinates in
 the same pixel space as the frames, so points are scaled by
 ``CaptureSession.pixel_ratio`` (physical / logical). NOTE: sessions recorded
 with capture >=0.5.4 persist ``pixel_ratio`` on the recording model itself, so
@@ -78,7 +78,7 @@
     silent-mis-conversion this detection exists to prevent);
   * takes frames as-is (already the client-window viewport);
   * validates the static viewport and bounds timeline, refuses a mid-session
-    resize (capture's MP4 and Flow recordings each have one fixed viewport),
+    resize (capture media and Flow recordings each have one fixed viewport),
     verifies extracted frame sizes, and screens every mouse action against
     that pixel space: window capture records out-of-window input at
     out-of-range coordinates instead of clamping, and such an action targeted
@@ -98,15 +98,14 @@
 A window-scoped session that declares a coordinate space this adapter does
 not understand is refused loudly rather than converted with guessed scaling.
 
-Frame selection: openadapt-capture records **action-gated** video (frames are
-encoded around user actions, not continuously). For an event at wall-clock time
-``T`` the *before* frame is ``get_frame_at(T)`` and the *after* frame is
+Frame selection: ``CaptureSession.get_frame_at`` abstracts captured media
+(default external-FFmpeg video and any supported frame store). For an event at
+wall-clock time ``T`` the *before* frame is ``get_frame_at(T)`` and the *after* frame is
 ``get_frame_at(T + settle_s)`` clamped to just before the next event — an
 approximation of the live Recorder's perceptual-hash settle wait (see
-docs/desktop/PHASE1.md). Because the video is action-gated, a per-action frame
-may be unavailable; a missing *before* frame for a click is fatal (the compiler
-requires it), a missing *after* frame simply yields no postconditions for that
-step.
+docs/desktop/PHASE1.md). A per-action frame may be unavailable; a missing
+*before* frame for a click is fatal (the compiler requires it), while a missing
+*after* frame simply yields no postconditions for that step.
 
 Scroll deltas: pynput reports wheel *notches* with positive ``dy`` = scroll up,
 while the flow recording stores *pixels* with positive ``dy`` = view down
@@ -121,7 +120,7 @@
 a recording of the SAME shape as a web recording, and it compiles into a valid
 bundle, but it CANNOT carry the ``structural`` locator (UIA ``AutomationId`` or
 AT-SPI accessible ID / role+name) that a DOM-armed web bundle gets: capture records only
-mouse/keyboard/video, so there is no live accessibility tree at conversion time
+mouse/keyboard/frame evidence, so there is no live accessibility tree at conversion time
 to read an element identity from. Every ``anchor.structural`` is therefore None
 and replay uses the VISUAL ladder (template/ocr/geometry). To get the
 deterministic structural top rung on desktop, record LIVE over ``WindowsBackend``
@@ -149,8 +148,8 @@
 # after each gesture).
 SCROLL_PIXELS_PER_NOTCH = 100
 
-# Search window (seconds) for get_frame_at. Capture is action-gated, so frames
-# cluster around action timestamps; a generous window finds the nearest one.
+# Search window (seconds) for get_frame_at. Frames cluster around action
+# timestamps; a generous window finds the nearest one.
 FRAME_TOLERANCE_S = 2.0
 
 # The one coordinate space a window-scoped capture session may declare today:
@@ -283,8 +282,8 @@ def _window_viewport_timeline(
     ``session.window_events`` accessor when the installed capture exposes one,
     else via the session's underlying recording model (same rows).
 
-    Flow recordings and capture's MP4 stream each have one fixed viewport.
-    Capture currently skips video frames after a window resize because they no
+    Flow recordings and capture's frame timeline each have one fixed viewport.
+    Capture currently skips frames after a window resize because they no
     longer match the stream size. Therefore a timeline viewport change cannot
     be represented faithfully and is refused instead of pairing coordinates in
     the new pixel space with a stale frame in the old pixel space.
@@ -662,7 +661,8 @@ def convert_capture(
 
     Args:
         capture_dir: An openadapt-capture session directory (contains
-            ``recording.db`` and an ``oa_recording-*.mp4`` video).
+            ``recording.db`` and frame media readable through
+            ``CaptureSession.get_frame_at``).
         out_recording_dir: Output recording directory (created if missing).
         params: Optional ``{param_name: demonstrated_value}`` map. A coalesced
             ``type`` event whose text equals a demonstrated value is marked as
@@ -683,7 +683,7 @@ def convert_capture(
             flow equivalent (drag, non-left click, modifier chord, unmapped
             named key, unknown input type), multiple params demonstrating the
             same value, or a click whose before frame is missing from the
-            action-gated video. Also, for a window-scoped session: on an
+            captured frame timeline. Also, for a window-scoped session: on an
             out-of-window action, an unknown declared coordinate space,
             malformed selector/viewport metadata, a mid-recording resize, or
             an extracted frame whose dimensions disagree with the recorded
@@ -769,9 +769,9 @@ def convert_capture(
                         )
             if event["kind"] in ("click", "double_click") and before_img is None:
                 raise ValueError(
-                    f"no video frame available for {event['kind']} at "
-                    f"t={ts - started_at:.3f}s: the action-gated capture video "
-                    "has no frame near this action, so its target cannot be "
+                    f"no captured frame available for {event['kind']} at "
+                    f"t={ts - started_at:.3f}s: capture has no frame near "
+                    "this action, so its target cannot be "
                     "anchored"
                 )
             if before_img is not None:

openadapt_flow/image_hash.py

@@ -0,0 +1,88 @@
+"""Deterministic perceptual hashes without SciPy.
+
+The algorithms and serialized hexadecimal representation intentionally match
+``ImageHash==4.3.2``.  OpenAdapt persisted those strings in compiled bundles,
+so this module is a compatibility implementation rather than a new hash
+format.
+"""
+
+from __future__ import annotations
+
+import cv2
+import numpy as np
+from PIL import Image
+
+
+def _binary_array_to_hex(bits: np.ndarray) -> str:
+    """Serialize a boolean array in ImageHash-compatible row-major order."""
+    flat = np.asarray(bits, dtype=np.bool_).ravel()
+    bit_string = "".join("1" if bit else "0" for bit in flat)
+    width = (len(bit_string) + 3) // 4
+    return f"{int(bit_string, 2):0{width}x}"
+
+
+def _unnormalized_dct2(pixels: np.ndarray) -> np.ndarray:
+    """Return SciPy fftpack-compatible unnormalized type-II 2-D DCT.
+
+    OpenCV computes an orthonormal DCT-II.  Scaling its frequency rows and
+    columns converts it to the unnormalized transform used by
+    ``scipy.fftpack.dct(..., type=2, norm=None)``.  Using OpenCV also preserves
+    exact zero-frequency symmetry for constant images; a direct cosine-matrix
+    implementation can leave tiny platform-dependent residuals that flip
+    median-threshold bits.
+    """
+    height, width = pixels.shape
+    if height != width:
+        raise ValueError("perceptual hash DCT input must be square")
+    normalized = cv2.dct(np.asarray(pixels, dtype=np.float64))
+    scale = np.full(width, np.sqrt(2.0 * width), dtype=np.float64)
+    scale[0] = 2.0 * np.sqrt(width)
+    return normalized * scale[:, np.newaxis] * scale[np.newaxis, :]
+
+
+def perceptual_hash(
+    image: Image.Image,
+    *,
+    hash_size: int = 8,
+    highfreq_factor: int = 4,
+) -> str:
+    """Return an ImageHash-compatible perceptual hash."""
+    if hash_size < 2:
+        raise ValueError("Hash size must be greater than or equal to 2")
+
+    image_size = hash_size * highfreq_factor
+    pixels = np.asarray(
+        image.convert("L").resize(
+            (image_size, image_size),
+            Image.Resampling.LANCZOS,
+        ),
+        dtype=np.float64,
+    )
+    dct = _unnormalized_dct2(pixels)
+    low_frequencies = dct[:hash_size, :hash_size]
+    return _binary_array_to_hex(low_frequencies > np.median(low_frequencies))
+
+
+def difference_hash(image: Image.Image, *, hash_size: int = 8) -> str:
+    """Return an ImageHash-compatible horizontal difference hash."""
+    if hash_size < 2:
+        raise ValueError("Hash size must be greater than or equal to 2")
+
+    pixels = np.asarray(
+        image.convert("L").resize(
+            (hash_size + 1, hash_size),
+            Image.Resampling.LANCZOS,
+        )
+    )
+    return _binary_array_to_hex(pixels[:, 1:] > pixels[:, :-1])
+
+
+def hash_distance(a: str, b: str) -> int:
+    """Return the Hamming distance between equal-width hexadecimal hashes."""
+    if len(a) != len(b):
+        raise TypeError(
+            "Image hashes must have the same bit length.",
+            len(a) * 4,
+            len(b) * 4,
+        )
+    return (int(a, 16) ^ int(b, 16)).bit_count()

openadapt_flow/recorder.py

@@ -17,7 +17,7 @@
       frames/{i:04d}_before.png
       frames/{i:04d}_after.png   # captured after the action settled
 
-The settle wait is implemented inline here (imagehash polling) on purpose:
+The settle wait is implemented inline here (perceptual-hash polling) on purpose:
 this module must not depend on `openadapt_flow.vision`.
 """
 
@@ -31,17 +31,17 @@
 from pathlib import Path
 from typing import Any, Callable, Optional
 
-import imagehash
 from PIL import Image, ImageDraw
 
 from openadapt_flow.backend import Backend
+from openadapt_flow.image_hash import perceptual_hash
 from openadapt_flow.ir import StructuralLocator
 
 
-def _phash(png: bytes) -> imagehash.ImageHash:
+def _phash(png: bytes) -> str:
     """Perceptual hash of a PNG frame."""
     with Image.open(io.BytesIO(png)) as img:
-        return imagehash.phash(img)
+        return perceptual_hash(img)
 
 
 class Recorder:
@@ -373,7 +373,7 @@ def _wait_settled(self) -> bytes:
             time.sleep(self._settle_interval_s)
             png = self._backend.screenshot()
             cur = _phash(png)
-            if cur - prev == 0:
+            if cur == prev:
                 consecutive += 1
             else:
                 consecutive = 1

openadapt_flow/validation/pixel_identity_probe.py

@@ -56,6 +56,12 @@
 import numpy as np
 from PIL import Image
 
+from openadapt_flow.image_hash import (
+    difference_hash,
+    hash_distance,
+    perceptual_hash,
+)
+
 # Reuse the dense-surface fixture's RENDERING unchanged (read it, call it,
 # never edit it): the HTML/CSS that defines exactly how an MRN cell paints.
 from openadapt_flow.validation.dense_surface import (
@@ -373,18 +379,20 @@ def m_ssim(a: np.ndarray, b: np.ndarray) -> float:
 
 
 def m_phash(a: np.ndarray, b: np.ndarray) -> float:
-    import imagehash
-
     return float(
-        imagehash.phash(_pil(a), hash_size=16) - imagehash.phash(_pil(b), hash_size=16)
+        hash_distance(
+            perceptual_hash(_pil(a), hash_size=16),
+            perceptual_hash(_pil(b), hash_size=16),
+        )
     )
 
 
 def m_dhash(a: np.ndarray, b: np.ndarray) -> float:
-    import imagehash
-
     return float(
-        imagehash.dhash(_pil(a), hash_size=16) - imagehash.dhash(_pil(b), hash_size=16)
+        hash_distance(
+            difference_hash(_pil(a), hash_size=16),
+            difference_hash(_pil(b), hash_size=16),
+        )
     )
 
 

openadapt_flow/vision/hashing.py

@@ -1,4 +1,4 @@
-"""Perceptual hashing helpers (imagehash-based).
+"""Perceptual hashing helpers.
 
 Used for change detection (settle polling) and REGION_STABLE postconditions.
 
@@ -15,9 +15,9 @@
 
 import io
 
-import imagehash
 from PIL import Image, ImageFilter
 
+from openadapt_flow.image_hash import hash_distance, perceptual_hash
 from openadapt_flow.ir import Region
 
 
@@ -47,9 +47,9 @@ def phash_png(png: bytes, region: Region | None = None) -> str:
             raise ValueError(f"region {region} is empty after clamping")
         img = img.crop((x0, y0, x1, y1))
     edges = img.convert("L").filter(ImageFilter.FIND_EDGES)
-    return str(imagehash.phash(edges))
+    return perceptual_hash(edges)
 
 
 def phash_distance(a: str, b: str) -> int:
     """Return the Hamming distance between two hex phash strings."""
-    return imagehash.hex_to_hash(a) - imagehash.hex_to_hash(b)
+    return hash_distance(a, b)

public-artifacts.json

@@ -76,19 +76,19 @@
     },
     {
       "path": ".github/workflows/canvas-nodom-ladder.yml",
-      "sha256": "ffcd2c3d1db82eec5bd92012f079c1eee4fbff69cf0c726465ebce2dddc91838"
+      "sha256": "3fe1fe1277df4bcd85a4ba00ffbb75f34395c31b26fd3038b7d0b6f7b31cc27c"
     },
     {
       "path": ".github/workflows/ci.yml",
       "sha256": "1268fdfe0dce27c106cd1833a0ef9023211ca59df755b0088562c7197a3b2699"
     },
     {
       "path": ".github/workflows/citrix-workspace-standin.yml",
-      "sha256": "166d41756f2c7b0d29965729c4251296e3312cb713cb1ea3889b49d048f36896"
+      "sha256": "7ffdf4e530dd85d13f101ae43ca210034a6f6fae8aaf6e86664d4db867f0d67b"
     },
     {
       "path": ".github/workflows/docker-rdp-vision-ladder.yml",
-      "sha256": "fadd77ccfbc1e868ac3306a7abb5d8f2ac020e8cf09c6fe7668343a4beb3210d"
+      "sha256": "bf53a9a348f6c1b539e33826530cd78287047799583dce180622a0a56ee303fb"
     },
     {
       "path": ".github/workflows/paper.yml",

pyproject.toml

@@ -19,7 +19,6 @@ dependencies = [
     "numpy>=1.26",
     "opencv-python-headless>=4.9",
     "pillow>=10.0",
-    "imagehash>=4.3",
     "rapidocr-onnxruntime>=1.3",
     # The reference backend and the CLI's demo-record/replay self-serve
     # path drive a Playwright browser; core, not a dev extra.