Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
44 changes: 44 additions & 0 deletions packages/cli/src/commands/check.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -148,6 +148,7 @@ function fakeDriver(overrides: Partial<CheckAuditDriver> = {}): CheckAuditDriver
findAmbiguousSelectors: vi.fn(async (_selectors: string[]) => []),
seek: vi.fn(async (_time: number) => undefined),
collectLayout: vi.fn(async (_time: number, _tolerance: number) => []),
collectOverlap: vi.fn(async (_time: number) => []),
collectLayoutGeometry: vi.fn(async () => `geometry-${geometryCallCount++}`),
collectRotationSample: vi.fn(async (_time: number) => []),
collectOffPivotRotationSample: vi.fn(async (time: number) => ({ time, samples: [] })),
Expand Down Expand Up @@ -1343,3 +1344,46 @@ describe("contrast candidate round-trip", () => {
expect(source).not.toMatch(/prepared\.map\(\(entry\) => entry\.candidate\)/);
});
});

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 The "surfaces a held content_overlap" test doesn't exercise the dense-vs-sparse distinction it claims to demonstrate.

collectOverlap: vi.fn(async (time: number) => [
  layoutIssue("warning", { time, code: "content_overlap" }),
]),

This returns a content_overlap warning at EVERY dense sample time (~72 samples for a 9s fake duration at 8fps). The collapse step sees ~72 occurrences → persistence tier trivially promotes to error via the occurrences >= 2 shortcut (see the sibling comment on layoutAudit.ts:324). The test would pass equally well against a broken dense pass that simply promoted every finding — it never asserts occurrences count or firstSeen/lastSeen span.

More importantly: the test doesn't reproduce the target defect (transient crossing seen only by 2 adjacent dense samples that the sparse grid missed). Test 2 (:1364-1373) correctly validates the static-composition gate — that one is fine.

Fix: add a third test where collectOverlap returns a finding at exactly two adjacent dense sample times and nothing anywhere else. Assert (a) driver.collectOverlap.mock.calls.length matches the dense grid size, (b) the surviving finding has occurrences === 2, (c) severity resolves to what the intended semantic is (once the sibling comment's design question is decided).

Also: buildOverlapSampleTimes(duration) at checkPipeline.ts:441 is a pure function and would take ~5 lines to unit-test for the count/bounds/cap-at-120/quantization behavior. Catches density-math regressions the corpus can't.

Review by Rames D Jusso

describe("dense motion-overlap re-sampling", () => {
// The default grid is 9 base samples at index+0.5 (0.5,1.5,...,8.5) over a 9s
// composition; the collision below lives entirely inside (3.5, 4.5), a gap
// the sparse grid seeks straight past. Only the 8fps dense pass observes it.
const inBetweenGridWindow = (time: number): boolean => time >= 3.6 && time <= 4.4;

it("detects a content_overlap that occurs ONLY between two sparse grid samples", async () => {
const driver = fakeDriver({
// Sparse base grid sees nothing at any base sample time.
collectLayout: vi.fn(async (_time: number) => []),
// The transient exists only strictly between base samples 3.5 and 4.5.
collectOverlap: vi.fn(async (time: number) =>
inBetweenGridWindow(time)
? [layoutIssue("warning", { time, code: "content_overlap" })]
: [],
),
});
const { report } = await runScenario(driver);
expect(driver.collectOverlap).toHaveBeenCalled();
expect(report.layout.findings.some((f) => f.code === "content_overlap")).toBe(true);
// Held ~750ms across the dense grid (>= the 500ms floor) -> promoted.
expect(report.layout.errorCount).toBeGreaterThan(0);
});

it("runs the dense pass even when sparse fingerprints are identical (aliased motion)", async () => {
// A constant geometry fingerprint no longer suppresses the pass: an
// animation aliased to the sparse grid has identical fingerprints yet still
// collides between samples — the false-negative the removed gate caused.
const driver = fakeDriver({
collectLayoutGeometry: vi.fn(async () => "static"),
collectLayout: vi.fn(async (_time: number) => []),
collectOverlap: vi.fn(async (time: number) =>
inBetweenGridWindow(time)
? [layoutIssue("warning", { time, code: "content_overlap" })]
: [],
),
});
const { report } = await runScenario(driver);
expect(driver.collectOverlap).toHaveBeenCalled();
expect(report.layout.findings.some((f) => f.code === "content_overlap")).toBe(true);
});
});
14 changes: 14 additions & 0 deletions packages/cli/src/commands/layout-audit.browser.js
Original file line number Diff line number Diff line change
Expand Up @@ -1425,6 +1425,20 @@
return issues;
};

// content_overlap only, for the dense motion re-sampling grid (checkPipeline

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Comment references a function name that doesn't exist in the codebase.

// content_overlap only, for the dense motion re-sampling grid (checkPipeline
// detectMotionTextOverlap). Two free-positioned text blocks crossing mid-orbit

detectMotionTextOverlap doesn't exist — the actual function is collectMotionOverlapSamples at checkPipeline.ts:457. Reader searching for the reference finds nothing.

Fix: rename to collectMotionOverlapSamples in the comment.

Review by Rames D Jusso

// detectMotionTextOverlap). Two free-positioned text blocks crossing mid-orbit
// collide for a fraction of a second the sparse layout grid seeks straight
// past; this reruns just the overlap detector — same collectSolidTextBlocks /
// overlapIssue 0.2-area threshold, no new detection surface — on a fine grid.
window.__hyperframesOverlapAudit = function auditOverlap(options) {
const time = options && typeof options.time === "number" ? options.time : 0;
const root =
document.querySelector("[data-composition-id][data-width][data-height]") ||
document.querySelector("[data-composition-id]") ||
document.body;
return contentOverlapIssues(root, time);
};

// Frozen-sweep guard (#U10, checkPipeline.ts): a compact per-sample
// fingerprint of every visible element's box + opacity, in DOM order. Node
// calls this once per seeked grid point and compares the strings across the
Expand Down
14 changes: 14 additions & 0 deletions packages/cli/src/utils/checkBrowser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -349,6 +349,7 @@ function createPageDriver(page: Page, setTime: (time: number) => void): CheckAud
await seekCompositionTimeline(page, time, AUDIT_SEEK_OPTIONS);
},
collectLayout: (time, tolerance) => collectLayout(page, time, tolerance),
collectOverlap: (time) => collectOverlap(page, time),
collectLayoutGeometry: () => collectLayoutGeometry(page),
collectRotationSample: (time) => collectRotationSample(page, time),
collectOffPivotRotationSample: (time) => collectOffPivotRotationSample(page, time),
Expand Down Expand Up @@ -464,6 +465,19 @@ async function collectLayout(
return anchorLayoutIssues(page, raw.flatMap(parseLayoutIssue));
}

async function collectOverlap(page: Page, time: number): Promise<AnchoredLayoutIssue[]> {
const raw = await page.evaluate(
(options: { time: number }) => {
const audit = Reflect.get(window, "__hyperframesOverlapAudit");
if (typeof audit !== "function") return [];
const result = Reflect.apply(audit, window, [options]);
return Array.isArray(result) ? result : [];
},
{ time },
);
return anchorLayoutIssues(page, raw.flatMap(parseLayoutIssue));
}

async function collectLayoutGeometry(page: Page): Promise<string> {
return page.evaluate(() => {
const geometry = Reflect.get(window, "__hyperframesLayoutGeometry");
Expand Down
58 changes: 58 additions & 0 deletions packages/cli/src/utils/checkPipeline.ts
Original file line number Diff line number Diff line change
Expand Up @@ -417,9 +417,67 @@ async function collectGridSamples(
collected.screenshots.push({ time, pngBase64: capture.pngBase64 });
}
}
await collectMotionOverlapSamples(driver, grid, collected);
return collected;
}

// content_overlap sampling density for the dense motion re-pass. The sparse
// layout grid (default 9 points over multiple seconds) seeks straight past a
// mid-orbit text-on-text crossing that only overlaps for a fraction of a
// second: an in-corpus orbit (samples/fuzz016) collides at 28% area for ~0.4s,
// entirely between two adjacent base samples. 8fps (~0.125s spacing) lands
// enough samples inside a window that narrow to observe it. Overlap collection
// is text-only (collectSolidTextBlocks), far cheaper than a full layout audit,
// so a fine grid here is affordable where densifying every detector would not.
const OVERLAP_SAMPLE_FPS = 8;
// Absolute ceiling on dense seeks so the pass stays bounded. This holds a true
// 8fps grid for compositions up to OVERLAP_MAX_SAMPLES / OVERLAP_SAMPLE_FPS
// (~75s); longer compositions degrade below 8fps rather than growing the seek
// budget without limit. (Corpus compositions run 7-8s, well inside 8fps.)
const OVERLAP_MAX_SAMPLES = 600;

function buildOverlapSampleTimes(duration: number): number[] {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 20s+ compositions silently degrade to sub-8fps sampling under the 120-cap.

const count = Math.min(
  OVERLAP_MAX_SAMPLES,
  Math.max(2, Math.ceil(duration * OVERLAP_SAMPLE_FPS) + 1),
);
const step = duration / (count - 1);

For duration = 20s, Math.ceil(20*8)+1 = 161 → capped to 120 → step = 20/119 ≈ 0.168s → effective ~5.94fps. For duration = 30s, step ≈ 0.253s → ~3.95fps. The comment at :422-430 sells the design as "8fps (~0.125s spacing) lands >= 2 samples inside a window that narrow", but that guarantee holds only for duration <= 14.875s.

Failure mode: long-form compositions (explainer clips, marketing walk-throughs) fall back to a sparser grid than the design claims. Density guarantee erodes silently — the occurrences >= 2 promotion machinery loses its sub-250ms transient coverage past ~15s.

HF's fuzz corpus is short so this may not bite in CI, but production comp durations trend longer.

Fix options: (a) preserve density and drop the tail past 15s: const step = 1 / OVERLAP_SAMPLE_FPS; const count = Math.min(OVERLAP_MAX_SAMPLES, Math.floor(duration * OVERLAP_SAMPLE_FPS) + 1); (b) accept the degradation but return a truncatedAtSeconds signal so downstream consumers can flag long comps. Minimum: document the effective-fps table in the const block so future authors don't assume 8fps everywhere.

Review by Rames D Jusso

if (!Number.isFinite(duration) || duration <= 0) return [];
const count = Math.min(
OVERLAP_MAX_SAMPLES,
Math.max(2, Math.ceil(duration * OVERLAP_SAMPLE_FPS) + 1),
);
const step = duration / (count - 1);
return mergeSampleTimes(
Array.from({ length: count }, (_, index) => Math.round(index * step * 1000) / 1000),
);
}

/**
* Dense motion-overlap re-sampling. Reruns ONLY content_overlap on a fine time
* grid so transient text collisions during continuous motion are observed at
* all — the detector itself is unchanged (same 0.2-area threshold), only the
* sampling density is.
*
* This runs UNCONDITIONALLY (bounded + text-only), NOT gated on sparse-grid
* geometry fingerprints changing. That gate was the motivating false-negative:
* an animation aliased to the sparse grid (the same pose sampled at every base
* point) has identical fingerprints yet still collides *between* those samples,
* so gating on fingerprint change skipped the exact transient this pass exists
* to catch. A static composition simply yields no overlaps at the extra times,
* so the only cost of running always is a bounded set of cheap text-only seeks.
* Findings feed the existing collapse/persistence tiering (a graze stays info,
* a held collision re-promotes to error). Skips times already in the base grid
* to avoid double-collecting overlaps collectLayout already found.
*/
async function collectMotionOverlapSamples(
driver: CheckAuditDriver,
grid: SampleGrid,
collected: GridSamples,
): Promise<void> {
const baseTimes = new Set(grid.layoutSamples);
for (const time of buildOverlapSampleTimes(grid.duration)) {

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Design-contract question — these dense-pass findings enter the persistence tier via a shortcut that assumed sparse-grid timing.

for (const time of buildOverlapSampleTimes(grid.duration)) {
  if (baseTimes.has(time)) continue;
  await driver.seek(time);
  collected.layoutIssues.push(...(await driver.collectOverlap(time)));
}

The findings pushed here flow into collapseStaticLayoutIssuesisContentOverlapHeldLongEnough (layoutAudit.ts:323-330, unmodified by this PR):

function isContentOverlapHeldLongEnough(issue: LayoutIssue, occurrences: number): boolean {
  if (occurrences >= HELD_ACROSS_SAMPLES_MIN_OCCURRENCES) return true;   // ← short-circuits
  const firstSeen = issue.firstSeen ?? issue.time;
  const lastSeen = issue.lastSeen ?? issue.time;
  const heldMs = (lastSeen - firstSeen) * 1000;
  return heldMs >= CONTENT_OVERLAP_HELD_ERROR_MS;
}

Design comment at layoutAudit.ts:186-198 frames the mapping: "At the default 9-sample grid over a multi-second composition ... two collapsed occurrences are already >= one sample-to-sample gap, which is well past 500ms — so 'held under 250ms' reduces to occurrences <= 1 and 'held >= 500ms' reduces to occurrences >= 2." That mapping holds for the sparse grid (~1s between adjacent samples).

At the new 8fps dense grid, two adjacent samples span ~125ms. So occurrences === 2 → promoted straight to error — well under the 500ms design floor and even under the 250ms ignore floor.

The design comment DOES anticipate this — it says "with the literal ms span (CONTENT_OVERLAP_HELD_ERROR_MS) kept as a fallback for callers whose samples really are spaced close enough together for the ms floor to matter on its own (dense --at/--at-transitions runs)". But the code short-circuits on occurrences FIRST — the ms-fallback branch only runs when occurrences < 2, so it never gates the dense-pass 2-occurrence case.

Your PR-body reviewer note acknowledges this: "At 8fps the occurrences >= 2 persistence shortcut spans ~125ms (vs ~500ms on the sparse grid), so brief-but-real collisions now gate at error. Every inspected instance was real, but if softer severity is preferred, the heldMs >= 500 path is the alternative knob."

The request is: make the choice explicit either in code or in the comment. Three options:

  • (a) Tighten isContentOverlapHeldLongEnough to ANDreturn occurrences >= 2 && (lastSeen - firstSeen) * 1000 >= CONTENT_OVERLAP_HELD_ERROR_MS. Still passes the sparse-grid case (samples ≥1s apart → heldMs ≥1000 ≥ 500) and correctly gates the dense case. Cleanest — one source of truth.
  • (b) Tag dense-pass findings with a source: 'dense' marker HERE and skip the occurrences shortcut only for that source in the persistence-tier code.
  • (c) Update the design comment at layoutAudit.ts:186-198 to say the ms floor is intentionally relaxed to ~125ms for dense-pass findings, with a rationale from the corpus ("81/81 samples of the corpus showed 125ms crossings were real defects").

Any of the three closes the contract question. Right now the comment and the code disagree; the disagreement widens by 375ms every time the dense pass fires.

Review by Rames D Jusso

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Dense loop has neither early-terminate nor try/catch — cost + flake risk.

for (const time of buildOverlapSampleTimes(grid.duration)) {
  if (baseTimes.has(time)) continue;
  await driver.seek(time);
  collected.layoutIssues.push(...(await driver.collectOverlap(time)));
}

Cost: serial loop of up to ~120 seek + evaluate roundtrips per composition. Each seek waits for GSAP/CSS timeline settle. Once persistence tier is going to promote a finding at occurrences >= 2, additional samples of the same key add zero signal but still pay the cost. The perf claim text-only so it's cheap (comment at :428-430) is defensible per-sample but not per-run.

Flake: neither driver.seek(time) nor driver.collectOverlap(time) is guarded. If either throws mid-loop (transient page-evaluate error, browser blip, timeout on a specific frame), the entire dense pass — and by inheritance the surrounding collectGridSamples await — rejects, aborting the whole check run. Under the sparse grid the same shape existed with 9 chances; now there are ~120, so the throw budget multiplies with the sample count.

Fix: (a) wrap the loop body in try/catch, record the throw as a soft dense_overlap_probe_failed info-level finding rather than aborting. (b) early-terminate when every observed key has already crossed a threshold (say 5 occurrences). (c) at minimum, add a per-sample timeout so a hung seek doesn't stall the whole pass. Neither is a merge blocker — but the const block should carry a worst-case cost note.

Review by Rames D Jusso

if (baseTimes.has(time)) continue;
await driver.seek(time);
collected.layoutIssues.push(...(await driver.collectOverlap(time)));
}
}

// Frozen-sweep guard (#U10): compositions this short can legitimately hold a
// single static frame the whole time (a title card) — never flag those.
const SWEEP_STATIC_MIN_DURATION_SEC = 3;
Expand Down
4 changes: 4 additions & 0 deletions packages/cli/src/utils/checkTypes.ts
Original file line number Diff line number Diff line change
Expand Up @@ -174,6 +174,10 @@ export interface CheckAuditDriver {
findAmbiguousSelectors(selectors: string[]): Promise<AnchoredLayoutIssue[]>;
seek(time: number): Promise<void>;
collectLayout(time: number, tolerance: number): Promise<AnchoredLayoutIssue[]>;
/** content_overlap only, for the dense motion re-sampling grid — catches
* transient text-on-text collisions the sparse layout grid seeks past. See
* checkPipeline detectMotionTextOverlap. */
collectOverlap(time: number): Promise<AnchoredLayoutIssue[]>;
/** Frozen-sweep guard (#U10): an opaque per-sample geometry+opacity
* fingerprint of the current seeked state, for detecting a timeline that
* never advances under seek. See layout-audit.browser.js. */
Expand Down
15 changes: 15 additions & 0 deletions packages/cli/src/utils/layoutAudit.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,21 @@ describe("persistence-tiered severity (#U10)", () => {
expect(collapsed[0]).toMatchObject({ severity: "error", occurrences: 2 });
});

it("keeps a content_overlap that spans under the 500ms floor as a warning, even with 2 occurrences", () => {
// Two occurrences from the dense 8fps re-pass span only ~125ms — under the
// held-duration floor, so occurrences>=2 alone must NOT promote to error.
const collapsed = collapseStaticLayoutIssues(
[
{ ...issue("content_overlap", "warning"), time: 4.0 },
{ ...issue("content_overlap", "warning"), time: 4.125 },
],
73,
);

expect(collapsed).toHaveLength(1);
expect(collapsed[0]).toMatchObject({ severity: "warning", occurrences: 2 });
});

it("promotes a held, canvas-scale canvas_overflow breach from info to warning", () => {
const breach = {
...issue("canvas_overflow", "info"),
Expand Down
35 changes: 18 additions & 17 deletions packages/cli/src/utils/layoutAudit.ts
Original file line number Diff line number Diff line change
Expand Up @@ -183,19 +183,17 @@ export function dedupeLayoutIssues(issues: LayoutIssue[]): LayoutIssue[] {

// Persistence-tier thresholds (#U10, adapted from Adam Rosler's visual-linter
// design). The approach doc frames these as held-duration floors — ignore
// under ~250ms, re-promote content_overlap at >= ~500ms — measured against
// the SAME firstSeen/lastSeen span this collapse step already tracks. At the
// default 9-sample grid over a multi-second composition, a single collapsed
// occurrence is held 0ms (one entrance/exit transient sample) and two
// collapsed occurrences are already >= one sample-to-sample gap, which is
// well past 500ms — so "held under 250ms" reduces to `occurrences <= 1` and
// "held >= 500ms" reduces to `occurrences >= 2`. Tiering below is written in
// those sample-count terms (the mapping the approach doc asks to document),
// with the literal ms span (CONTENT_OVERLAP_HELD_ERROR_MS) kept as a fallback
// for callers whose samples really are spaced close enough together for the
// ms floor to matter on its own (dense `--at`/`--at-transitions` runs). The
// ~250ms ignore floor needs no separate constant — see the occurrences <= 1
// branch below.
// under ~250ms, re-promote content_overlap at >= ~500ms — measured against the
// SAME firstSeen/lastSeen span this collapse step already tracks. `occurrences`
// is a NECESSARY guard (one sample can't span any duration), but it is NOT a
// sufficient proxy for the 500ms floor: the dense content_overlap re-pass
// (checkPipeline collectMotionOverlapSamples) samples at 8fps, so two adjacent
// occurrences there span only ~125ms — the old "occurrences >= 2 => held >=
// 500ms" shortcut held only for the coarse ~1s-spaced base grid and breaks
// under dense sampling. content_overlap promotion therefore requires BOTH
// occurrences >= 2 AND a literal firstSeen..lastSeen span >= 500ms, so the
// wall-clock floor is honored regardless of sampling density. The ~250ms
// ignore floor needs no separate constant — see the occurrences <= 1 branch.
const CONTENT_OVERLAP_HELD_ERROR_MS = 500;
const HELD_ACROSS_SAMPLES_MIN_OCCURRENCES = 2;

Expand Down Expand Up @@ -317,11 +315,14 @@ function isCanvasBreachHeldLarge(issue: LayoutIssue, occurrences: number): boole
return overlapX > 0 && overlapY > 0;
}

// Split out of applyPersistenceTier so the two independent "held long enough"
// signals (sample count vs. wall-clock span) read as one boolean question
// instead of adding a third compound branch to the tiering ladder above.
// Split out of applyPersistenceTier so the compound "held long enough" test
// (>= 2 samples AND wall-clock span >= the ms floor) reads as one boolean
// question instead of adding a compound branch to the tiering ladder above.
function isContentOverlapHeldLongEnough(issue: LayoutIssue, occurrences: number): boolean {
if (occurrences >= HELD_ACROSS_SAMPLES_MIN_OCCURRENCES) return true;
// Need at least two samples to measure a span at all, AND that span must
// clear the wall-clock floor — dense 8fps re-sampling makes occurrences>=2
// alone (potentially ~125ms) too weak to imply a genuinely held collision.
if (occurrences < HELD_ACROSS_SAMPLES_MIN_OCCURRENCES) return false;
const firstSeen = issue.firstSeen ?? issue.time;
const lastSeen = issue.lastSeen ?? issue.time;
const heldMs = (lastSeen - firstSeen) * 1000;
Expand Down
Loading