-
Notifications
You must be signed in to change notification settings - Fork 3.5k
feat(lint): dense motion re-sampling for content_overlap #2746
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: xuanru/needle-pivot-offset-check
Are you sure you want to change the base?
Changes from all commits
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -1425,6 +1425,20 @@ | |
| return issues; | ||
| }; | ||
|
|
||
| // content_overlap only, for the dense motion re-sampling grid (checkPipeline | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Comment references a function name that doesn't exist in the codebase. // content_overlap only, for the dense motion re-sampling grid (checkPipeline
// detectMotionTextOverlap). Two free-positioned text blocks crossing mid-orbit
Fix: rename to — Review by Rames D Jusso |
||
| // detectMotionTextOverlap). Two free-positioned text blocks crossing mid-orbit | ||
| // collide for a fraction of a second the sparse layout grid seeks straight | ||
| // past; this reruns just the overlap detector — same collectSolidTextBlocks / | ||
| // overlapIssue 0.2-area threshold, no new detection surface — on a fine grid. | ||
| window.__hyperframesOverlapAudit = function auditOverlap(options) { | ||
| const time = options && typeof options.time === "number" ? options.time : 0; | ||
| const root = | ||
| document.querySelector("[data-composition-id][data-width][data-height]") || | ||
| document.querySelector("[data-composition-id]") || | ||
| document.body; | ||
| return contentOverlapIssues(root, time); | ||
| }; | ||
|
|
||
| // Frozen-sweep guard (#U10, checkPipeline.ts): a compact per-sample | ||
| // fingerprint of every visible element's box + opacity, in DOM order. Node | ||
| // calls this once per seeked grid point and compares the strings across the | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -417,9 +417,67 @@ async function collectGridSamples( | |
| collected.screenshots.push({ time, pngBase64: capture.pngBase64 }); | ||
| } | ||
| } | ||
| await collectMotionOverlapSamples(driver, grid, collected); | ||
| return collected; | ||
| } | ||
|
|
||
| // content_overlap sampling density for the dense motion re-pass. The sparse | ||
| // layout grid (default 9 points over multiple seconds) seeks straight past a | ||
| // mid-orbit text-on-text crossing that only overlaps for a fraction of a | ||
| // second: an in-corpus orbit (samples/fuzz016) collides at 28% area for ~0.4s, | ||
| // entirely between two adjacent base samples. 8fps (~0.125s spacing) lands | ||
| // enough samples inside a window that narrow to observe it. Overlap collection | ||
| // is text-only (collectSolidTextBlocks), far cheaper than a full layout audit, | ||
| // so a fine grid here is affordable where densifying every detector would not. | ||
| const OVERLAP_SAMPLE_FPS = 8; | ||
| // Absolute ceiling on dense seeks so the pass stays bounded. This holds a true | ||
| // 8fps grid for compositions up to OVERLAP_MAX_SAMPLES / OVERLAP_SAMPLE_FPS | ||
| // (~75s); longer compositions degrade below 8fps rather than growing the seek | ||
| // budget without limit. (Corpus compositions run 7-8s, well inside 8fps.) | ||
| const OVERLAP_MAX_SAMPLES = 600; | ||
|
|
||
| function buildOverlapSampleTimes(duration: number): number[] { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟠 20s+ compositions silently degrade to sub-8fps sampling under the 120-cap. const count = Math.min(
OVERLAP_MAX_SAMPLES,
Math.max(2, Math.ceil(duration * OVERLAP_SAMPLE_FPS) + 1),
);
const step = duration / (count - 1);For Failure mode: long-form compositions (explainer clips, marketing walk-throughs) fall back to a sparser grid than the design claims. Density guarantee erodes silently — the HF's fuzz corpus is short so this may not bite in CI, but production comp durations trend longer. Fix options: (a) preserve density and drop the tail past 15s: — Review by Rames D Jusso |
||
| if (!Number.isFinite(duration) || duration <= 0) return []; | ||
| const count = Math.min( | ||
| OVERLAP_MAX_SAMPLES, | ||
| Math.max(2, Math.ceil(duration * OVERLAP_SAMPLE_FPS) + 1), | ||
| ); | ||
| const step = duration / (count - 1); | ||
| return mergeSampleTimes( | ||
| Array.from({ length: count }, (_, index) => Math.round(index * step * 1000) / 1000), | ||
| ); | ||
| } | ||
|
|
||
| /** | ||
| * Dense motion-overlap re-sampling. Reruns ONLY content_overlap on a fine time | ||
| * grid so transient text collisions during continuous motion are observed at | ||
| * all — the detector itself is unchanged (same 0.2-area threshold), only the | ||
| * sampling density is. | ||
| * | ||
| * This runs UNCONDITIONALLY (bounded + text-only), NOT gated on sparse-grid | ||
| * geometry fingerprints changing. That gate was the motivating false-negative: | ||
| * an animation aliased to the sparse grid (the same pose sampled at every base | ||
| * point) has identical fingerprints yet still collides *between* those samples, | ||
| * so gating on fingerprint change skipped the exact transient this pass exists | ||
| * to catch. A static composition simply yields no overlaps at the extra times, | ||
| * so the only cost of running always is a bounded set of cheap text-only seeks. | ||
| * Findings feed the existing collapse/persistence tiering (a graze stays info, | ||
| * a held collision re-promotes to error). Skips times already in the base grid | ||
| * to avoid double-collecting overlaps collectLayout already found. | ||
| */ | ||
| async function collectMotionOverlapSamples( | ||
| driver: CheckAuditDriver, | ||
| grid: SampleGrid, | ||
| collected: GridSamples, | ||
| ): Promise<void> { | ||
| const baseTimes = new Set(grid.layoutSamples); | ||
| for (const time of buildOverlapSampleTimes(grid.duration)) { | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟠 Design-contract question — these dense-pass findings enter the persistence tier via a shortcut that assumed sparse-grid timing. for (const time of buildOverlapSampleTimes(grid.duration)) {
if (baseTimes.has(time)) continue;
await driver.seek(time);
collected.layoutIssues.push(...(await driver.collectOverlap(time)));
}The findings pushed here flow into function isContentOverlapHeldLongEnough(issue: LayoutIssue, occurrences: number): boolean {
if (occurrences >= HELD_ACROSS_SAMPLES_MIN_OCCURRENCES) return true; // ← short-circuits
const firstSeen = issue.firstSeen ?? issue.time;
const lastSeen = issue.lastSeen ?? issue.time;
const heldMs = (lastSeen - firstSeen) * 1000;
return heldMs >= CONTENT_OVERLAP_HELD_ERROR_MS;
}Design comment at At the new 8fps dense grid, two adjacent samples span ~125ms. So The design comment DOES anticipate this — it says "with the literal ms span (CONTENT_OVERLAP_HELD_ERROR_MS) kept as a fallback for callers whose samples really are spaced close enough together for the ms floor to matter on its own (dense Your PR-body reviewer note acknowledges this: "At 8fps the The request is: make the choice explicit either in code or in the comment. Three options:
Any of the three closes the contract question. Right now the comment and the code disagree; the disagreement widens by 375ms every time the dense pass fires. — Review by Rames D Jusso There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟠 Dense loop has neither early-terminate nor try/catch — cost + flake risk. for (const time of buildOverlapSampleTimes(grid.duration)) {
if (baseTimes.has(time)) continue;
await driver.seek(time);
collected.layoutIssues.push(...(await driver.collectOverlap(time)));
}Cost: serial loop of up to ~120 Flake: neither Fix: (a) wrap the loop body in try/catch, record the throw as a soft — Review by Rames D Jusso |
||
| if (baseTimes.has(time)) continue; | ||
| await driver.seek(time); | ||
| collected.layoutIssues.push(...(await driver.collectOverlap(time))); | ||
| } | ||
| } | ||
|
|
||
| // Frozen-sweep guard (#U10): compositions this short can legitimately hold a | ||
| // single static frame the whole time (a title card) — never flag those. | ||
| const SWEEP_STATIC_MIN_DURATION_SEC = 3; | ||
|
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
🟠 The "surfaces a held content_overlap" test doesn't exercise the dense-vs-sparse distinction it claims to demonstrate.
This returns a
content_overlapwarning at EVERY dense sample time (~72 samples for a 9s fake duration at 8fps). The collapse step sees ~72 occurrences → persistence tier trivially promotes to error via theoccurrences >= 2shortcut (see the sibling comment onlayoutAudit.ts:324). The test would pass equally well against a broken dense pass that simply promoted every finding — it never assertsoccurrencescount orfirstSeen/lastSeenspan.More importantly: the test doesn't reproduce the target defect (transient crossing seen only by 2 adjacent dense samples that the sparse grid missed). Test 2 (:1364-1373) correctly validates the static-composition gate — that one is fine.
Fix: add a third test where
collectOverlapreturns a finding at exactly two adjacent dense sample times and nothing anywhere else. Assert (a)driver.collectOverlap.mock.calls.lengthmatches the dense grid size, (b) the surviving finding hasoccurrences === 2, (c) severity resolves to what the intended semantic is (once the sibling comment's design question is decided).Also:
buildOverlapSampleTimes(duration)at checkPipeline.ts:441 is a pure function and would take ~5 lines to unit-test for the count/bounds/cap-at-120/quantization behavior. Catches density-math regressions the corpus can't.— Review by Rames D Jusso