fix(cua): use scroll notch count (wheel units) in all computer-use templates#129
Merged
dprevoznik merged 16 commits intomainfrom Mar 3, 2026
Merged
fix(cua): use scroll notch count (wheel units) in all computer-use templates#129dprevoznik merged 16 commits intomainfrom
dprevoznik merged 16 commits intomainfrom
Conversation
…template - Send delta_x/delta_y as signed notch count (kernel-images uses delta as wheel-event repeat count, not pixels) - Return tool output: "Scrolled N wheel unit(s) direction." - Add system prompt line: scroll_amount and result are in wheel units Made-with: Cursor
…late - Send delta_x/delta_y as signed notch count (kernel-images uses delta as wheel-event repeat count, not pixels) - Return tool output: "Scrolled N wheel unit(s) direction." - Add system prompt line: scroll_amount and result are in wheel units Made-with: Cursor
…plate - Send delta_x/delta_y as signed notch count (kernel-images uses delta as wheel-event repeat count, not pixels) - SCROLL_DOCUMENT: use 3 notches instead of 500 pixels - SCROLL_AT: treat magnitude as notch count instead of denormalizing to pixels Made-with: Cursor
- Send delta_x/delta_y as signed notch count (kernel-images uses delta as wheel-event repeat count, not pixels) - SCROLL_DOCUMENT: use 3 notches instead of 500 pixels - SCROLL_AT: treat magnitude as notch count instead of denormalizing to pixels Made-with: Cursor
…plate - Send delta_x/delta_y as signed notch count (kernel-images uses delta as wheel-event repeat count, not pixels) - Return tool output: "Scrolled N wheel unit(s) direction." Made-with: Cursor
- Send delta_x/delta_y as signed notch count (kernel-images uses delta as wheel-event repeat count, not pixels) - Return tool output: "Scrolled N wheel unit(s) direction." Made-with: Cursor
- Send delta_x/delta_y as signed notch count (kernel-images uses delta as wheel-event repeat count, not pixels) - Support left/right scroll directions - Default scroll_amount changed from 100 (pixels) to 3 (notches) Made-with: Cursor
…7 notches) - scroll_document and scroll_at: magnitude → notches with PX_PER_NOTCH=60, MAX_NOTCHES_PER_ACTION=17, single API call - Remove chunking; default magnitude 400 - KERNEL_INVOCATION guard so invokes use payload query Made-with: Cursor
- Scroll: same as Python (PX_PER_NOTCH 60, MAX_NOTCHES_PER_ACTION 17, single call) - loop.ts: Environment.ENVIRONMENT_BROWSER, candidate?.content, fc.name guard, content check in pruneOldScreenshots - session.ts: session_id/liveViewUrl/replayViewUrl ?? null for string | null - index.ts: KERNEL_INVOCATION guard for payload query Made-with: Cursor
- OpenAGI/Lux emits N scroll actions for amount N; treat each as 1 notch - Document in handler docstring; no coalescing Made-with: Cursor
…lates Made-with: Cursor
dprevoznik
changed the title
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Scroll amount was stored but never used; the only call to _execute_scroll hardcoded notches=1. Align with docstring: 1 scroll event = 1 notch, model controls amount by emitting N scroll actions. Remove parameter and assignment to fix misleading API. Made-with: Cursor
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
…mplates Made-with: Cursor
- Template key: yutori-computer-use -> yutori in pkg/create/templates.go - Rename pkg/templates/python/yutori-computer-use -> python/yutori - Rename pkg/templates/typescript/yutori-computer-use -> typescript/yutori - Update .cursor/commands/qa.md to use -t yutori Made-with: Cursor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
updates scrolling logic across all computer use templates in the CLI.
fix-cua-templates-scroll-behaviorMade with Cursor
Note
Medium Risk
Changes scroll semantics across multiple computer-use templates from pixel-based deltas to wheel-notch units, which can materially alter agent navigation behavior and task success. Also includes a few runtime guards (local execution gating, null-handling) that are low risk but touch invocation paths.
Overview
Unifies scroll behavior across computer-use templates by switching from pixel-based scroll deltas to wheel unit (notch) counts in Anthropic (TS/Python), Gemini (TS/Python), Yutori (TS/Python), and OpenAGI handlers, and updating tool outputs/prompts to reflect the new units.
Gemini templates now convert
magnitude(px) to capped notch counts (viaPX_PER_NOTCH/MAX_NOTCHES_PER_ACTION) and add a few safety checks (e.g., missing function names/content). Local test entrypoints in Gemini are gated onKERNEL_INVOCATION, and the TS Gemini session adds null-safe handling for returned URLs/IDs.Template/QA naming is aligned by changing the yutori template key from
yutori-computer-usetoyutoriin docs (qa.md) andpkg/create/templates.go.Written by Cursor Bugbot for commit 6f3501d. This will update automatically on new commits. Configure here.