feat: add smooth Bezier movement to dragMouse#172
feat: add smooth Bezier movement to dragMouse#172ulziibay-kernel wants to merge 5 commits intomainfrom
Conversation
Covers click, type, press key, scroll, and drag mouse with performance-first algorithms (zero additional xdotool process spawns). Includes the existing Bezier curve mouse movement as reference. Co-authored-by: Cursor <cursoragent@cursor.com>
- Add smooth (boolean, default false) and duration_ms (integer, 50-10000ms) to DragMouseRequest in OpenAPI spec - Add GenerateMultiSegmentTrajectory to mousetrajectory library: chains per-pair Bezier curves with proportional point distribution and screen bounds clamping - Refactor doDragMouse into doDragMouseLinear (existing behavior) and doDragMouseSmooth (Bezier path with jitter delays) - Add multi-segment trajectory tests and visual demo script Made-with: Cursor
- Change smooth default from false to true (matching moveMouse behavior) - Add drag_demo.html with draggable ball and persistent path visualization - Fix waypoint marker coordinates (viewport vs screen offset) - Add smooth_drag_demo.gif for PR illustration Made-with: Cursor
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 2 potential issues.
Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
| } | ||
| func (s *ApiService) doDragMouseSmooth(ctx context.Context, log *slog.Logger, body oapi.DragMouseRequest, btn string, screenWidth, screenHeight int) error { | ||
| if body.DurationMs != nil && (*body.DurationMs < 50 || *body.DurationMs > 10000) { | ||
| return &validationError{msg: "duration_ms must be between 50 and 10000"} |
There was a problem hiding this comment.
Late validation causes unintended mouse click on invalid input
Medium Severity
The duration_ms validation inside doDragMouseSmooth runs after Phase 1 has already executed mousedown. When the validation fails, the parent error handler sends mouseup as cleanup — effectively producing an unintended click at the start position before returning a 400 error. The duration_ms bounds check needs to happen before Phase 1's mousedown to avoid this side effect.
Additional Locations (1)
| points[i][1] = maxY | ||
| } | ||
| } | ||
| } |
There was a problem hiding this comment.
Duplicated clampPoints utility across two packages
Low Severity
The new unexported clampPoints function in mousetrajectory.go is functionally identical to the existing clampPoints in computer.go. Having two copies of the same logic increases maintenance burden and risk of inconsistent fixes. Since GenerateMultiSegmentTrajectory now handles clamping internally, consider exporting it from one location.
Drop drag_demo.html, smooth_drag_demo.gif, test_smooth_drag.py, and humanize-computer-endpoints.md from this branch. Made-with: Cursor
- Replace per-step Go-side sleeps (O(n) fork+exec) with a single xdotool arg chain using inline sleep directives (1 fork+exec) - Add smoothstep easing (3t²-2t³) for step delays: slow at pickup and placement, fast in the middle, matching natural drag behavior - Aligns with the performance-first design in the humanize plan Made-with: Cursor
1 participant
Summary
smooth(boolean, defaulttrue) andduration_ms(integer, 50-10000ms) toDragMouseRequestsmooth=true, drag movement uses human-like Bezier curves between path waypoints instead of linear interpolation (steps_per_segmentandstep_delay_msare ignored)GenerateMultiSegmentTrajectoryto themousetrajectorylibrary: chains per-pair Bezier curves with proportional point distribution and screen bounds clampingdoDragMouseintodoDragMouseLinear(existing behavior) anddoDragMouseSmooth(Bezier path with jitter delays)This is the counterpart to #148 (smooth moveMouse) — same Bezier approach, now applied to drag operations. A corresponding PR in
kernel/kernelwill forwardsmoothandduration_msthrough the public API. Overall project tracked at https://linear.app/onkernel/issue/KERNEL-1070/humanize-all-computer-interaction-endpointsDemo
Blue =
smooth: true(Bezier curves) | Red =smooth: false(linear interpolation)Test plan
mousetrajectorymulti-segment tests (3+ waypoints, 2 waypoints, duration_ms, screen clamping, single point, path continuity)test_smooth_drag.py) produces recording showing curved vs linear drag pathsMade with Cursor
Note
Medium Risk
Changes the runtime behavior and timing of
DragMouseby defaulting to a new smooth trajectory generator, which could affect automation reliability and edge-case cursor positioning despite added validation/clamping.Overview
DragMousenow supports human-like dragging via a newsmoothflag (defaulttrue) and optionalduration_ms, switching movement from fixed linear relative steps to a multi-segment Bezier trajectory with eased/jittered per-step delays.The drag implementation is refactored into
doDragMouseLinear(previous behavior) anddoDragMouseSmooth, with the smooth path built from a newmousetrajectory.GenerateMultiSegmentTrajectoryhelper that chains waypoint-to-waypoint curves, distributes points by segment distance, clamps points to screen bounds, and derives a per-step delay fromduration_ms.OpenAPI/OAPI models are updated accordingly (including regenerated embedded
swaggerSpec), and new unit tests cover multi-waypoint trajectory generation, duration behavior, clamping, and path continuity.Written by Cursor Bugbot for commit 3884879. This will update automatically on new commits. Configure here.