Skip to content

Multicam: draw the selected detection on other cameras without switching first#1737

Merged
BryonLewis merged 12 commits into
mainfrom
dev/cross-camera-continuation-draw
Jul 3, 2026
Merged

Multicam: draw the selected detection on other cameras without switching first#1737
BryonLewis merged 12 commits into
mainfrom
dev/cross-camera-continuation-draw

Conversation

@mattdawkins

Copy link
Copy Markdown
Member

Motivation

In multicam/stereo manual annotation, after drawing a line (or rectangle) on one camera, drawing it on the other camera required left-clicking that camera (switching the selected camera, which also dropped edit mode) and re-entering creation mode. For stereo measurement workflows this adds several clicks per detection.

Change

While in edit mode, any non-selected camera that is missing the selected track's geometry at the current frame now keeps a live creation cursor. Drawing there commits to that camera under the same track id — so a line drawn on the left camera can immediately be placed on the right camera with two more clicks.

This generalizes the existing "seamless multicam creation" mechanism (which handled brand-new tracks with no geometry anywhere) to the continuation case.

Implementation

LayerManager.vue

  • New cameraAwaitingGeometry() helper — true when, in edit mode, the selected track has no geometry on this camera at this frame (or doesn't exist on it). Complements isCreatingNewDetection().
  • updateLayers: the non-selected-camera creation-cursor branch now also fires for this case.
  • update:geojson routing: for the continuation case, the same-id track is created on the drawn-on camera before selectCamera is called. Ordering matters: since the track then exists, selectCamera skips its trackEdit toggle — which would call finalizeInProgress() and discard the in-progress 1-point line replica, breaking the 2-click line completion. The brand-new-track path (trackAdd()) is unchanged.

Viewer.vue

  • changeCamera no longer consumes a left-mousedown on a camera awaiting geometry (that click is the start of a draw there). Right-click camera switching (mouseup.right) and the camera dropdown are unaffected. Escape exits edit mode, restoring normal left-click switching.

Point/segmentation mode is excluded from all of the above — segmentation has its own cross-camera machinery (stereo warp).

Side benefit: when the second camera's line completes, onStereoAnnotationComplete fires as usual, so with "update lengths when modified" enabled the stereo length is computed immediately once both sides have lines.

Verification

  • eslint clean on both files.
  • yarn test: 197 passed / 2 failed — the failures are identical on the pristine tree (Electron install + missing File global in the node test env), unrelated to this change.

…ing first

Previously, after drawing a line or rectangle on one camera during manual
creation, annotating the other camera required clicking it (switching the
selected camera) and re-entering edit mode before drawing.

Now, while in edit mode, any non-selected camera missing the selected
track's geometry at the current frame keeps a live creation cursor, so the
detection can be drawn on each camera in turn -- e.g. two more clicks to
place the line on the other stereo camera -- committed under the same
track id.

Implementation:
- LayerManager: new cameraAwaitingGeometry() complements the existing
  isCreatingNewDetection() (brand-new track) mechanism, both for enabling
  the creation cursor on non-selected cameras and for routing a landed
  draw. The routing creates the same-id track on the drawn-on camera
  BEFORE calling selectCamera, so selectCamera skips its trackEdit toggle
  (which would finalize and discard an in-progress 1-point line replica,
  breaking 2-click completion).
- Viewer: changeCamera no longer consumes a left-mousedown on a camera
  awaiting geometry (that click starts the draw). Right-click switching
  and the camera dropdown are unaffected.

Point/segmentation mode is excluded; segmentation has its own
cross-camera machinery.
Two fixes for point-click segmentation on stereo/multicam datasets:

1. Segmentation prompt points (the green/red click markers) were rendered
   by every camera's layer manager with no camera filter, so each click
   also drew its marker on the other camera at the same pixel location --
   which looked like an unwarped point being auto-created there,
   regardless of the auto-compute setting or calibration. The markers now
   render only on the selected camera.

2. Point mode was excluded from cross-camera continuation drawing. Now a
   single click on the other camera starts a segmentation there for the
   same track id: 'awaiting geometry' for Point mode means the camera has
   no segmentation polygon yet (so a box-only detection still accepts a
   click), and a new segmentationFinalizePending handler -- invoked on
   any camera switch -- locks in the pending mask on the source camera
   and clears the recipe's accumulated prompt points, which belong to the
   source camera's image and must not leak into the other camera's
   prediction. Unlike the right-click confirm path it does not deselect,
   so the flow continues uninterrupted. The predict image path already
   resolves the selected camera at call time, so post-switch predictions
   run against the correct camera's image.

Line, rectangle, and polygon continuation behavior is unchanged.
@mattdawkins

Copy link
Copy Markdown
Member Author

Added a second commit extending the continuation mechanism to point-click segmentation:

  • Prompt-point markers no longer render on every camera. The green/red click markers were drawn by each camera's layer manager with no camera filter, so a click also showed its marker on the other camera at the same pixel location — easily mistaken for an unwarped auto-created point (it appeared regardless of the auto-compute setting or calibration). They now render only on the selected camera.
  • One-click segmentation on the other camera. Point mode was previously excluded from cross-camera continuation. Now a click on the other camera segments there under the same track id. "Awaiting geometry" for Point mode means no segmentation polygon on that camera yet, so box-only detections accept a click. A new segmentationFinalizePending handler (invoked on any camera switch) locks in the source camera's pending mask and clears the recipe's accumulated prompt points — they belong to the source camera's image and must not leak into the other camera's prediction. Unlike right-click confirm, it does not deselect, so the flow continues uninterrupted.

Line/rectangle/polygon continuation behavior is unchanged.

@BryonLewis BryonLewis self-requested a review July 2, 2026 15:36
…ing first

Previously, after drawing a line or rectangle on one camera during manual
creation, annotating the other camera required clicking it (switching the
selected camera) and re-entering edit mode before drawing.

Now, while in edit mode, any non-selected camera missing the selected
track's geometry at the current frame keeps a live creation cursor, so the
detection can be drawn on each camera in turn -- e.g. two more clicks to
place the line on the other stereo camera -- committed under the same
track id.

Implementation:
- LayerManager: new cameraAwaitingGeometry() complements the existing
  isCreatingNewDetection() (brand-new track) mechanism, both for enabling
  the creation cursor on non-selected cameras and for routing a landed
  draw. The routing creates the same-id track on the drawn-on camera
  BEFORE calling selectCamera, so selectCamera skips its trackEdit toggle
  (which would finalize and discard an in-progress 1-point line replica,
  breaking 2-click completion).
- Viewer: changeCamera no longer consumes a left-mousedown on a camera
  awaiting geometry (that click starts the draw). Right-click switching
  and the camera dropdown are unaffected.

Point/segmentation mode is excluded; segmentation has its own
cross-camera machinery.
Two fixes for point-click segmentation on stereo/multicam datasets:

1. Segmentation prompt points (the green/red click markers) were rendered
   by every camera's layer manager with no camera filter, so each click
   also drew its marker on the other camera at the same pixel location --
   which looked like an unwarped point being auto-created there,
   regardless of the auto-compute setting or calibration. The markers now
   render only on the selected camera.

2. Point mode was excluded from cross-camera continuation drawing. Now a
   single click on the other camera starts a segmentation there for the
   same track id: 'awaiting geometry' for Point mode means the camera has
   no segmentation polygon yet (so a box-only detection still accepts a
   click), and a new segmentationFinalizePending handler -- invoked on
   any camera switch -- locks in the pending mask on the source camera
   and clears the recipe's accumulated prompt points, which belong to the
   source camera's image and must not leak into the other camera's
   prediction. Unlike the right-click confirm path it does not deselect,
   so the flow continues uninterrupted. The predict image path already
   resolves the selected camera at call time, so post-switch predictions
   run against the correct camera's image.

Line, rectangle, and polygon continuation behavior is unchanged.
…deselect, stereo startup

Three fixes for stereo/multicam annotation editing:

- Edit mode is now active on every camera showing the selected track, not
  just the selected one, so a stereo detection can be adjusted on either
  camera without selecting it first. The mousedown that grabs an edit
  handle switches the selected camera (preserving edit mode since the
  track exists there) without preventDefault killing the drag, and the
  edit-layer reset is deferred while the button is held. A routing
  fallback commits edits from a non-selected camera to that camera's own
  track.

- A right-click while editing now finalizes and deselects the detection
  in a single press, matching single-camera behavior, instead of merely
  switching cameras and leaving it selected until a second right-click.
  Also fixes finalizeInProgress emitting the garbage GeoJS trace for a
  LineString still in creation mode (the real feature lives in
  shapeInProgress), which could commit a bogus line.

- Interactive stereo startup no longer silently fails to produce lengths.
  Entering a sequence with a stereo feature on shows the persistent,
  annotation-blocking loading dialog until the service is ready; enable
  failures are always surfaced instead of degrading silently; and if a
  measurement is requested while the service is wanted but not running,
  the handler kicks off the enable itself and waits, so drawing a line
  always forces the service up rather than producing no measurement.
@mattdawkins mattdawkins force-pushed the dev/cross-camera-continuation-draw branch from 44ff630 to 26663be Compare July 2, 2026 17:59
@mattdawkins

Copy link
Copy Markdown
Member Author

Rebased on top of main (picks up #1736 stereo auto-enable, #1735, and the deployment-docs commit) and added a third commit with fixes from interactive testing:

  • Edit mode on both cameras. Edit handles are now live on every camera showing the selected track, so a stereo detection can be adjusted on either camera without selecting it first. The mousedown that grabs a handle switches the selected camera (edit mode preserved) without preventDefault killing the drag; the edit-layer reset is deferred while the button is held; and a routing fallback commits an edit from a non-selected camera to that camera's own track.
  • Single-press right-click deselect. A right-click while editing finalizes and deselects the detection in one press (matching single-camera behavior), instead of merely switching cameras and leaving it selected until a second right-click. Also fixes finalizeInProgress emitting the garbage GeoJS trace for a LineString still in creation mode (the real feature lives in shapeInProgress), which could commit a bogus line.
  • Stereo startup. Entering a sequence with a stereo feature on now shows the persistent, annotation-blocking loading dialog until the service is ready; enable failures are surfaced instead of degrading silently; and if a measurement is requested while the service is wanted but not running, the handler starts the enable itself and waits — so drawing a line always forces the interactive stereo service up rather than silently producing no measurement.

mattdawkins and others added 7 commits July 2, 2026 15:00
…ning

A failed cross-camera transfer showed two popups in sequence: the
'Computing stereo correspondence...' loading dialog flashed and vanished
on its own, immediately followed by a separate 'Stereo Transfer Error'
prompt. Reuse the loading dialog's own error state instead -- the spinner
morphs in place into the error (title + message + Close) rather than
hiding and spawning a second dialog.

A transfer that finds no stereo match is a common, recoverable event, so
its alert is now a softer 'warning' (amber) rather than the hard red
'error' reserved for service-startup failures.
A right-click near the head/tail line of a tight (skinny) bounding box
lands inside both the rectangle polygon and the line feature, so
RectangleLayer and LineLayer both emit annotation-right-clicked for the
one physical click. Both are wired to Clicked -> handler.trackEdit, which
toggles edit mode, so the second emit immediately undid the first: the
detection entered line edit mode and then dropped straight back to merely
selected.

Add a same-tick guard in Clicked (mirroring the existing
justFinalizedCreation flag): the first emit for a physical click is
handled and any further emits in the same synchronous tick are ignored,
cleared on the next macrotask so distinct user clicks are unaffected.
Both overlapping features belong to the same track, so first-wins is
unambiguous.
@BryonLewis

Copy link
Copy Markdown
Collaborator

I've made some updates to resolve some minor issues and some other fixes:

  • Toolbar button alignment: I didn't like the way the collapsed vs uncollapsed additional icons (like the delete buttons or reset in segmentation) didn't align properly so I've updated it to align with the collapsed version better. Also there was a divider in segmentation mode that was too large (stretched above and below the toolbar line) so I removed it
  • The segmentation reation used to rotate the magic marker/wand icon. I changed it to use a standard spinner instead of having that icon rotate during the segmentation creation process.
  • Multicamera creation/editing Adjustments:
    • Polygon creation didn't work across cameras so it has been udpated so you can just click and start creating the new polygon on the other camera
    • Updated some editing issues with bboxes where the editing bbox would snap back when selecting an editing handle on another camera

@BryonLewis BryonLewis left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested and made some updates to fix minor issues and some nagging UI/UX issues.

@BryonLewis BryonLewis merged commit 5e55550 into main Jul 3, 2026
3 checks passed
@BryonLewis BryonLewis deleted the dev/cross-camera-continuation-draw branch July 3, 2026 14:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants