UI Freeze: Area selection overlay triggers unexpectedly and locks the screen when exiting Settings in Instant Mode

[wubie052]

Description

When I select the area recording option in the Cap UI, switch to Instant mode, and then open the Settings menu, a logical bug occurs when I try to exit Settings. The system immediately triggers the area selection overlay, even though I haven't clicked 'Start Recording' or the 'Area' button.
At this point, the selection interface freezes, making the computer appear completely unresponsive. My entire desktop is covered by an invisible, semi-transparent overlay. All mouse interactions and most keyboard inputs are completely intercepted. I can't see anything, and right-clicking anywhere brings up the Cap context menu. The only way to recover is to force-kill the application or click 'Refresh' in the Cap context menu.

Reproduction

1. Select the area recording option in the Cap UI.
2. Switch to Instant mode.
3. Open the Settings menu, and then exit it.
4. Observe the screen after exiting the Settings menu.

Additional Context

• Cap version:0.5.1
• Operating system, version:Windows 11 - 10.0.26200
• Device (optional): Dual monitors with identical resolution and size（HKC Display）

[linear]

CAP-738

[tembo]

The root cause is in the Settings window close handler at apps/desktop/src-tauri/src/lib.rs:5138-5162. When the Settings window is destroyed, it unconditionally calls show_overlay() for any existing TargetSelectOverlay windows:

CapWindowId::Settings => {
    for (label, window) in app.webview_windows() {
        if let Ok(id) = CapWindowId::from_str(&label) {
            match id {
                CapWindowId::TargetSelectOverlay { .. } => {
                    show_overlay(&window);  // Shows overlay regardless of current targetMode
                }
                // ...
            }
        }
    }
}

The show_overlay() function (windows.rs:108-111) enables cursor events and shows the window:

pub fn show_overlay(window: &WebviewWindow) {
    let _ = window.set_ignore_cursor_events(false);
    let _ = window.show();
}

The issue is that the overlay window may still exist from when the user selected area mode, but the frontend targetMode state has changed. When Settings closes, the backend restores the overlay without checking whether the frontend actually wants it displayed.

A fix would need to either:

1. Check the current targetMode from RecordingSettingsStore before calling show_overlay() and only show if targetMode is still "area"
2. Or emit a RequestSetTargetMode event with the current stored target mode so the frontend can decide whether to show the overlay
3. Or close the overlay windows when switching away from area mode, rather than just hiding them

This is similar to the pattern used for the Main window restoration in the same handler, where it properly coordinates with restore_main_window_inputs(). The overlay restoration needs similar coordination with the frontend state.

Related: #1915 discusses area picker issues on Windows with multi-monitor setups, and PR #1412 covers the Settings window close behavior.

[wubie052]

Yes，That's definitely the exact same issue. I'm also running Windows on a dual-monitor setup and selected 'Area Mode'
Hardware: Dual monitors with identical resolution and size（HKC Display）

[wubie052]

Also, just to add to this: the bug is also triggered when returning from the Settings window while in Studio mode, or when Display mode is selected.