Skip to content

Dialog detection: subprocess timeout fix and documentation#24

Draft
Niaobu wants to merge 5 commits intomainfrom
feature/dialog-detection
Draft

Dialog detection: subprocess timeout fix and documentation#24
Niaobu wants to merge 5 commits intomainfrom
feature/dialog-detection

Conversation

@Niaobu
Copy link
Contributor

@Niaobu Niaobu commented Mar 3, 2026

Summary

Follow-up to the dialog detection feature (a72c210, committed directly to main):

  • Subprocess timeout: Fix osascript hanging indefinitely on macOS when the calling process lacks Accessibility permissions. Uses async ReadToEnd + WaitForExit(5s) with kill on timeout, preventing the CLI from blocking.
  • Progress bar detection (all platforms): Extend dialog detection to also detect Unity progress bars and the startup splash window. Progress bars (DisplayProgressBar, DisplayCancelableProgressBar) are treated as dialogs with optional progress (0.0-1.0) and description fields.
  • Documentation: Comprehensive docs/dialog-detection.md covering platform support (Windows/macOS/Linux), Safe Mode behavior, agent workflow patterns, window classes, progress bar detection, and implementation details.

Progress bar detection

Detected as dialogs with optional description and progress fields (nullable, omitted from JSON when absent for backwards compatibility):

{"title":"Building Player (busy for 35s)...","buttons":["Cancel","Skip Transcoding"],"description":"Write asset files","progress":1.0}

Windows: Detects msctls_progress32 child controls via PBM_GETPOS/PBM_GETRANGE, Static text labels for descriptions. Matches both #32770 (standard Win32 dialog) and UnitySplashWindow (editor startup loading screen).

macOS: Reads progress indicator elements and their value property, static text elements for descriptions. Now detects windows with progress indicators even if they have no buttons.

Linux: Reads ROLE_PROGRESS_BAR via pyatspi queryValue(), ROLE_LABEL for descriptions. The pyatspi-only path now detects any window with a progress bar regardless of AT-SPI role.

macOS finding

All macOS APIs that can read window content or interact with UI elements (AXUIElement, osascript, JXA, NSWorkspace) require Accessibility permissions. No workaround exists. The current approach (osascript + graceful timeout) is the right trade-off — full functionality from local terminals, silent degradation over SSH.

Test plan

  • Windows: dialog detection and dismissal verified E2E (1/2/3-button dialogs, safe mode dialog)
  • Windows: progress bar detection verified with real Unity operations:
    • Editor startup splash (UnitySplashWindow with live progress updates)
    • BuildPipeline.BuildPlayer (full build lifecycle: compiling scripts → building player → shader variants → incremental build)
    • EditorUtility.DisplayProgressBar and DisplayCancelableProgressBar
  • Windows: status command shows progress bars inline with dialogs
  • Windows: "dismiss" hint only shown when dialogs have buttons
  • macOS via SSH: dialog list returns empty gracefully without hanging (5s timeout works)
  • macOS via SSH: status, dialog dismiss degrade gracefully
  • macOS via SSH: all non-dialog commands unaffected (script eval, editor run/stop, wait)
  • dotnet build passes, dotnet test 163/163 pass

Niaobu added 5 commits March 3, 2026 21:31
Add cross-platform detection and dismissal of Unity Editor modal dialogs
(Windows Win32 API, macOS AppleScript, Linux xdotool/pyatspi). Dialogs
block Unity's main thread, causing all RPC commands to hang with no
feedback — this feature lets agents detect and dismiss them.

- New `dialog list` and `dialog dismiss` commands
- `status` command now surfaces detected popups with a yellow warning
- DialogInfo DTO for JSON output
- EditorCommands process helpers made internal for reuse

Remove SuppressSafeModeDialog — it silently forced Unity into Safe Mode
on compilation errors, which prevents the plugin from connecting. The
dialog system is strictly better: agents can detect the "Enter Safe
Mode?" dialog and click "Ignore" to stay in normal mode.
osascript on macOS can hang indefinitely if the calling process lacks
Accessibility permissions. Use async ReadToEnd + WaitForExit(5s) with
kill on timeout to prevent blocking the parent command.
Covers platform support (Windows/macOS/Linux), Safe Mode behavior,
agent workflow patterns, and implementation details.
Extend dialog detection to also detect Unity progress bars and the
startup splash window. Progress bars (EditorUtility.DisplayProgressBar,
DisplayCancelableProgressBar) and the UnitySplashWindow are now treated
as dialogs with optional progress/description fields.

On Windows, detects msctls_progress32 child controls (reading position
via PBM_GETPOS/PBM_GETRANGE) and Static text labels for descriptions.
Also matches UnitySplashWindow class in addition to #32770 dialogs.

DialogInfo DTO gains nullable description and progress fields, omitted
from JSON when not present for backwards compatibility.
macOS: Read progress indicators and static text via AppleScript
System Events API. Windows with buttons or progress indicators are
now detected (previously only windows with buttons were included).

Linux: Read ROLE_PROGRESS_BAR via pyatspi queryValue() and
ROLE_LABEL for descriptions. The pyatspi-only path now detects any
window with a progress bar regardless of role (not just ROLE_DIALOG
and ROLE_ALERT). GetButtonsPyatspi replaced with GetWindowInfoPyatspi
returning buttons, progress, and description as a tuple.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant