Skip to content

Modal-aware invoke + display keep-awake - #1

Merged
otykier merged 4 commits into
mainfrom
feature/modal-aware-invoke
Aug 19, 2026
Merged

Modal-aware invoke + display keep-awake#1
otykier merged 4 commits into
mainfrom
feature/modal-aware-invoke

Conversation

@otykier

@otykier otykier commented Aug 19, 2026

Copy link
Copy Markdown

Modal-aware invoke + display keep-awake

Two features that make long, unattended automation runs against WinForms/DevExpress apps (e.g. Tabular Editor 3) actually survive: clicks that open modal dialogs no longer hang the server, and Windows no longer locks the screen mid-run.

Modal dialog detection (e943ed7)

UIA pattern calls (Invoke/Toggle/Select) are synchronous cross-process calls. In WinForms/DevExpress apps, a click handler that opens a modal dialog (ShowDialog()) does not return until the dialog closes — so the pattern call, and with it the target app's entire UIA provider, stays blocked, and every subsequent tool call hits the 30s timeout.

What changed:

  • windows_click now runs the pattern call on a background thread while watching the target process's top-level windows via non-blocking Win32 APIs. When a modal is detected (new top-level window, or owner window disabled), the tool returns immediately with the dialog's title and interaction guidance instead of hanging.
  • While a provider is blocked by a pending call, ref-based tools (windows_snapshot, windows_get_text, windows_click, windows_type, windows_fill, ref-based windows_send_keys) and windows_batch actions targeting that app fail fast with guidance instead of timing out.
  • windows_list_windows, windows_focus and windows_close now use Win32 (EnumWindows / SetForegroundWindow / WM_CLOSE), so they are never blocked. Window handles are also stable across windows_list_windows calls now.
  • windows_screenshot falls back to a Win32 window-bounds capture while the provider is blocked.
  • Tools that don't need UIA keep working throughout — so a modal can always be seen (screenshot) and dismissed (ref-less send_keys/type).

Typical flow: click a button → "modal dialog opened" → screenshot to see it → send keys to dismiss it → snapshot works again.

New components: ModalAwareInvoker, PendingInvokeTracker, Win32Desktop.

Display keep-awake (c218870)

Long automation runs generate no keyboard or mouse input, so Windows turns off the display and shows the lock screen mid-run — breaking screenshots and interaction.

What changed:

  • While tool calls are arriving, the server holds a Windows power availability request (PowerCreateRequest/PowerSetRequest with DisplayRequired + SystemRequired) — the same signal video players and conferencing apps send. Visible in powercfg /requests (elevated) as "FlaUI-MCP is driving Windows UI automation".
  • The request uses a sliding hold: the first tool call acquires it, every call extends it, and it is released after 5 minutes without activity — an idle server does not keep the screen on.
  • Configurable via the FLAUI_MCP_KEEP_AWAKE_SECONDS environment variable (positive value = idle period, 0 = disabled).

Limitation (documented in README): a domain group policy enforcing a hard machine inactivity limit ("Interactive logon: Machine inactivity limit") locks on raw input idle time and cannot be suppressed by availability requests.

New components: KeepAwake (sliding-hold controller), PowerAvailabilityRequest (Win32 wrapper); ToolRegistry gained an onToolActivity callback.

Housekeeping

  • Release-notes README link in release.yml now points at this fork instead of upstream shanselman/FlaUI-MCP (837201f).
  • Local publish/ output is now gitignored (9153469).

Testing

  • Unit tests: ModalAwareInvokerTests, PendingInvokeTrackerTests, KeepAwakeTests (sliding-hold semantics, dispose behavior, and a live round-trip through the real PowerCreateRequest API), plus a ToolRegistry activity-callback test — 32 tests passing.
  • Desktop integration test (ModalDialogTests) exercises the WinForms test app's modal dialog end-to-end (requires an interactive Windows session).
  • Manually verified against Tabular Editor 3: modal dialogs are detected and dismissible, and the machine no longer locks during long runs.

🤖 Generated with Claude Code

otykier and others added 4 commits August 12, 2026 15:24
UIA pattern calls (Invoke/Toggle/Select) are synchronous cross-process calls. In
WinForms/DevExpress apps a click handler that opens a modal dialog (ShowDialog)
does not return until the dialog closes, so the pattern call - and with it the
target app's entire UIA provider - stays blocked, causing every tool to hit the
30s timeout.

windows_click now runs the pattern call on a background thread while watching the
target process's top-level windows via non-blocking Win32 APIs. When a modal is
detected (new top-level window, or owner window disabled) it returns immediately
with the dialog title instead of hanging.

While a provider is blocked by a pending call, ref-based tools (snapshot, get_text,
click, type, fill, send_keys) and batch actions fail fast with guidance instead of
timing out. windows_list_windows/focus/close now use Win32 (never blocked), and
windows_screenshot falls back to a Win32 window-bounds capture. Tools that do not
need UIA - screenshot, ref-less send_keys/type - keep working throughout, so a
modal can be seen and dismissed.

Adds ModalAwareInvoker, PendingInvokeTracker, Win32Desktop, unit tests, and a
desktop integration test exercising the WinForms test app's modal dialog.
Long automation runs generate no keyboard or mouse input, so Windows turns
off the display and shows the lock screen mid-run, breaking screenshots and
interaction. The server now holds a Windows power availability request
(PowerCreateRequest/PowerSetRequest with DisplayRequired + SystemRequired) -
the same signal video players and conferencing apps send - while tool calls
are arriving, visible in `powercfg /requests` with a diagnostic reason.

The request uses a sliding hold: the first tool call acquires it, every call
extends it, and it is released after 5 minutes without activity so an idle
server does not keep the screen on. Configure the idle period (or disable
with 0) via FLAUI_MCP_KEEP_AWAKE_SECONDS.

Note: a group-policy machine inactivity limit locks on raw input idle time
and cannot be suppressed by availability requests.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@otykier
otykier merged commit 1df570e into main Aug 19, 2026
2 checks passed
@otykier
otykier deleted the feature/modal-aware-invoke branch August 19, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant