Modal-aware invoke + display keep-awake

.github/workflows/release.yml

@@ -180,7 +180,7 @@ jobs:
           }
           ```
 
-          See [README](https://github.com/shanselman/FlaUI-MCP#readme) for full documentation.
+          See [README](https://github.com/TabularEditor/FlaUI-MCP#readme) for full documentation.
         files: artifacts/*.zip
         draft: ${{ steps.release.outputs.draft }}
         prerelease: ${{ steps.release.outputs.prerelease }}

.gitignore

@@ -18,6 +18,7 @@ obj/
 # Build results
 [Dd]ebug/
 [Rr]elease/
+publish/
 x64/
 x86/
 build/

CHANGELOG.md

@@ -7,6 +7,16 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+- The server now keeps the display awake while tools are actively being called, so Windows does not turn off the screen or show the lock screen in the middle of a long automation run. Implemented with a Windows power availability request (`PowerCreateRequest`/`PowerSetRequest` with `PowerRequestDisplayRequired` + `PowerRequestSystemRequired`) — the same mechanism video players and conferencing apps use, visible in `powercfg /requests`. The request is released after 5 minutes without a tool call; configure the idle period (or disable with `0`) via the `FLAUI_MCP_KEEP_AWAKE_SECONDS` environment variable.
+
+### Fixed
+- `windows_click` no longer hangs (and then times out) when the clicked element's handler opens a modal dialog. UIA pattern calls (Invoke/Toggle/Select) are synchronous cross-process calls: a WinForms/DevExpress handler that calls `ShowDialog()` does not return until the dialog closes, blocking the target app's entire UIA provider. The click now runs on a background thread while non-blocking Win32 APIs watch for the modal signature (new top-level window, or owner window disabled) and returns immediately with the dialog's title and interaction guidance.
+- While an app's UIA provider is blocked by such a pending call, `windows_snapshot`, `windows_get_text`, `windows_click`, `windows_type`, `windows_fill`, `windows_send_keys` (ref-based) and `windows_batch` actions targeting that app now fail fast with guidance (use `windows_screenshot` / `windows_send_keys` without ref) instead of hanging until the 30s global timeout.
+- `windows_list_windows` now enumerates windows via Win32 instead of walking the UIA desktop tree, so it keeps working even while some app's UIA provider is blocked. Window handles are also stable across calls now (previously every call registered new handles for the same windows).
+- `windows_focus` and `windows_close` (by handle) now use Win32 (`SetForegroundWindow` / `WM_CLOSE`) and work while a provider is blocked.
+- `windows_screenshot` with a window handle falls back to a Win32 window-bounds capture while the app's provider is blocked.
+
 ## [0.2.0] - 2026-07-08
 
 ### Fixed

README.md

@@ -122,6 +122,27 @@ It can also save screenshots with `savePath`, which must be an absolute local
 Tool calls have a 30-second timeout so a blocked UI Automation provider or modal
 dialog returns an actionable error instead of hanging the MCP server forever.
 
+### Keeping the Screen Awake
+
+Long automation runs generate no keyboard or mouse input, so Windows would
+normally turn off the display and show the lock screen mid-run — which breaks
+screenshots and can freeze rendering. While tools are actively being called,
+FlaUI-MCP holds a Windows *power availability request* (the same signal video
+players and conferencing apps send) that keeps the display on and suppresses
+the idle lock. The request appears in `powercfg /requests` (run as admin) with
+the reason "FlaUI-MCP is driving Windows UI automation".
+
+The request is released after **5 minutes** without a tool call, so an idle MCP
+server does not keep your screen on. Configure via the
+`FLAUI_MCP_KEEP_AWAKE_SECONDS` environment variable: a positive value changes
+the idle period, `0` disables keep-awake entirely.
+
+Note: this covers the common idle-lock paths (display timeout, screensaver,
+sleep). A domain group policy that enforces a hard machine inactivity limit
+("Interactive logon: Machine inactivity limit") locks based on input idle time
+and is not suppressed by availability requests — no application can override
+that policy.
+
 ### Tool Examples
 
 Send a keyboard chord to a target element:
@@ -215,6 +236,18 @@ This comes from **Windows UI Automation** - the same API screen readers use. Eac
 
 FlaUI-MCP uses accessibility because it's what screen readers use - it's designed for programmatic UI interaction.
 
+### Modal Dialogs
+
+UI Automation pattern calls (Invoke, Toggle, Select) are synchronous cross-process calls. When a click handler opens a **modal dialog** (`ShowDialog()` in WinForms/WPF), the handler — and therefore the pattern call and the app's entire UIA provider — stays blocked until the dialog closes.
+
+FlaUI-MCP handles this instead of hanging:
+
+- `windows_click` runs the pattern call on a background thread and watches the app's top-level windows via non-blocking Win32 APIs. If a modal appears, the tool returns immediately with the dialog's title.
+- While the call is pending, UIA-based tools targeting that app (`windows_snapshot`, `windows_get_text`, ref-based typing/clicking) **fail fast** with guidance instead of timing out.
+- Tools that don't need UIA keep working throughout: `windows_screenshot`, `windows_send_keys` / `windows_type` *without a ref* (pure keyboard input), `windows_list_windows`, `windows_focus`, and `windows_close`.
+
+Typical flow: click a button → "modal dialog opened" → screenshot to see it → send keys (e.g. `Enter` or `Tab`+`Enter`) to dismiss it → snapshot works again.
+
 ## Building from Source
 
 ```powershell

src/FlaUI.Mcp/Core/ElementRegistry.cs

@@ -10,6 +10,7 @@ public class ElementRegistry
 {
     private readonly Dictionary<string, AutomationElement> _elements = new();
     private readonly Dictionary<string, int> _windowCounters = new();
+    private readonly Dictionary<string, int> _windowProcessIds = new();
 
     /// <summary>
     /// Clear all elements for a window (called before new snapshot)
@@ -55,4 +56,26 @@ public bool HasElement(string refId)
     {
         return _elements.ContainsKey(refId);
     }
+
+    /// <summary>
+    /// Record the process id owning a window's elements. Called during snapshot
+    /// building (when the provider is known to be responsive) so tools can later
+    /// check for a blocked provider without touching UI Automation.
+    /// </summary>
+    public void SetWindowProcessId(string windowHandle, int processId)
+    {
+        _windowProcessIds[windowHandle] = processId;
+    }
+
+    /// <summary>
+    /// Get the process id for an element ref (e.g. "w1e5" -> pid of window "w1").
+    /// Returns 0 if unknown.
+    /// </summary>
+    public int GetProcessIdForRef(string refId)
+    {
+        var separator = refId.LastIndexOf('e');
+        if (separator <= 0) return 0;
+        var windowHandle = refId[..separator];
+        return _windowProcessIds.TryGetValue(windowHandle, out var pid) ? pid : 0;
+    }
 }

src/FlaUI.Mcp/Core/KeepAwake.cs

@@ -0,0 +1,234 @@
+using System.Runtime.InteropServices;
+
+namespace PlaywrightWindows.Mcp.Core;
+
+/// <summary>
+/// Holds a resource (typically a Windows power availability request) while tool
+/// calls are actively arriving, releasing it after a sliding idle period.
+///
+/// <see cref="Poke"/> acquires the resource on first call and extends the hold
+/// on every subsequent call; when no poke arrives for the hold duration, the
+/// resource is released so the machine returns to its normal power/lock policy.
+/// </summary>
+public sealed class KeepAwake : IDisposable
+{
+    private readonly object _lock = new();
+    private readonly TimeSpan _holdDuration;
+    private readonly Action _acquire;
+    private readonly Action _release;
+    private readonly IDisposable? _ownedResource;
+    private readonly System.Threading.Timer _timer;
+    private long _deadlineTicks;
+    private bool _active;
+    private bool _disposed;
+
+    public KeepAwake(TimeSpan holdDuration, Action acquire, Action release, IDisposable? ownedResource = null)
+    {
+        if (holdDuration <= TimeSpan.Zero)
+        {
+            throw new ArgumentOutOfRangeException(nameof(holdDuration), "Hold duration must be greater than zero.");
+        }
+
+        _holdDuration = holdDuration;
+        _acquire = acquire;
+        _release = release;
+        _ownedResource = ownedResource;
+        _timer = new System.Threading.Timer(_ => OnTimerFired(), null, Timeout.InfiniteTimeSpan, Timeout.InfiniteTimeSpan);
+    }
+
+    /// <summary>
+    /// Create a KeepAwake that tells Windows the display is in use (the same
+    /// signal video players and conferencing apps send), preventing display
+    /// timeout, sleep and the idle-triggered lock screen while tools are active.
+    /// Returns null if the power request could not be created.
+    /// </summary>
+    public static KeepAwake? CreateDisplayKeepAwake(TimeSpan holdDuration, string reason)
+    {
+        var request = PowerAvailabilityRequest.Create(reason);
+        if (request == null)
+        {
+            return null;
+        }
+        return new KeepAwake(holdDuration, request.Set, request.Clear, request);
+    }
+
+    public bool IsActive
+    {
+        get
+        {
+            lock (_lock)
+            {
+                return _active;
+            }
+        }
+    }
+
+    /// <summary>
+    /// Signal activity: acquire the resource if not already held, and extend
+    /// the hold so it is released only after the idle period elapses.
+    /// </summary>
+    public void Poke()
+    {
+        lock (_lock)
+        {
+            if (_disposed)
+            {
+                return;
+            }
+
+            if (!_active)
+            {
+                _acquire();
+                _active = true;
+            }
+
+            _deadlineTicks = Environment.TickCount64 + (long)_holdDuration.TotalMilliseconds;
+            _timer.Change(_holdDuration, Timeout.InfiniteTimeSpan);
+        }
+    }
+
+    private void OnTimerFired()
+    {
+        lock (_lock)
+        {
+            if (_disposed || !_active)
+            {
+                return;
+            }
+
+            // A Poke may have raced with this callback; only release once the
+            // most recently extended deadline has actually passed.
+            var remainingMs = _deadlineTicks - Environment.TickCount64;
+            if (remainingMs > 0)
+            {
+                _timer.Change(TimeSpan.FromMilliseconds(remainingMs), Timeout.InfiniteTimeSpan);
+                return;
+            }
+
+            _release();
+            _active = false;
+        }
+    }
+
+    public void Dispose()
+    {
+        lock (_lock)
+        {
+            if (_disposed)
+            {
+                return;
+            }
+            _disposed = true;
+            _timer.Dispose();
+            if (_active)
+            {
+                try { _release(); } catch { }
+                _active = false;
+            }
+            _ownedResource?.Dispose();
+        }
+    }
+}
+
+/// <summary>
+/// Wraps a Windows power availability request (PowerCreateRequest) that, while
+/// set, tells the OS the display is required. This suppresses display timeout,
+/// automatic sleep and the inactivity lock screen - the same mechanism browsers
+/// use during video playback. The request is visible in `powercfg /requests`
+/// together with the reason string.
+/// </summary>
+public sealed class PowerAvailabilityRequest : IDisposable
+{
+    private readonly nint _handle;
+    private bool _disposed;
+
+    private PowerAvailabilityRequest(nint handle)
+    {
+        _handle = handle;
+    }
+
+    /// <summary>
+    /// Create a power request with the given diagnostic reason, or null on failure.
+    /// </summary>
+    public static PowerAvailabilityRequest? Create(string reason)
+    {
+        var reasonPtr = Marshal.StringToHGlobalUni(reason);
+        try
+        {
+            var context = new REASON_CONTEXT
+            {
+                Version = POWER_REQUEST_CONTEXT_VERSION,
+                Flags = POWER_REQUEST_CONTEXT_SIMPLE_STRING,
+                SimpleReasonString = reasonPtr
+            };
+
+            var handle = PowerCreateRequest(ref context);
+            if (handle == 0 || handle == INVALID_HANDLE_VALUE)
+            {
+                return null;
+            }
+            return new PowerAvailabilityRequest(handle);
+        }
+        finally
+        {
+            Marshal.FreeHGlobal(reasonPtr);
+        }
+    }
+
+    /// <summary>Activate the request: display must stay on, system must stay awake.</summary>
+    public void Set()
+    {
+        PowerSetRequest(_handle, POWER_REQUEST_TYPE.PowerRequestDisplayRequired);
+        PowerSetRequest(_handle, POWER_REQUEST_TYPE.PowerRequestSystemRequired);
+    }
+
+    /// <summary>Deactivate the request, restoring normal power/lock policy.</summary>
+    public void Clear()
+    {
+        PowerClearRequest(_handle, POWER_REQUEST_TYPE.PowerRequestDisplayRequired);
+        PowerClearRequest(_handle, POWER_REQUEST_TYPE.PowerRequestSystemRequired);
+    }
+
+    public void Dispose()
+    {
+        if (_disposed)
+        {
+            return;
+        }
+        _disposed = true;
+        try { Clear(); } catch { }
+        CloseHandle(_handle);
+    }
+
+    private const uint POWER_REQUEST_CONTEXT_VERSION = 0;
+    private const uint POWER_REQUEST_CONTEXT_SIMPLE_STRING = 0x1;
+    private static readonly nint INVALID_HANDLE_VALUE = -1;
+
+    private enum POWER_REQUEST_TYPE
+    {
+        PowerRequestDisplayRequired = 0,
+        PowerRequestSystemRequired = 1,
+        PowerRequestAwayModeRequired = 2,
+        PowerRequestExecutionRequired = 3
+    }
+
+    [StructLayout(LayoutKind.Sequential)]
+    private struct REASON_CONTEXT
+    {
+        public uint Version;
+        public uint Flags;
+        public nint SimpleReasonString;
+    }
+
+    [DllImport("kernel32.dll", SetLastError = true)]
+    private static extern nint PowerCreateRequest(ref REASON_CONTEXT context);
+
+    [DllImport("kernel32.dll", SetLastError = true)]
+    private static extern bool PowerSetRequest(nint powerRequest, POWER_REQUEST_TYPE requestType);
+
+    [DllImport("kernel32.dll", SetLastError = true)]
+    private static extern bool PowerClearRequest(nint powerRequest, POWER_REQUEST_TYPE requestType);
+
+    [DllImport("kernel32.dll", SetLastError = true)]
+    private static extern bool CloseHandle(nint handle);
+}