Gate the macOS window probe on the code, not on the runner's session

A macos-14 runner was measured to have no ordinary application windows at
all, so a probe that asserted a count would be asserting a property of the
runner image rather than of this project — and would go red the day that
image happens to open one. It now asserts that the backend is selected, the
Quartz query runs and every window it does return can be described, and
reports the counts either side of the layer filter so an empty session and a
filter that dropped everything stay told apart.

Author: JE-Chen

.github/workflows/platform-smoke.yml

@@ -84,8 +84,5 @@ jobs:
       # So the flag is off and this is a gate now: EXPECTED in the script
       # holds what was measured, and a capability appearing or disappearing
       # turns this red and names which one.
-      # Back in --measure for one round: the window-management probe is new
-      # and its value is not in EXPECTED yet. Asserting a value nobody has
-      # measured would be the guess this whole script exists to avoid.
-      - name: Measure the macOS backend against a real window server
-        run: python test/verify/macos_verify.py --measure
+      - name: Verify the macOS backend against a real window server
+        run: python test/verify/macos_verify.py

test/verify/macos_verify.py

@@ -61,6 +61,10 @@
     "keyboard-post": True,
     "accessibility-tree": True,
     "recorder-absent": True,
+    # True means "the code answered", not "the runner had windows": a
+    # macos-14 runner was measured to have none at the application layer, so
+    # the count is reported and not asserted.
+    "window-management": True,
 }
 
 _results: List[Tuple[str, bool, str]] = []
@@ -215,25 +219,32 @@ def probe_recorder() -> Outcome:
 def probe_window_management() -> Outcome:
     """Quartz lists windows without a grant; acting on one needs Accessibility.
 
-    Listing is what this measures, because it is the half that has to work
-    before any of the rest means anything — and because a runner with no
-    windows at all would make every window command untestable here.
+    What is asserted is that the *code* answers — the backend is selected, the
+    Quartz query runs, and every window it returns can be described. What is
+    only *reported* is how many there are, because that is a property of the
+    runner's session rather than of this project: a GitHub macOS runner was
+    measured to have no ordinary application windows at all. Gating on a
+    count would go red the day the runner image happens to open one.
     """
     import je_auto_control as ac
     from je_auto_control.wrapper.window_backends import get_backend
 
     backend = get_backend()
     if not backend.available:
         return Outcome(False, f"backend {backend.name!r} reports unavailable")
+    # Before the layer filter, so "the session is empty" and "the filter
+    # dropped everything" are told apart rather than guessed at.
+    raw = len(backend._window_info())
     windows = ac.list_windows()
-    if not windows:
-        return Outcome(False, f"backend {backend.name!r} listed no windows")
-    window_id, title = windows[0]
-    rect = backend.window_rect(window_id)
-    pid = backend.window_process_id(window_id)
-    return Outcome(rect is not None and pid > 0,
-                   f"{len(windows)} window(s); first {title!r} "
-                   f"rect={rect} pid={pid}")
+    described = []
+    for window_id, title in windows[:3]:
+        described.append((title, backend.window_rect(window_id),
+                          backend.window_process_id(window_id)))
+    complete = all(rect is not None and pid > 0
+                   for _title, rect, pid in described)
+    return Outcome(complete,
+                   f"{raw} on-screen window(s) from Quartz, {len(windows)} at "
+                   f"the application layer; described {described}")
 
 
 PROBES: List[Tuple[str, Callable[[], Outcome]]] = [