Expand manual verification across web and Apple apps

skills/manual-verify/SKILL.md

@@ -1,31 +1,87 @@
 ---
 name: manual-verify
-description: Verify changes through real user workflows when hands-on testing would add confidence, including web apps in a browser and iOS apps in Simulator.
+description: Verify changes through real user workflows when hands-on testing would add confidence, including web apps in a browser and native iOS or macOS apps.
 ---
 
 # Manual Verify
 
-Think like someone who uses this product: who are they, what are they trying to
-get done, and how could these changes affect them?
+Exercise the product through the interface its users operate. Judge whether the
+workflow works and remains usable; a successful command or build is supporting
+evidence, not the verdict.
 
-1. **Choose workflows.** Read the changes and identify the affected audience
-   and tasks. Use judgment to pick realistic workflows, including nearby behavior
-   that could break. Cover relevant error or edge cases without turning every
-   change into a full product audit.
+## Workflow
 
-2. **Use the product.** Use a browser for web apps and iOS Simulator for iOS apps.
-   For other tools, use the interface their users would use. Carry out the chosen
-   workflows and compare what happens with what the user needs. Check that the
-   result makes sense and is usable, beyond whether the action succeeds.
+1. **Choose workflows.** Read the request and changes. Identify the affected
+   users, their goal, and nearby behavior the change could break. Select the
+   smallest useful set of realistic workflows covering the main path and any
+   material error or edge case. State the expected result of each workflow.
 
-   If login or another step needs the user, tell them exactly what to do and
-   where, then resume once they finish. Continue any independent checks meanwhile.
-   If the environment cannot support a needed check, report the gap.
+2. **Choose the interface.** Use the product surface its users use. For a web
+   app, read [web verification](references/web.md). For an iOS app, read
+   [iOS verification](references/ios.md). For a native Mac app, read
+   [macOS verification](references/macos.md). Use the same observe-act-observe
+   loop for other interactive products with the best available interface tooling.
 
-3. **Report what happened.** Briefly state the workflows tested, expected and
-   observed results, and any failures or gaps. Include screenshots or other
-   evidence when useful. Base conclusions on what you exercised; distinguish
-   untested behavior from passing checks.
+3. **Observe, act, observe.** Start from a known state and wait for the interface
+   to settle. Identify the environment and external effects before exercising a
+   consequential workflow. Use disposable accounts, fixtures, and provider test
+   modes when available. Complete an irreversible or externally visible action
+   only when the task authorizes it; otherwise stop at the last safe step and
+   report the remaining gap.
 
-   After UI work, use `showroom` to package useful visual checkpoints. Keep
-   workflow selection and the pass or fail judgment here.
+   Inspect both the rendered appearance and semantic representation when
+   available. Perform one meaningful interaction, then inspect the resulting
+   state before continuing. Prefer semantic targets such as roles, labels, and
+   identifiers; use coordinates only when the interface exposes no stable target.
+
+   If an interaction produces no expected change, record that outcome before any
+   retry. Retry once only when observable evidence shows that the target moved or
+   the interface had not settled, and count the retry as a separate attempt.
+   Treat a repeated failure as evidence instead of retrying until it disappears.
+
+   When the reported behavior is intermittent, choose a bounded attempt count
+   before testing. Restore an equivalent known state with disposable or uniquely
+   identified data before every planned attempt so earlier outcomes cannot affect
+   later ones. Record every outcome. Recovery retries are separate from the
+   planned attempt count and must be recorded separately. Report `reproduced` or
+   `not reproduced in N attempts`; do not turn one successful attempt into a pass
+   for the intermittent report.
+
+4. **Judge the result.** Compare the observed result with the workflow's expected
+   result. Check function and usability, including relevant layout, readability,
+   focus, input, navigation, loading, empty, disabled, and error states. Distinguish:
+
+   - functional failures, such as the wrong destination or an unresponsive action;
+   - visual failures, such as clipped, overlapping, unreadable, or misplaced content;
+   - crashes and unexpected exits;
+   - transient states that settle correctly; and
+   - expected states, such as a disabled submit action for incomplete input.
+
+   When the expected result includes persisted data or an output artifact, verify
+   it through a fresh read path rather than relying on the current screen alone.
+
+5. **Report evidence.** For each workflow, state what was exercised and the
+   expected and observed result. Give each required acceptance assertion a status:
+
+   - `passed` when the observed result matched the expected assertion;
+   - `failed` when the observed result differed from the expected assertion;
+   - `blocked` when a missing prerequisite prevented the check; or
+   - `untested` when the selected check was not exercised.
+
+   Report defect reproduction separately as `reproduced` or
+   `not reproduced in N attempts`. A reproduced defect makes its affected
+   assertion and the overall workflow `failed`. When the defect is not reproduced,
+   report that outcome beside the acceptance status and state that it does not
+   establish that the defect is fixed.
+
+   A workflow passes when every required assertion passed. Otherwise report every
+   non-passing assertion and use `failed` when any assertion failed, then
+   `blocked`, then `untested` as the overall status. Include screenshots, logs, or
+   semantic snapshots when they explain the conclusion. Name any untested
+   behavior or environmental gap rather than treating it as passing.
+
+If login or another step requires the user, tell them exactly what to do and
+where, then resume after they finish. Continue independent checks meanwhile.
+
+After UI work, use `showroom` to package useful visual checkpoints. Keep workflow
+selection and the pass or fail judgment in this skill.

skills/manual-verify/references/ios.md

@@ -0,0 +1,50 @@
+# iOS Verification
+
+Use a Simulator that supports the app's deployment target. Identify the project
+or workspace, scheme, build configuration, Simulator model, and runtime used for
+the check. Prefer XcodeBuildMCP when it is available; otherwise use the host's
+Simulator and Xcode tooling.
+
+## Build and launch
+
+- Build and run the intended scheme. If the build fails, preserve the failure and
+  determine whether it comes from the product or verification environment. Report
+  it as the workflow result. Fix code only when the active task includes fixing
+  failures, then restart verification against the changed revision.
+- After launch, confirm that the expected app and screen are visible. Capture a
+  screenshot and inspect the accessibility hierarchy when the available tooling
+  exposes it.
+- Preserve the workflow's data assumptions. Reset, reinstall, seed, or relaunch
+  only when the required starting state calls for it, and report material setup.
+
+## Observe and interact
+
+- Target controls by accessibility identifier or label when supported. When the
+  available tooling exposes only rendered state, interact from the current view,
+  recapture after every action, and report semantic or accessibility coverage as
+  untested. Derive coordinate targets from current hierarchy or bounds when they
+  are available rather than from a stale screenshot.
+- Recapture the hierarchy after navigation, presentation, rotation, animation, or
+  any interaction that may move the target.
+- Exercise text input, gestures, scrolling, orientation, permissions, and
+  background or foreground transitions when they belong to the selected workflow.
+- When a save or update should persist, verify it through a fresh read path by
+  relaunching and refetching, inspecting the stored value through an independent
+  interface, or reopening through a path known to create a new model and reload
+  durable storage. Record the persisted value observed.
+- Check both function and presentation: hit targets, clipping, Dynamic Type where
+  relevant, safe areas, keyboard avoidance, and expected system dialogs.
+- Capture application logs when the app crashes, exits, hangs, or behaves
+  differently from the visible state. Use the process and bundle identifier to
+  separate app output from unrelated Simulator noise.
+
+Keep an explicitly requested Simulator check in Simulator. Exercise the portions
+the Simulator supports and name the exact device-only step that remains unverified.
+Use a physical device when the request includes device verification and suitable
+hardware is available.
+
+## Evidence
+
+Record the scheme, Simulator model and runtime, workflow states, and relevant
+screenshots or logs. A successful build proves that the app compiled; the
+interaction and resulting state prove whether the workflow worked.

skills/manual-verify/references/macos.md

@@ -0,0 +1,49 @@
+# macOS Verification
+
+Build and launch the intended app configuration. Record the macOS version, app
+build or revision, and any fixture or account state that affects the check. Use
+the host's native app or computer-use tooling, with accessibility inspection when
+it is available.
+
+## Observe and interact
+
+- Confirm that the expected app and window are active before interacting. Inspect
+  the rendered interface and accessibility hierarchy when the tooling exposes it.
+- Target controls by accessibility role, label, or identifier when supported.
+  With rendered-state tooling, recapture after each action and report semantic or
+  accessibility coverage as untested.
+- Exercise menus, keyboard shortcuts, focus, window resizing, sheets, popovers,
+  multiple windows, and full-screen behavior when they belong to the selected
+  workflow. Give multiple windows distinct observable states, confirm which one
+  is active, then inspect both the active window's expected change and the
+  inactive window's unchanged state.
+- For open, save, import, export, or drag-and-drop workflows, use disposable
+  copies while preserving the source fixtures. Record the paths and expected
+  changes. To verify an explicit save action, capture an independent on-disk
+  baseline and the document's dirty state immediately before invoking it, then
+  observe the command-specific state transition and inspect the on-disk result.
+  If autosave removes the dirty precondition before the command or makes
+  attribution impossible, report persistence separately and mark the save-action
+  assertion `blocked`. Mark it `untested` when the command was not exercised. If
+  the document was dirty, the command was exercised, and neither the expected
+  transition nor persisted output appears after settling, mark it `failed`.
+  Closing and reopening the recorded path can provide additional persistence
+  evidence. Record the persisted value observed.
+- Treat permission prompts and other system UI as part of the workflow. Record
+  any permission state that a later run would need to reproduce. A controlled
+  denial requested by the workflow is a test precondition: keep it denied unless
+  the task authorizes changing it, then check the app's error or fallback behavior
+  and confirm that it did not produce restricted output.
+- Capture application logs or crash reports when the app exits, hangs, or behaves
+  differently from the visible state.
+
+When the verifier cannot establish or observe a required state because hardware,
+services, entitlements, permissions, or system configuration are unavailable,
+exercise the supported portion and mark only the affected assertions blocked.
+
+## Evidence
+
+Record the macOS version, app build or revision, workflow states, affected and
+unaffected windows, and relevant screenshots, inspected output files, or logs. A
+successful build proves that the app compiled; the interaction and resulting
+state prove whether the workflow worked.

skills/manual-verify/references/web.md

@@ -0,0 +1,26 @@
+# Web Verification
+
+Use the browser environment already attached to the task when possible. Start or
+locate the development server, open the exact changed route, and record the
+viewport or device mode when it affects the result.
+
+## Observe and interact
+
+- Capture a browser snapshot or inspect the DOM and accessibility tree before
+  interacting. Pair semantic inspection with a screenshot when appearance matters.
+- Target elements by role, accessible name, label, or another stable locator.
+  Reinspect the page after navigation, re-rendering, or layout changes before
+  reusing a target.
+- Exercise pointer, keyboard, focus, scrolling, and responsive behavior when the
+  change affects them. Test only the viewports and input methods relevant to the
+  requested workflow and likely regression risk.
+- Wait for visible loading to settle. If an action produces no clear result,
+  inspect the console and failed network requests before deciding whether the
+  product or the environment failed.
+
+## Evidence
+
+Capture the states that establish the result: the initial condition, the outcome,
+and any failure. Record the route, viewport, and relevant console or network
+evidence. Do not infer success from a screenshot when behavior required an
+interaction.