Skip to content

feat(cli): expose simulator accessibility trees - #88

Open
eksdev007 wants to merge 1 commit into
Mastersam07:devfrom
eksdev007:feat/engine-ui-tree
Open

eksdev007 wants to merge 1 commit into
Mastersam07:devfrom
eksdev007:feat/engine-ui-tree

Conversation

@eksdev007

Copy link
Copy Markdown

Summary

Adds odhub ui-tree <udid> to inspect the accessibility tree of the foreground iOS app on a booted simulator. The output includes element attributes and coordinates that can be passed directly to odhub tap.

This enables scripts and automation tools to discover controls by their labels or identifiers before interacting with them.

Usage

Print the complete tree with indentation:

odhub ui-tree <udid>

Show only elements with a label, title, identifier, or value:

odhub ui-tree <udid> --labelled

Export the complete tree as JSON, with frames in guest points:

odhub ui-tree <udid> --json

Text output includes normalized tap coordinates:

Button "Save" id=save @ 0.50,0.91

Tap that control with:

odhub tap <udid> --x 0.50 --y 0.91

Implementation

  • Adds accessibilityTree(_:) to SimulatorAdapter, returning a Codable element tree containing roles, role descriptions, labels, titles, identifiers, values, frames, and children.
  • Loads macOS’s AccessibilityPlatformTranslation framework dynamically and bridges its requests to the guest through CoreSimulator.
  • Reads the tree on a dedicated serial queue, keeping translator calls off the main thread and serializing access to its shared instance.
  • Checks private selectors before use and returns an error when a required selector is unavailable.
  • Bounds individual guest-request waits and caps traversal depth at 64.
  • Reads the guest’s display rotation and converts displayed element centers into the portrait-native coordinates expected by tap.
  • Omits tap coordinates for empty frames and elements whose centers are off screen.
  • Documents the new command in the README and removes the shell-window accessibility limitation.

Validation

  • Project build through scripts/build.sh passed.
  • Full test suite passed: 563 tests, zero failures.
  • Unit coverage checks coordinate normalization, all four orientations, empty and off-screen frames, JSON round trips, framework loading, and selector availability.
  • Integration tests verify reading the foreground app, calling the API from the main thread, and rejecting shutdown devices.
  • Live CLI checks in Safari confirmed that coordinates emitted in both landscape orientations hit the expected controls.

The simulator remained booted during validation.

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant