fix(tap): Detect switch-like controls for coordinate taps - #67
fix(tap): Detect switch-like controls for coordinate taps#67BariBariGood wants to merge 2 commits into
Conversation
Coordinate taps (tap -x/-y) with the default automatic tap style always used the simulator tapAt event, which SwiftUI Toggle and UIKit UISwitch controls ignore. The tap printed a success message while the toggle never changed state; only --tap-style physical worked. Selector taps already resolve the target element and switch to physical touch for switch-like controls. Extend the same behavior to coordinate taps: when the style is automatic, look up the accessibility element at the requested point and use physical touch if it is a switch-like control. Explicit --tap-style simulator and --tap-style physical are unchanged, and if the point lookup fails the tap falls back to the simulator event as before. Batch coordinate taps get the same handling.
Point lookup only sees the accessibility role that the element exposes at the tapped coordinate. SwiftUI Toggle reports a switch role there, but UISwitch internals report as plain groups, so coordinate taps on a UISwitch still need --tap-style physical or a selector tap. Say so in the changelog entry and skill docs instead of implying both are covered.
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (7)
WalkthroughCoordinate taps now resolve through accessibility inspection when tap style is automatic. Switch-like elements use physical touch, while inspection failures use simulator tapping. Explicit tap styles bypass inspection. Accessibility data decoding accepts one object or a singleton array. Batch coordinate taps use the new resolver. Tests cover decoding and switch detection. CLI guidance and the changelog document the updated behaviour and UIKit 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Independent reproduction on AXe 1.8.0 with Xcode 26.6 (17F113) and an iOS 26.5 Simulator:
This also affects callers that pre-resolve semantic refs into coordinate tap steps (including batch callers). A post-action accessibility assertion was necessary to distinguish the no-op from actual delivery. No desktop capture was used. |
2 participants
Summary
tap -x/-y) with the defaultautomatictap style always used the simulatortapAtevent, which SwiftUITogglecontrols ignore — the tap printed "✓ Tap ... completed successfully" while the toggle never changed state; only--tap-style physicalworked.automaticstyle, coordinate taps now look up the accessibility element at the requested point and use physical touch when it reports a switch-like role. Explicit--tap-style simulatorand--tap-style physicalare unchanged, and if the point lookup fails the tap falls back to the simulator event as before. Batch coordinate taps get the same handling.Validation
swift testpasses (adds decoding tests for single-element point-lookup payloads).tap -x/-yat the SwiftUI toggle's centre reports success while the value stays Off; with this change the same tap flips it to On.--tap-style simulatorand--tap-style physicalbehave as before.Notes
UIKit
UISwitchinternals report as plain groups at the tapped point, so coordinate taps on aUISwitchstill need--tap-style physicalor a selector tap; this limitation is documented in the changelog entry and skill docs.