fix(describe): flag an Android read that saw only system chrome - #672
Open
filip131311 wants to merge 1 commit into
Open
fix(describe): flag an Android read that saw only system chrome#672filip131311 wants to merge 1 commit into
filip131311 wants to merge 1 commit into
Conversation
With the display off, or a lock screen up, describe returns a bare ROOT Screen (0,0,1,1) — every node on screen belongs to com.android.systemui and prunes away. Nothing marks it, so an empty screen and an unreadable one are indistinguishable, and await-ui-element's isBlindRead has no flag to key on: its only Android guard is everMatched, which is false if the wait starts after the screen is already dark. Measured on a device: the same element, unmoved, reported visible with the screen on and hidden with it off, both succeeding in 5ms. An agent gating an action on 'the dialog is gone' got a green light against a screen nobody could read. The signal is two counts the helper already returns and describe threw away, so this costs no extra round trip and no probe: nodeCount is what the accessibility layer handed over, and the rendered child count is what survived pruning. Everything pruned means the only window was system chrome. Measured across four states — app 62 nodes/2 children, launcher 64/7, screen off 28/0, keyguard 73/0 — the rule separates them exactly. windowCount is deliberately NOT the signal: it stays 1 with the display off, because the keyguard window is still there, so a check for zero windows would never fire. That was worth measuring rather than assuming. Unknown or zero nodeCount fails open. An older helper that reports no count must not turn every sparse screen into a blind read — that would be worse than the bug, and the existing 'hidden succeeds when the node is gone' test covers exactly that shape. hint rather than a throw: it already flows into isBlindRead, which stops hidden resolving, and into the timeout note, so no consumer changes. Six callers of describeAndroid keep working, and describe on a sleeping device stays more useful as an empty tree plus 'wake it' than as an error. Note this makes a previously instant hidden wait run to its timeout and fail, which halts a run-sequence — that is the point, but it is a behaviour change for any sequence that depended on the false pass.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #627.
The false pass, measured
With the display off,
describereturns a bareROOT Screen (0,0,1,1)— no error, no hint. The same element, never moved:await-ui-element visible "Go to home screen"{"success": true, "elapsed": 5}await-ui-element hidden "Go to home screen"{"success": true, "elapsed": 5}An agent gating an action on "the dialog is gone" gets a green light against a screen nobody can read.
await-ui-element'sisBlindReadhas no flag to key on here — its only Android guard iseverMatched, which is false when the wait starts after the screen is already dark. The code comment said as much: "Android / Chromium never set these flags."The signal was already on the wire
getHierarchy()returnsnodeCountalongsidexml, anddescribedestructured onlyxml.nodeCount > 0with zero rendered children means every node on screen belonged tocom.android.systemuiand pruned away — which is exactly what a powered-off display and a lock screen look like.Measured across four states on a Pixel_9 / API 36:
So: no adb probe, no new module, no extra round trip, no
dumpsysparsing to go stale across OEMs, and nothing added to the hot path.windowCountis deliberately not the signal. The obvious guess is "no windows ⇒ blind", but it stays 1 with the display off, because the keyguard window is still there — it would never fire. Worth measuring rather than assuming.I verified the keyguard case separately (set a PIN and disabled the screen timeout so the device couldn't sleep mid-test, since an earlier attempt was unattributable):
mWakefulness=Awake,SCREEN_STATE_ON,mIsShowing=true, lock screen visibly on the display — and the tree still comes back empty. A display-power-only check would have missed it entirely.Fails open
Unknown or zero
nodeCount⇒ no hint. An older device helper that reports no count must not turn every sparse screen into a blind read — that is worse than the bug. This is not hypothetical: the existinghiddensucceeds when the node is gone test uses a stub with nonodeCount, and my first draft flagged it, becauseundefined <= 0isfalse. It now passes unmodified.hint, not a throwhintalready flows into bothisBlindRead(sohiddencan no longer resolve) and the timeout note — so the false pass closes with zero consumer changes, and the six callers ofdescribeAndroidkeep working. On a sleeping device an empty tree plus "wake it" is more useful than a hard error.Live, after the fix:
Screen on: no hint, full tree, unchanged.
Behaviour change to know about
A
hiddenwait that previously returned instantly now runs to its timeout and fails, and an unmet wait halts arun-sequence. That is the intent, but it will change any recorded sequence that (wrongly) depended on the instant pass.Interaction with #591
#591 (draft) touches the same two files. No textual collision: its
android/index.tshunk is@@ -52,7 +52,34 @@and mine is the destructure at:54plus a new helper above the TV hint — the one adjacent line resolves trivially either merge order. Inawait-ui-element/index.tsits hunks are at 187-195 / 217-223 / 354-360; my only edit is the now-false comment at:151-165, which had to be corrected since it documented precisely the gap this closes. The newawait-ui-elementtest is a separate file because #591 also extendstest/await-ui-element.test.ts.#519 is the iOS half of this class; it throws inside
flow-ios-tree.ts, whose contract is "helpers throw". The true Android mirror isflows/flow-android-tree.ts, which does not route throughdescribeAndroid— so flowassert hiddenstill false-passes there. Left as a follow-up so it can be reviewed next to #519.Checks
await-ui-element/describe-tv/await-screen-idletests pass unmodified.