fix(gestures): reject off-screen finger positions instead of dispatching them - #671
Open
filip131311 wants to merge 1 commit into
Open
fix(gestures): reject off-screen finger positions instead of dispatching them#671filip131311 wants to merge 1 commit into
filip131311 wants to merge 1 commit into
Conversation
…ing them
gesture-pinch computes two finger positions from a center, a distance and an
angle, then dispatches them with no bounds check. centerY 0.35 with
startDistance 0.75 at 90 degrees puts a finger at y = -0.025; Android reads the
off-screen touch as a status-bar pull, opens the notification shade, and the
tool returns { pinched: true }. gesture-rotate has the same shape and the same
gap — verified at y = -0.15.
Both now build every frame first, check it, and only then dispatch, so a
rejected call sends nothing. Rejecting after the first Down would leave a
synthetic finger held on the glass with no matching Up, which gesture-rotate's
abort path already guards against.
The check runs over the frames that will actually be sent rather than the
endpoints. A pinch is affine in time so its endpoints would bound it, but an arc
is not: 0 to 180 degrees starts and ends on the centre line while passing a full
radius away at 90. Checking swept frames also keeps a short arc near an edge
valid, which a whole-circle check would refuse.
Bounds are inclusive: a full-width gesture legitimately lands exactly on 0 or 1,
and the arithmetic that gets there drifts a few ulps, so the comparison carries
a small epsilon.
Clamping was the issue's first suggestion and does not work. Clamping -0.025 to
0 produces a finger at exactly y = 0 — which I dispatched, and it opened the
notification shade just the same. Clamping would also change the effective
separation and move the zoom anchor off centerY, so the caller would get a
different gesture than it asked for and still be told it succeeded. The tool
descriptions now say that a finger merely NEAR an edge can still be taken by a
system gesture, since enforcing the inset the flow layer uses would refuse
deliberate edge gestures — that layer treats those zones as prefer-not, never
reject.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #628.
Reproduced first try
0.35 − 0.75/2 = −0.025. Android reads the off-screen touch as a status-bar pull, so the app never sees the pinch, and the tool reports success.gesture-rotatehas the identical gap (not in the issue) — verified live aty = −0.15, returning{rotated: true}. It didn't happen to open the shade on that path, which is what makes it worse to leave: the same invalid input is sometimes silently harmful.Clamping — the issue's first suggestion — does not work
I tested the clamped outcome directly, by picking parameters that land a finger exactly where clamping would put it:
A finger at exactly y = 0.0 — perfectly in bounds — still trips the system gesture. So clamping
−0.025 → 0.0converts an off-screen touch into an in-bounds touch with the same observable failure, while still reporting success. It would also change the effective separation (0.75 → 0.725) and shift the zoom anchor offcenterY, so the caller silently gets a different gesture than it asked for.The repo already knows
[0,1]isn't safe:flow-pinch-geometry.ts:17definesSCREEN_EDGE_INSET = 0.02and:37-40encodes system-gesture zones (top: 0.08, from a Pixel measurement).So: reject, and additionally say in both tool descriptions that a finger merely near an edge can still be taken by a system gesture. I did not promote those inset constants into the tool layer — the flow layer that owns them deliberately treats them as "prefer-not-reject no-start zones" and demotes in ranking rather than rejecting, and hard-rejecting them here would refuse deliberate edge gestures.
Implementation
Both tools build every frame, validate, and only then dispatch — a rejected call sends zero touch events. Rejecting after the first
Downwould strand a synthetic finger with no matchingUp, whichgesture-rotate's abort path already guards against andgesture-pinchhas no protection for.The check runs over the frames that will actually be sent, not the endpoints. A pinch is affine in
tso endpoints would bound it — but an arc is not:0°→180°starts and ends on the centre line while passing a full radius away at 90°. There's a test for exactly that. Checking swept frames also keeps a legitimate short arc near an edge valid, which a "whole circle must fit" check would refuse (also tested).Bounds are inclusive with a small epsilon: a full-width gesture legitimately produces exactly
0or1, and a real rotate frame already yields0.09999999999999998.Live output:
Uses
InvalidToolInputError, which surfaces the bare message at HTTP 400. A zod.refine()would have been JSON-serialized into zod's issue array, burying the coordinate — and refinements are dropped from the advertised JSON Schema anyway, asgesture-rotate/index.ts:65-67already notes.Scope
Only the two tools that derive positions the caller can't see.
gesture-tap/swipe/customtake coordinates directly — there the value is the caller's stated intent, nothing is computed and hidden.Worth a follow-up, noted while checking:
gesture-tap's Chromium path clamps to the viewport while its native path doesn't. Andrun-sequencecatches a step throw and returns HTTP 200 with partial results, so a rejected gesture inside a sequence stops the remainder — the message is written to read standalone.Verified live on a Pixel_9
{pinched: true}, gesture reached the appChecks