Skip to content

fix(flow): let a flow launch an app that can never be instrumented - #677

Open
filip131311 wants to merge 1 commit into
mainfrom
filip/flow-launch-non-injectable
Open

fix(flow): let a flow launch an app that can never be instrumented#677
filip131311 wants to merge 1 commit into
mainfrom
filip/flow-launch-non-injectable

Conversation

@filip131311

Copy link
Copy Markdown
Collaborator

Fixes #623.

Reproduced

[0] launch  error  could not connect to native devtools for com.apple.Preferences.
                   Re-run to relaunch the app and retry… restart the argent server…
[1] echo    skip
[2] tap     skip          ok=false errored=1 skipped=2

Control — the identical flow without the launch step passes (ok=true). So the runner was fine; the gate alone made system apps undriveable, and it gave advice that can never work for them.

The signal already existed: isInjectableBundleId is used by native-devtools-status (whose docs call injectable: false terminal — do NOT restart/retry) and by describe. The flow gate never consulted it, so it burned the full 8 s NATIVE_READY_TIMEOUT_MS waiting for a connection that cannot occur.

After:

[0] launch  pass   ⚠ com.apple.Preferences is an Apple system app… coordinate steps work;
                     selector-based steps cannot resolve for this app.
[1] echo    pass
[2] tap     pass          ok=true, ~3s total

Removing the wait exposes a race the wait was hiding — so this closes it too

This is the part I'd want a reviewer to look at hardest.

Selector steps reach resolveNativeTargetApp, which auto-targets whatever app is connected. chooseFrontmostConnectedApp has a weak tier accepting applicationState === "inactive" — exactly what an app reports while it backgrounds. Today the first tree read lands ~9.5 s after launch (8 s gate + 1.5 s settle), comfortably past that window. Without the wait it lands at ~1.5 s, inside it.

So a selector step could resolve against a different app's tree. Most conditions then fail noisily, but hidden-shaped ones (assert: { hidden: X }, await: { hidden: X }, a when: { hidden: X } guard reporting a green skip) would pass. The fix records that the launched app cannot be instrumented and fails tree reads immediately with a terminal reason, which closes it.

Without that guard the later path was also worse than the gate it replaced: NATIVE_TARGET_NO_CONNECTED_APPS says "Launch or restart the app first", after a 3 s settle.

[1] await  fail  `com.apple.Preferences` is an Apple system app, so … selector-based steps cannot
                 resolve. This is terminal — relaunching or restarting the argent server will not
                 change it. Target this screen by coordinate (`tap: { x, y }`) instead.

Design choices

Guard on tree reads, not step kindssettleTree + waitForCondition. That covers tap/long-press/await/assert/scroll-to/snapshot in two places, and leaves selector-less pinch/rotate working (they degrade to a default aspect and never read a tree). Guarding by step kind would have missed scroll-to, which calls settleTree directly.

No pre-flight "does this flow use selectors" scan. launch can appear anywhere, including inside a nested fragment, and run: fragments are read lazily at execution time — so "does this flow contain selectors" is ill-posed (which steps? after this launch? under a when: that may not be entered?).

Pass, not fail. The launch genuinely succeeded — the issue's own control proves the rest of such a flow runs. Failing it would re-introduce the bug for coordinate flows.

No 5th status. The CLI maps status with no fallback, so an unknown value renders undefined. Used the existing warning field instead: ⚠ replaces the pass glyph, the text prints under the step, and it's counted in the summary. Both renderers already handle it as legacy wire-compat, so an old CLI renders it correctly — those comments are updated, since it's now actually produced.

Also fixed

The create-flow skill's recording walkthrough used com.apple.Preferences with selector steps (tap: { text: General }). That could never have been captured (selector capture reads the same tree) or replayed. Switched to a third-party app.

Not fixed, deliberately

A fragment with no launch step, run while a system app is frontmost, never sets the flag and still gets the raw message. That's the issue's own control case.

Checks

  • 3088 tests pass; 2 new, both failing against the pre-fix source. All 542 existing flow tests pass unmodified, including flow-composition.test.ts:213, which pins the retry-worded failure for a genuinely injectable app.
  • CLI (277) and MCP (78) renderer suites pass.
  • One unrelated flake on a first full run that passed on re-run.
  • Skills gate 10.0; extract-tools 46/46; prettier, eslint, both typechecks clean; lock untouched.

A flow's launch step waits for native devtools on iOS, and an Apple system app
is a platform binary with library validation, so that connection can never
happen. The step burned the full 8s timeout, failed, and took the rest of the
flow with it — even when nothing in the flow needed the view hierarchy. The
advice it gave ("re-run", "restart the argent server") could never work, which
is the retry loop #453 set out to remove.

The signal was already there: isInjectableBundleId, used by
native-devtools-status and by describe, whose own docs call injectable:false a
terminal state. The gate never consulted it. It does now, and the launch passes
with a warning saying what does and does not work for such an app.

Removing the wait exposes something the wait was accidentally hiding, so the
same change has to close it. Selector steps reach resolveNativeTargetApp, which
auto-targets whatever app is connected and accepts one reporting "inactive" as
foreground-like. An app the user was driving reports exactly that while it
backgrounds. Today the first tree read lands ~9.5s after launch, past that
window; without the wait it lands at ~1.5s, inside it — so a selector could
resolve against a different app's tree, and a hidden assertion against it would
pass. The run now records that the launched app cannot be instrumented and
fails tree reads immediately with a terminal reason.

The guard sits on settleTree and waitForCondition rather than on step kinds, so
it covers tap, long-press, await, assert, scroll-to and snapshot at once, and
leaves selector-less pinch and rotate working — those degrade to a default
aspect and never read a tree.

Reporting a pass rather than a failure is deliberate: the launch did succeed,
and the control case in the issue proves the rest of such a flow runs fine. The
warning field was already rendered by both the CLI and MCP as legacy
wire-compat; it is now produced, so those comments no longer describe it as
something only an old server sends.

The recording walkthrough in the create-flow skill used com.apple.Preferences
with selector steps, which could never have been captured or replayed. It now
uses a third-party app.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

flow launch step requires native devtools, so a coordinate-only flow can never launch a non-injectable app (com.apple.*)

1 participant