perf: faster page reads and waits (ideas from jev-ultrafast)

[karngyan]

Three browser-side techniques from browser-use/jev-ultrafast (MIT) that should make plain reins faster and more reliable. No API key or model involved: this is purely how we read the page and wait on it. See their snapshot.js and browser.py.

Their measured result on one Google Flights task: median browser protocol calls went from 1,092 to 101 and task time dropped 25%. That's their loop, not ours, so we need our own numbers.

Already fine in reins, so not part of this issue: the debugger session stays attached between commands (IDLE_DETACH_MS, packages/extension/src/lib/cdp.ts:89), and snapshot is already a single Runtime.evaluate.

1. Stable element identity in the snapshot (WeakMap instead of data-reins-ref)

Today (SNAPSHOT_EXPR, cdp.ts:278): every snapshot renumbers from e1 and writes data-reins-ref attributes into the page DOM. Refs are then resolved by CSS selector.

Problems:

• The same element gets a different ref in each snapshot, so an agent can't carry refs across snapshots.
• A stale ref can silently match a different element that picked up the same number in a later snapshot.
• We mutate the page DOM (site MutationObservers see it, and it's visible in the page's markup).
• Visibility uses offsetParent === null, which also drops every position: fixed element: modals, sticky headers, cookie banners.
• The output has no field values, checked, selected or expanded state, so the agent needs extra calls to learn them.

Proposal: keep a page-side cache: WeakMap<Element, id>, plus Map<id, WeakRef/Element> for lookup.

• Same node gives the same ref across snapshots.
• A ref whose node is gone (!isConnected) fails loudly with "ref is stale, snapshot again" and never matches a stand-in.
• No DOM writes.
• Use checkVisibility({checkOpacity, checkVisibilityCSS}) plus viewport/rect checks for visibility.
• Include value, checked, selected and expanded state in the snapshot.

Note: a node that React replaces is still a new node and gets a new ref. The win is stable numbering, no false matches, and no DOM mutation. It won't carry a ref over to a replacement node.

Open question: host the cache in an isolated world (Page.createIsolatedWorld) rather than a page global like their window.__jevFast, so pages can't see or tamper with it.

2. Frame-driven waits instead of fixed 100 ms polls

Today, several loops poll on fixed timers, costing up to 100 ms of dead time per iteration:

• actionability retry loop: setTimeout(r, 100) (actionability.ts:143)
• wait_for: 100 ms poll (cdp.ts:521)
• ensureVisible: 50 ms × 20 (cdp.ts:335)

Proposal: run the check inside the page as a single Runtime.evaluate with awaitPromise: true, re-checking on requestAnimationFrame (and/or a MutationObserver), and resolve as soon as it passes. Keep the existing timeout as the cap, and keep a timer fallback for when rAF doesn't fire.

Also consider a post-action settle like jev's: after a click or type, wait at most 2 animation frames or 50 ms. For combobox input, wait up to 200 ms for visible role="option" suggestions. Then the next snapshot sees the result instead of the half-rendered state.

3. Focus emulation so background tabs keep rendering (spike)

Today ensureVisible (cdp.ts:325) brings a hidden tab to the front with chrome.tabs.update({active: true}), because Chromium holds CDP input for hidden tabs. That steals the user's view whenever an agent acts on a background tab.

jev calls Emulation.setFocusEmulationEnabled({enabled: true}) on its tab "to keep rAF/menus rendering in an owned background tab, without activating the user's Chrome tab."

Unverified for us: jev creates its tab with Target.createTarget({background: true}) through browser-harness. Test before building:

• Does this work through chrome.debugger from an extension?
• Does it make trusted Input.dispatchMouseEvent / key events land in a hidden tab, or does it only un-throttle rAF?
• Does document.visibilityState change?
• Side effects: the page will believe it has focus (document.hasFocus(), focus/blur handlers). The emulation should reset when the debugger detaches; confirm that.

If input still requires visibility, this may still let snapshot/text/waits run on background tabs without throttling, and we keep ensureVisible for input only.

Done when

• Benchmark first: time snapshot → click → snapshot on 2–3 real SPAs (e.g. Google Flights, GitHub, a React form) and count CDP calls, before and after
• 1: stable refs, stale-ref errors, fixed-position elements included, form state in output; pointer.browser.test.ts-style browser tests for moved, replaced, hidden and covered elements
• 2: fixed polls replaced; post-action settle
• 3: spike results written up here; ship only if input or rendering works without activating the tab