test: Upgrade Playwright to 1.63 and cover soft navigation web vitals - #24273
Draft
logaretm wants to merge 7 commits into
Draft
test: Upgrade Playwright to 1.63 and cover soft navigation web vitals#24273logaretm wants to merge 7 commits into
logaretm wants to merge 7 commits into
Conversation
Contributor
size-limit report 📦
|
logaretm
force-pushed
the
awad/soft-nav-web-vitals-e2e
branch
from
September 9, 2026 22:59
6b626d2 to
6bf5302
Compare
logaretm
force-pushed
the
awad/soft-nav-web-vitals-e2e
branch
from
September 9, 2026 23:18
6bf5302 to
edec2c5
Compare
logaretm
added this pull request to stack #24274
September 9, 2026 23:18
…itals
Playwright 1.56 bundles Chromium 141, which has no Soft Navigations API, so
`supportsSoftNavigations()` is false and the soft navigation web vitals path never
runs under test. 1.63 bundles Chromium 153, which supports it unflagged.
Adds an e2e test asserting that a soft navigation's web vitals are attributed to the
navigation span they were measured on, through `reactRouterV6BrowserTracingIntegration`,
which starts its navigation span from a layout effect rather than from the history change.
The INP test needed a longer wait on the newer Chromium. web-vitals defers processing an
interaction's event entries to `requestIdleCallback(..., { timeout: 1000 })`; Chromium 141
ran that well inside the test's 500ms wait, Chromium 153 only runs it on the timeout. Hiding
the page before then forced a report while the metric was still unset, so no INP was emitted.
…e hiding the page
web-vitals defers processing an interaction's event entries into
`requestIdleCallback(..., { timeout: 1000 })`. Chromium 141 ran that well before the tests
hid the page; Chromium 153 only runs it once the timeout elapses, so hiding first forced a
report while the metric was still unset and no vital was emitted at all.
`hidePage` now waits for an idle callback of its own first. Idle callbacks run in scheduling
order, so one queued at hide time running means web-vitals' earlier callback has too.
CLS lands on the pageload span when it ends on the idle timeout, so nothing has to force the page away to finalize it. The reload raced the envelope instead: on Chromium 153 the page reloads before the in-flight send leaves, so no transaction was ever received and the test timed out. It also looped, since the reloaded page re-ran the same subject.
web-vitals finalizes LCP on the first trusted input or visibility change and checks `isTrusted`, so the synthetic `visibilitychange` this test dispatched never finalized it. It passed anyway while `reportAllChanges` was on, since every LCP entry was reported as it arrived. Soft navigation reporting turns that off, so on a browser that supports the Soft Navigations API no LCP span was emitted at all. Clicking finalizes it the way a real user does.
logaretm
force-pushed
the
awad/soft-nav-web-vitals-e2e
branch
from
September 10, 2026 03:32
edec2c5 to
55c4475
Compare
`page.route` resolves with a `Disposable` as of Playwright 1.63, so returning it from a
`Promise<void>` helper no longer type-checks.
The INP tests in react-17, react-17-static and the react-router 7/8 SPA apps hid the page
500ms after the interaction, which is the same race already fixed in react-router-6:
web-vitals defers entry processing to `requestIdleCallback(..., { timeout: 1000 })` and
Chromium 153 only runs it on the timeout, so the forced report found an unset metric.
solid-tanstack-router's LCP test dispatched a synthetic `visibilitychange`, which web-vitals
ignores for finalization because it checks `isTrusted`. It passed only while
`reportAllChanges` was on. It now finalizes with a real click, as in the browser integration
suite.
Playwright 1.63 ships a WebKit that reports `deliveryType`, so the attribute is no longer Chromium-only and the expectation moves out of the non-WebKit branch. That WebKit also resolves the intercepted routes without measurable delay, so the TTFB range check keeps its upper bound (which is what verifies we report seconds, not milliseconds) but only holds the other engines to a non-zero value.
Two assumptions in the bfcache app stopped holding on newer Chromium, and both traced back to `unload`. An unload listener still makes a page ineligible, but Chrome no longer hands out the `unload-listener` reason for it, only the privacy-masked one. The test waited for both, so it hung on a reason that never arrives. It now asserts the masked reason, which is what the browser actually reports and which older versions reported too. The child-frame test used an unload listener inside the iframe purely as a blocker, so it lost its reason as well. It blocks with a held-up IndexedDB version upgrade instead, mirroring the top-level botcher, which keeps the child-frame classification covered.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Upgrades Playwright to 1.63 so the soft navigation web vitals path is testable at all, and adds an e2e test for it.
Three existing tests needed fixing for the newer browser:
requestIdleCallback(..., { timeout: 1000 }), which Chromium 153 only runs on the timeout. Hiding the page after 500ms reported an unset metric.hidePagenow waits for its own idle callback first.visibilitychangenever did. It passed purely becausereportAllChangesreported every entry; soft nav reporting turns that off. Now finalized with a real click.