Skip to content

Block GA measurement in Chromium's network stack - #37

Merged
mauanga merged 3 commits into
mainfrom
block-known-ga-hosts
Sep 11, 2026
Merged

mauanga merged 3 commits into
mainfrom
block-known-ga-hosts

Conversation

@mauanga

@mauanga mauanga commented Sep 10, 2026

Copy link
Copy Markdown
Contributor

Why

Route interception can be raced. Playwright stops answering routes the moment the
page closes, and Chromium then releases every still-undecided request to the
network, so measurement hits could escape a policy that was active.

What changed

  • Chromium applies the blocked list with Network.setBlockedURLs, inside its own
    network stack, where there is no callback to race. Other engines keep the route
    fallback.
  • The policy is applied on the context's page event, so it covers every page the
    context creates, including ones the site opens itself.
  • Host scope is unchanged: every path on google-analytics.com and
    analytics.google.com, and only /{batch,g,j,mp,r}/collect on google.com and
    stats.g.doubleclick.net. Advertising endpoints (/ccm/collect,
    /rmkt/collect/…) stay reachable.
  • Tests now match real URLs against the generated patterns, covering both
    over-blocking and under-blocking.

@mauanga
mauanga force-pushed the block-known-ga-hosts branch 3 times, most recently from fd16591 to 6bb0d26 Compare September 11, 2026 10:08
@mauanga mauanga changed the title fix(render): block GA measurement endpoints in Chromium's network stack Block GA measurement in Chromium's network stack Sep 11, 2026
@mauanga
mauanga marked this pull request as ready for review September 11, 2026 10:15
@mauanga
mauanga requested a review from rpanfili September 11, 2026 10:16
equivalent, so they fall back to the route and inherit that weakness.
"""
patterns = build_blocked_url_patterns()
if self._engine_name() == "chromium":

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

__enter__ (line 65, unchanged by this PR) only ever calls self._playwright.chromium.launch(...) — there's no code path anywhere in the SDK that launches firefox/webkit. That makes _engine_name() always return "chromium" (or "unknown" on error), so this else branch — the non-Chromium page.route fallback — can never execute. This contradicts the PR description ("Other engines keep the route fallback") and the AGENTS.md/docs updates in this same PR. Either wire up real multi-engine support, or drop the dead branch and the docs claim about it.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I opted to keep this. Although it is dead-code at the moment, it cost little and is kept in case in the future we ever utilize a non-chromium browser. I've revised the PR to restore the regex approach we settled on previously for this branch, so it is no longer a regression.

Comment thread wordlift_sdk/render/browser.py Outdated
session.send("Network.enable")
session.send("Network.setBlockedURLs", {"urls": patterns})
return
for pattern in patterns:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Beyond being currently unreachable (see comment on line 65), this fallback would be weaker than intended if it ever ran: it reuses build_blocked_url_patterns() verbatim for page.route(), but Playwright's own glob dialect doesn't let a single * cross / the way CDP's Network.setBlockedURLs does (which the tests explicitly model via fnmatch). A URL like https://google.com/g/collect?redir=http://x/y would match under CDP semantics but fail to match Playwright's route glob, since the embedded / breaks the trailing *.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Resolved by splitting the pattern strategy for each branch. The glob dialect mismatch is no longer expressible.

Comment thread wordlift_sdk/render/network_policy.py Outdated

def build_blocked_url_patterns() -> list[str]:
return [
f"https://{subdomain}{host}/**"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Apex-host patterns (no subdomain wildcard) have nothing to absorb userinfo, so https://user:pass@google-analytics.com/g/collect matches no generated pattern and bypasses the block. It only happens to work for the www. case because the *. subdomain wildcard incidentally swallows user:pass@www too — that's accidental, not deliberate. The old regex handled this explicitly with (?:[^/?#@]*@)?; worth restoring equivalent coverage (or confirming Chromium/CDP never sees userinfo-bearing URLs in practice).

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed on both paths. The regex keeps (?:[^/?#@]*@)?, and the glob authorities gained a *@ form (*. already absorbs credentials on a subdomain, so only the apex needed it)

Comment thread wordlift_sdk/render/network_policy.py Outdated
for host in _MEASUREMENT_ONLY_HOSTS
for subdomain in _SUBDOMAIN_FORMS
] + [
f"https://{subdomain}{host}{path}*"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Two regressions vs. the old regex on this line:

  1. Port bypass: no pattern includes a port wildcard, so https://google-analytics.com:8443/g/collect (or the www. form) isn't blocked at all. The old _HOST_TAIL = r"\.?(?::\d+)?" covered this and it wasn't carried over.
  2. Missing path terminator: the trailing bare * has no boundary, so https://google.com/g/collectData123 would incorrectly match .../g/collect* — the old _TERMINATOR = r"(?:[/?#]|$)" prevented this. Low real-world risk since Google doesn't serve such paths today, but it's a real loosening on the _MIXED_HOSTS, which is exactly where over-blocking is the concern this module is trying to avoid.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've restored the old regex for non-Chromium route.abort() path.

@mauanga mauanga Sep 11, 2026

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Port is fixed on both paths - _HOST_TAIL on the regex, a :* authority form on the globs.

The terminator is fixed on the route path only (_TERMINATOR is back), and it isn't expressible in setBlockedURLs. That API matches a pattern anywhere in the URL, anchored to neither end, pattern collect blocks /g/collect, and pattern /g/collect blocks /g/collectData123. There's no syntax for "the path ends here". The one character that could serve as a boundary is ?, which is a literal rather than a wildcard (/g/collect?* doesn't match /g/collectX), but a pattern ending in ? would stop blocking requests that carry no query string.

We don't expect the residue to bite. It only applies to google.com and stats.g.doubleclick.net - the measurement-only hosts block every path by design - and it needs Google to serve a path that starts with /batch/collect, /g/collect, /j/collect, /mp/collect or /r/collect and then continues. That's Google's own measurement namespace, and no such path exists today.

Comment thread tests/test_render_browser.py Outdated
self.closed = False
self.kwargs = None
self.context = _FakeContext()
self.browser_type = type("_Type", (), {"name": "chromium"})()

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Every fake browser here hard-codes browser_type.name = "chromium", so the page.route fallback branch in _block_measurement_endpoints (browser.py:127-128) has zero test coverage — worth a case with a non-chromium engine name once/if that branch becomes reachable, so a future engine-name change can't silently ship the glob-dialect mismatch noted on that line.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added test_non_chromium_engines_use_the_route_fallback: fake engine name firefox, asserts the route is registered with the regex and no CDP call is made.

@mauanga
mauanga requested a review from rpanfili September 11, 2026 12:58
@mauanga
mauanga force-pushed the block-known-ga-hosts branch from e53aa15 to 7066d77 Compare September 11, 2026 13:41
@mauanga
mauanga merged commit 44e1acb into main Sep 11, 2026
12 checks passed
@mauanga
mauanga deleted the block-known-ga-hosts branch September 11, 2026 13:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants