Block GA measurement in Chromium's network stack - #37
Conversation
fd16591 to
6bb0d26
Compare
| equivalent, so they fall back to the route and inherit that weakness. | ||
| """ | ||
| patterns = build_blocked_url_patterns() | ||
| if self._engine_name() == "chromium": |
There was a problem hiding this comment.
__enter__ (line 65, unchanged by this PR) only ever calls self._playwright.chromium.launch(...) — there's no code path anywhere in the SDK that launches firefox/webkit. That makes _engine_name() always return "chromium" (or "unknown" on error), so this else branch — the non-Chromium page.route fallback — can never execute. This contradicts the PR description ("Other engines keep the route fallback") and the AGENTS.md/docs updates in this same PR. Either wire up real multi-engine support, or drop the dead branch and the docs claim about it.
There was a problem hiding this comment.
I opted to keep this. Although it is dead-code at the moment, it cost little and is kept in case in the future we ever utilize a non-chromium browser. I've revised the PR to restore the regex approach we settled on previously for this branch, so it is no longer a regression.
| session.send("Network.enable") | ||
| session.send("Network.setBlockedURLs", {"urls": patterns}) | ||
| return | ||
| for pattern in patterns: |
There was a problem hiding this comment.
Beyond being currently unreachable (see comment on line 65), this fallback would be weaker than intended if it ever ran: it reuses build_blocked_url_patterns() verbatim for page.route(), but Playwright's own glob dialect doesn't let a single * cross / the way CDP's Network.setBlockedURLs does (which the tests explicitly model via fnmatch). A URL like https://google.com/g/collect?redir=http://x/y would match under CDP semantics but fail to match Playwright's route glob, since the embedded / breaks the trailing *.
There was a problem hiding this comment.
Resolved by splitting the pattern strategy for each branch. The glob dialect mismatch is no longer expressible.
|
|
||
| def build_blocked_url_patterns() -> list[str]: | ||
| return [ | ||
| f"https://{subdomain}{host}/**" |
There was a problem hiding this comment.
Apex-host patterns (no subdomain wildcard) have nothing to absorb userinfo, so https://user:pass@google-analytics.com/g/collect matches no generated pattern and bypasses the block. It only happens to work for the www. case because the *. subdomain wildcard incidentally swallows user:pass@www too — that's accidental, not deliberate. The old regex handled this explicitly with (?:[^/?#@]*@)?; worth restoring equivalent coverage (or confirming Chromium/CDP never sees userinfo-bearing URLs in practice).
There was a problem hiding this comment.
Fixed on both paths. The regex keeps (?:[^/?#@]*@)?, and the glob authorities gained a *@ form (*. already absorbs credentials on a subdomain, so only the apex needed it)
| for host in _MEASUREMENT_ONLY_HOSTS | ||
| for subdomain in _SUBDOMAIN_FORMS | ||
| ] + [ | ||
| f"https://{subdomain}{host}{path}*" |
There was a problem hiding this comment.
Two regressions vs. the old regex on this line:
- Port bypass: no pattern includes a port wildcard, so
https://google-analytics.com:8443/g/collect(or thewww.form) isn't blocked at all. The old_HOST_TAIL = r"\.?(?::\d+)?"covered this and it wasn't carried over. - Missing path terminator: the trailing bare
*has no boundary, sohttps://google.com/g/collectData123would incorrectly match.../g/collect*— the old_TERMINATOR = r"(?:[/?#]|$)"prevented this. Low real-world risk since Google doesn't serve such paths today, but it's a real loosening on the_MIXED_HOSTS, which is exactly where over-blocking is the concern this module is trying to avoid.
There was a problem hiding this comment.
I've restored the old regex for non-Chromium route.abort() path.
There was a problem hiding this comment.
Port is fixed on both paths - _HOST_TAIL on the regex, a :* authority form on the globs.
The terminator is fixed on the route path only (_TERMINATOR is back), and it isn't expressible in setBlockedURLs. That API matches a pattern anywhere in the URL, anchored to neither end, pattern collect blocks /g/collect, and pattern /g/collect blocks /g/collectData123. There's no syntax for "the path ends here". The one character that could serve as a boundary is ?, which is a literal rather than a wildcard (/g/collect?* doesn't match /g/collectX), but a pattern ending in ? would stop blocking requests that carry no query string.
We don't expect the residue to bite. It only applies to google.com and stats.g.doubleclick.net - the measurement-only hosts block every path by design - and it needs Google to serve a path that starts with /batch/collect, /g/collect, /j/collect, /mp/collect or /r/collect and then continues. That's Google's own measurement namespace, and no such path exists today.
| self.closed = False | ||
| self.kwargs = None | ||
| self.context = _FakeContext() | ||
| self.browser_type = type("_Type", (), {"name": "chromium"})() |
There was a problem hiding this comment.
Every fake browser here hard-codes browser_type.name = "chromium", so the page.route fallback branch in _block_measurement_endpoints (browser.py:127-128) has zero test coverage — worth a case with a non-chromium engine name once/if that branch becomes reachable, so a future engine-name change can't silently ship the glob-dialect mismatch noted on that line.
There was a problem hiding this comment.
Added test_non_chromium_engines_use_the_route_fallback: fake engine name firefox, asserts the route is registered with the regex and no CDP call is made.
e53aa15 to
7066d77
Compare
Why
Route interception can be raced. Playwright stops answering routes the moment the
page closes, and Chromium then releases every still-undecided request to the
network, so measurement hits could escape a policy that was active.
What changed
Network.setBlockedURLs, inside its ownnetwork stack, where there is no callback to race. Other engines keep the route
fallback.
pageevent, so it covers every page thecontext creates, including ones the site opens itself.
google-analytics.comandanalytics.google.com, and only/{batch,g,j,mp,r}/collectongoogle.comandstats.g.doubleclick.net. Advertising endpoints (/ccm/collect,/rmkt/collect/…) stay reachable.over-blocking and under-blocking.