Block GA measurement in Chromium's network stack

AGENTS.md

@@ -242,10 +242,13 @@
 - Playwright ingestion is async-loop-safe in cloud workflows: when a caller thread
   already runs an asyncio event loop, rendering is offloaded away from that thread
   before Sync Playwright APIs are invoked.
-- All Playwright rendering contexts disable service workers and apply the mandatory
-  context-wide Google Analytics measurement policy before creating pages: every
-  path on the Analytics measurement hosts, plus `/{g,j,mp,r,batch}/collect` on
-  enumerated mixed-purpose Google hosts; Google Tag Manager, advertising and
+- All Playwright rendering contexts disable service workers, and every page applies
+  the mandatory Google Analytics measurement policy: every path on the Analytics
+  measurement hosts, and the measurement paths (`/{batch,g,j,mp,r}/collect`) on
+  `google.com` and `stats.g.doubleclick.net`. The same hosts and paths are rendered
+  as URL globs for Chromium's `Network.setBlockedURLs` and as a regex for the
+  Playwright route other engines fall back to; only Chromium is launched today, so
+  that route is not currently reached. Google Tag Manager, advertising and
   third-party hosts remain available.
 - Playwright ingestion default `wait_until` is `domcontentloaded` (override with
   `PLAYWRIGHT_WAIT_UNTIL`), and navigation timeout now falls back to partial DOM

CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## 8.4.6 - 2026-09-11
+
+### Fixed
+
+- Google Analytics measurement traffic is now blocked inside Chromium's network
+  stack with `Network.setBlockedURLs` instead of Playwright route interception,
+  which stopped answering once a page closed and let undecided requests through.
+  The policy applies to every page a context opens, and covers credentialed and
+  non-default-port URLs.
+
 ## 8.4.5 - 2026-09-09
 
 ### Fixed

docs/render.md

@@ -12,23 +12,39 @@ Renders a URL using `Browser` and converts HTML to XHTML with `HtmlConverter`.
 ### Browser
 Thin wrapper around Playwright that opens a page and returns the page, response, elapsed time, and resource list.
 
-Each browser context blocks service workers and installs a mandatory context-wide
-route that aborts Google Analytics measurement traffic. Two host groups are
-treated differently:
-
-- `*.google-analytics.com` and `*.analytics.google.com` exist only to collect,
-  so every path on them is blocked.
-- `*.google.com` and `*.stats.g.doubleclick.net` also serve traffic that must
-  stay reachable, so only the measurement paths are blocked there:
-  `/g/collect`, `/j/collect`, `/mp/collect`, `/r/collect` and `/batch/collect`.
-  The path rule is needed because the Google tag sends the same GA4 payload to
-  `www.google.com/g/collect` when the measurement hosts are unreachable.
+Each browser context blocks service workers, and every page blocks Google
+Analytics measurement traffic as it is created.
+
+The policy is defined once as hosts and measurement paths:
+
+- every path on `google-analytics.com` and `analytics.google.com`, which exist
+  only to collect measurement
+- the measurement paths `/batch/collect`, `/g/collect`, `/j/collect`,
+  `/mp/collect` and `/r/collect` on `google.com` and `stats.g.doubleclick.net`,
+  which also serve traffic that must stay reachable
+
+Each host is listed with and without a subdomain wildcard.
+
+`google.com` is on the list because the Google tag sends the same GA4 payload to
+`www.google.com/g/collect` when the measurement hosts are unreachable.
 
 Hosts are enumerated deliberately. A path rule applied to any host cannot be
 bounded, because third-party endpoint names are unpredictable, so third-party
 hosts are out of scope whatever they call their paths. Google Tag Manager and
 advertising conversion endpoints (`/ccm/collect`, `/rmkt/collect/<id>/`) remain
-available. Blocked URLs and payloads are not logged or added to the
+available.
+
+On Chromium it is rendered as URL globs and applied with
+`Network.setBlockedURLs`, which blocks inside the browser's own network stack.
+Other engines fall back to a Playwright route over the equivalent regex. That
+fallback is weaker: a route decision is a round trip out of the browser,
+Playwright stops answering as soon as the page is closed, and the browser then
+sends any request it has not been told to block. The regex is the more precise
+of the two -- it bounds the path and covers credentials, ports and `http://`,
+which the glob syntax cannot express. Only Chromium is launched today, so the
+route is not currently reached.
+
+Blocked URLs and payloads are not logged or added to the
 response-resource list. Customer-specific first-party or server-side tagging
 gateways that route measurement through their own path are outside this policy.
 Disabling service workers intentionally trades PWA offline caching and

pyproject.toml

@@ -1,6 +1,6 @@
 [tool.poetry]
 name = "wordlift-sdk"
-version = "8.4.5"
+version = "8.4.6"
 description = "Python toolkit for orchestrating WordLift imports and structured data workflows."
 authors = ["David Riccitelli <david@wordlift.io>"]
 readme = "README.md"

tests/test_render_browser.py

@@ -4,17 +4,25 @@
 
 import wordlift_sdk.render.browser as browser_module
 from wordlift_sdk.render.browser import Browser, BrowserOperationError
+from wordlift_sdk.render.network_policy import (
+    GOOGLE_ANALYTICS_URL_PATTERN,
+    build_blocked_url_patterns,
+)
 
 
 class _FakePage:
     def __init__(self, should_raise=False, error_message="boom"):
         self._handlers = {}
         self._should_raise = should_raise
         self._error_message = error_message
+        self.routes = []
 
     def on(self, name, handler):
         self._handlers[name] = handler
 
+    def route(self, pattern, handler):
+        self.routes.append((pattern, handler))
+
     def goto(self, url, wait_until, timeout):
         if self._should_raise:
             raise browser_module.PlaywrightError(self._error_message)
@@ -32,36 +40,49 @@ class _Req:
         return response
 
 
+class _FakeCdpSession:
+    def __init__(self):
+        self.sent = []
+
+    def send(self, method, params=None):
+        self.sent.append((method, params))
+
+
 class _FakeContext:
     def __init__(self):
         self.closed = False
         self.script = None
         self.page = _FakePage()
-        self.route_matcher = None
-        self.route_handler = None
+        self.cdp = _FakeCdpSession()
         self.events = []
+        self._page_handlers = []
+
+    def on(self, event, handler):
+        if event == "page":
+            self._page_handlers.append(handler)
 
-    def route(self, matcher, handler):
-        self.route_matcher = matcher
-        self.route_handler = handler
-        self.events.append("route")
+    def new_cdp_session(self, page):
+        return self.cdp
 
     def add_init_script(self, script):
         self.script = script
 
     def new_page(self):
         self.events.append("new_page")
+        for handler in self._page_handlers:
+            handler(self.page)
         return self.page
 
     def close(self):
         self.closed = True
 
 
 class _FakeBrowser:
-    def __init__(self):
+    def __init__(self, engine="chromium"):
         self.closed = False
         self.kwargs = None
         self.context = _FakeContext()
+        self.browser_type = type("_Type", (), {"name": engine})()
 
     def new_context(self, **kwargs):
         self.kwargs = kwargs
@@ -72,9 +93,9 @@ def close(self):
 
 
 class _FakePlaywright:
-    def __init__(self):
+    def __init__(self, engine="chromium"):
         self.chromium = self
-        self.browser = _FakeBrowser()
+        self.browser = _FakeBrowser(engine)
         self.stopped = False
 
     def launch(self, headless):
@@ -122,21 +143,11 @@ def test_browser_enter_exit_and_open(monkeypatch: pytest.MonkeyPatch):
         assert pw.browser.kwargs["viewport"]["width"] == 1200
         assert pw.browser.kwargs["ignore_https_errors"] is True
         assert pw.browser.kwargs["service_workers"] == "block"
-        assert pw.browser.context.events == ["route", "new_page"]
-        assert pw.browser.context.route_matcher.search(
-            "https://region1.google-analytics.com/g/collect"
+        assert pw.browser.context.events == ["new_page"]
+        assert ("Network.setBlockedURLs", {"urls": build_blocked_url_patterns()}) in (
+            pw.browser.context.cdp.sent
         )
 
-        class _FakeRoute:
-            aborted_with = None
-
-            def abort(self, error_code):
-                self.aborted_with = error_code
-
-        route = _FakeRoute()
-        pw.browser.context.route_handler(route)
-        assert route.aborted_with == "blockedbyclient"
-
     assert pw.browser.context.closed is True
     assert pw.browser.closed is True
     assert pw.stopped is True
@@ -174,3 +185,34 @@ def test_browser_open_requires_initialized_context():
     browser = Browser(headless=True, timeout_ms=50, wait_until="load")
     with pytest.raises(RuntimeError, match="not initialized"):
         browser.open("https://example.org")
+
+
+def test_pages_the_site_opens_are_covered(monkeypatch: pytest.MonkeyPatch):
+    # A popup the page opens itself is a new page in the same context, and the
+    # policy has to reach it too -- not just the page `open()` creates.
+    pw = _FakePlaywright()
+    monkeypatch.setattr(browser_module, "sync_playwright", lambda: _Manager(pw))
+
+    with Browser(headless=True, timeout_ms=100, wait_until="load"):
+        context = pw.browser.context
+        context.cdp.sent.clear()
+        for handler in context._page_handlers:
+            handler(_FakePage())
+
+    assert ("Network.setBlockedURLs", {"urls": build_blocked_url_patterns()}) in (
+        context.cdp.sent
+    )
+
+
+def test_non_chromium_engines_use_the_route_fallback(monkeypatch: pytest.MonkeyPatch):
+    pw = _FakePlaywright(engine="firefox")
+    monkeypatch.setattr(browser_module, "sync_playwright", lambda: _Manager(pw))
+
+    with Browser(headless=True, timeout_ms=100, wait_until="load") as browser:
+        browser.open("https://example.org")
+
+    context = pw.browser.context
+    assert [pattern for pattern, _ in context.page.routes] == [
+        GOOGLE_ANALYTICS_URL_PATTERN
+    ]
+    assert context.cdp.sent == []