Skip to content

Stop app unit tests from reaching the network - #1504

Merged
anth-volk merged 4 commits into
mainfrom
unit-test-network-guard
Oct 30, 2026
Merged

anth-volk merged 4 commits into
mainfrom
unit-test-network-guard

Conversation

@anth-volk

@anth-volk anth-volk commented Oct 30, 2026 •

Copy link
Copy Markdown
Collaborator

Fixes #1503

What this changes

ReportLoadStates.test.tsx no longer reaches production. Its UK report test rendered the report page, which since #1434 starts the UK local-area breakdown alongside the national run. Nothing mocked that request, so every run sent GET /uk/economy/98557/over/2 to the production Public API: 213 requests since 27 October, 202 of them from CI. The test now mocks fetchLocalAreaBreakdown with a request that stays in flight (as it already does for metadata), and its UK report uses UK fixture policies instead of US ones relabelled as UK.

App tests can no longer reach the network. app/vitest.setup.mjs installs a guard from app/test-utils/unmockedRequestGuard.ts before each test file:

  • The global fetch rejects at once with an UnmockedRequestError naming the request's method and URL. In jsdom suites, XMLHttpRequest's send throws the same error (jsdom's XMLHttpRequest makes real requests, and Plotly uses it).
  • Each refused request is recorded. A check after each test, and after each file's last test, fails the test with the list of refused requests, so a test fails even when the code under test catches the error, as the report page did here.
  • A test's own mock replaces the guard as before: vi.stubGlobal('fetch', …), vi.spyOn(globalThis, 'fetch'), assigning global.fetch, or mocking the API module. vi.unstubAllGlobals() brings the guard back.
  • fetchRespondingTo({ [url]: () => response }), exported from @test-utils, is a fetch mock that answers the listed URLs and refuses every other request as the guard does.

src/tests/unit/test-utils/unmockedRequestGuard.test.ts covers the guard. Its requests go to a reserved .test host, so even without the guard they could not reach a real server.

Other tests the guard found in CI

Test file Request Fix
libs/flagship/v3CleanSlate.test.tsx GET https://api.policyengine.org/us/economy/77/over/2, sent to the production Public API by runFlagshipReport Mock fetchSocietyWideCalculation with a request that stays in flight
pages/flagship/BillReport.test.tsx POST /api/model-validation, /api/scorecard-comparisons, /api/calibration-targets Answer the calculator's own routes with 404, as outside calculator-app
pages/report-output/congressional-district/AbsoluteChangeByDistrict.test.tsx and RelativeChangeByDistrict.test.tsx GET /data/geojson/congressional_districts.geojson Answer it with the district map test's GeoJSON fixture

The relative URLs never left the test process (Node's fetch cannot parse them), but the tests depended on that failure without saying so; they now mock those requests explicitly.

Commits

  1. The guard and its test. On its own, this commit fails ReportLoadStates.test.tsx with the unmocked breakdown request.
  2. The ReportLoadStates.test.tsx fix.
  3. fetchRespondingTo, with its test.
  4. Mocks for the four test files above.

Tests

Run individually: bun run vitest -- src/tests/unit/test-utils/unmockedRequestGuard.test.ts src/tests/unit/pages/flagship/ReportLoadStates.test.tsx src/tests/unit/libs/flagship/v3CleanSlate.test.tsx src/tests/unit/pages/flagship/BillReport.test.tsx src/tests/unit/pages/report-output/congressional-district/AbsoluteChangeByDistrict.test.tsx src/tests/unit/pages/report-output/congressional-district/RelativeChangeByDistrict.test.tsx (6 files, 55 tests passed). Type check, ESLint and Prettier pass. CI's Tests job passes: 464 app test files (6,118 passed, 10 skipped) and 58 website test files.

🤖 Generated with Claude Code

anth-volk and others added 2 commits October 30, 2026 18:33
The shared Vitest setup now installs a guard as the global fetch and as
XMLHttpRequest's send. A request that no test mocked is refused at once
with an error naming its method and URL, and a check after each test
(and after each file's last test) fails the test that sent it, even when
the code under test catches the error. A test that mocks fetch or the
API module that sends the request replaces the guard, as before.

With this alone, ReportLoadStates' UK report test fails: it sends the
local-area breakdown request to the production Public API.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Since #1434 the report page starts a UK report's local-area breakdown
alongside its national run. ReportLoadStates' UK report test mocked
neither fetchLocalAreaBreakdown nor the hook that calls it, so every run
sent GET /uk/economy/98557/over/2 to the production Public API.

The test now mocks fetchLocalAreaBreakdown with a request that stays in
flight, as it already does for the metadata request, and its UK report
uses UK fixture policies (current law 1, reform 98746) in place of the
US ones it relabelled as UK.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 30, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
policyengine-calculator-next Ready Ready Preview Oct 30, 2026 3:02pm UTC
policyengine-website Ready Ready Preview Oct 30, 2026 3:02pm UTC

Request Review

anth-volk and others added 2 commits October 30, 2026 19:01
A fetch mock that answers the URLs a test lists and refuses every other
request as the guard does, so a test can mock the requests it expects
without letting any other one through.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- v3CleanSlate: runFlagshipReport starts the report's calculation
  without waiting for it, and nothing mocked fetchSocietyWideCalculation,
  so the test sent GET /us/economy/77/over/2 to the production Public
  API. It now mocks that function with a request that stays in flight.
- BillReport: the page asks the calculator's own /api/model-validation,
  /api/scorecard-comparisons and /api/calibration-targets routes. Their
  relative URLs could not be sent from the test environment, and the
  page read the failure as "unavailable". The tests now answer them with
  404, as outside calculator-app, and refuse any other request.
- AbsoluteChangeByDistrict and RelativeChangeByDistrict: the district
  map loads /data/geojson/congressional_districts.geojson, which failed
  the same way. The tests now answer it with the map test's GeoJSON
  fixture.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk marked this pull request as ready for review October 30, 2026 16:19
@anth-volk
anth-volk merged commit a075643 into main Oct 30, 2026
10 checks passed
@anth-volk
anth-volk deleted the unit-test-network-guard branch October 30, 2026 16:20
anth-volk added a commit that referenced this pull request Nov 2, 2026
main's unit tests now refuse network requests nothing mocked (#1504),
and its AbsoluteChangeByDistrict and RelativeChangeByDistrict tests
answer the district map's GeoJSON request. This branch rewrote both
tests around fakeStateDistrictApi, which replaces fetch, so they keep
this branch's version, and the fake API now answers the boundary file
with the map test's GeoJSON fixture instead of a 404.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch was successfully deployed

2 active deployments
Preview – policyengine-website — ca51d217 Deployed Oct 30, 2026 by vercel[bot]
Preview – policyengine-calculator-next — ca51d217 Deployed Oct 30, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

App unit tests send real requests to the production Public API

1 participant