Chore/retire screenshot suite - #154
Merged
Merged
Conversation
✅ Deploy Preview for boldare-tech ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
It existed to answer one question -- does the upgraded site still look like the site as it was? -- and the final deploy preview answered it: 18/18 across six pages and three viewports, plus 9/9 journeys and widget metrics against the deployed site, and 27 of 28 content pages structurally identical to production. Keeping it past that point would mean comparing forever against a site that no longer exists. Once this merges, production becomes this code: every future intentional change fails the diff, and re-baselining turns it into a change detector with no notion of what is correct. It also earned its removal on the evidence. Three CI failures came from the suite itself -- monospace resolving differently on every machine, thresholds picked too loose then too tight -- and three real regressions went straight through it: action bar icons at 43x43 instead of 48x48, thumbnails at 90x51 instead of 90x90, and the page background turning white. The first two are rounding errors on a full-page image; the third is invisible by construction, because toHaveScreenshot's per-pixel `threshold` defaults to 0.2 and #fafafa against #ffffff is 5/255 per channel. What it was actually good at is now in e2e/widget-metrics.spec.js as computed assertions: deterministic, environment-independent, no container, and they hold the numbers the screenshots could not see. Those comments are reframed too -- "what production renders" stops meaning anything the moment this ships, so they now read as the record of the pre-upgrade design. Removed with it: the three viewport projects, the Playwright container step in CI and the ~2GB image pull it needed every run, 1.9MB of baseline PNGs, and the container discipline the README had to teach anyone regenerating them. CI keeps lint, build, smoke and e2e. tests/BASELINE.md keeps the record of what was verified. Local after removal: lint 0 errors, 149/149 pages, 15 passed / 1 skipped in 41s (was ~2min). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
danbski
force-pushed
the
chore/retire-screenshot-suite
branch
from
September 3, 2026 06:50
37b088a to
e365645
Compare
An unfiltered `push:` alongside `pull_request:` ran the whole job twice for every commit on a PR branch — once for the push, once for the synchronize. That duplication was the bulk of the CI spend, and both runs tested the same tree. The trade-off is that nothing runs on the merge commit, so the suite never sees exactly what lands on master. Netlify absorbs most of that: it builds master on merge, so a broken build surfaces as a failed deploy. What it does not cover is smoke and e2e, which is where a rendering or routing regression would show. The two lines that close the gap are in the comment if it ever matters. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No description provided.