Rebuild the screenshots with one command, and guard them - #18
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The screenshots went stale silently. For a whole release the README hero and the landing page's only image showed a filter wall and a two-row tab bar that no longer existed, plus a typo that had been fixed. Nothing failed, because nothing checked. A screenshot is a build artifact and should be rebuildable.
npm run mediaDrives a real headless browser over the DevTools protocol against a throwaway synthetic store, and rewrites
docs/media. No dependencies: Node ships a WebSocket client, and Chromium is a tool on the machine rather than something this project installs (CHROME=/pathoverrides discovery; it fails with the list of paths it tried).Two of the five shots are driven, not staged. The filter image clicks
+ Filter, picks a dimension and ticks two values, and throws if no chip appears. The palette image presses the real hotkey and throws if it opens with no commands. So the script cannot produce a screenshot of a state the UI could not reach, which is exactly how the closed-palette bug would have been caught.MEDIAis the single source of truth for the shot list.MEDIA_NOT_SHOT_HERErecords the three images that cannot be produced here and why: the two menu bar popovers need the Swift target's off-screen preview path, and the architecture hero is drawn by hand.The guards
test/media.test.jsholds the three things that let this go unnoticed, and each was mutation-tested against its own defect rather than assumed to work:What it deliberately does not claim
It is not idempotent, and a diff is not a signal. The mock corpus is seeded and stable, but its window ends on the day it runs, so the dates and every pixel shift daily. That is stated in the script header and in the README, so nobody wires it into CI expecting a clean diff. No test can tell whether a screenshot is current; only a person looking at it can, and the script says so when it finishes.
841 tests, lint clean over 169 files, zero type errors. Ran end to end: all five images regenerated and reviewed.