From 8ab69bbb05d5dbcc7310403a7cbfa6cc41fc1889 Mon Sep 17 00:00:00 2001 From: Antoine BERNIER Date: Thu, 13 Aug 2026 17:24:39 +0200 Subject: [PATCH] A visual change is Chromatic's to report, not this job's `chromatic-job` fails on a changed build today, which #186 introduced on purpose: a change nobody accepted should not read as green. It is the right rule enforced in the wrong place. Chromatic already posts a UI Tests check on the commit, red until the build is accepted, and it clears itself the moment someone accepts -- no re-run of anything. This job failing on top of that adds nothing and costs two things. A pull request that meant its change reads as broken rather than as waiting for review. And the one failure worth seeing here -- an upload that did not work, a token that expired, an archive that never arrived -- looks exactly like 68 snapshots pending review. So `--exit-zero-on-changes` off main: this job answers "did the upload work?", Chromatic answers "should this have changed?". This needs the Chromatic GitHub app installed on the repository -- it is what publishes UI Tests. Without it a changed build now reports nothing at all, which is how build 42 went green with a change waiting unreviewed. Co-Authored-By: Claude Opus 5 --- .github/workflows/ci.yml | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 9c2f40bb..42d1007c 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -167,12 +167,18 @@ jobs: # secrets. Main has to publish too, or a pull request would have no # baseline to compare to. # - # No `--exit-zero-on-changes`: a visual change fails the pull request - # until someone accepts it in the Chromatic UI and re-runs this job. The - # snapshots are deterministic, so a change is never noise -- either the - # pull request meant it, and accepting is the review working, or it - # touched nothing near that example and red is the right answer. Build 42 - # went green with exactly such a change waiting unreviewed. + # `--exit-zero-on-changes`: a visual change is Chromatic's to report, not + # this job's. The job answers "did the upload work?"; the UI Tests check + # Chromatic posts on the commit answers "should this have changed?", and + # it stays red until someone accepts the build -- no re-run of this job + # needed. Failing here on top of that said "broken" where the truth was + # "waiting for you", and it hid a real one: a `pnpm chromatic` that + # cannot upload looks exactly like 68 snapshots pending review. + # + # That requires the Chromatic GitHub app to be installed on the + # repository, which is what publishes the UI Tests check. Without it a + # changed build reports nothing at all, and this flag is how changes pile + # up unreviewed -- build 42 went green with exactly one waiting. # # `--auto-accept-changes` on main, and only there. What lands on main has # already been accepted on its pull request, so asking a second time @@ -185,7 +191,7 @@ jobs: # times, and the nightly is what keeps asking. Chromatic answers "should # this have changed?", which only makes sense once the answer to "did # anything change by itself?" is no. - - run: pnpm chromatic ${{ github.ref == 'refs/heads/main' && '--auto-accept-changes' || '' }} + - run: pnpm chromatic ${{ github.ref == 'refs/heads/main' && '--auto-accept-changes' || '--exit-zero-on-changes' }} if: ${{ env.CHROMATIC == 'true' }} env: CHROMATIC_PROJECT_TOKEN: ${{ secrets.CHROMATIC_PROJECT_TOKEN }}