Skip to content

Commit 607c870

Browse files
claude[bot]claude
andauthored
fix(ci): stop the ci-failure self-test minting a failure annotation on green runs (#12018)
`scripts/pm/ci-failure.mjs --self-test` printed a line of prose that named a workflow-command token literally, inside backticks, documenting what the tool anchors on. The Actions runner does not know a token is being quoted: it parsed it and minted `annotation_level: failure` on a check run whose conclusion was SUCCESS. Measured on `d63b01436` (check-run 97678882948, `Lint & Repo Gates`), that artifact was the ONLY annotation the run carried, and the step has no `if:` and the workflow no `paths:` filter, so it fired on every pull request. Two facts measured on real runs shape the fix and the gate: * the legacy `##[...]` form is parsed ANYWHERE in a printed line -- the token sat at column 18 and was still consumed, with everything before it dropped and the rest of the line taken as the message; * the `::...::` form is parsed only at LINE START -- two mid-sentence ones print on every PR from `check-prerelease-pin-watch` and mint nothing. The prose is reworded so the printed bytes carry no token; splitting the JS string would have changed nothing, because the runner reads the printed line. `scripts/check-self-test-workflow-commands.mjs` closes the class rather than the instance: it runs the self-tests CI runs and scans their real stdout, so the question "is this token printed?" is answered by printing it rather than by a rule about source. A static prefilter only selects which to run, which keeps the step at ~14s against the ~4min the whole population costs. A sweep of all 128 found exactly one live site; the two inert ones redden the moment they move. Claude-Session: https://claude.ai/code/session_015ahemw8RcTgqtxrj15PEZx Co-authored-by: Claude <noreply@anthropic.com>
1 parent 1318ba3 commit 607c870

3 files changed

Lines changed: 455 additions & 3 deletions

File tree

.github/workflows/lint.yml

Lines changed: 43 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -788,6 +788,49 @@ jobs:
788788
node scripts/check-self-test-wired.mjs --self-test
789789
node scripts/check-self-test-wired.mjs
790790
791+
# Self-test output vs. the runner's command parser (#11886). Sibling of
792+
# the step above and sharing its population: that one asks whether CI RUNS
793+
# each self-test, this one asks what each self-test PRINTS while it does.
794+
#
795+
# The defect it closes was live on every PR in this repo. A line of
796+
# `scripts/pm/ci-failure.mjs`'s self-test prose named a workflow-command
797+
# token literally, inside backticks, as documentation of what that tool
798+
# anchors on. The runner does not know a token is being quoted: it parsed
799+
# it and minted `annotation_level: failure` on a check run whose
800+
# conclusion was SUCCESS. On `d63b01436` that artifact was the ONLY
801+
# annotation the `Lint & Repo Gates` run carried. A failure-level
802+
# annotation on a green required check is a claim that something failed,
803+
# and a repo that mints one on every PR is teaching its readers to skim
804+
# exactly the surface other gates report findings into.
805+
#
806+
# ⚠️ INVISIBLE TO EVERY LOCAL RUN, which is the whole reason it is a gate:
807+
# the text is ordinary prose on disk and in a terminal, and only a runner
808+
# transforms it. It was found by reading one job's annotations through the
809+
# API, not by anything in this file.
810+
#
811+
# Two facts were measured on real runs and the gate's shape follows them:
812+
# the legacy `##[...]` form is parsed ANYWHERE in a printed line (the
813+
# measured token sat at column 18 and was still consumed), while the
814+
# `::...::` form is parsed only at LINE START (two mid-sentence ones print
815+
# on every PR from `check-prerelease-pin-watch` and mint nothing). So the
816+
# first detector is unanchored and the second is anchored — flagging inert
817+
# prose would be a false positive the next author would be right to delete.
818+
#
819+
# It RUNS the self-tests rather than reasoning about their source, because
820+
# "is this token printed?" is only answerable by printing it. A static
821+
# prefilter picks which ones to run — over-selecting freely, since the
822+
# verdict never comes from it — which is what keeps this step at ~14 s
823+
# instead of the ~4 min the whole population costs.
824+
#
825+
# Invoked as `node scripts/…` rather than through a `pnpm check:*` alias:
826+
# see the GATE INVOCATION IDIOM note at the top of this file. Reads
827+
# `scripts/` and `.github/workflows/` off disk and spawns the selected
828+
# self-tests; no network.
829+
- name: Self-test workflow-command gate
830+
run: |
831+
node scripts/check-self-test-workflow-commands.mjs --self-test
832+
node scripts/check-self-test-workflow-commands.mjs
833+
791834
# Verify-lock entry-point self-test (#9661). `scripts/pm/os-verify-lock.sh`
792835
# is the ONE way an agent takes the container's shared heavy-verify lock,
793836
# and it is the enforcement of a rule that used to live only in prose: the

0 commit comments

Comments
 (0)