Skip to content

test(ci): fixture the no-major changeset guard and wire it into the exemption-free self-test step (#6923) - #7008

Merged
os-project-manager merged 1 commit into
mainfrom
claude/issue-6923-changeset-no-major-self-test
Aug 9, 2026
Merged

test(ci): fixture the no-major changeset guard and wire it into the exemption-free self-test step (#6923)#7008
os-project-manager merged 1 commit into
mainfrom
claude/issue-6923-changeset-no-major-self-test

Conversation

@os-project-manager

Copy link
Copy Markdown
Collaborator

Fixes #6923

scripts/check-changeset-no-major.mjs was the one member of the changeset-gate family with no --self-test, which is why PR #6917 could not wire it into lint.yml alongside the other two. This adds one, following the idiom of check-empty-changeset.mjs and check-adr-0087-registration.mjs (pure exported judge, fixtures that drive the real decision, missing-input-is-a-failure), then uses those fixtures to measure the enforcing half instead of reasoning about it.

Premise re-verified on origin/main first

Both halves of the card still held at d3e53f2d8:

  • grep -n "self-test\|selfTest" scripts/check-changeset-no-major.mjs — no output.
  • .changeset/pre.json is "mode": "pre", so the real scan takes the RC exemption branch and exits 0 on every run.

Which half the fixtures exercise, and what is still unexecuted

Stating this plainly because it is the thing most easily overstated.

The measurement that changed the diff

The dialect fixtures are measured against @changesets/parse@0.4.3 — the version this repo resolves — rather than written from the regex. That found a real divergence, so this PR is not fixtures-only:

input @changesets/parse guard, before after
"@objectstack/spec": major major caught caught
'@objectstack/spec': major major caught caught
docs: major (unquoted) major caught caught
CRLF major caught caught
a leading blank line before the fence major MISSED caught
"@objectstack/spec": MAJOR throws caught caught
no closing fence throws caught caught
"@objectstack/spec": major # keep major MISSED MISSED (#7004)

The parser required its fence on line 1 (if (lines[0]?.trim() !== '---') return []), so a changeset opening with one blank line declared nothing to this guard while changesets honoured its major and would have promoted the entire lockstep group. Both sibling parsers already skipped leading blanks, and all three carry a comment saying the three read the same block — so this was also the one place that comment was false. One line, aligned with the siblings; it can only ever add offenders, and today it adds only ::notice lines.

The trailing-comment row is a family-wide gap in a shared regex with a different consequence in each of the three scripts, so it is filed (#7004) rather than fixed here. The fixture pins the current behaviour with a message telling the next author to flip it, not delete it.

Reverse verification — predictions written before running

Ten ablations, each with its expected failure count stated up front. Nine matched exactly; one did not, and is reported as it happened rather than reshaped.

# ablation predicted observed
R1 delete the leading-blank-line tolerance 3 3 — the three leading-blank fixtures
R2 pre?.mode === 'pre' becomes !== 'exit' 5 crash, not 5 — see below
R2b same, with the null-deref repaired 5 5 — the exact five named
R3 enforcing half exits 0 1 1
R4a unwire from check:changeset-gate-self-tests 1 1
R4b same, seen by the sibling's pin 1 1 (check-empty-changeset)
R4c wire the REAL scan into the shared step 2 2
R5 delete the real scan from pr-automation.yml 1 1
R6 stop excluding .changeset/README.md 2 2
R7 rename lint: so the slice anchor goes stale 2 2
R8 unreadable dir returns an empty Map, not null 1 1

R2 diverged and the divergence is worth recording. The prediction was five assertion failures. What happened was a TypeError: Cannot read properties of null (reading 'tag') — the loosened test lets pre === null reach the exempt branch, which then dereferences pre.tag. Still red, but by crashing before any assertion ran, so it proved less than intended. R2b repeats it with pre?.tag and produces exactly the predicted five: no pre.json at all, pre.json that did not parse, pre.json with no mode key, mode: some future spelling, mode: PRE (wrong case) — with mode: exit correctly staying green.

R7 is the vacuous-green trap this family keeps hitting, checked deliberately after reading check-required-contexts.mjs (#6983). The wiring block slices lint.yml by anchor; a stale anchor yields an empty slice that every assertion below would pass against. R7 renames the job, and the guard fires by name (the lint: job could not be sliced out of lint.yml — its anchors went stale) rather than passing silently.

Wiring

check:changeset-gate-self-tests gains a third entry. Two halves land in deliberately different places:

check-empty-changeset.mjs is edited for one reason: its consumer assertion stated in its own message that this script is "deliberately absent: it has no --self-test to run". That was true when #6917 wrote it and false as of this commit, so leaving it would make the repo assert something false. Replaced with an assertion that this script IS wired, and its --self-test-on-every-invocation regex extended to cover the new entry (R4b/R4c confirm both are load-bearing).

What was measured and deliberately NOT acted on

Per the card's boundary — fixtures may document the enforcing half; changing release-mode semantics is not this PR's to do. .changeset/pre.json is untouched.

The enforcing half reads the whole .changeset stock with no branch point, unlike its diff-scoped sibling. Measured at d3e53f2d8: 1552 changesets on disk, 171 of them declaring major (222 package entries), still present because pre-mode changeset version records consumed changesets in pre.json (1279) rather than deleting them. changeset pre exit rewrites the mode to "exit" (@changesets/pre@2.0.2), so from that commit until the final Version PR merges, every unlabelled PR would fail this guard listing 171 files it never touched, with allow-major as the only route out. That is #6129's direction by another route and it decides what the guard means, so it is filed (#7005) for triage, not improvised here.

Findings filed (unassigned, out of this PR's scope)

Verification

$ pnpm check:changeset-gate-self-tests
✓ check-empty-changeset --self-test: 65 assertions over real temp git repos (real scan() path)
✓ check-adr-0087-registration --self-test: 107 assertions over real temp git repos (real scan()/assertInputs() path)
✓ check-changeset-no-major --self-test: 70 assertions (frontmatter dialects measured against
  @changesets/parse + the pre/exit exemption switch in both directions + the #4690 reader pins + the wiring).

$ node scripts/check-changeset-no-major.mjs      # real scan, output unchanged
✓ Changesets is in pre-release mode (tag: rc) — `major` bumps are the expected product of an RC window; ...

$ pnpm lint          # exit 0
$ pnpm check:nul-bytes
check-nul-bytes: OK (scanned 6473 tracked text file(s); ... no raw ASCII control bytes).

No changeset: this PR touches scripts/ and the root package.json only and publishes nothing, so it carries skip-changeset.


Generated by Claude Code

…xemption-free self-test step (#6923)

`scripts/check-changeset-no-major.mjs` was the one member of the changeset-gate
family with no `--self-test`, which is why PR #6917 could not wire it into
`lint.yml` alongside the other two. Refactor it into a pure `judge()` plus a
pure `render()` (verdict -> lines + exit code), add 70 fixtures, and add the
script to `check:changeset-gate-self-tests`.

The fixtures were measured against `@changesets/parse@0.4.3` rather than
reasoned about, and that measurement found a real divergence: the parser
required its fence on line 1, so a changeset opening with a blank line declared
nothing to this guard while changesets honoured its `major` and would have
promoted the whole lockstep group. Both sibling parsers already skipped leading
blanks, and all three carry a comment claiming the three read the same block --
so this was also the one place that comment was false. Fixed, and pinned.

The enforcing half is still unexecuted on CI: `.changeset/pre.json` is
`"mode": "pre"`, so the real scan takes the exemption branch on every run. What
changed is that `enforce` is now produced by fixtures on every PR, in a job with
no label exemption. Fixtured is not executed, and the header says so.

Also corrects `check-empty-changeset.mjs`, whose consumer assertion stated in
its own message that this script is "deliberately absent: it has no
`--self-test` to run" -- true when #6917 wrote it, false as of this commit.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01F8q5J1MQyocgtNspb15fSn
@vercel

vercel Bot commented Aug 9, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
objectstack Ignored Ignored Aug 9, 2026 8:39am

Request Review

@github-actions github-actions Bot added the size/l label Aug 9, 2026
@os-project-manager os-project-manager added skip-changeset PR has no user-facing published change; bypasses the changeset gate and removed size/l labels Aug 9, 2026 — with Claude
@github-actions github-actions Bot added the dependencies Pull requests that update a dependency file label Aug 9, 2026
@os-project-manager
os-project-manager marked this pull request as ready for review August 9, 2026 10:21
@os-project-manager
os-project-manager added this pull request to the merge queue Aug 9, 2026
Merged via the queue into main with commit 8db3fce Aug 9, 2026
32 checks passed
@os-project-manager
os-project-manager deleted the claude/issue-6923-changeset-no-major-self-test branch August 9, 2026 10:38
akarma-synetal pushed a commit to akarma-synetal/framework that referenced this pull request Aug 17, 2026
…6984) (objectstack-ai#7104)

`scan()` 原本只枚举 `git ls-files`,也就是索引。一个刚写出来、还没
`git add` 的文件根本不在扫描集里,门禁照样 exit 0 并打印它惯常的成功
行 —— 读起来是「这棵树没有裸控制字节」,而不是「你刚写的那个文件没被
看过」。

这个洞的形状恰好等于本门禁自己的事故源:每一例裸字节都来自编辑工具在
作者「正在写关于这个字节的内容」时把转义落成真字节(objectstack-ai#4763 / objectstack-ai#4890 /
PR objectstack-ai#5140 / objectstack-ai#5460 的 issue 正文),而落点必然是一个**新文件**。于是
agent 指令要求的那次「push 前跑一遍」,正是最可能在看一个索引枚举够不
到的文件的那次运行。objectstack-ai#6984 实测过:一个未跟踪文件里躺着裸 0x1b,
AGENTS.md 的 self-scan 找得到,本门禁绿,唯一的差别是索引。

- 扫描集改为「索引 + `--others --exclude-standard`」,即 git 既不跟踪
  也不忽略的工作区文件。
- 成功行改为同时报出两半:`scanned N text file(s) -- N tracked,
  M untracked-not-ignored`。M 为 0 时**照样打印**,否则 CI 的绿和
  objectstack-ai#6984 之前那种绿会渲染成同一句话,而后者正是本单要消掉的假信心。
- 未跟踪的命中在报错行上标 `[untracked]`——「还没 add」是作者拿到
  一个被拒文件后第一个需要知道的事实。
- `--list` 逐条列出未跟踪的那一半路径。

三项实测,不是断言:

1. `--exclude-standard` 让本地垃圾留在外面。本仓库在完整 `pnpm install`
   之后,未跟踪路径 76122 条,过 `--exclude-standard` 之后 0 条;它是剪
   枝而不是遍历,16ms vs 索引枚举的 5ms。EXCLUDED 仍叠加在上面,所以有
   人把 node_modules 取消忽略也点不红。
2. CI 里是 no-op:workflow 检出的是一个 commit,全树受跟踪,未跟踪那一
   半恒为空。`--self-test` 用「把样本 stage 掉再扫一遍」证明,而不是在
   注释里声称。
3. 反向:回退 `enumerate()` 的未跟踪那一半,6 条新断言立刻转红。

`--self-test` 断言数 48 → 75。临时仓库额外把 `core.excludesFile` 指向
一个空文件,否则开发者的全局 ignore 会让「EXCLUDED 仍然生效」那条样本
因为另一个原因通过。

工具链改动,不涉及任何 package,不发版 —— 走 `skip-changeset`,与近期
scripts/ + workflow-only PR 的做法一致(objectstack-ai#7048 / objectstack-ai#7008 / objectstack-ai#6983 / objectstack-ai#6917)。


Claude-Session: https://claude.ai/code/session_01F8q5J1MQyocgtNspb15fSn

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

dependencies Pull requests that update a dependency file skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Observation: check-changeset-no-major.mjs has no --self-test, and its enforcing half is unexecuted on every run today (pre-mode)

2 participants