Skip to content

Commit 8da6368

Browse files
os-zhuangclaude
andauthored
fix(publish-smoke): derive the tarball pin set from the publishable population, not a by-name exclusion (#11257)
* fix(publish-smoke): derive the tarball pin set from the publishable population, not a scope/exclusion The packed-tarball smoke built its pin set as "publishable MINUS a hand-written exclusion", and the exclusion held `create-objectstack` on the rationale that no @objectstack/* manifest depends on it. That rationale expired when @objectstack/cli took a dependency on the scaffolder: the unscoped name was neither packed nor pinned, pnpm fell back to the registry, and the smoke died on ERR_PNPM_NO_MATCHING_VERSION for a version that by definition is not published yet — a deterministic false red on every release candidate. - publish-smoke-pack.mjs: the set is `private !== true`, full stop. No scope filter, no exclusion, no hand list. `assertPinSetTotal` re-checks pin set == publishable set in BOTH directions at the derivation site and names the offending packages, so a future exclusion cannot reopen the hole silently. - publish-smoke.sh: the registry-leak assertion drove off a `@objectstack/*` grep, which made it blind in exactly the case it existed to catch. It now drives off the override map's own names, scoped and unscoped alike. - A --self-test (unscoped package survives derivation; both directions of the equality assertion) wired as `check:publish-smoke-pin` in lint.yml, so a regression reddens on the PR that causes it rather than on a release run. Co-Authored-By: Claude <noreply@anthropic.com> * fix(publish-smoke): de-duplicate the registry-leak diagnostic across lockfile sections A name is keyed in both `packages:` and `snapshots:`, so counting lines listed every offender twice and inflated the tarball-resolved total. Co-Authored-By: Claude <noreply@anthropic.com> --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 7345308 commit 8da6368

5 files changed

Lines changed: 230 additions & 25 deletions

File tree

.github/workflows/lint.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -416,6 +416,16 @@ jobs:
416416
- name: Part-of closing-keyword guard self-test
417417
run: pnpm check:partof-closing-keyword
418418

419+
# Publish-smoke tarball pin-set self-test. The assertion it pins lives on
420+
# the RELEASE path (scripts/publish-smoke-pack.mjs runs only inside the
421+
# packed-tarball smoke), so without this step a regression in it would be
422+
# discovered by a release candidate — which is exactly how the unscoped
423+
# `create-objectstack` hole surfaced: as a red on the operator's own
424+
# release run, not on the PR that opened it. Pure functions, synthetic
425+
# fixtures, no pnpm/workspace/network; ~0.05s.
426+
- name: Publish-smoke pin-set self-test
427+
run: pnpm check:publish-smoke-pin
428+
419429
# Single-claim path guard self-test (#9402). Same split as the step above
420430
# and for the same reason: the guard is a PR-scoped blocking check in its
421431
# own workflow, because its question is about OTHER open PRs and needs a

.github/workflows/publish-smoke.yml

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,9 @@
1313
#
1414
# pack-smoke SMOKE_MODE=pack — `pnpm pack` every publishable package
1515
# (pack applies the same manifest rewrites as publish),
16-
# scaffold a fresh project OUTSIDE the workspace, pin
17-
# @objectstack/* to the tarballs via the project's own pnpm
16+
# scaffold a fresh project OUTSIDE the workspace, pin every
17+
# publishable package — scoped and unscoped alike — to the
18+
# tarballs via the project's own pnpm
1819
# overrides, and smoke it. This is "what 15.1.0 would have
1920
# failed": the release-candidate combination, no workspace
2021
# overrides in sight.

package.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,7 @@
5959
"check:pm-half-states": "node scripts/pm/check-half-states.mjs --self-test",
6060
"check:pm-governed-merges": "node scripts/pm/check-governed-merges.mjs --self-test",
6161
"check:pm-governed-prose": "node scripts/pm/check-governed-prose.mjs --self-test && node scripts/pm/check-governed-prose.mjs",
62+
"check:publish-smoke-pin": "node scripts/publish-smoke-pack.mjs --self-test",
6263
"check:partof-closing-keyword": "node scripts/check-partof-closing-keyword.mjs --self-test",
6364
"check:single-claim-paths": "node scripts/check-single-claim-paths.mjs --self-test",
6465
"check:pnpm-filter-targets": "node scripts/pnpm-filter-targets.mjs --self-test && node scripts/check-pnpm-filter-targets.mjs --self-test && node scripts/check-pnpm-filter-targets.mjs",

scripts/publish-smoke-pack.mjs

Lines changed: 166 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -16,29 +16,94 @@
1616
* Packing everything keeps the overrides map total — a package missing from
1717
* it would make the smoke project resolve that name from the npm registry,
1818
* silently testing a published version instead of the candidate one.
19+
*
20+
* THE SET IS THE PUBLISHABLE POPULATION — never a scope glob, never a hand
21+
* list, never an exclusion. This script used to carve `create-objectstack`
22+
* out by name, on the rationale that "no @objectstack/* manifest depends on
23+
* it". That rationale expired the day `@objectstack/cli` took a dependency on
24+
* the scaffolder: cli@17.2.0 declared `create-objectstack@17.2.0`, the name
25+
* was neither packed nor pinned, pnpm fell back to the registry, and the
26+
* release candidate's own smoke died on ERR_PNPM_NO_MATCHING_VERSION for a
27+
* version that by definition does not exist yet — a chicken-and-egg red on
28+
* every release candidate from that day on. The general shape of that bill:
29+
* whether a workspace package is *reachable* from some other manifest is a
30+
* fact about the dependency graph AT ONE MOMENT, and it is not the question
31+
* this script gets to ask. Publishable is the question, `private !== true`
32+
* is the answer, and `assertPinSetTotal` below re-checks it on every run so
33+
* a future exclusion cannot re-open the hole silently.
34+
*
35+
* Source of truth: the workspace itself. The Changesets `fixed` group in
36+
* .changeset/config.json enumerates the same 69 names, but it is a DERIVED
37+
* declaration validated against the workspace by scripts/check-changeset-fixed.mjs
38+
* (which reddens both when a public package is missing from the group and
39+
* when a group name no longer exists) — deriving from the group would mean
40+
* reading a copy that a gate keeps honest, rather than the thing itself.
1941
*/
2042

2143
import { execFile } from 'node:child_process';
2244
import { mkdirSync, writeFileSync } from 'node:fs';
2345
import { resolve } from 'node:path';
2446
import { promisify } from 'node:util';
2547

26-
const execFileP = promisify(execFile);
48+
import { isEntrypoint } from './invoked-as.mjs';
2749

28-
// Not consumed as npm dependencies by a scaffolded project:
29-
// create-objectstack — the scaffolder itself; the smoke runs it straight
30-
// from the repo's built bin, and no @objectstack/* manifest depends on it.
31-
const EXCLUDE = new Set(['create-objectstack']);
50+
const execFileP = promisify(execFile);
3251

3352
const CONCURRENCY = 8;
3453

54+
/**
55+
* The publishable population: every workspace member npm would receive.
56+
* No scope filter — `create-objectstack` is unscoped and publishable, and
57+
* the next unscoped public package must land in the set on its own.
58+
*
59+
* @param {{name?: string, private?: boolean}[]} all workspace members
60+
* @returns {{name: string}[]}
61+
*/
62+
export function selectPublishable(all) {
63+
return all.filter((p) => p.name && p.private !== true);
64+
}
65+
66+
/**
67+
* The pin set MUST equal the publishable set, both directions. A hole in
68+
* either direction makes the smoke test something other than the candidate:
69+
* a missing pin resolves that name from the registry (the bill above), and a
70+
* surplus pin points the smoke project at a tarball for a name npm will never
71+
* publish. Both are reported BY NAME — "the map is incomplete" without the
72+
* name is the diagnostic the release operator had to reverse-engineer.
73+
*
74+
* @param {string[]} pinned names present in the overrides map
75+
* @param {string[]} publishable names of the publishable population
76+
*/
77+
export function assertPinSetTotal(pinned, publishable) {
78+
const pinnedSet = new Set(pinned);
79+
const publishableSet = new Set(publishable);
80+
const missing = publishable.filter((n) => !pinnedSet.has(n)).sort();
81+
const surplus = pinned.filter((n) => !publishableSet.has(n)).sort();
82+
if (missing.length === 0 && surplus.length === 0) return;
83+
const lines = ['tarball pin set != publishable set'];
84+
if (missing.length > 0) {
85+
lines.push(
86+
` publishable but NOT pinned (${missing.length}): ${missing.join(', ')}`,
87+
' → the smoke project would resolve these from the npm registry, so it',
88+
' would test PUBLISHED code, or die on a version not published yet.',
89+
);
90+
}
91+
if (surplus.length > 0) {
92+
lines.push(
93+
` pinned but NOT publishable (${surplus.length}): ${surplus.join(', ')}`,
94+
' → pinning a name npm will never publish; the smoke would pass on a',
95+
' resolution no real user can reproduce.',
96+
);
97+
}
98+
throw new Error(lines.join('\n'));
99+
}
100+
35101
async function listPublicPackages(repoRoot) {
36102
const { stdout } = await execFileP('pnpm', ['-r', 'list', '--depth', '-1', '--json'], {
37103
cwd: repoRoot,
38104
maxBuffer: 64 * 1024 * 1024,
39105
});
40-
const all = JSON.parse(stdout);
41-
return all.filter((p) => p.name && p.private !== true && !EXCLUDE.has(p.name));
106+
return selectPublishable(JSON.parse(stdout));
42107
}
43108

44109
async function packOne(pkg, destDir) {
@@ -89,6 +154,11 @@ async function main() {
89154
});
90155
await Promise.all(workers);
91156

157+
assertPinSetTotal(
158+
Object.keys(overrides),
159+
packages.map((p) => p.name),
160+
);
161+
92162
const sorted = Object.fromEntries(
93163
Object.entries(overrides).sort(([a], [b]) => a.localeCompare(b)),
94164
);
@@ -97,7 +167,92 @@ async function main() {
97167
console.log(`Wrote ${Object.keys(sorted).length} override(s) → ${outPath}`);
98168
}
99169

100-
main().catch((err) => {
101-
console.error(err.stack ?? String(err));
102-
process.exit(1);
103-
});
170+
/**
171+
* Self-test — runs without pnpm, a workspace, or a network. It pins the two
172+
* properties the release smoke depends on, and both are ABLATION-CHECKED
173+
* (2026-08-23): restoring `EXCLUDE = new Set(['create-objectstack'])` and
174+
* filtering it out of `selectPublishable` turns case 1 red by name; deleting
175+
* the `missing`/`surplus` branch of `assertPinSetTotal` turns cases 2/3 red.
176+
*
177+
* Case 1 is not "some package survives the filter" — it is specifically that
178+
* an UNSCOPED public package does, because every form this defect has taken
179+
* (a `@objectstack/*` scope glob in the pinning prose, a by-name exclusion in
180+
* the derivation) is invisible to any fixture whose names all start with `@`.
181+
*/
182+
function selfTest() {
183+
const cases = [];
184+
const check = (name, fn) => {
185+
try {
186+
fn();
187+
cases.push(` ok — ${name}`);
188+
} catch (err) {
189+
cases.push(` FAIL — ${name}\n ${(err.message ?? String(err)).split('\n').join('\n ')}`);
190+
process.exitCode = 1;
191+
}
192+
};
193+
const assert = (cond, msg) => {
194+
if (!cond) throw new Error(msg);
195+
};
196+
197+
check('an unscoped public package is in the derived set', () => {
198+
const picked = selectPublishable([
199+
{ name: '@objectstack/cli', private: false },
200+
{ name: 'create-objectstack' }, // no `private` key at all — the real manifest
201+
{ name: '@objectstack/internal-fixtures', private: true },
202+
{ name: undefined },
203+
]).map((p) => p.name);
204+
assert(
205+
picked.includes('create-objectstack'),
206+
`unscoped public package dropped from the set: ${JSON.stringify(picked)}`,
207+
);
208+
assert(
209+
!picked.includes('@objectstack/internal-fixtures'),
210+
'a private package leaked into the publishable set',
211+
);
212+
assert(picked.length === 2, `expected 2 publishable, got ${picked.length}`);
213+
});
214+
215+
check('set == publishable set is accepted', () => {
216+
assertPinSetTotal(['create-objectstack', '@objectstack/cli'], ['@objectstack/cli', 'create-objectstack']);
217+
});
218+
219+
check('a MISSING member reddens, by name', () => {
220+
let msg = '';
221+
try {
222+
assertPinSetTotal(['@objectstack/cli'], ['@objectstack/cli', 'create-objectstack']);
223+
} catch (err) {
224+
msg = err.message;
225+
}
226+
assert(msg !== '', 'a pin set missing a publishable member was accepted');
227+
assert(
228+
msg.includes('create-objectstack'),
229+
`the diagnostic does not name the missing package: ${msg}`,
230+
);
231+
});
232+
233+
check('a SURPLUS member reddens, by name', () => {
234+
let msg = '';
235+
try {
236+
assertPinSetTotal(['@objectstack/cli', '@objectstack/gone'], ['@objectstack/cli']);
237+
} catch (err) {
238+
msg = err.message;
239+
}
240+
assert(msg !== '', 'a pin set with a non-publishable member was accepted');
241+
assert(msg.includes('@objectstack/gone'), `the diagnostic does not name the surplus package: ${msg}`);
242+
});
243+
244+
console.log('publish-smoke-pack self-test');
245+
for (const line of cases) console.log(line);
246+
console.log(process.exitCode === 1 ? 'SELF-TEST FAILED' : `SELF-TEST PASSED (${cases.length} cases)`);
247+
}
248+
249+
if (isEntrypoint(import.meta.url)) {
250+
if (process.argv.includes('--self-test')) {
251+
selfTest();
252+
} else {
253+
main().catch((err) => {
254+
console.error(err.stack ?? String(err));
255+
process.exit(1);
256+
});
257+
}
258+
}

scripts/publish-smoke.sh

Lines changed: 50 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -460,7 +460,7 @@ if [ "$SMOKE_MODE" = "pack" ]; then
460460
# Anything NOT in the override map (transitive deps, better-auth, hono, …)
461461
# resolves from the registry exactly as it would for a real user; that
462462
# unpinned resolution is the thing under test.
463-
log "Pinning @objectstack/* to local tarballs via project-local overrides"
463+
log "Pinning every publishable package to local tarballs via project-local overrides"
464464
node - "$SMOKE_ROOT/tarballs/overrides.json" "$APP_DIR/pnpm-workspace.yaml" <<'EOF'
465465
const { existsSync, readFileSync, writeFileSync } = require('node:fs');
466466
const [overridesPath, wsPath] = process.argv.slice(2);
@@ -484,7 +484,8 @@ const lines = [
484484
base,
485485
'',
486486
'# ── appended by scripts/publish-smoke.sh ─────────────────────────────────',
487-
'# @objectstack/* pinned to the about-to-publish tarballs; everything above',
487+
'# every publishable package pinned to its about-to-publish tarball (scoped',
488+
'# and unscoped alike); everything above',
488489
'# is what the template ships, everything else resolves from the registry.',
489490
'overrides:',
490491
...Object.entries(overrides).map(([name, spec]) => ` '${name}': '${spec}'`),
@@ -518,17 +519,54 @@ EOF
518519
log "Installing (pnpm, tarball-pinned)"
519520
(cd "$APP_DIR" && pnpm install --no-frozen-lockfile)
520521

521-
# Belt-and-braces: if any @objectstack/* resolved from the REGISTRY the
522+
# Belt-and-braces: if any package we PINNED resolved from the REGISTRY the
522523
# override map has a hole and the smoke would silently test published code.
523-
# Registry-resolved lockfile keys read '@objectstack/<name>@<version>';
524-
# tarball-pinned ones read '@objectstack/<name>@file:…' (with possible
525-
# peer suffixes containing their own @<version>, hence the [^'@] name part).
526-
log "Asserting no @objectstack/* leaked to the registry"
527-
if grep -En "'@objectstack/[^'@]+@[0-9]" "$APP_DIR/pnpm-lock.yaml"; then
528-
fail "some @objectstack/* packages resolved from the registry (see above) — publish-smoke-pack.mjs override map is incomplete"
529-
fi
530-
TARBALL_COUNT=$(grep -cE "'@objectstack/[^'@]+@file:" "$APP_DIR/pnpm-lock.yaml" || true)
531-
echo " ok — $TARBALL_COUNT tarball-resolved @objectstack/* lockfile entries"
524+
# Registry-resolved lockfile keys read '<name>@<version>'; tarball-pinned
525+
# ones read '<name>@file:…' (with possible peer suffixes containing their
526+
# own @<version>, hence the name-anchored match).
527+
#
528+
# The names come from the override map, NOT from a `@objectstack/*` glob.
529+
# This assertion used to grep the scope, which made it blind in exactly the
530+
# case it existed to catch: `create-objectstack` is unscoped, so when it went
531+
# unpinned and resolved from the registry, this guard reported "ok" and the
532+
# smoke died 200 lines later inside pnpm with ERR_PNPM_NO_MATCHING_VERSION.
533+
# A guard whose alphabet is narrower than the set it guards is not a guard.
534+
log "Asserting no pinned package leaked to the registry"
535+
node - "$SMOKE_ROOT/tarballs/overrides.json" "$APP_DIR/pnpm-lock.yaml" <<'EOF'
536+
const { readFileSync } = require('node:fs');
537+
const [overridesPath, lockPath] = process.argv.slice(2);
538+
const names = Object.keys(JSON.parse(readFileSync(overridesPath, 'utf8')));
539+
const lock = readFileSync(lockPath, 'utf8').split(/\r?\n/);
540+
541+
const esc = (s) => s.replace(/[.*+?^${}()|[\]\\]/g, '\\$&');
542+
// Sets, not arrays: a name is keyed in BOTH the `packages:` and `snapshots:`
543+
// sections, so counting lines would report every package twice.
544+
const leaked = new Set();
545+
const pinned = new Set();
546+
for (const name of names) {
547+
// Lockfile key lines: optional quote, the exact package name, '@', spec.
548+
const key = new RegExp(`^\\s*'?${esc(name)}@([^']+?)'?:\\s*$`);
549+
for (const line of lock) {
550+
const m = key.exec(line);
551+
if (!m) continue;
552+
if (m[1].startsWith('file:')) pinned.add(name);
553+
else if (/^[0-9]/.test(m[1])) leaked.add(`${name}@${m[1]}`);
554+
}
555+
}
556+
557+
if (leaked.size > 0) {
558+
console.error('::error::these PINNED packages resolved from the npm registry:');
559+
for (const l of [...leaked].sort()) console.error(` ${l}`);
560+
console.error(
561+
'The publish-smoke-pack.mjs override map has a hole, so the smoke tested ' +
562+
'PUBLISHED code instead of the release candidate.',
563+
);
564+
process.exit(1);
565+
}
566+
console.log(
567+
` ok — ${pinned.size}/${names.length} pinned packages resolved from tarballs, 0 registry leaks`,
568+
);
569+
EOF
532570
else
533571
log "Scaffolding $APP_NAME with published create-objectstack@latest"
534572
(cd "$SMOKE_ROOT" && npx -y create-objectstack@latest "$APP_NAME" --skip-install --skip-skills)

0 commit comments

Comments
 (0)