Skip to content

Commit f0c9ffa

Browse files
os-litantclaude
andauthored
test(cli): name a callback arrow's site after its call, so sibling it() blocks key apart (#12544)
`enclosingFunctionName()` recognised four shapes -- a function declaration, a method, and an arrow/function expression bound to a variable or a property -- and none of them is the one that dominates a test directory: an arrow passed DIRECTLY as a call argument. `it('...', () => {})`, `beforeAll(() => {})` and `describe(...)` matched none, so the walk ran past the callback to the source file and attributed the site to `(top-level)`. That was never merely an ugly label. `siteKey(row)` is `file::fn` and the DELIBERATE registry is keyed by it, so two deliberate bulk copies in two `it()` blocks of ONE file both keyed to `<file>::(top-level)` -- one entry would have silenced BOTH, and the second would never have been reviewed as part of the PR that needed it. That is the carve-out-by-accident shape the gate's own header says the registry exists to prevent. Latent, not live: both current DELIBERATE entries name real functions (`childEnv`, `leakedEnv`), so nothing was mis-keyed and the live census is byte-identical across this change. What is closed is the NEXT entry's problem. An arrow whose parent is a call with an IDENTIFIER callee is now named after that callee plus its first string-literal argument, falling back to the callee alone when there is no literal. Rule 2 reads the same walk and has no registry to mis-key, so it gets the better site in its MESSAGES from the same change. `audit()` takes an optional registry, defaulting to DELIBERATE. The property under test -- ONE entry silences ONE site -- cannot be asserted against a live registry whose two entries sit in two different files, so the self-test synthesises one and drives the real classification path with it. Self-test: 78 -> 92 cases. Claude-Session: https://claude.ai/code/session_01UjujZN219uFzBhSYfMykCd Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1 parent 092b9da commit f0c9ffa

1 file changed

Lines changed: 163 additions & 3 deletions

File tree

scripts/check-cli-test-child-env.mjs

Lines changed: 163 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -390,12 +390,67 @@ function isBulkUse(node) {
390390
return true;
391391
}
392392

393+
/**
394+
* The name a call-argument arrow takes from the call it is passed to:
395+
* `it("boots the child")`, or `beforeAll` when the call carries no title.
396+
*
397+
* DOUBLE quotes around the title, deliberately: the product is a
398+
* {@link DELIBERATE} key, and this file's string literals are single-quoted, so
399+
* a key pasted into that registry needs no escaping.
400+
*/
401+
function callbackSiteName(owner) {
402+
if (!owner || !ts.isCallExpression(owner) || !ts.isIdentifier(owner.expression)) return null;
403+
const callee = owner.expression.text;
404+
const title = owner.arguments.find((arg) => ts.isStringLiteralLike(arg));
405+
return title ? `${callee}("${title.text.replace(/\s+/g, ' ').trim()}")` : callee;
406+
}
407+
393408
/**
394409
* The nearest enclosing named function, which is the stable half of a site key.
395410
*
396411
* A line number is not: any edit above a site moves it, so a registry keyed by
397412
* line would go stale on unrelated changes and teach its readers to re-run
398413
* `--update` without looking.
414+
*
415+
* ## Why a call-argument arrow is named after its CALL (#12531)
416+
*
417+
* The four shapes below -- a function declaration, a method, and an
418+
* arrow/function expression bound to a variable or to a property -- miss the
419+
* one that dominates a TEST directory: an arrow passed DIRECTLY as a call
420+
* argument. `it('boots the child', async () => { ... })`, `beforeAll(async () =>
421+
* { ... })` and `describe(...)` match none of them, so the walk ran straight
422+
* past the callback to the source file and attributed the site to
423+
* `(top-level)`.
424+
*
425+
* That was never merely an ugly label. {@link siteKey} is `file::fn` and
426+
* {@link DELIBERATE} is keyed by it, so two deliberate bulk copies in two
427+
* `it()` blocks of ONE file both keyed to `<file>::(top-level)` -- and a single
428+
* registry entry would have silenced BOTH, leaving the second unreviewed by the
429+
* PR that needed it. That is exactly the carve-out-by-accident shape this
430+
* file's header says the registry exists to prevent. ⚠️ It was LATENT and not
431+
* live when it was found: both entries in the registry name real functions
432+
* (`childEnv`, `leakedEnv`), so nothing was ever mis-keyed. What is closed here
433+
* is the NEXT entry's problem.
434+
*
435+
* So an arrow whose parent is a call with an IDENTIFIER callee is named after
436+
* that callee plus its first string-literal argument, falling back to the
437+
* callee alone when there is no literal. Sibling blocks in one file therefore
438+
* key differently, which is the whole repair -- and rule 2, which has no
439+
* registry to mis-key, gets the same site in its MESSAGES for free.
440+
*
441+
* ⚠️ Three limits, named here so they are pinned rather than discovered:
442+
*
443+
* 1. The callee must be an IDENTIFIER. `it.skip('...')` and
444+
* `it.each(table)('...')` call through a property access or through
445+
* another call, and still walk past to `(top-level)`. That is a choice:
446+
* widening to a property-access callee would also capture
447+
* `promise.then(() => ...)` and `rows.map(() => ...)`, whose callee is a
448+
* WORSE site name than the enclosing test block the walk reaches today.
449+
* 2. The title's whitespace is collapsed so the name stays one line, but it
450+
* is ⛔ never truncated -- two long titles sharing a prefix would collide
451+
* again, which is the defect being closed.
452+
* 3. The nearest NAMED binding still wins, unchanged: a helper declared
453+
* inside an `it()` block is named after the helper, not after the block.
399454
*/
400455
export function enclosingFunctionName(node, sourceFile) {
401456
let cursor = node.parent;
@@ -407,6 +462,8 @@ export function enclosingFunctionName(node, sourceFile) {
407462
if (owner && ts.isVariableDeclaration(owner) && owner.name) return owner.name.getText(sourceFile);
408463
if (owner && ts.isPropertyAssignment(owner) && owner.name) return owner.name.getText(sourceFile);
409464
if (cursor.name) return cursor.name.text;
465+
const called = callbackSiteName(owner);
466+
if (called) return called;
410467
}
411468
cursor = cursor.parent;
412469
}
@@ -658,8 +715,16 @@ export const siteKey = (row) => `${row.file}::${row.fn}`;
658715
* never an empty finding list standing in for a population it could not read.
659716
*
660717
* @param {string} root A directory containing the population root.
718+
* @param {Record<string, {why: string}>} registry The declaration registry to
719+
* classify against. Defaults to {@link DELIBERATE}, which is what every
720+
* caller but the self-test uses. The parameter exists because the property
721+
* the site key has to hold -- that ONE entry silences ONE site -- cannot be
722+
* asserted against the live registry, whose two entries sit in two different
723+
* files and name real functions. A synthesised registry is what lets the
724+
* #12531 collision be shown GONE through the real classification path,
725+
* rather than read off a key string.
661726
*/
662-
export function audit(root) {
727+
export function audit(root, registry = DELIBERATE) {
663728
const populationDir = join(root, POPULATION_ROOT);
664729
let exists = false;
665730
try {
@@ -705,7 +770,7 @@ export function audit(root) {
705770
spawners += 1;
706771
for (const hit of bulkEnvReferences(abs, source)) {
707772
const row = { file: rel, ...hit };
708-
(Object.hasOwn(DELIBERATE, siteKey(row)) ? deliberate : findings).push(row);
773+
(Object.hasOwn(registry, siteKey(row)) ? deliberate : findings).push(row);
709774
}
710775
const calls = envlessSpawnCalls(abs, source);
711776
spawnCalls += calls.calls;
@@ -980,6 +1045,13 @@ export function selfTest() {
9801045
const spawner = (body) => `import { execFile } from 'node:child_process';\n\nexport function runCli(cwd: string) {\n${body}\n}\n`;
9811046
/** The same body in a file that imports nothing that spawns. */
9821047
const plain = (body) => `export function helper(cwd: string) {\n void cwd;\n${body}\n}\n`;
1048+
/**
1049+
* A spawner file whose body sits at TOP LEVEL rather than inside a named
1050+
* function -- the shape a real test file has, and the one the site-naming
1051+
* cases below need: wrapping them in `runCli` would name every site `runCli`
1052+
* and measure nothing about the walk.
1053+
*/
1054+
const suite = (body) => `import { execFile } from 'node:child_process';\nvoid execFile;\n${body}\n`;
9831055

9841056
/** Findings from a one-file spawner tree. */
9851057
const scan = (name, sources) => {
@@ -1104,6 +1176,85 @@ export function selfTest() {
11041176
carvedNeighbour.findings?.length === 1 && carvedNeighbour.findings[0].fn === 'somethingElse',
11051177
JSON.stringify(carvedNeighbour.findings));
11061178

1179+
// -- (8b) SITE NAMING (#12531): a callback arrow is named after its CALL
1180+
// An arrow passed DIRECTLY as a call argument matches none of the four
1181+
// shapes the walk recognises, so before this it ran past `it(...)` and
1182+
// `beforeAll(...)` to the source file and reported `(top-level)`.
1183+
1184+
/** The site names a one-file spawner tree attributes its bulk copies to. */
1185+
const siteNames = (name, sources) => JSON.stringify(scan(name, sources).findings?.map((row) => row.fn));
1186+
const names = (...expected) => JSON.stringify(expected);
1187+
const BULK = ' const e = { ...process.env };\n void e;';
1188+
1189+
t('a bulk copy inside an it() block is named after the BLOCK, not (top-level)',
1190+
siteNames('site-it', { 'a.e2e.test.ts': suite(`it('boots the child', () => {\n${BULK}\n});`) })
1191+
=== names('it("boots the child")'));
1192+
t('a callback with no string-literal title falls back to the callee alone',
1193+
siteNames('site-beforeall', { 'a.e2e.test.ts': suite(`beforeAll(async () => {\n${BULK}\n});`) })
1194+
=== names('beforeAll'));
1195+
1196+
// ⭐ THE case. Two sibling blocks in ONE file must not share a key.
1197+
t('two it() blocks in one file get TWO DISTINCT names -- the collision this closes',
1198+
siteNames('site-siblings', {
1199+
'a.e2e.test.ts': suite(`it('first', () => {\n${BULK}\n});\nit('second', () => {\n${BULK}\n});`),
1200+
}) === names('it("first")', 'it("second")'));
1201+
1202+
t('the NEAREST block wins inside a describe()',
1203+
siteNames('site-nested', {
1204+
'a.e2e.test.ts': suite(`describe('outer', () => {\n it('inner', () => {\n${BULK}\n });\n});`),
1205+
}) === names('it("inner")'));
1206+
t('a named helper declared inside an it() block still wins over the block',
1207+
siteNames('site-helper', {
1208+
'a.e2e.test.ts': suite(`it('x', () => {\n function build() {\n return { ...process.env };\n }\n void build;\n});`),
1209+
}) === names('build'));
1210+
t('a variable-bound arrow inside an it() block still wins too',
1211+
siteNames('site-arrow-binding', {
1212+
'a.e2e.test.ts': suite(`it('x', () => {\n const build = () => ({ ...process.env });\n void build;\n});`),
1213+
}) === names('build'));
1214+
t('a bulk copy at TOP LEVEL is still (top-level) -- the fallback survives',
1215+
siteNames('site-top', { 'a.e2e.test.ts': suite('export const e = { ...process.env };') })
1216+
=== names('(top-level)'));
1217+
1218+
t('a template-literal title is a string literal too',
1219+
siteNames('site-template', { 'a.e2e.test.ts': suite(`it(\`boots\`, () => {\n${BULK}\n});`) })
1220+
=== names('it("boots")'));
1221+
t('...but a title with SUBSTITUTIONS is not, so the callee alone names it',
1222+
siteNames('site-template-sub', { 'a.e2e.test.ts': suite(`it(\`boots \${n}\`, () => {\n${BULK}\n});`) })
1223+
=== names('it'));
1224+
t('a title spanning lines is collapsed to one line, and never truncated',
1225+
siteNames('site-title-wrap', { 'a.e2e.test.ts': suite(`it(\`boots\nthe child\`, () => {\n${BULK}\n});`) })
1226+
=== names('it("boots the child")'));
1227+
1228+
// The named limit, pinned rather than discovered: a property-access callee
1229+
// is deliberately NOT captured, because `promise.then(...)` and `rows.map(
1230+
// ...)` would be worse site names than the test block the walk reaches.
1231+
t('an it.skip() callee is a property access, not an identifier, so it still reads (top-level)',
1232+
siteNames('site-property-callee', { 'a.e2e.test.ts': suite(`it.skip('x', () => {\n${BULK}\n});`) })
1233+
=== names('(top-level)'));
1234+
1235+
// ⭐ The collision END TO END, through the real classification path in
1236+
// audit(). One synthesised entry, two sibling blocks. Before this fix
1237+
// both rows keyed to `<file>::(top-level)`, so ONE entry classified BOTH
1238+
// as deliberate and the second was never reviewed by the PR that needed
1239+
// it -- the carve-out-by-accident this file's header names.
1240+
const siblingRoot = tree('carve-siblings', {
1241+
'a.e2e.test.ts': suite(`it('first', () => {\n${BULK}\n});\nit('second', () => {\n${BULK}\n});`),
1242+
});
1243+
const fnsOf = (result) => JSON.stringify({
1244+
deliberate: result.deliberate.map((row) => row.fn),
1245+
findings: result.findings.map((row) => row.fn),
1246+
});
1247+
1248+
const onlyFirst = audit(siblingRoot, { 'packages/cli/test/a.e2e.test.ts::it("first")': { why: 'synthetic' } });
1249+
t('a registry entry for ONE it() block silences THAT block and leaves its sibling a finding',
1250+
fnsOf(onlyFirst) === JSON.stringify({ deliberate: ['it("first")'], findings: ['it("second")'] }),
1251+
fnsOf(onlyFirst));
1252+
1253+
const topLevelEntry = audit(siblingRoot, { 'packages/cli/test/a.e2e.test.ts::(top-level)': { why: 'synthetic' } });
1254+
t('...and a (top-level) entry silences NEITHER -- before this fix that ONE entry silenced BOTH',
1255+
fnsOf(topLevelEntry) === JSON.stringify({ deliberate: [], findings: ['it("first")', 'it("second")'] }),
1256+
fnsOf(topLevelEntry));
1257+
11071258
// -- (9) RULE 2 (#11595): a spawn CALL that leaves its env undeclared ---
11081259
// The reds here are the whole point of the rule, so they are pinned by
11091260
// REASON as well as by count: "reds for some reason" would still pass if
@@ -1199,6 +1350,14 @@ export function selfTest() {
11991350
t('a commented-out env-less spawn is not a call site',
12001351
undeclared('envless-prose', { ...COMPANION, 'a.e2e.test.ts': spawner(' // execFile(\'x\', []);\n void cwd;') }).length === 0);
12011352

1353+
// Rule 2 has no registry, so it cannot MIS-KEY -- but it names sites in its
1354+
// messages, and `(top-level)` was as unhelpful there. Both rules read the
1355+
// same walk, so this is ONE change and not two (#12531).
1356+
t('rule 2 names the it() block too, so an undeclared env is not reported at (top-level)',
1357+
JSON.stringify(undeclared('envless-in-it', {
1358+
'a.e2e.test.ts': suite('it(\'spawns a probe\', () => {\n execFile(\'x\', []);\n});'),
1359+
}).map((row) => row.fn)) === names('it("spawns a probe")'));
1360+
12021361
// -- (10) the ratchet, in every direction it must move -----------------
12031362
const one = [{ file: 'packages/cli/test/a.ts', fn: 'runCli', line: 1, text: 'x' }];
12041363
const two = [...one, { file: 'packages/cli/test/a.ts', fn: 'runOther', line: 2, text: 'x' }];
@@ -1361,7 +1520,8 @@ export function selfTest() {
13611520
+ '(a bare spread in a spawner reds OUT OF PROCESS and the childEnv() form exits zero through the same entry point; '
13621521
+ 'an env-less spawn reds out of process too, by reason, through every spelling that reaches a spawn API; '
13631522
+ 'a legitimate non-spawner bulk copy stays green and the same body reds once the file spawns; '
1364-
+ 'every member read stays green; the two rules red independently of each other; the carve-out is site-scoped; '
1523+
+ 'every member read stays green; the two rules red independently of each other; the carve-out is site-scoped, '
1524+
+ 'and a callback arrow is named after its CALL so two it() blocks in one file cannot share a registry key; '
13651525
+ 'the ratchet fails in both directions; and all four refusals are paired with a tree that still returns a verdict).',
13661526
);
13671527
return 0;

0 commit comments

Comments
 (0)