Skip to content

Commit fdb408a

Browse files
committed
ci(devx): floor and handshake the watchdog pin's self-test; drop a comment its own trigger falsified
Review found two things and both are correct. 1. The workflow carried a comment claiming NO pull_request and NO merge_group trigger, four lines under the pull_request trigger this branch added, and false about the pin besides. Replaced with the true statement. 2. The new gate's self-test decided success by `failures.length === 0` and printed an UNCOMPARED case count, and the dispatch discarded its completion — so an early return would have printed `0 assertions` and exited 0. Landing a new scripts/** gate in that shape re-opens two closed sets by one. Brought to the landed shape: a frozen SELF_TEST_BATTERIES roster of eight battery names with per-battery floors, the roster's own size pinned, registerCase() attribution to the most recently opened battery, the floor evaluated at the verdict site before the green line, and the reached-verdict handshake read at the --self-test dispatch. Measured, not asserted: `measure-self-test-floor.mjs --json` now reads ROSTER for this file (was COUNT), and `--probe --only` reads HELD (0 DEFEATED, 1 HELD, 0 ACCIDENT). Every floor equals its measured count; 42 assertions across 8 batteries. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PU9zBGbH2s2ZtxSyu963M3
1 parent 993d85a commit fdb408a

2 files changed

Lines changed: 105 additions & 12 deletions

File tree

.github/workflows/platform-checklist-watchdog.yml

Lines changed: 6 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -120,10 +120,12 @@ on:
120120
pull_request:
121121
paths:
122122
- '.github/workflows/platform-checklist-watchdog.yml'
123-
# ⛔ NO `pull_request:` and ⛔ NO `merge_group:` here, deliberately — see the
124-
# header. `scripts/check-platform-checklist-watchdog.mjs` fails when either
125-
# appears, and its self-test proves that refusal fires rather than asserting
126-
# it into the void.
123+
# ⛔ NO `merge_group:` and ⛔ NO `pull_request_target:` here, ever — those two
124+
# are refused OUTRIGHT, because no `paths:` filter makes either safe in this
125+
# workflow. `scripts/check-platform-checklist-watchdog.mjs` fails when either
126+
# appears, and fails just as loudly when the `pull_request:` trigger above
127+
# loses its filter or gains a second path; its self-test drives every one of
128+
# those cases rather than asserting them into the void.
127129

128130
# Least privilege. The gate is read-only against the filesystem by construction
129131
# (no socket, no token), and this job writes issues and nothing else — no label

scripts/check-platform-checklist-watchdog.mjs

Lines changed: 99 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -121,6 +121,47 @@ const BOARD_WRITE_CALLS = Object.freeze([
121121
'issues.addLabels(',
122122
]);
123123

124+
// ── The self-test's own battery roster and floor ───────────────────────────
125+
//
126+
// `failures.length === 0` alone cannot tell "every case held" from "the cases
127+
// never ran": both print the same green line, and a printed `checked` count
128+
// that nothing COMPARES is evidence, not proof. So each battery declares a
129+
// FLOOR and the roster is compared as a SET — a set difference names WHICH
130+
// battery stopped, where a count says only that something did.
131+
//
132+
// The counts are a floor, not an equality: adding cases is ordinary work and
133+
// must not red. A battery BELOW its floor means cases stopped running, and the
134+
// remedy is to find what stopped registering — ⛔ never to lower the number.
135+
//
136+
// ⛔ A pinned TOTAL is not the repair either: one battery dropping from 12
137+
// cases to 2 keeps a total "right" the moment a sibling grows.
138+
const SELF_TEST_BATTERIES = Object.freeze({
139+
'the positive control — a compliant workflow yields no finding': 2,
140+
'clause 1 — the file exists, is non-empty, and parses as a workflow': 5,
141+
'clause 2 — schedule (with a real cron) and workflow_dispatch': 5,
142+
'clause 3 — merge_group and pull_request_target refused outright': 6,
143+
'clause 4 — a pull_request trigger paths-filtered to this file ALONE': 10,
144+
'clause 4b — no board write reachable from a pull_request run': 6,
145+
'clauses 5 and 6 — the package script, and no inlined copy of it': 5,
146+
'the manifest clause — the package script exists to be invoked': 3,
147+
});
148+
149+
// DELETING an entry silences that battery's floor exactly as effectively as
150+
// zeroing it, so the roster's own size is pinned beside the floors.
151+
const SELF_TEST_BATTERY_FLOOR = 8;
152+
153+
// The key an assertion is filed under when no battery is open. It is not a
154+
// declared battery, so it reds by the same set difference rather than silently
155+
// inflating whichever battery happened to run last.
156+
const UNATTRIBUTED_BATTERY = '(no battery open)';
157+
158+
// Set by `selfTest()` only after a verdict is printed — EITHER verdict — and
159+
// read at the dispatch: a `return` that leaves the function above those lines
160+
// prints nothing and still exits 0, so a self-test that never finished would
161+
// report as one that passed. The failure path sets it too, so the refusal fires
162+
// only when NEITHER verdict was printed, never on a genuine red.
163+
let selfTestReachedVerdict = false;
164+
124165
/** The tail every trigger refusal carries, so one reason is stated once. */
125166
const DECISION_TAIL = 'A standing maintainer decision keeps `check:platform-checklist` off the per-PR path, in its own words so that "an unrelated PR is never blocked by checklist drift"; this workflow changes the reporting channel and must never undo that.';
126167

@@ -376,9 +417,22 @@ const PR_BLOCK = " pull_request:\n paths:\n - '.github/workflows/platfo
376417
const withPullRequest = (block) => GOOD.replace(PR_BLOCK, block);
377418

378419
export function selfTest(parse) {
420+
// `battery()` opens a battery; every assertion below is attributed to the one
421+
// most recently opened, so a section that stops running stops registering and
422+
// names ITSELF at the floor rather than going quiet.
423+
const seen = new Map();
424+
let openBattery = null;
425+
const battery = (name) => {
426+
openBattery = name;
427+
};
428+
const registerCase = () => {
429+
const b = openBattery ?? UNATTRIBUTED_BATTERY;
430+
seen.set(b, (seen.get(b) ?? 0) + 1);
431+
};
379432
const failures = [];
380433
let checked = 0;
381434
const t = (what, ok) => {
435+
registerCase();
382436
checked += 1;
383437
if (!ok) failures.push(what);
384438
};
@@ -387,26 +441,30 @@ export function selfTest(parse) {
387441
return f.some((m) => m.includes(needle));
388442
};
389443

444+
battery('the positive control — a compliant workflow yields no finding');
390445
// The positive control. Every clause must be SILENT on a good workflow --
391446
// without this, a rule that fires on everything would pass every case below.
392447
const good = judgeWorkflow(GOOD, parse);
393448
t('the good fixture must produce zero findings (positive control)', good.failures.length === 0);
394449
t('the good fixture must report its triggers', Array.isArray(good.triggers) && good.triggers.includes('schedule'));
395450

451+
battery('clause 1 — the file exists, is non-empty, and parses as a workflow');
396452
// Clause 1 -- absence and unreadability.
397453
t('a missing workflow ⇒ names the file', fires(null, WORKFLOW_REL));
398454
t('a missing workflow ⇒ says the gate has no other channel', fires(null, 'is visible to nobody'));
399455
t('an empty workflow ⇒ fires', fires('', 'is empty'));
400456
t('unparseable YAML ⇒ fires with the parse error', fires('jobs:\n a:\n \tbad: [', 'could not be read as YAML'));
401457
t('a YAML scalar ⇒ fires', fires('just a string', 'does not parse to a workflow mapping'));
402458

459+
battery('clause 2 — schedule (with a real cron) and workflow_dispatch');
403460
// Clause 2 -- the positive triggers.
404461
t('no `on:` block at all ⇒ fires', fires('name: x\njobs: {}\n', 'declares no trigger block'));
405462
t('no `schedule:` ⇒ fires', fires(GOOD.replace(/ schedule:\n - cron: '51 2 \* \* \*'\n/, ''), 'declares no `schedule:` trigger'));
406463
t('no `workflow_dispatch:` ⇒ fires', fires(GOOD.replace(' workflow_dispatch: {}\n', ''), 'declares no `workflow_dispatch:` trigger'));
407464
t('a `schedule:` with no cron ⇒ fires', fires(GOOD.replace(" - cron: '51 2 * * *'\n", ' - {}\n'), 'no usable `cron:`'));
408465
t('a `schedule:` with an empty cron ⇒ fires', fires(GOOD.replace("'51 2 * * *'", "''"), 'no usable `cron:`'));
409466

467+
battery('clause 3 — merge_group and pull_request_target refused outright');
410468
// Clause 3 -- REFUSED OUTRIGHT. Each fires on its own, and the good fixture
411469
// above proves neither fires without cause.
412470
for (const trigger of REFUSED_TRIGGERS) {
@@ -415,6 +473,7 @@ export function selfTest(parse) {
415473
t(`a \`${trigger}:\` trigger ⇒ cites the decision it protects`, fires(withTrigger(trigger), 'never blocked by checklist drift'));
416474
}
417475

476+
battery('clause 4 — a pull_request trigger paths-filtered to this file ALONE');
418477
// Clause 4 -- THE NARROWED CLAUSE. `pull_request:` is permitted, and ONLY
419478
// while its `paths:` filter names this workflow and nothing else. The good
420479
// fixture carries exactly that and is silent (asserted above), so each case
@@ -448,6 +507,7 @@ export function selfTest(parse) {
448507
GOOD.replace('on:\n', '# the pull_request trigger below is paths-filtered; merge_group is refused\non:\n'), parse,
449508
).failures.length === 0);
450509

510+
battery('clause 4b — no board write reachable from a pull_request run');
451511
// Clause 4's other half -- a board write reachable from a pull_request run.
452512
const UNGUARDED = GOOD.replace(" if: steps.gate.outputs.exit_code != '0' && github.event_name != 'pull_request'\n", " if: steps.gate.outputs.exit_code != '0'\n");
453513
t('⭐ a board write whose `if:` drops the pull_request guard ⇒ fires', fires(UNGUARDED, 'could write to the board'));
@@ -461,6 +521,7 @@ export function selfTest(parse) {
461521
t('a step that writes nothing needs no guard (no false positive)', judgeBoardWrites(
462522
parse('jobs:\n j:\n steps:\n - name: read only\n run: echo hi\n')).length === 0);
463523

524+
battery('clauses 5 and 6 — the package script, and no inlined copy of it');
464525
// Clause 4 / 5 -- the invocation.
465526
t('no `pnpm check:platform-checklist` ⇒ fires', fires(
466527
GOOD.replace('pnpm check:platform-checklist', 'pnpm check:something-else'),
@@ -483,27 +544,57 @@ export function selfTest(parse) {
483544
`never invokes \`pnpm ${PACKAGE_SCRIPT}\``,
484545
));
485546

547+
battery('the manifest clause — the package script exists to be invoked');
486548
// The manifest clause.
487549
t('a manifest without the script ⇒ fires', judgeManifest('{"scripts":{"lint":"eslint ."}}').length === 1);
488550
t('a manifest with the script ⇒ silent', judgeManifest(`{"scripts":{"${PACKAGE_SCRIPT}":"node x.mjs"}}`).length === 0);
489551
t('an unparseable manifest ⇒ fires', judgeManifest('{').length === 1);
490552

491-
return { failures, checked };
553+
// ── The floor, evaluated BEFORE either verdict is printed ───────────────
554+
// A set difference over battery NAMES, so a battery that stopped running
555+
// names itself instead of hiding inside a smaller total.
556+
const declaredBatteries = Object.keys(SELF_TEST_BATTERIES);
557+
if (declaredBatteries.length < SELF_TEST_BATTERY_FLOOR) {
558+
failures.push(`SELF_TEST_BATTERIES declares ${declaredBatteries.length} batteries, below the pinned ${SELF_TEST_BATTERY_FLOOR} — a battery deleted from the roster takes its own floor with it.`);
559+
}
560+
for (const [name, count] of seen) {
561+
if (declaredBatteries.includes(name)) continue;
562+
failures.push(`self-test battery "${name}" registered ${count} case(s) but is not declared in SELF_TEST_BATTERIES — an assertion attributed to no declared battery is one nothing floors.`);
563+
}
564+
for (const name of declaredBatteries) {
565+
const count = seen.get(name) ?? 0;
566+
if (count >= SELF_TEST_BATTERIES[name]) continue;
567+
failures.push(count === 0
568+
? `self-test battery "${name}" DID NOT RUN — 0 cases registered, ${SELF_TEST_BATTERIES[name]} pinned. The verdict would otherwise have claimed those cases hold.`
569+
: `self-test battery "${name}" registered ${count} case(s), below its pinned floor of ${SELF_TEST_BATTERIES[name]} — cases that used to run no longer do (⛔ MAINTAINER-ONLY: lowering a floor is not the repair).`);
570+
}
571+
572+
if (failures.length > 0) {
573+
console.error(`\nx check-platform-checklist-watchdog self-test: ${failures.length} of ${checked} assertions failed\n`);
574+
for (const f of failures) console.error(` - ${f}`);
575+
console.error('');
576+
selfTestReachedVerdict = true;
577+
return 1;
578+
}
579+
console.log(`OK check-platform-checklist-watchdog self-test: ${checked} assertions across ${declaredBatteries.length} floored batteries, every clause driven by a fixture that makes it fire.`);
580+
selfTestReachedVerdict = true;
581+
return 0;
492582
}
493583

494584
async function main(argv) {
495585
const { parse } = await requireDependency('yaml', () => import('yaml'), import.meta.url);
496586

497587
if (argv.includes('--self-test')) {
498-
const { failures, checked } = selfTest(parse);
499-
if (failures.length > 0) {
500-
console.error(`\nx check-platform-checklist-watchdog self-test: ${failures.length} of ${checked} assertions failed\n`);
501-
for (const f of failures) console.error(` - ${f}`);
502-
console.error('');
588+
const code = selfTest(parse);
589+
if (!selfTestReachedVerdict) {
590+
console.error(
591+
'\nx check-platform-checklist-watchdog self-test: selfTest() returned without reaching its\n'
592+
+ 'verdict, so neither line was printed and its battery floors never ran. Exiting 0 here\n'
593+
+ 'would report a self-test that never finished as a self-test that passed.\n',
594+
);
503595
process.exit(1);
504596
}
505-
console.log(`OK check-platform-checklist-watchdog self-test: ${checked} assertions, every clause driven by a fixture that makes it fire.`);
506-
return;
597+
process.exit(code);
507598
}
508599

509600
const workflowPath = join(ROOT, WORKFLOW_REL);

0 commit comments

Comments
 (0)