Skip to content

Commit 408ff1b

Browse files
committed
One command a judge can paste, and the figure count it caught
npm run verify runs every gate CI runs, in the same order, with zero credentials and no network: decision-surface purity (I10), the 80-test suite, the survey re-derivation, and the Solidity tests. An optional gate that cannot run reports SKIP, never PASS. A judge who clones this and lacks foundry should see that plainly rather than read a green line that checked nothing. Writing it surfaced a stale number: rederive.py asserts 24 published figures and the README said 22. Corrected in both places, and the count is no longer hardcoded in the verify output so it cannot drift again.
1 parent 272cc2f commit 408ff1b

3 files changed

Lines changed: 137 additions & 6 deletions

File tree

β€ŽREADME.mdβ€Ž

Lines changed: 21 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -62,7 +62,7 @@ can concatenate signatures. Knowing which 7.4% are real β€” and then which of th
6262
land β€” is the hard part, and it is measurable.
6363

6464
**Every number in that table is reproducible from this repo.** The scripts that made the calls, the
65-
raw responses they returned, and a re-derivation that asserts all 22 published figures are committed
65+
raw responses they returned, and a re-derivation that asserts all 24 published figures are committed
6666
at [`survey/`](survey/) β€” and CI runs it, so the prose cannot drift away from the measurement:
6767

6868
```bash
@@ -302,12 +302,28 @@ cryptography. A tool that promises to clear those is advertising drains that can
302302

303303
## πŸƒ Run it yourself
304304

305-
Only the first command runs with **zero credentials** β€” and it is the one that exercises the whole
306-
decision surface.
305+
**One command checks everything.** Zero credentials, no network, under a second:
307306

308307
```bash
309-
git clone <this repo> && cd gavel
308+
git clone https://github.com/edycutjong/gavel && cd gavel
310309
npm install
310+
npm run verify
311+
```
312+
313+
It runs every gate CI runs, in the same order, and prints PASS/FAIL per gate:
314+
315+
| Gate | What it proves |
316+
|---|---|
317+
| Decision surface is pure | `src/assemble.mjs` contains no `import`, `eval`, `fetch`, `Date.now`, `Math.random` or `process.env` β€” invariant I10 |
318+
| JS test suite | 80 tests, `node --test` |
319+
| Published figures re-derive | all 24 README figures, re-derived from 1.3 MB of committed raw responses |
320+
| Solidity tests | `forge test` β€” skipped **loudly** if foundry is absent, never passed quietly |
321+
322+
An optional gate that cannot run reports `SKIP`, not `PASS`. A gate you can't run must never look green.
323+
324+
If you would rather run the pieces yourself:
325+
326+
```bash
311327
npm test # 80 tests, ~0.08 s, no network, no keys
312328
```
313329

@@ -373,7 +389,7 @@ refusing correctly is a success, not an error.
373389
| [`scripts/bench.py`](scripts/bench.py) | Reduces those rows to p50/p95 duration and gas cost. Python 3 stdlib, no network β€” a reducer, never a harness: with no receipts there is no output |
374390
| [`contracts/`](contracts/) | `MockUSDC.sol`, the testnet stand-in, and [`contracts/test/`](contracts/test/) β€” 22 tests at 100% coverage on every metric, dependency-free |
375391
| [`test/`](test/) | 80 tests: unit fixtures, the live-response regression file, manifest/roster invariants, and the coverage-gap suite ([`COVERAGE.md`](test/COVERAGE.md)) |
376-
| [`survey/`](survey/) | The 1,299-Safe measurement: collectors, 1.3 MB of raw responses, and `rederive.py`, which asserts all 22 published figures offline |
392+
| [`survey/`](survey/) | The 1,299-Safe measurement: collectors, 1.3 MB of raw responses, and `rederive.py`, which asserts all 24 published figures offline |
377393
| [`DX-REPORT.md`](DX-REPORT.md) | Seven reproducible KeeperHub findings, dated as they were hit |
378394
| [`workflows/`](workflows/) | Generated `gavel-drain` graph, 11 nodes (1 trigger + 10 actions), 10 edges |
379395
| [`docs/rehearsal-11155111.md`](docs/rehearsal-11155111.md) | Rehearsal log. Labelled NOT EVIDENCE |

β€Žpackage.jsonβ€Ž

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,8 @@
88
"node": ">=20"
99
},
1010
"scripts": {
11-
"test": "node --test"
11+
"test": "node --test",
12+
"verify": "node scripts/verify.mjs"
1213
},
1314
"license": "MIT",
1415
"dependencies": {

β€Žscripts/verify.mjsβ€Ž

Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
#!/usr/bin/env node
2+
// One command a judge can paste. Runs every gate CI runs, in the same order,
3+
// with zero credentials and zero network. Optional toolchains SKIP loudly
4+
// rather than passing quietly β€” a gate that cannot run must never look green.
5+
//
6+
// Exit 0 only if every REQUIRED gate passed.
7+
8+
import { spawnSync } from 'node:child_process';
9+
import { readFileSync } from 'node:fs';
10+
11+
const B = '\x1b[1m', D = '\x1b[2m', G = '\x1b[32m', R = '\x1b[31m', Y = '\x1b[33m', X = '\x1b[0m';
12+
const results = [];
13+
14+
function has(cmd, args = ['--version']) {
15+
return spawnSync(cmd, args, { stdio: 'ignore' }).status === 0;
16+
}
17+
18+
function gate({ name, required = true, skipIf, run, detail }) {
19+
if (skipIf) {
20+
results.push({ name, state: 'SKIP', detail: skipIf, required });
21+
return;
22+
}
23+
const t = Date.now();
24+
const ok = run();
25+
results.push({
26+
name,
27+
state: ok ? 'PASS' : 'FAIL',
28+
detail: `${detail} Β· ${((Date.now() - t) / 1000).toFixed(2)}s`,
29+
required,
30+
});
31+
}
32+
33+
const sh = (cmd, args) =>
34+
spawnSync(cmd, args, { stdio: ['ignore', 'pipe', 'pipe'], encoding: 'utf8' });
35+
36+
console.log(`\n${B}gavel Β· verify${X} ${D}every CI gate, no credentials, no network${X}\n`);
37+
38+
// ── 1 Β· the decision surface is pure ────────────────────────────────────────
39+
// Invariant I10. assemble.mjs is injected verbatim into a sandboxed code node,
40+
// so an impure call would fail at runtime on-chain. Asserted here and in CI.
41+
gate({
42+
name: 'Decision surface is pure',
43+
detail: 'src/assemble.mjs β€” no import/eval/fetch/Date.now/Math.random/process.env',
44+
run: () => {
45+
const s = readFileSync(new URL('../src/assemble.mjs', import.meta.url), 'utf8');
46+
const banned = [
47+
/^\s*import\s/m, /\bimport\s*\(/, /\beval\s*\(/, /\bnew\s+Function\s*\(/,
48+
/\bfetch\s*\(/, /\bMath\.random\s*\(/, /\bDate\.now\s*\(/, /\bprocess\.env/,
49+
];
50+
const hit = banned.filter((r) => r.test(s));
51+
if (hit.length) console.error(` ${R}impure:${X} ${hit.map(String).join(', ')}`);
52+
return hit.length === 0;
53+
},
54+
});
55+
56+
// ── 2 Β· the test suite ──────────────────────────────────────────────────────
57+
gate({
58+
name: 'JS test suite',
59+
detail: 'node --test',
60+
run: () => {
61+
const r = sh('node', ['--test']);
62+
const m = /^# pass (\d+)/m.exec(r.stdout ?? '');
63+
if (m) console.log(` ${D}${m[1]} tests passed${X}`);
64+
if (r.status !== 0) console.error(r.stdout?.slice(-1500) ?? '');
65+
return r.status === 0;
66+
},
67+
});
68+
69+
// ── 3 Β· the published numbers re-derive from committed data ─────────────────
70+
// This is the one that matters most: it proves the README's figures were
71+
// measured rather than asserted.
72+
gate({
73+
name: 'Published survey figures re-derive',
74+
detail: 'survey/rederive.py β€” every published figure vs 1.3 MB of committed responses',
75+
skipIf: has('python3') ? null : 'python3 not found β€” install it to check the survey numbers',
76+
run: () => {
77+
const r = sh('python3', ['survey/rederive.py']);
78+
const n = (r.stdout?.match(/^\s*ok\s/gm) ?? []).length;
79+
if (n) console.log(` ${D}${n} figures re-derived${X}`);
80+
if (r.status !== 0) console.error(r.stdout?.slice(-1200) ?? '');
81+
return r.status === 0;
82+
},
83+
});
84+
85+
// ── 4 Β· the rehearsal token ─────────────────────────────────────────────────
86+
// Optional: needs foundry. Dependency-free otherwise β€” no forge-std, no lib/.
87+
gate({
88+
name: 'Solidity tests',
89+
required: false,
90+
detail: 'forge test β€” MockUSDC, 100% coverage on all four metrics',
91+
skipIf: has('forge') ? null : 'foundry not installed β€” optional, JS gates cover the decision surface',
92+
run: () => sh('forge', ['test']).status === 0,
93+
});
94+
95+
// ── report ──────────────────────────────────────────────────────────────────
96+
const pad = Math.max(...results.map((r) => r.name.length));
97+
console.log('');
98+
for (const r of results) {
99+
const tag = r.state === 'PASS' ? `${G}PASS${X}` : r.state === 'FAIL' ? `${R}FAIL${X}` : `${Y}SKIP${X}`;
100+
console.log(` ${tag} ${r.name.padEnd(pad)} ${D}${r.detail}${X}`);
101+
}
102+
103+
const failed = results.filter((r) => r.state === 'FAIL' && r.required);
104+
const skipped = results.filter((r) => r.state === 'SKIP');
105+
console.log('');
106+
if (failed.length) {
107+
console.log(`${R}${B}VERIFY FAILED${X} β€” ${failed.length} required gate(s) did not pass.\n`);
108+
process.exit(1);
109+
}
110+
console.log(
111+
`${G}${B}VERIFY PASSED${X}` +
112+
(skipped.length ? ` ${Y}(${skipped.length} optional gate skipped)${X}` : '') +
113+
`\n${D}Nothing above touched the network or read a credential.${X}\n`,
114+
);

0 commit comments

Comments
Β (0)