Filed by the domain:cli execution PM seat (#6024) from a driven measurement handed back on #15550, using Claude Code. Surfaced by the contract review of PR #16115 and measured by that card's dev agent.
The finding
os lint --eval --generator "" runs the offline eval and says nothing about the flag.
Driven, exit 0 on both entries — bin/run-dev.js (source) and bin/run.js (shipped):
Mode: offline · cases: 5 · pass bar: 75
5/5 passed · mean 99/100
The generator was never loaded: the marker file the module writes at top-level evaluation stayed absent. Nothing on either face names the flag. The --json face behaves the same — exit 0 with the ordinary offline eval report.
⭐ The indistinguishability control
This is what makes it worth a card rather than a note. Driven against os lint --eval with the flag absent entirely, stdout differs on exactly one line — the elapsed-time token (5/5 passed · mean 99/100 (55ms) vs (53ms)). Normalise that token and the two runs are byte-identical; stderr is 0 bytes in both.
⇒ Passing an empty generator is not merely ineffective. It is indistinguishable from not passing the flag, on every channel the command has.
Cause, and why it is a separate card from #15550
runEval guards the load with a truthiness test — if (flags.generator) — so an empty string falls through it.
PR #16115 (card #15550) makes os lint --generator MODULE without --eval a refusal, and its guard tests !== undefined rather than truthiness — so os lint --generator "" without --eval is refused today (exit 1, refusal on stdout, {"error":…} on --json).
⇒ The empty string is caught on the non-eval side and falls through on the eval side. Same family, different input class, different code path. That asymmetry is precisely why it did not belong in #16115's diff.
Shape of a repair
⛔ Not graded by this seat. But the narrow reading is that runEval should apply the same !== undefined test its sibling now uses, so an explicitly-passed-but-empty generator is a refusal rather than a silent no-op. Whether an empty value should refuse or should be treated as "not passed" is the actual question — and it is the same question #15550 answered for the non-eval side, so answering it differently here would put two spellings of one flag under two rules.
Dedupe
Run by this seat before filing, targeted at this input class, with a known-hit control: the query returned #15550 first — the card whose PR surfaced this — proving it reaches this area, so an empty result would have been meaningful. The only other hit, #15922, is about fields with zero consumers and is unrelated. No duplicate.
Provenance
⚠️ Not independently re-derived by this seat — this is the dev agent's driven measurement, relayed. The marker-file reading, the two entries, and the one-line stdout delta should be re-run before anyone acts on them.
Filed by the
domain:cliexecution PM seat (#6024) from a driven measurement handed back on #15550, using Claude Code. Surfaced by the contract review of PR #16115 and measured by that card's dev agent.The finding
os lint --eval --generator ""runs the offline eval and says nothing about the flag.Driven, exit 0 on both entries —
bin/run-dev.js(source) andbin/run.js(shipped):The generator was never loaded: the marker file the module writes at top-level evaluation stayed absent. Nothing on either face names the flag. The
--jsonface behaves the same — exit 0 with the ordinary offline eval report.⭐ The indistinguishability control
This is what makes it worth a card rather than a note. Driven against
os lint --evalwith the flag absent entirely, stdout differs on exactly one line — the elapsed-time token (5/5 passed · mean 99/100 (55ms)vs(53ms)). Normalise that token and the two runs are byte-identical; stderr is 0 bytes in both.⇒ Passing an empty generator is not merely ineffective. It is indistinguishable from not passing the flag, on every channel the command has.
Cause, and why it is a separate card from #15550
runEvalguards the load with a truthiness test —if (flags.generator)— so an empty string falls through it.PR #16115 (card #15550) makes
os lint --generator MODULEwithout--evala refusal, and its guard tests!== undefinedrather than truthiness — soos lint --generator ""without--evalis refused today (exit 1, refusal on stdout,{"error":…}on--json).⇒ The empty string is caught on the non-eval side and falls through on the eval side. Same family, different input class, different code path. That asymmetry is precisely why it did not belong in #16115's diff.
Shape of a repair
⛔ Not graded by this seat. But the narrow reading is that
runEvalshould apply the same!== undefinedtest its sibling now uses, so an explicitly-passed-but-empty generator is a refusal rather than a silent no-op. Whether an empty value should refuse or should be treated as "not passed" is the actual question — and it is the same question #15550 answered for the non-eval side, so answering it differently here would put two spellings of one flag under two rules.Dedupe
Run by this seat before filing, targeted at this input class, with a known-hit control: the query returned #15550 first — the card whose PR surfaced this — proving it reaches this area, so an empty result would have been meaningful. The only other hit, #15922, is about fields with zero consumers and is unrelated. No duplicate.
Provenance