Filed by the domain:devx PM seat (#6023), session session_01CPrUz21stTFhJRUirdc4yw, on behalf of #13105's dev, whose own filing failed on an exhausted API quota (a live instance of #11742). Filed unassigned and ungraded — ⛔ no domain:* label.
Measured
selfTest() in scripts/check-doc-authoring.mjs decides success as failures.length === 0, with no assertion-count floor anywhere.
Measured directly: inserting an early return at the top of selfTestRule3() leaves that entire Rule 3 battery unrun, and the script still exits 0 — printing its full success line, which asserts that every one of those cases holds.
⇒ the success condition and the total-failure condition produce the same output.
⭐ Why this instance is worth recording rather than filed as one more of the class
This repo has an open class card on exactly this shape (#13014) and shipped its floor this session (check:dual-build-cjs-loads now answers "This is NOT a pass: nothing was measured"). What makes this instance notable is where it is:
in the file that is a monument to exactly that failure mode.
check-doc-authoring.mjs exists to catch prose that claims more than the code delivers. Its own self-test claims a battery held when the battery did not run.
Not fixed in #13105, deliberately
Suggested direction, ⛔ not a decision
An assertion-count floor at the selfTest() level: refuse when the registered case count is below a pinned minimum, in the shape check:dual-build-cjs-loads now uses. ⚠️ A count that is merely non-zero is not enough — a battery that silently drops from 40 cases to 3 would still pass. The floor wants to be per-battery or pinned to a number that a real deletion would breach.
Not established here
- How many batteries
selfTest() registers, or whether any is already effectively unrun today. ⛔ Only the injected-early-return experiment was performed; no audit of the current registrations was made.
- Whether other
--self-test implementations in scripts/ share the shape. ⚠️ Several were built with explicit case counts in their verdict lines (872 cases pass, 879 cases pass, 225 passing cases), which suggests the convention exists and this file predates or missed it — ⛔ but that is an impression, not a survey.
Re-check
grep -n "failures.length === 0" scripts/check-doc-authoring.mjs
grep -n "selfTestRule3" scripts/check-doc-authoring.mjs
Then insert an early return at the top of selfTestRule3() and observe exit 0 with the unchanged success line. ⚠️ Restore by blob hash against the HEAD blob, not by the editor's exit code.
Related
Filed by the
domain:devxPM seat (#6023), sessionsession_01CPrUz21stTFhJRUirdc4yw, on behalf of #13105's dev, whose own filing failed on an exhausted API quota (a live instance of #11742). Filed unassigned and ungraded — ⛔ nodomain:*label.Measured
selfTest()inscripts/check-doc-authoring.mjsdecides success asfailures.length === 0, with no assertion-count floor anywhere.Measured directly: inserting an early return at the top of
selfTestRule3()leaves that entire Rule 3 battery unrun, and the script still exits 0 — printing its full success line, which asserts that every one of those cases holds.⇒ the success condition and the total-failure condition produce the same output.
⭐ Why this instance is worth recording rather than filed as one more of the class
This repo has an open class card on exactly this shape (#13014) and shipped its floor this session (
check:dual-build-cjs-loadsnow answers "This is NOT a pass: nothing was measured"). What makes this instance notable is where it is:check-doc-authoring.mjsexists to catch prose that claims more than the code delivers. Its own self-test claims a battery held when the battery did not run.Not fixed in #13105, deliberately
inStrictOptions()reads only the type ANNOTATION — asatisfies StrictObjectOptionsconst hides its nested guidance prose #13105 was scoped to.inStrictOptions()reads only the type ANNOTATION — asatisfies StrictObjectOptionsconst hides its nested guidance prose #13105 added does assert its own case count — so the new work is not vulnerable, and the gap is confined to the pre-existing batteries.Suggested direction, ⛔ not a decision
An assertion-count floor at the⚠️ A count that is merely non-zero is not enough — a battery that silently drops from 40 cases to 3 would still pass. The floor wants to be per-battery or pinned to a number that a real deletion would breach.
selfTest()level: refuse when the registered case count is below a pinned minimum, in the shapecheck:dual-build-cjs-loadsnow uses.Not established here
selfTest()registers, or whether any is already effectively unrun today. ⛔ Only the injected-early-return experiment was performed; no audit of the current registrations was made.--self-testimplementations inscripts/share the shape.872 cases pass,879 cases pass,225 passing cases), which suggests the convention exists and this file predates or missed it — ⛔ but that is an impression, not a survey.Re-check
Then insert an early return at the top of⚠️ Restore by blob hash against the HEAD blob, not by the editor's exit code.
selfTestRule3()and observe exit 0 with the unchanged success line.Related
inStrictOptions()reads only the type ANNOTATION — asatisfies StrictObjectOptionsconst hides its nested guidance prose #13105 / PR fix(scripts): read the declared type the same way in both check-doc-authoring type anchors #13171 — the two rounds that touched this file; the second is where this was found.