Repository navigation
create-vale-rule: a rule about a command passes its fixtures and catches nothing #167
Description
Activity
Two measurements from using the rule in anger, both of which change what the recipe should recommend.
scopeis per-rule, and onerawrule does not affect the othersWorth stating explicitly, because Vale assembles one config for the whole run and it is reasonable to assume the scopes interact. They do not. Same document, same run:
rule scope prose inline code fenced block no-hedgingdefault fires — quiet docs-npx-cli[raw, code, text]fires fires fires The scope table, measured
rawsubsumes the other two, so[raw, code, text]and a barerawbehave identically.scopeprose inline code fenced block text✓ code✓ [code, text]✓ ✓ raw✓ ✓ ✓ rawand Vale's suppression comments are mutually exclusiveThis is the part worth documenting, because the obvious escape hatch is not available:
scope<!-- vale Style.Rule = NO -->honored[raw, code, text]no [code, text]yes Under
rawVale reads the unparsed text, so a markup-level directive is invisible to it. A rule that reaches into fenced blocks therefore cannot be annotated away on a case-by-case basis, and a command rule needsrawprecisely because every command in a README is fenced.The practical consequence, hit on
taskless/cliitself: two READMEs documentednpxandpnpm dlxside by side as deliberate alternatives. The rule fires on thepnpm dlxline, the directive cannot silence it, and the only resolutions are to change the docs or accept a permanent finding. We changed the docs, which was right here — but a repository that genuinely wants to document both package managers has no way to express that.So the worked rule this issue proposes should carry the trade-off, not just the scope value:
rawbuys fenced-block coverage and spends the ability to annotate an exception.Folded the general scope findings into #170, which collects Vale authoring pitfalls that pass
verifyandtestand are only visible when someone notices a rule has never reported anything. This issue stays the specific case: a rule about a command, inert because commands are fenced.Delivered by #174 (
a596e54, "docs(cli): close the create-vale-rule gaps that produce a silent rule"), merged tomain. Verified againstpackages/cli/src/agent/create-vale-rule.txtonmainrather than inferred from the PR title.Everything this issue asked for is in the recipe:
- The false-negative framing. The markup notes no longer present code-span skipping only as false-positive protection. Step 2 now states that a rule about a command, flag, package name or env var has a subject that lives in fenced blocks in every real README, and that the default scope cannot see fenced blocks at all.
- The fixture instruction. "When the rule's subject normally appears in code, the
fail/fixture must carry it three ways" — inline in a code span, inside a fenced block, and in ordinary prose, in one document. That is exactly the check this issue identified as the one that would have caughtdocs-npx-clibefore it shipped. - The measurement, carried rather than described. "the default scope found one of three,
rawfound three", with the diagnostic rule stated in the form an author can act on: if the fixture fires on the prose line and not the other two, the answer isscope: raw. - The cost is stated too, which the issue did not ask for: a
raw-scoped rule cannot be silenced by an in-document Vale directive, because directives are applied to the parsed document andrawreads the unparsed one.
One refinement worth recording. This issue proposed
scope: [raw, code, text], which is whatdocs-npx-cliwas authored with. The recipe landed onscope: rawinstead, becauserawwas measured to subsumecodeandtext: on one document holding the token in all three places,textfound one,codefound one,[code, text]found two, andrawfound all three. So the list form is not wrong, just redundant —rawis not "a bit wider", it is everything.The general scope findings live in #170 as noted above; this issue's specific case is closed.
Summary
The
create-vale-rulerecipe leads an author straight into a rule that is green onverify, green ontest, and catches zero real violations — because in markdown Vale skips code spans and fenced blocks by default, and a rule about a shell command is a rule about text that is almost always inside one.What happened
Authoring
docs-npx-cli(flagpnpm dlx @taskless/cli, prefernpx @taskless/cli) by following the recipe:substitutionrule and fixtures per the worked examples.verifyok,testok.checkagainst the real repository: zero findings, despite two READMEs containing the exact string — both inside```bashblocks.scope: inline code not flagged, fenced block not flagged, plain prose flagged.scope: raw,scope: code, andscope: texteach alone still missed at least one form. Onlyscope: [raw, code, text]caught all three — and then the two real violations surfaced.Why the recipe does not prevent this
It does say "URLs and code spans are not prose" under the markup notes, which is accurate. But:
`Github/docs`"), not as the false-negative it becomes when the rule's whole subject is a command.substitutionwith no scope guidance, andscopeis introduced later as a way to narrow, never to widen into code.pass/fixture must contain the same phrase outside that place") catches over-firing. Nothing prompts the author to put the phrase in a code span infail/, which is what would have caught this.The result is a rule that passes every local gate and is inert.
Suggested change
Add a worked rule for the command/identifier case — the one where a multi-scope value is required:
With the "goes wrong" line the other nine have: omitting the scope. Measured, the rule then passes its fixtures and reports nothing on a real repository, because every command in a README is in a code span or a fenced block.
Worth pairing with a fixture instruction: when a rule is about something that normally appears in code, the
fail/document must contain it inline, fenced, and in prose. That is the check that would have caught this before it shipped.