Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .changeset/vale-config-schema.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,8 @@
"@taskless/cli": patch
---

`verify` validates a Vale rule's `.vale.ini` against a schema and names the constraint each rejection violates. The config is parsed into an ordered structure and checked there: an assignment above the first matcher, a matcher without its `tskl) rule` breadcrumb, a key naming another rule, a value other than `YES`/`NO`, a non-empty `BasedOnStyles`, a config with no matcher or no `YES`, and a `NO` matcher that precedes every `YES` are each rejected under a `vale-config-*` constraint that `verify --json` reports in `violations[]` and `reference.json` publishes. A repeated key, a `[*]` matcher, and a `.taskless/**` matcher are reported as a notice without failing the rule. `check` is unchanged.
`verify` validates a Vale rule's `.vale.ini` against a schema and names the constraint each rejection violates. The config is parsed into an ordered structure and checked there: an assignment above the first matcher, a matcher without its `tskl) rule` breadcrumb, a key naming another rule, a value other than `YES`/`NO`, a non-empty `BasedOnStyles`, a config with no matcher or no `YES`, and a `NO` matcher that precedes every `YES` (both judged by each matcher's final verdict, so a `YES` a later `NO` in the same matcher overrides does not count) are each rejected under a `vale-config-*` constraint that `verify --json` reports in `violations[]` and `reference.json` publishes. A repeated key, a `[*]` matcher, and a `.taskless/**` matcher are reported as a notice without failing the rule.

`check` runs the same schema before assembling the Vale run config, and a rejected config refuses the Vale engine for that run: the failure names the rule and the line, reaches the exit code, and ast-grep still runs. A rule is never silently left out of the assembled config. Accepted configs are written verbatim under their breadcrumb, so the one string edit assembly used to make (dropping a copied-in `StylesPath`) is gone; that line is now a rejection. Advisories reach `check`'s notices. To find every rejected line at once, run `taskless verify`.

This is still `patch`. The package is `0.y.z`, and every config the schema refuses was already being misread by Vale: a rule enabled nowhere with a `W101` on stderr, a rule silently overriding a neighbour, a disable the following enable cancelled. The release surfaces a defect the consumer already had rather than introducing one, the same call as linting files over 128 KB again in 0.11.3.
Original file line number Diff line number Diff line change
Expand Up @@ -10,9 +10,9 @@

## 2. Slice 2: assembly refusal, dispatch, recipe, ledger (tip)

- [ ] 2.1 In `rules/assemble.ts`, delete `ruleConfigBody` and `sectionPatternsOf`; `assembleValeConfig` validates each config, and on any rejection returns a refusal naming the rule and line without writing the file. Accepted configs are written verbatim under their breadcrumb; `sections` comes from the AST. Verify the "Assembly order is stable" test still asserts byte-identical output, and a new test asserts a foreign key leaves `.taskless/.vale.ini` unwritten.
- [ ] 2.2 In `rules/dispatch.ts`, a refusal becomes the Vale engine's `failure` (reaches the exit code); advisories join `notices`. Verify a mixed-engine test shows ast-grep results alongside the Vale failure and a non-zero exit.
- [ ] 2.3 Recipe `packages/cli/src/agent/create-vale-rule.md`: state that the config is schema-checked, list what is rejected and what is advised, drop any `.taskless/**` matcher from its examples, and bump the topic version. Verify `pnpm build && pnpm cli agent create-vale-rule` renders and `pnpm cli check` is clean over the prose.
- [ ] 2.4 `packages/cli/src/agent/update.md`: 0.11.3 ledger entry — a config `check` used to tolerate now refuses the Vale run, with the `verify` command that names the line; bump the topic version.
- [ ] 2.5 Extend the changeset with the refusal. Run `pnpm typecheck`, `pnpm lint`, `pnpm --filter @taskless/cli test`; open PR 2 against PR 1's branch.
- [ ] 2.6 Archive on this tip: wip commit, `pnpm openspec archive vale-config-schema -y`, confirm every prior scenario in `cli-vale-rule-engine` (23 before, 32 after) and `cli-rule-validation` (26 before, 30 after) survives, reset to the wip SHA, then archive for real.
- [x] 2.1 In `rules/assemble.ts`, delete `ruleConfigBody` and `sectionPatternsOf`; `assembleValeConfig` validates each config, and on any rejection returns a refusal naming the rule and line without writing the file. Accepted configs are written verbatim under their breadcrumb; `sections` comes from the AST. Verify the "Assembly order is stable" test still asserts byte-identical output, and a new test asserts a foreign key leaves `.taskless/.vale.ini` unwritten.
- [x] 2.2 In `rules/dispatch.ts`, a refusal becomes the Vale engine's `failure` (reaches the exit code); advisories join `notices`. Verify a mixed-engine test shows ast-grep results alongside the Vale failure and a non-zero exit.
- [x] 2.3 Recipe `packages/cli/src/agent/create-vale-rule.md`: state that the config is schema-checked, list what is rejected and what is advised, drop any `.taskless/**` matcher from its examples, and bump the topic version. Verify `pnpm build && pnpm cli agent create-vale-rule` renders and `pnpm cli check` is clean over the prose.
- [x] 2.4 `packages/cli/src/agent/update.md`: 0.11.3 ledger entry — a config `check` used to tolerate now refuses the Vale run, with the `verify` command that names the line; bump the topic version.
- [x] 2.5 Extend the changeset with the refusal. Run `pnpm typecheck`, `pnpm lint`, `pnpm --filter @taskless/cli test`; open PR 2 against PR 1's branch.
- [x] 2.6 Archive on this tip: wip commit, `pnpm openspec archive vale-config-schema -y`, confirm every prior scenario in `cli-vale-rule-engine` (23 before, 32 after) and `cli-rule-validation` (26 before, 30 after) survives, reset to the wip SHA, then archive for real.
42 changes: 36 additions & 6 deletions openspec/specs/cli-rule-validation/spec.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,11 +42,11 @@ The two commands split because they have different preconditions. An agent part-

Per engine, `verify` SHALL check:

| Engine | Components |
| --------- | ---------------------------------------------------------------------------------------------------- |
| `sg` | `<id>.yml` against the ast-grep schema and the Taskless required fields |
| `vale` | `<id>.yml` against the Vale rule schema and the Taskless required fields, and the rule's `.vale.ini` |
| `runtime` | `check.ts` present, and at least one capture rule under `captures/` |
| Engine | Components |
| --------- | ------------------------------------------------------------------------------------------------------------------------------ |
| `sg` | `<id>.yml` against the ast-grep schema and the Taskless required fields |
| `vale` | `<id>.yml` against the Vale rule schema and the Taskless required fields, and the rule's `.vale.ini` against the config schema |
| `runtime` | `check.ts` present, and at least one capture rule under `captures/` |

The `vale` row previously read "against Vale's own validation." Measured against the pinned 3.18.0 binary, that covers less than it claims: `level: bananas` is reported, while `extends: nonsense` and `scope: fenced` both verify clean and produce a rule that matches nothing. Vale validates a rule when it _runs_ one, and it runs one field at a time — so a name it does not recognize is not an error, it is a check that never fires. Schema validation is therefore its own layer for `vale`, as it already is for `sg`.

Expand Down Expand Up @@ -80,6 +80,29 @@ The `vale` row previously read "against Vale's own validation." Measured against

The failure it prevents is not a local one: Vale reports this as `E201: has invalid keys` and reads one assembled config per run, so a single rule with a stray field suppresses every other Vale rule's findings.

#### Scenario: A rule config that never enables the rule is rejected

- **WHEN** a Vale rule's `.vale.ini` declares matchers but no `<id>.<id> = YES`
- **THEN** `verify` SHALL report that the rule is present but off, naming the file

#### Scenario: A rule config that assigns a foreign key is rejected

- **WHEN** a Vale rule's `.vale.ini` assigns a `<style>.<check>` key naming a different rule
- **THEN** `verify` SHALL report it, naming the key and the line
- **AND** it SHALL NOT report the rule as valid

#### Scenario: A rule config advisory does not fail verify

- **WHEN** a Vale rule's `.vale.ini` assigns the same key twice inside one matcher, and another matcher still enables the rule
- **THEN** `verify` SHALL report the rule as valid
- **AND** the repeat SHALL be printed as a notice on that rule

#### Scenario: A rule config whose only enable is overridden is rejected

- **WHEN** a Vale rule's `.vale.ini` assigns `<id>.<id> = YES` and then `<id>.<id> = NO` in its only matcher
- **THEN** `verify` SHALL report that the rule is present but off, under `vale-config-enabled-somewhere`
- **AND** the repeat SHALL still be printed as a notice on that rule

### Requirement: Test runs a rule's fixtures and runs verify first

`test` SHALL execute a rule against its test material — ast-grep test cases, Vale `pass`/`fail` fixture buckets, or the runtime harness — and SHALL run `verify` first, stopping on a verify failure without running the fixtures.
Expand Down Expand Up @@ -209,7 +232,8 @@ A documented value that never fires is a trap an author walks into with the docs

`verify --json` and `test --json` SHALL report, per rule, the constraints a
rejection violated, pairing a `constraintId` drawn from the published
`RULE_CONSTRAINTS` with the message that reports it.
`RULE_CONSTRAINTS` with the message that reports it. Vale config rejections are
attributable in the same way, under `vale-config-*` constraint ids.

The existing `errors` array SHALL continue to carry every failure message,
including those that are attributable. A consumer reading only `errors` SHALL
Expand Down Expand Up @@ -239,3 +263,9 @@ error message is not a breaking change.

- **WHEN** `verify --json` accepts a rule
- **THEN** its violations SHALL be empty

#### Scenario: A Vale config rejection is attributed

- **WHEN** `verify --json` rejects a Vale rule whose config assigns a key naming another rule
- **THEN** the rule's result SHALL carry a violation with `constraintId` `vale-config-own-key-only`
- **AND** the violation's message SHALL also appear in `errors`
Loading
Loading