Skip to content

Ratchet the inline noqa comments, so their number can only fall - #75

Merged
anth-volk merged 7 commits into
mainfrom
claude/noqa-ratchet
Nov 3, 2026
Merged

anth-volk merged 7 commits into
mainfrom
claude/noqa-ratchet

Conversation

@anth-volk

@anth-volk anth-volk commented Nov 3, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #73

What this does

The total number of inline suppressions may not grow: a ratcheted rule lists every suppression comment Ruff applies, and its count may only fall against the base branch, so a pull request may add a suppression only by removing another. As Anthony decided in round 1, the comparison counts comments only: it does not stop a swap at the same total, a renamed file, or a code added to an existing comment, once the baseline lists the result. 16.1 · 16 in the plan. It was stacked on #71 (gh stack #76); #71 has merged, and this pull request now targets main.

Starting count: 140 suppression comments in 49 Python files, all # noqa, on #71's final head 6925db1, which is also main now that #71 has merged. The most common codes are BLE001 (60), T201 (16), PLR0913 (15), S105 (13), ARG002 (12) and TRY004 (8). Before #71, main had 125 in 42 files, after #69's merge, so #71 added 15 in 7 files. The repository has none of Ruff's own forms.

  • The guard, scripts/rules/noqa_comments.py (noqa-comments):
    • What it lists: each suppression comment Ruff applies in a tracked Python file, as <path>: <codes> with the codes sorted, or <path>: (all) for a blanket one. That covers an inline # noqa; Ruff's ruff: ignore[...], ruff: disable[...] and ruff: file-ignore[...], labelled by form (an enable[...] is not one); and an own-line file-level ruff: noqa or flake8: noqa (round 1, bug 1). It finds comments with tokenize, so text in a string doesn't count, and neither does a comment that names noqa without the directive.
    • How it reads codes: as Ruff's lexer does (round 1). BLE001S110 is two codes (bug 3), and # noqa : BLE001 names BLE001. A directive Ruff rejects and does not apply is not counted: lowercase codes, or a file-level ruff: noqa after code.
    • The comparison: it counts each entry and compares it with the rule's baseline file. It fails on a comment the baseline does not list, and on an entry with fewer comments left than the baseline lists, so the baseline changes with the comments. The baseline's total is what the base-branch comparison checks.
    • Moves: a comment moved within its file still matches, because entries carry no line numbers.
    • Packaging: it uses only the standard library, like the rest of the rules tool, and run_quality_guards.py runs it.
  • The rule, emitter.quality.inline-suppressions-only-shrink:
    • Baseline: ratcheted, with [rule.baseline] naming rules/baselines/noqa-comments.txt, which holds 140 entries, one per comment.
    • The total: the rules tool's existing baseline comparison refuses a count that grows against the base branch, as the Public API uses baselines, with or without a [rule.downgrade]. The tool already supported ratcheted rules and that comparison, so only the guard is new. One helper, baseline.baseline_lines, now reads every baseline file's entries, with an explicit encoding (round 1).
    • Proof: the rule has a proof, two TOML manifests the proof test writes into a temporary directory. They are TOML rather than Python, so their comments are not counted here.
  • A downgrade, naming Reduce the emitter's inline noqa comments #74. The rules tool refuses a new rule that starts ratcheted with tolerated entries unless it has a [rule.downgrade] naming an open issue. I opened Reduce the emitter's inline noqa comments #74, "Reduce the emitter's inline noqa comments", for that purpose.
  • For Anthony to confirm: the registry guide reserves both for a person: the [rule.downgrade] section naming Reduce the emitter's inline noqa comments #74, and the 140-entry baseline. The tracker task asks for this ratcheted rule, so I added both; change the reason, the issue or the baseline if you prefer.
  • The README's development section and the registry guide say how the rule works. The guide's advice on inline suppressions now follows the tool: a new justified suppression needs another removed in the same pull request, and raising the total is a person's decision, since the guard refuses a grown count with or without a downgrade.

Test-first record

Commit Result
b9bd74f The tests and proof fixtures Red: python -m unittest scripts.rules.tests.test_noqa_comments fails to import scripts.rules.noqa_comments.
9c76f80 The guard, the rule, the baseline and the docs Green: the file's four test classes pass, run by class (proof; new and removed suppressions; comment parsing; this repository's baseline matching every comment). All six guards pass with GITHUB_BASE_REF=main, and guard --base-ref origin/claude/ruff-complexity passes with the notes that the rule is new, starts ratcheted with 140 entries, and that #74 is open. ruff check and ruff format --check pass on the changed Python files.

Review round 1:

Commit Result
d3883ae Merge of main #75's branch was rebased onto #71's merge outside this session; this brings main's release. render changed nothing, and the guards pass.
b9aa036 Tests Red, on assertions: 7 of the 8 tests for Ruff's forms and code reading fail. The guard returns no entry for the Ruff forms, one code for BLE001S110, and (all) for # noqa : BLE001 and lowercase codes. The same-total tests pin Anthony's decision and pass: an added code, a swapped code and a rename pass at the same total, a higher total fails, and a comment moved between two versions of a file keeps its entry.
cde122e Ruff's forms, Ruff's code reading, one baseline reader Green: all seven test classes pass by name, including this repository's baseline, still 140.
cde3bbd Test of the count-only message Red, on assertions: the proof test and test_a_new_noqa_comment_fails expect the new wording.
0deef32 The wording Green: all seven classes pass. The six guards pass with GITHUB_BASE_REF=main, and guard --base-ref origin/main passes. ruff check passes on the changed Python files.

No other test was run.

CI

No workflow, trigger or script changes. The guard runs in the existing "Rules registry" job (pr.yaml's rules job, through run_quality_guards.py). The proof test runs in that job's unittest discover -s scripts/rules/tests. The downgrade's issue lookup uses that step's existing GH_TOKEN, which the step's env: sets from github.token. No new environment variable or secret.

Release

The fragment is changelog.d/noqa-ratchet.changed.md, a patch bump: no shipped code changes, so the wheels are unchanged.

🤖 Generated with Claude Code

@anth-volk
anth-volk added this pull request to stack #76 November 3, 2026 20:47
Base automatically changed from claude/ruff-complexity to main November 3, 2026 21:15
anth-volk and others added 2 commits November 4, 2026 01:15
The noqa-comments guard should list each `# noqa` comment in a tracked
Python file as `<path>: <codes>`, compare them with the rule's baseline
file as a count per entry, and fail on a comment the baseline does not
list (a new suppression) and on an entry with no comment left (asking
for the baseline to shrink). A move within the file still matches. One
test checks that this repository's baseline lists every current comment,
and the proof fixtures are TOML manifests, so the guard does not count
their comments. All fail now: scripts.rules.noqa_comments does not exist.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- scripts/rules/noqa_comments.py, the noqa-comments quality guard,
  lists each inline `noqa` comment in a tracked Python file as
  `<path>: <codes>` (tokenize finds comments only) and compares the
  counts with the rule's baseline: an unlisted comment fails, and so
  does an entry with fewer comments left than it lists.
- The ratcheted rule emitter.quality.inline-suppressions-only-shrink
  lists the 140 comments in 49 files in
  rules/baselines/noqa-comments.txt; the rules guard already refuses a
  baseline count that grows against the base branch. As a new rule that
  starts ratcheted with entries, it names #74 in [rule.downgrade].
- run_quality_guards.py runs the guard, the README says how the rule
  works, and the rules views are rendered.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk force-pushed the claude/noqa-ratchet branch from 5406846 to a5bdc0c Compare November 3, 2026 21:15
anth-volk and others added 5 commits November 4, 2026 01:19
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…cases

- Ruff's `# ruff: ignore[...]`, `# ruff: disable[...]` (its `enable`
  is not one), `# ruff: file-ignore[...]` and own-line `# ruff: noqa`
  and `# flake8: noqa` are entries, labelled by form; a file-level
  `# ruff: noqa` after code, which Ruff does not apply, is not.
- Codes are read as Ruff reads them: `BLE001S110` is two codes, a space
  before the colon still names the codes, and lowercase codes, which
  Ruff rejects and does not apply, are not counted.
- As Anthony decided, the base-branch comparison counts comments only:
  an added code, a swapped code and a renamed file pass at the same
  total, a higher total fails, and a comment moved between two versions
  of a file keeps its entry.

Red: 7 of the 8 new form and code tests fail on their assertions (the
guard returns no entry for the Ruff forms, one code for the squashed
pair, and `(all)` for the other two); the same-total tests pass.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- The guard counts Ruff's ignore[...], disable[...] and file-ignore[...]
  comments and an own-line file-level ruff: noqa or flake8: noqa,
  labelled by form; an enable[...] and a file-level noqa after code,
  which Ruff does not apply, are not counted.
- codes_after reads codes as Ruff's lexer does: BLE001S110 is two codes,
  a space before the colon still names the codes, and a directive whose
  codes Ruff rejects, such as lowercase ones, applies nothing and is not
  counted.
- One helper, baseline.baseline_lines, reads every baseline file's
  entries, for the guard, validation and the ci-scripts baselines, each
  with an explicit UTF-8 encoding.

The tests from b9aa036 pass, as does this repository's baseline, still
140 entries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
As Anthony decided, the base-branch comparison counts comments only, so
a new suppression is allowed when another goes. The message for an
unlisted comment should say so: fix the finding, or remove another
suppression and list this one, since the total may not grow. Red: the
proof test and test_a_new_noqa_comment_fails fail on the old wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ot grow

As Anthony decided, the base-branch comparison stays count only, so the
wording now matches what the check does:

- The rule statement, README, changelog fragment and the guard's
  message say the total may not grow: a pull request may add a
  suppression only by removing another, and the check does not stop a
  swap at the same total, a rename or a code added to an existing
  comment. The statement puts `<path>: <codes>` in backticks, so the
  generated document shows it.
- The registry guide says a new justified suppression needs another
  removed in the same pull request, and raising the total is a person's
  decision, since the guard refuses a grown baseline count with or
  without a [rule.downgrade].
- The baseline file's header names Ruff's own forms.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@anth-volk
anth-volk marked this pull request as ready for review November 3, 2026 21:57
@anth-volk
anth-volk merged commit 3009068 into main Nov 3, 2026
22 of 24 checks passed
@anth-volk
anth-volk deleted the claude/noqa-ratchet branch November 3, 2026 21:57
anth-volk added a commit that referenced this pull request Nov 4, 2026
Count the remaining suppression forms and say the total may not grow (follow-up to #75)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Ratchet the emitter's inline noqa comments, so their number can only fall

1 participant