Skip to content

Set Scoped's obligation by weight rather than by coverage - #372

Merged
ikuwow merged 3 commits into
mainfrom
scoped-drop-completeness
Aug 20, 2026
Merged

Set Scoped's obligation by weight rather than by coverage#372
ikuwow merged 3 commits into
mainfrom
scoped-drop-completeness

Conversation

@ikuwow

@ikuwow ikuwow commented Aug 20, 2026

Copy link
Copy Markdown
Owner

Purpose

Scoped asked that every part of the diff sit inside the stated intent. That is a claim about each part, and a claim about each part gets discharged by whatever enumeration answers it most cheaply — naming the changed files answers it, and a change inside a file the body already names is then outside the property's reach. The property came to test whether the body listed the areas it touched rather than whether it conveyed the change.

Key changes

  • Scoped now asks that a reader take the whole of the change from the body: the intent behind it, the parts needed to hold that intent, and the rest left to the diff
    • the test is whether the diff matches what the body led the reader to expect, so a list of files does not answer it, and no threshold is needed to say which parts must appear
    • what is given up is the per-part obligation: the body is no longer asked to account for all of the diff, and a change a reader was already prepared for needs no line of its own
  • /pr-selfcheck's Fix condition asked the same question per part and moves with it, so the rule and the checker's blocking condition state one requirement

Evidence this rests on

Three rounds of /pr-selfcheck against frozen PRs are recorded in #371, which is a measurement branch and is not proposed for merge. Rounds 1 and 2 ran the rules as they stand, over sixteen runs between them; round 3 ran a different candidate.

Fourteen of those sixteen runs resolved Scoped to the changed-file list, unanimously across round 1's ten. The two that read the diff finer are the two that found a rule the diff added and the body named nowhere. That observation is what this change acts on, and it holds whichever base a run compared against, which is why it is the only measurement cited here.

Round 3's recall figures are not usable and neither support this change nor contradict it. Its fixture lets a run compare against a main that already carries the change under test, and across it and round 2 five of twelve runs took that comparison and none of them reached the addition.

Verification

  • The per-part obligation is gone from both sites that carried it → git grep -n -E 'accounts for all of it|every part of the diff|the body never accounts for' over the tree returns nothing, against one hit each before the change
    • across the whole diff no longer appears in the rule; the property is stated as one about the body rather than about each part
  • The property definition, the rule, and the severity table state the same requirement, checked by reading the three against each other
  • Every figure cited above is reproducible from Measure how much of pr-selfcheck output survives repetition #371's branch → rounds 1 to 3 and the preregistration are committed there
  • Whether the change moves what /pr-selfcheck reports is not measured here — the fixture that would measure it is confounded, and the intended check is to watch live PRs after merge

Notes

An earlier revision of this branch bounded the obligation by a defined threshold instead of removing it, naming the tests a change had to meet to be owed a mention. It is worth recording because it was rejected on a design ground rather than a measured one: a rule that defines a threshold and its exemptions builds a per-change adjudication, which is the shape that produced the file-list reading in the first place. The wording that shipped states the property and leaves the judgement where it was.

That revision also carried one line this one does not — that naming the file a change sits in does not name the change. It closed the measured failure directly, and its loss is the part of this change most worth watching: whether the property alone keeps a body from answering with the areas it touched.

#369 records the instability that prompted this work and is not closed by this change. Run-to-run disagreement is round 1's result, which involves no planted defect and no base selection: no defect was reported by all ten runs, and four of five were reported by exactly one.

`Scoped` asked that the body account for all of the diff and that every part
of it sit inside the stated intent. A universal obligation is discharged at
whatever unit makes it cheapest, and across sixteen recorded runs of
`/pr-selfcheck` fourteen discharged it against the changed-file list, which
any body naming its changed areas satisfies whatever those files hold. The two
runs that read the diff finer are the two that found a rule one of those files
added and the body named nowhere.

The obligation now reaches the changes that carry weight — one that moves
behavior, one large enough to redirect a review — and says a mechanical
consequence or an edit too small to change what a reviewer would do is owed no
mention. The threshold is what the property turns on, so the file list no
longer answers it, and the body is no longer pushed toward the diff paraphrase
`Necessary` forbids.

The severity table's `Fix` condition carried the same coverage framing and
moves with it, so the checker's blocking condition and the rule state one
requirement rather than two.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
ikuwow and others added 2 commits August 20, 2026 17:40
The rule reached weight through two tests that do not coincide: a change
large enough to redirect how a reviewer reads the diff, and, on the exempting
side, one too small to change what a reviewer would do. A change that redirects
attention without changing a decision satisfies one and fails the other, and
two readers land on opposite verdicts. Weight is now one test — a reviewer
would do something different for knowing the change — with behavior and
consequence as the reasons it holds rather than as separate thresholds.

"Moves behavior" also read against the file's other uses of the word, where a
change moves behind a link and an author moves the body or the diff, so a
literal reading made it the relocation that carries no weight at all.

The bullet on where weight hides now leads with what it exists to rule out,
since naming the containing file is the reading it has to close.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The property asked that every part of the diff sit inside the stated intent,
which is a claim about each part and is discharged by whatever enumeration
answers it most cheaply. Naming the changed files answers it, and a change
inside a file the body names is then outside the property's reach.

`Scoped` now asks what the body is for: that a reader take the whole of the
change from it, at the altitude where the parts needed to hold the intent are
visible and the rest belongs to the diff. The test is whether the diff matches
what the body led the reader to expect, so a file list does not answer it —
a list of files conveys no change — and no threshold is needed to say which
parts must appear.

The severity table's `Fix` condition asked the same question per part and
moves with it.

Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant