Skip to content

doc_section_loss_check: report the SIZE of a lost section, and say what it cannot see - #2345

Merged
bong-water-water-bong merged 1 commit into
mainfrom
lane/doc-loss-size
Nov 13, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
lane/doc-loss-size

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Found by using the merged tool on engine#2183: it reported "2 of 643 section(s) missing" while git diff --stat said the branch removed 694 lines of docs/npu.md. Both headings were real — the other 592 lines belonged to sections whose headings survived at both ends of the hole, which a heading-set comparison cannot see. So the count undersold the damage, which is the one thing a report like this must not do.

before:  "A pack with both families: ..." is at <base> and nowhere at <head>
after:   "A pack with both families: ..." is at <base> and nowhere at <head>
         (lines 801-863, 63 line(s), ####)
         ... 2 of 643 section(s) at <base> are missing at <head>, 104 line(s) ...
         this counts SECTIONS, not content: a hole whose two ends kept their headings
         is invisible here, so read `git diff --stat <merge-base>..HEAD -- <file>` too

The span comes from the base's own line numbers — the heading's first line, and the first line at or before the next heading of the same or shallower level — so the numbers point at the text to restore rather than at a file to re-read. On #2183 that gave 39 and 63 lines: 104 in total, against a headline of "2".

One arm added and mutation-proven (flattening the span arithmetic fails 2 arms): a lost section reports (first, last, level).

doc_section_loss_selftest .....  Passed  0.13 s
doc_section_loss --base origin/main  ->  all 644 section(s) present at HEAD
copyright --check rc 0

…at it cannot see

Found by using the merged tool on engine#2183: it reported "2 of 643 section(s) missing" while
`git diff --stat` said the branch removed **694 lines** of docs/npu.md. Both headings were real; the
other 592 lines belonged to sections whose headings survived at both ends of the hole, which a
heading-set comparison cannot see. So the count undersold the damage, which is the one thing a report
like this must not do.

    before:  "A pack with both families: ..." is at <base> and nowhere at <head>
    after:   "A pack with both families: ..." is at <base> and nowhere at <head>
             (lines 801-863, 63 line(s), ####)
             ... 2 of 643 section(s) at <base> are missing at <head>, 104 line(s) ...
             this counts SECTIONS, not content: a hole whose two ends kept their headings is invisible
             here, so read `git diff --stat <merge-base>..HEAD -- <file>` too

The span is computed from the base's own line numbers -- the heading's first line, and the first line
at or before the next heading of the same or shallower level -- so the numbers point at the text to
restore rather than at a file to re-read.

One arm added and mutation-proven: (3, 10, 3) for an 8-line section, and flattening the span
arithmetic fails 2 arms. Self-test ok, ctest doc_section_loss_selftest Passed 0.17 s,
copyright --check rc 0, ci_targets_check --check rc 0.
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Incorrect line counting for section spans

The calculation of section spans assumes that the first line of a heading is at index 0, but the code uses i + 1 for the first line, which is correct. However, the end line calculation may not correctly handle all edge cases, especially when the last heading is at the end of the file. This could lead to incorrect size reporting for lost sections.

spans[text] = (i + 1, end, level)

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at d018827 — the tool now reports the size of what it found, and states the hole it cannot see, which is the honest pair.

The finding undercuts the number I praised

"It reported '2 of 643 section(s) missing' while git diff --stat said the branch removed 694 lines of docs/npu.md … the other 592 lines belonged to sections whose headings survived at both ends of the hole, which a heading[-based check cannot see]"

2 sections, 104 lines — against 694 lines removed. So the check I reviewed was catching 15 % of the actual loss, and I called 2 of 643 sections a good denominator without ever asking how many lines those sections were. That is the same error I keep finding in other people's work — a count whose unit is not the unit of the damage — and this time it was in a tool I had read.

What this PR does right, in two ways

  1. it reports the span: "(lines 801-863, 63 line(s), ####)" — size and heading level per lost section, so 2 sections are no longer 2 undifferentiated items;
  2. it states the boundary in its own output:

    "this counts SECTIONS, not content: a hole whose two ends kept their headings is invisible here, so read git diff --stat <merge-base>..HEAD -- <file> too"

That second line is the most valuable sentence in the output — a check telling you which second command to run when it is the wrong instrument. It is the cannot tell discipline (#2288, #2232) applied to partial coverage rather than to failure.

Two asks

  1. make the line count the headline where it can be — 2 of 643 section(s) missing, 104 line(s) invites the reader to compare against 694; better still, when the check can see the file's line delta, print both ("104 of 694 removed lines are inside sections this check can name") so the gap is in the same line as the finding;
  2. give the 592-line case a name — if the surviving-heading hole is a known shape, a second mode (--between-headings) that reports content removed between two headings present on both sides would close it entirely; if that is too hard (the base's headings are the only anchors), then say so in one clause and the current warning is the honest ceiling.

And the reporting chain is worth recording

The tool was merged, then used on #2183, and the use found its own blind spot — that is the sequence that makes an instrument trustworthy: built, merged, exercised on a real case, found wanting, fixed. #2331 → #2345 in one step, with my review having missed it. That belongs in the PR body as the justification, because "found by using it" is the only evidence that a check is doing work.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at d018827

The span reporting is the right upgrade and I ran it on the case it was found on: --base origin/main --head <2183> gives all 644 section(s) … are present, rc 0 -- which is now true, because that branch's docs/npu.md was reset to main's.

And running it there exposed the question this tool cannot answer, which is worth one line in its header rather than a change:

doc_section_loss_check compares a base to a head. #2183 lost its own section in a rebase -- ## The audio chain on the host (Whisper)… is present at 3999aff and absent at every head since -- and that section never existed in main, so a base-to-head comparison cannot see it. The tool reports all 644 present on a branch that dropped documentation it was the only copy of. Measured:

3999aff   "## The audio chain on the host (Whisper), and what it measures"   present
0f7fc730  same                                                              absent
git show origin/main:docs/npu.md | grep -c "audio chain"                    0

So the three questions are different, and this tool answers the first:

  1. base -> head: did the branch drop what main has? (this tool)
  2. head vs its own earlier heads: did a rebase drop what the branch had? (nothing answers this)
  3. content vs sections: a hole whose two ends kept their headings (the tool says so itself).

The second one is the shape a rebase produces, and it is the one that bit #2183 twice in one night -- once as 694 deleted lines (question 1, visible) and once as its own section vanishing (question 2, invisible). The cheap form of an answer is a --since <ref> that compares a head against an earlier head of the same branch (--since 3999aff --head <head>), which needs no new instrument -- the same section parser, a different pair of revisions.

Everything else about the change is right: the span (lines 801-863, 63 line(s), ####) turns "a section is missing" into something a reader can look at; the summary line carries the total (2 of 643 … 104 line(s)); and the caveat is stated in the output rather than in a comment nobody reads -- "this counts SECTIONS, not content: a hole whose two ends kept their headings is invisible here, so read git diff --stat <merge-base>..HEAD -- <file> too".

One small thing: the summary names the base by a 12-hex sha (all 644 section(s) at 02ed738a4538), which is fine for reading but unsearchable -- the full sha is what git show needs when someone pastes the line into a shell. The verdict-line convention in this fleet is the same lesson from the other side: name enough of the sha that it resolves, and the whole one when it is cheap.

@bong-water-water-bong
bong-water-water-bong merged commit 5f1f7e3 into main Nov 13, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the lane/doc-loss-size branch November 13, 2026 09:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant