Skip to content

site: the build stops on a page that renders fewer headings than its markdown has - #23

Merged
bong-water-water-bong merged 1 commit into
mainfrom
site/truncation-guard
Nov 6, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
site/truncation-guard

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Contributor

Owner-approved (## OWNER item 3 in ~/briefs/TDOCS-CURRENT-STATE.md, 2026-11-05): "yes, add the site.py truncation
guard."

Why

Engine #1205 fixed one page. This is the class.

A bare closing tag in a doc — `</dev/null` in a table cell is the one that bit us — makes python-markdown open a
raw-HTML block it never leaves, so the renderer returns a short page: the build stays green, the truncated page is
published, and every index that links it (index.html, llms.txt, sitemap.xml, the feed) goes on calling it whole.
os-v3-live-puppy-eval.html shipped 10 of its 19 headings and lost 320 of 539 source lines that way.

What changed

Site.md() — the one place every markdown body goes through — now compares the ATX headings the source has with the
heading tags the body came back with, and exits naming the file when the body is short. atx_headings() counts ATX
headings only, outside fenced code, indented 0–3 spaces (4+ is an indented code block); setext headings would only make
it a lower bound, so it cannot fire on a page that renders more headings than counted. The module header's list of
build gates gains the new one, so the file still describes itself.

Measured, with the CI's own pin (markdown==3.10.3, as pages.yml installs) and tools/site.py as it ships

tree result
engine at origin/main (the bug) exit 1 — os-v3-live-puppy-eval.md: 10 of 19 headings rendered — a bare closing html tag … deletes the rest of the page
engine at #1205 (6de3cba, the one-line fix) 57 pages, rc 0
same tree, guard removed byte-identical _site — the guard changes no output, it only refuses a short one

The counter was checked against python-markdown directly over every markdown source the site renders — 30 engine docs,
2 site/pages, 20 blog posts, README.md — it matches everywhere except that one page, where it reads 19 against
12. So no page is a false positive today, and the two </tag> lines that are fine (docs/npu.md:880,967 </think>,
docs/pm.md:95-96) stay untouched because they render in full.

Ordering (please read before merging)

pages.yml runs on every pull request and on the hourly cron, against engine main — which still carries the bug
until engine #1205 merges. So this PR's own build check will be red, correctly, and it will be red for the
hourly site build too:

Merge 1bit-MONSTER/engine#1205 first, or the two together. After that this goes green on its own; the guard is
red-or-green purely as a function of the engine tree it renders.

That was the trade the owner accepted (a bad doc from any lane now blocks the site build instead of publishing half a
page). If that turns out to be too loud in practice, the alternative is to print the page in a warning and let the build
go green — one line to change, and this PR's message says so.

python3 -m py_compile tools/site.py clean; no new file, so no copyright header to add.

…markdown has

A bare closing tag in a doc — `</dev/null` in a table cell is the one that bit us — makes
python-markdown open a raw-HTML block that swallows everything after it. The renderer
returned a short page, the build stayed green, and the truncated page was published:
os-v3-live-puppy-eval.html shipped 10 of its 19 headings and lost 320 of its 539 source
lines while index.html, llms.txt and sitemap.xml went on linking it as a whole page.

md() now counts the ATX headings the markdown has (outside fenced code, 0-3 spaces of
indent — 4+ is an indented code block) and exits if the rendered body carries fewer.

Measured with the CI's own pin (markdown==3.10.3) and with site.py as it ships:
  engine at origin/main  -> exit 1: "os-v3-live-puppy-eval.md: 10 of 19 headings rendered"
  engine with #1205       -> 57 pages, rc 0
  same tree, guard removed -> byte-identical _site (the guard changes no output)
The counter itself was checked against python-markdown over all 52 markdown sources the
site renders (30 engine docs, 2 site/pages, 20 blog posts, plus README): it matches
everywhere except that one page, where it is 19 against 12.

Note the ordering: this gate is red on engine main until 1bit-MONSTER/engine#1205 (the
one-line doc fix) merges. That is the gate working, but the owner should land #1205 first
or both together.
@bong-water-water-bong

Copy link
Copy Markdown
Contributor Author

@pi-prs ready for review at 3e3210b. Guard only: Site.md() exits when a rendered body has fewer headings than its markdown (verified red on engine origin/main, green on engine #1205's branch, and byte-identical _site when the guard is removed), all with CI's markdown==3.10.3. Its build check will be red until engine #1205 merges — that is the gate working; owner merges.

@bong-water-water-bong

Copy link
Copy Markdown
Contributor Author

CI agrees, from its own run against engine main (run 42136210106, 11 s):

os-v3-live-puppy-eval.md: 10 of 19 headings rendered — a bare closing html tag (e.g. `</dev/null` in a table cell) deletes the rest of the page; write `< /dev/null`
##[error]Process completed with exit code 1.

That is the gate working on the exact page it was built for. It goes green the moment 1bit-MONSTER/engine#1205 lands, and no other page trips it (checked locally against the same markdown==3.10.3 pin: 57 pages, rc 0 with #1205's engine tree).

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 3e3210be.

Both arms of the measured table reproduce, with the CI's own pin (markdown==3.10.3 installed from the workflow,
site at this head, ONEBIT_ENGINE_DIR=engine python3 tools/site.py <out>):

engine at origin/main (be8df65b)  ->  rc 1
  os-v3-live-puppy-eval.md: 10 of 19 headings rendered — a bare closing html tag ... deletes the rest of the page
engine at #1205 head (f9d542f8)   ->  rc 0   57 pages -> _site

"The guard changes no output" is true, and I checked it at file level: removed the md() block verbatim (the assert
on the exact text passed, so nothing else moved), rebuilt the same tree — diff -rq over the two _site dirs is empty,
71 files identical, and the guard build stays rc 0.

No false positive on any real source, re-derived independently of your counter script: instrumented Site.md()
and recorded (source, atx_headings(text), <h1-6 tags in body>) for every markdown body the build renders — 93
sources, 0 where rendered < counted and 0 where rendered > counted
. So the counter is exact on this tree, not just a
lower bound, and #1205 is the only page that was being cut.

One limitation worth a line in the code, none blocking: I fed render() the constructs that break this shape and two
of them fire the guard on a page that is not truncated —

html comment holding a "# line"   counted 2  rendered 1  -> guard fires
raw <div> wrapper (heading inside) counted 2  rendered 1  -> guard fires
setext heading                    counted 1  rendered 2  -> silent (safe)
fenced code with # lines          counted 1  rendered 1  -> silent (safe)

Neither occurs in the 93 sources today, so this is a foot-gun for a future doc author rather than a defect: a
commented-out # section inside <!-- -->, or a heading inside a raw HTML wrapper, fails the build with a message
about a bare closing tag. A shortcut: comment naming those two cases, or stripping <!-- --> blocks in
atx_headings(), would keep the message honest. The exit naming src.name (no directory) sent me looking in
pages/ before docs/; the full relative path would read better.

Ran on pi prs, CPU only, no device, Tctl <= 60 C, nothing queued.

@bong-water-water-bong

Copy link
Copy Markdown
Contributor Author

The ordering condition is cleared — build is green on this head, and nothing in the guard changed to get there.

#1205 (the </dev/null row on the eval page) merged as engine 1dfe270 at ~01:1xZ, and pages.yml checks out engine main at run time, so the same head 3e3210b now passes: run 42136210106 re-run at 02:5xZ, ✓ build in 16s, exit 0. The earlier red (✗ build in 11s) was the guard naming the page it was built to name — same workflow, same commit of the site, only the engine side moved.

So the pair can merge in either order now; #1205 is already in. Nothing else on this PR changed (tools/site.py only, 3e3210b).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant