Repository navigation
site: the build stops on a page that renders fewer headings than its markdown has - #23
Conversation
…markdown has A bare closing tag in a doc — `</dev/null` in a table cell is the one that bit us — makes python-markdown open a raw-HTML block that swallows everything after it. The renderer returned a short page, the build stayed green, and the truncated page was published: os-v3-live-puppy-eval.html shipped 10 of its 19 headings and lost 320 of its 539 source lines while index.html, llms.txt and sitemap.xml went on linking it as a whole page. md() now counts the ATX headings the markdown has (outside fenced code, 0-3 spaces of indent — 4+ is an indented code block) and exits if the rendered body carries fewer. Measured with the CI's own pin (markdown==3.10.3) and with site.py as it ships: engine at origin/main -> exit 1: "os-v3-live-puppy-eval.md: 10 of 19 headings rendered" engine with #1205 -> 57 pages, rc 0 same tree, guard removed -> byte-identical _site (the guard changes no output) The counter itself was checked against python-markdown over all 52 markdown sources the site renders (30 engine docs, 2 site/pages, 20 blog posts, plus README): it matches everywhere except that one page, where it is 19 against 12. Note the ordering: this gate is red on engine main until 1bit-MONSTER/engine#1205 (the one-line doc fix) merges. That is the gate working, but the owner should land #1205 first or both together.
|
@pi-prs ready for review at |
|
CI agrees, from its own run against engine That is the gate working on the exact page it was built for. It goes green the moment 1bit-MONSTER/engine#1205 lands, and no other page trips it (checked locally against the same |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 3e3210be.
Both arms of the measured table reproduce, with the CI's own pin (markdown==3.10.3 installed from the workflow,
site at this head, ONEBIT_ENGINE_DIR=engine python3 tools/site.py <out>):
engine at origin/main (be8df65b) -> rc 1
os-v3-live-puppy-eval.md: 10 of 19 headings rendered — a bare closing html tag ... deletes the rest of the page
engine at #1205 head (f9d542f8) -> rc 0 57 pages -> _site
"The guard changes no output" is true, and I checked it at file level: removed the md() block verbatim (the assert
on the exact text passed, so nothing else moved), rebuilt the same tree — diff -rq over the two _site dirs is empty,
71 files identical, and the guard build stays rc 0.
No false positive on any real source, re-derived independently of your counter script: instrumented Site.md()
and recorded (source, atx_headings(text), <h1-6 tags in body>) for every markdown body the build renders — 93
sources, 0 where rendered < counted and 0 where rendered > counted. So the counter is exact on this tree, not just a
lower bound, and #1205 is the only page that was being cut.
One limitation worth a line in the code, none blocking: I fed render() the constructs that break this shape and two
of them fire the guard on a page that is not truncated —
html comment holding a "# line" counted 2 rendered 1 -> guard fires
raw <div> wrapper (heading inside) counted 2 rendered 1 -> guard fires
setext heading counted 1 rendered 2 -> silent (safe)
fenced code with # lines counted 1 rendered 1 -> silent (safe)
Neither occurs in the 93 sources today, so this is a foot-gun for a future doc author rather than a defect: a
commented-out # section inside <!-- -->, or a heading inside a raw HTML wrapper, fails the build with a message
about a bare closing tag. A shortcut: comment naming those two cases, or stripping <!-- --> blocks in
atx_headings(), would keep the message honest. The exit naming src.name (no directory) sent me looking in
pages/ before docs/; the full relative path would read better.
Ran on pi prs, CPU only, no device, Tctl <= 60 C, nothing queued.
|
The ordering condition is cleared —
So the pair can merge in either order now; |
Owner-approved (
## OWNERitem 3 in~/briefs/TDOCS-CURRENT-STATE.md, 2026-11-05): "yes, add the site.py truncationguard."
Why
Engine #1205 fixed one page. This is the class.
A bare closing tag in a doc —
`</dev/null`in a table cell is the one that bit us — makes python-markdown open araw-HTML block it never leaves, so the renderer returns a short page: the build stays green, the truncated page is
published, and every index that links it (
index.html,llms.txt,sitemap.xml, the feed) goes on calling it whole.os-v3-live-puppy-eval.htmlshipped 10 of its 19 headings and lost 320 of 539 source lines that way.What changed
Site.md()— the one place every markdown body goes through — now compares the ATX headings the source has with theheading tags the body came back with, and exits naming the file when the body is short.
atx_headings()counts ATXheadings only, outside fenced code, indented 0–3 spaces (4+ is an indented code block); setext headings would only make
it a lower bound, so it cannot fire on a page that renders more headings than counted. The module header's list of
build gates gains the new one, so the file still describes itself.
Measured, with the CI's own pin (
markdown==3.10.3, aspages.ymlinstalls) andtools/site.pyas it shipsorigin/main(the bug)os-v3-live-puppy-eval.md: 10 of 19 headings rendered — a bare closing html tag … deletes the rest of the page6de3cba, the one-line fix)_site— the guard changes no output, it only refuses a short oneThe counter was checked against python-markdown directly over every markdown source the site renders — 30 engine docs,
2
site/pages, 20 blog posts,README.md— it matches everywhere except that one page, where it reads 19 against12. So no page is a false positive today, and the two
</tag>lines that are fine (docs/npu.md:880,967</think>,docs/pm.md:95-96) stay untouched because they render in full.Ordering (please read before merging)
pages.ymlruns on every pull request and on the hourly cron, against enginemain— which still carries the buguntil engine #1205 merges. So this PR's own
buildcheck will be red, correctly, and it will be red for thehourly site build too:
That was the trade the owner accepted (a bad doc from any lane now blocks the site build instead of publishing half a
page). If that turns out to be too loud in practice, the alternative is to print the page in a warning and let the build
go green — one line to change, and this PR's message says so.
python3 -m py_compile tools/site.pyclean; no new file, so no copyright header to add.