Smaller text: space glyph in the base-14 subsets, one text object per line (#288) - #289
Merged
Merged
Conversation
None of defaultfonts/*.subset.ttf had a glyph for U+0020 or U+00A0. The HTML renderer fell back to a system font for every space, which ended the glyph run at every word (two text objects per word) and embedded a second, machine-dependent font (HelveticaNeue on macOS). A space has no outline, so this adds no font data: an empty glyph 'space', appended so existing glyph ids keep their numbers, with the standard base-14 advance (Helvetica 278, Times 250, Courier 600, Symbol 250, ZapfDingbats 278 per 1000 em), mapped from U+0020 and U+00A0. scripts/add_space_glyphs.py does it reproducibly (gzip mtime 0) and is idempotent. Helvetica's exact coverage goes from 213 to 215 codepoints. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- src/text_merge.rs, run on every page's ops when PdfSaveOptions::optimize is set: consecutive text objects with only text state between them become one (only when the next one sets Tm before it shows or moves anything, as BT resets the text matrix); a font or fill colour already in effect is not set again (forgotten at Q and gs); /ActualText spans that follow each other on one baseline become one span with the joined text, so copy-paste of a line is unchanged and line breaks survive. - The same-baseline glyph-run merger rounds kerns to whole thousandths of an em (carrying the exact pen, so the error stays below half a unit instead of adding up) and writes glyphs without a kern between them as one hex string, instead of a sub-unit kern after almost every glyph. On a 1000-word OCR'd book page the compressed content stream goes from 45 KB to 17 KB (positioned words) and from 32 KB to 12 KB (flowing paragraphs), together with the space glyph from the previous commit. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`cargo test --no-default-features` failed to compile the file: PdfDocument::from_html only exists with `html`. Gated per test, as tests/issue_repros.rs does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WyGP18BMbfHvnTRYqNVjbP
fschutt
added a commit
that referenced
this pull request
Oct 2, 2026
…kept beside this one) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WyGP18BMbfHvnTRYqNVjbP
fschutt
added a commit
that referenced
this pull request
Oct 2, 2026
…r alpha) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WyGP18BMbfHvnTRYqNVjbP
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #288.
HTML text in a base-14 family was written as two text objects per word: the word, then its space in a system fallback font, each with its own
Tf BT /Span <</ActualText …>> BDC Tm TJ EMC ET. Three changes, measured on 20 OCR'd book pages (~1000 words each) rendered withfrom_html:Per page, the compressed content stream goes from 45 KB to 17 KB (positioned) and from 32 KB to 12 KB (flow). A Helvetica line is now one
BT,Tf,TmandTJ. The extracted text of the rendered pages is unchanged.1.
defaultfonts/: the space (54b4493)None of the 14 subsets had a glyph for U+0020 / U+00A0, so every space fell back to whatever system font azul found (HelveticaNeue on macOS). That ended the glyph run at each word and embedded a second, machine-dependent font. A space has no outline, so
scripts/add_space_glyphs.pyadds an emptyspaceglyph with the standard base-14 advance, mapped from U+0020 and U+00A0. It's appended so existing glyph ids are unchanged, and the script is reproducible and idempotent. Helvetica's exact coverage goes 213 → 215.Still missing and still falling back:
!(U+0021),”(U+201D), soft hyphen, and Ž ž € in Times-Italic / Courier-BoldOblique. These need outlines, so the subsets would have to be re-cut from their sources.2.
src/text_merge.rs: op pass underoptimize(c2ab8d8)Tf, colour,Tc/Tw/Tz/TL/Ts/Tr) lies between them. This only happens when the next object setsTmbefore showing or moving anything, sinceBTresets the text matrix.Qandgs./ActualText: spans that follow each other on the same baseline become one span with the joined text. Spans on different baselines stay separate, so line breaks survive copy-paste.3. Glyph-run merger: whole-unit kerns, one string per unkerned sequence (
c2ab8d8)The merger wrote a kern after almost every glyph (
[<0030> -0.16767642<0041> -0.15256259<0052> …]) for the sub-unit difference between the layout's positions and the integer/Wwidths. Kerns are now rounded to whole thousandths of an em. The exact pen is carried along, so the error stays below half a unit instead of accumulating, and glyphs without a kern between them share one hex string. Theglyph_run_mergetests use exact integer kerns and pass unchanged.Tests
tests/text_run_merge.rs(7):optimize: falsewrites the ops as givenBT's matrix is left aloneTfis set again afterQAll 207 tests pass.
🤖 Generated with Claude Code