Skip to content

Smaller text: space glyph in the base-14 subsets, one text object per line (#288) - #289

Merged
fschutt merged 3 commits into
masterfrom
perf/merge-text-sections
Oct 2, 2026
Merged

fschutt merged 3 commits into
masterfrom
perf/merge-text-sections

Conversation

@fschutt

@fschutt fschutt commented Oct 1, 2026

Copy link
Copy Markdown
Owner

Fixes #288.

HTML text in a base-14 family was written as two text objects per word: the word, then its space in a system fallback font, each with its own Tf BT /Span <</ActualText …>> BDC Tm TJ EMC ET. Three changes, measured on 20 OCR'd book pages (~1000 words each) rendered with from_html:

0.12.8 + text-object merge + whole-unit kerns + space glyph
positioned words (absolutely placed spans) 612 KiB 608 436 257 KiB
flowing paragraphs 461 KiB 457 370 184 KiB

Per page, the compressed content stream goes from 45 KB to 17 KB (positioned) and from 32 KB to 12 KB (flow). A Helvetica line is now one BT, Tf, Tm and TJ. The extracted text of the rendered pages is unchanged.

1. defaultfonts/: the space (54b4493)

None of the 14 subsets had a glyph for U+0020 / U+00A0, so every space fell back to whatever system font azul found (HelveticaNeue on macOS). That ended the glyph run at each word and embedded a second, machine-dependent font. A space has no outline, so scripts/add_space_glyphs.py adds an empty space glyph with the standard base-14 advance, mapped from U+0020 and U+00A0. It's appended so existing glyph ids are unchanged, and the script is reproducible and idempotent. Helvetica's exact coverage goes 213 → 215.

Still missing and still falling back: ! (U+0021), ” (U+201D), soft hyphen, and Ž ž € in Times-Italic / Courier-BoldOblique. These need outlines, so the subsets would have to be re-cut from their sources.

2. src/text_merge.rs: op pass under optimize (c2ab8d8)

  • Text objects: consecutive ones merge when only text state (Tf, colour, Tc/Tw/Tz/TL/Ts/Tr) lies between them. This only happens when the next object sets Tm before showing or moving anything, since BT resets the text matrix.
  • State: a font or fill colour that is already in effect is dropped. The tracked state is forgotten at Q and gs.
  • /ActualText: spans that follow each other on the same baseline become one span with the joined text. Spans on different baselines stay separate, so line breaks survive copy-paste.

3. Glyph-run merger: whole-unit kerns, one string per unkerned sequence (c2ab8d8)

The merger wrote a kern after almost every glyph ([<0030> -0.16767642<0041> -0.15256259<0052> …]) for the sub-unit difference between the layout's positions and the integer /W widths. Kerns are now rounded to whole thousandths of an em. The exact pen is carried along, so the error stays below half a unit instead of accumulating, and glyphs without a kern between them share one hex string. The glyph_run_merge tests use exact integer kerns and pass unchanged.

Tests

tests/text_run_merge.rs (7):

  • same-baseline runs merge into one text object / one span per line
  • optimize: false writes the ops as given
  • a text object relying on BT's matrix is left alone
  • drawing between text objects keeps them apart
  • Tf is set again after Q
  • a Helvetica HTML line is one glyph run with a single embedded font
  • extracted text is unchanged by the pass

All 207 tests pass.

🤖 Generated with Claude Code

fschutt and others added 3 commits October 1, 2026 23:20
None of defaultfonts/*.subset.ttf had a glyph for U+0020 or U+00A0. The
HTML renderer fell back to a system font for every space, which ended the
glyph run at every word (two text objects per word) and embedded a second,
machine-dependent font (HelveticaNeue on macOS).

A space has no outline, so this adds no font data: an empty glyph 'space',
appended so existing glyph ids keep their numbers, with the standard base-14
advance (Helvetica 278, Times 250, Courier 600, Symbol 250, ZapfDingbats 278
per 1000 em), mapped from U+0020 and U+00A0. scripts/add_space_glyphs.py
does it reproducibly (gzip mtime 0) and is idempotent.

Helvetica's exact coverage goes from 213 to 215 codepoints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- src/text_merge.rs, run on every page's ops when PdfSaveOptions::optimize
  is set: consecutive text objects with only text state between them become
  one (only when the next one sets Tm before it shows or moves anything, as
  BT resets the text matrix); a font or fill colour already in effect is
  not set again (forgotten at Q and gs); /ActualText spans that follow each
  other on one baseline become one span with the joined text, so copy-paste
  of a line is unchanged and line breaks survive.
- The same-baseline glyph-run merger rounds kerns to whole thousandths of an
  em (carrying the exact pen, so the error stays below half a unit instead of
  adding up) and writes glyphs without a kern between them as one hex
  string, instead of a sub-unit kern after almost every glyph.

On a 1000-word OCR'd book page the compressed content stream goes from
45 KB to 17 KB (positioned words) and from 32 KB to 12 KB (flowing
paragraphs), together with the space glyph from the previous commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`cargo test --no-default-features` failed to compile the file: PdfDocument::from_html
only exists with `html`. Gated per test, as tests/issue_repros.rs does.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WyGP18BMbfHvnTRYqNVjbP
@fschutt
fschutt merged commit eb4e993 into master Oct 2, 2026
49 checks passed
fschutt added a commit that referenced this pull request Oct 2, 2026
…kept beside this one)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WyGP18BMbfHvnTRYqNVjbP
fschutt added a commit that referenced this pull request Oct 2, 2026
…r alpha)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WyGP18BMbfHvnTRYqNVjbP
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bundled base-14 fonts have no space glyph: every space falls back to a system font, two text objects per word

1 participant