fix(pdf): recover drop-caps, ligatures, and TJ word spaces

[asionesjia]

Summary

Confirmed #78 against the cited opendataloader-bench pages. Three separate PDF text bugs were producing the reported output.

1. Encoding /Differences names such as f_i, T_h, eight.oldstyle, and t.sc were looked up as raw AGL keys, so ligatures and small-caps vanished (field → eld, THIS COLLECTION OF SIX missing).
2. A drop-cap is a large one-letter run whose baseline sits on a later body line. headerLevel used that letter's size, so T / I became headings and the rest of the word (his report, ndia suffers) was left behind.
3. Word spaces encoded as NBSP (uni00A0) or as a large negative TJ adjustment (-250, -1000) were not turned into Markdown spaces (as ayoung, SOLAR10.7B, 1Introduction).

Changes

• Map glyph names with the AGL suffix / underscore / uniXXXX rules, expand fi/fl ligatures, and treat NBSP as a space.
• Glue a true drop-cap onto the top overlapping body line. Ignore large letters that sit in a same-size title run.
• Insert a space when TJ moves the text matrix by at least 0.12 em.
• Measure heading size from the line's body font, and pick the body size from character mass so footnote-heavy pages do not promote wrapped body lines.

Test plan

• bunx vitest run packages/pdf/test
• bun run lint
• Pre-commit bun test (547 pass)
• Rechecked the issue's bench pages: 007 ligatures and drop-cap, 009 false headings, 022 NBSP spaces, 079/080 drop-caps, 185 TJ word spaces

---

Note

Medium Risk
Changes core PDF text extraction and markdown structure heuristics; regressions could affect heading levels, spacing, and ligature decoding across many documents, though behavior is covered by expanded vitest cases.

Overview
Improves PDF-to-Markdown text recovery for encoding, layout, and heading heuristics that were mangling real documents.

Glyph names (glyphNameToUnicode) now follow AGL-style rules: strip period suffixes, split on _, resolve uniXXXX / uXXXX, expand Latin ligature presentation forms, and uppercase small-cap suffixes (t.sc → T). Invisible cleanup normalizes NBSP and similar width spaces to ordinary spaces and expands ligature code points in decoded strings.

Drop caps are detected as oversized single letters and reattached to the overlapping body line (with guards for multi-column and misplaced targets), so output reads This report… instead of a # T heading plus his report…. Line grouping uses a slightly looser vertical tolerance when merging runs on the same row.

TJ spacing: large negative adjustments (≥ ~0.12 em) insert a space between adjacent text runs, fixing glued tokens like SOLAR10.7B and 1Introduction.

Heading detection measures each line’s body font size (not the first run), estimates document body size from character mass (so footnote-heavy pages don’t skew base size), tightens false-heading cues (e.g. trailing comma), and skips very short isolated lines—reducing false # headings on wrapped body and drop-cap artifacts while keeping real titles like “Executive Summary”.

^{Reviewed by Cursor Bugbot for commit b3a6c7e. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	mdgate-demo	b3a6c7e	Aug 27 2026, 06:05 AM

[cursor]

Security review of added/modified lines found no MEDIUM or higher issues with verified exploitability. Glyph-name mapping, drop-cap merge, and TJ spacing stay inside untrusted-PDF-to-Markdown conversion and do not cross a privilege or sandbox boundary.

  

_{Sent by Cursor Security Agent: Security Reviewer}

[cursor]

Stale comment

Cursor Bugbot stayed pending past the 8 minute wait, so this run is not approving. Human review is needed; no reviewers were assigned.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Drop-cap glues onto the wrong line
  • findDropCapTarget now rejects body lines beyond a max x distance and prefers the nearest column so a higher right-column or farther run no longer receives the drop-cap.

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit 9f6dcfb. Configure here.}

[cursor]

Approved. Cursor Bugbot completed successfully on the current head with no unresolved findings that need human review. No reviewers were assigned.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}