fix(pdf): honor ActualText and skip ToUnicode FFFD

[asionesjia]

Summary

#80 is already merged. It closed #71 by keeping /Artifact text. The leftover is #81: after that merge, 01030000000001.pdf still starts with 3�4 Yarrow instead of gold 314 / YARROW.

Confirmed. The 314 run is Type1 GKDCHH+Brill-Roman. ToUnicode maps 0x13 to U+FFFD and overwrites /Differences /one.SP. The producer also wraps that glyph in /Span << /ActualText (UTF-16BE "1") >> BDC. @mdgate/pdf ignored both.

YARROW vs Yarrow is the small-cap half of the same font. Glyph-name suffix mapping from #88 already turns /Y.c2sc and /a.smcp into uppercase. This PR keeps that path when ToUnicode is missing or FFFD.

Changes

• Skip ToUnicode entries that are empty or only U+FFFD, so Encoding Differences names (one.SP, Y.c2sc) remain.
• Treat FFFD cmap values as unmapped at decode time and fall through to Differences / Adobe CID / ASCII.
• Honor marked-content /ActualText from inline BDC dicts, named /Properties, and StructTreeRoot MCID entries. Replace the shown glyphs; keep artifact graphics skipped.

Test plan

• bunx vitest run packages/pdf/test/encoding.test.ts packages/pdf/test/layout.test.ts
• bunx vitest run packages/pdf/test (169 pass; images needs dist)
• bun run lint
• Pre-commit bun test (565 pass)

---

Note

Medium Risk
Changes core PDF text decoding and marked-content extraction paths; regressions could affect markdown output for many PDFs, but scope is limited to font mapping and accessibility metadata, not security or I/O.

Overview
Improves PDF-to-markdown text extraction when producers lie in the glyph stream or ToUnicode maps codes to U+FFFD.

ToUnicode / encoding: Entries that are empty or only replacement characters are no longer applied to the font cmap or treated as successful decode results, so Encoding /Differences glyph names (e.g. one.SP, small-cap names) and other fallbacks can still produce the intended Unicode.

Marked content /ActualText: While a marked-content span carries actual text (inline BDC dict, named /Properties, or StructTreeRoot via MCID—including shared parent ActualText across sibling/nested MCIDs), shown glyphs are not emitted; layout is tracked and ActualText is output once when the span closes. /Artifact handling is refactored to a frame stack but behavior stays the same for skipping decorative graphics.

Tests cover FFFD + Differences, all ActualText sources, and small-cap Differences mapping.

^{Reviewed by Cursor Bugbot for commit 41d8720. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	mdgate-demo	41d8720	Aug 27 2026, 12:04 PM

[cursor]

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Autofix Details

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Parent ActualText repeats on child MCIDs
  • Parent /ActualText is now bound to the first descendant MCID and later child sequences are suppressed, so the replacement emits once.

_{You can send follow-ups to the cloud agent here.}

[cursor]

Stale comment

Left a non-blocking comment. Cursor Bugbot finished as skipped and reported one unresolved Medium finding (parent ActualText repeating on child MCIDs), so this run did not approve. Human review is needed; no reviewer was assigned because the only assignable user is the PR author.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}

[cursor]

Stale comment

Left a non-blocking comment. Cursor Bugbot finished as skipped and left one unresolved Medium finding on the current head (parent ActualText can drop text), so this run did not approve. Human review is needed; no reviewer was assigned because the only assignable user is the PR author.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Bugbot Autofix prepared a fix for the issue found in the latest run.

• ✅ Fixed: Nested MCIDs emit ActualText too early
  • Shared glyph capture now lives on StructActual and ActualText emits only after the last open frame that uses that bind, so nested MCIDs keep the outer start position.

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit bcb4577. Configure here.}

[cursor]

Stale comment

Left a non-blocking comment. Cursor Bugbot finished as skipped and left one unresolved Medium finding on the current head (nested MCIDs emit ActualText too early), so this run did not approve. Human review is needed; no reviewer was assigned because no assignable reviewer other than the PR author is available.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}

[cursor]

Approved. Cursor Bugbot completed successfully on the current head with no unresolved findings that need human review, so this run approved and did not assign reviewers.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}