fix(pdf): keep footnotes out of body reading order

[asionesjia]

Summary

Confirmed #74 against the cited opendataloader-bench pages. The PDF converter had no footnote region detector, so raised markers became ordinary digits in the wrong sentence, and numbered note blocks mixed into body columns or glued onto the last paragraph.

This change:

• Attaches raised/small digit markers to the preceding word even when that word ends in punctuation (FY2019.⁴, equilibrium?¹²).
• Peels numbered footnote lines (and matching stray marker lines) before XY-cut and table detection, then emits them after the page body so two-column notes do not interleave.
• Starts a new paragraph at each note so footer notes are not space-joined onto the last body sentence.

Verified on the issue examples (01030000000008, 57, 86, 97, 135). Notes stay out of the body. Remaining gaps (a page number that still sits mid-page on 08, a wrapped note line ordered before its start on 135) are outside the mix-into-body failure.

fixes #74

---

Note

Medium Risk
Changes core PDF reading-order and paragraph assembly heuristics; misclassification could still affect some layouts, but scope is limited to the PDF package with broad new test coverage.

Overview
Improves PDF-to-markdown so footnote markers and bottom-of-page note blocks no longer pollute main body text or column reading order.

Superscript merge in pdf.ts is reworked: small digit runs attach to the nearest preceding text item using position and punctuation rules (e.g. FY2019.⁴, equilibrium?¹²), not only when the parent ends in a letter.

Footnote peeling adds exported peelFootnoteLines in layout.ts, which per page infers note regions from superscript markers, smaller type, and numbered note starts (with guards for page numbers, figure captions, and false positives when too much of the page would be classified as notes). Peeled lines are excluded from table detection; markdown flow emits body → notes → footer per page with note/footer roles so two-column notes do not interleave with columns. Note lines get explicit paragraph breaks and skip heading heuristics so footer notes are not space-joined onto the last body sentence or list item.

Unit and integration tests cover peeling heuristics and end-to-end markdown behavior.

^{Reviewed by Cursor Bugbot for commit 90928eb. Bugbot is set up for automated code reviews on this repo. Configure here.}

[cloudflare-workers-and-pages]

Deploying with    Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status	Name	Latest Commit	Updated (UTC)
✅ Deployment successful! View logs	mdgate-demo	90928eb	Aug 26 2026, 05:27 AM

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 4 potential issues.

Bugbot Autofix prepared fixes for all 4 issues found in the latest run.

• ✅ Fixed: Notes glue onto trailing list items
  • roleChange and noteStart now close an open list before note emission, so a following note starts a new block instead of continuing the last item.
• ✅ Fixed: Matching page numbers get dropped
  • Footer-only lines are skipped as note starts, and bottom page numbers are classified as footers before orphan-marker drop.
• ✅ Fixed: Figure caption exclusion never matches
  • The figure/table guard now tests the text after the leading number, so numbered captions are no longer treated as notes.
• ✅ Fixed: Trailing marker scan is overbroad
  • Trailing marker collection now requires two or more letters before 1-3 digits at a token end, so Fig.1, p.45, v2, and decimals no longer peel body lists.

_{You can send follow-ups to the cloud agent here.}

^{Reviewed by Cursor Bugbot for commit 25cac30. Configure here.}

[cursor]

Stale comment

Not approved: Cursor Bugbot finished as skipped and left 4 unresolved findings that need human review. No reviewers were assigned (the only recent editor of these files is the PR author). Human review is needed.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}

[cursor]

Approved: Cursor Bugbot passed on the current head with no unresolved findings that need human review. No reviewers were assigned.

  

_{Sent by Cursor Approval Agent: Pull Request Router and Approver}