fix(streaming): preserve heading IDs and reference links

[farnabaz]

What

Force a full reparse for document-scoped constructs while streaming (reference definitions, heading ID counters), omit reference tokens from the AST, and clear stale frontmatter when a stream switches documents.

Why

Incremental streaming reuse skips the completed prefix, but heading IDs and reference links depend on full-document state. Without this, streamed tails can emit wrong IDs, broken links, or leaked reference nodes. Also avoid carrying old frontmatter when the source is replaced.

Cherry-picked from #396 (core parse fixes only) for separation of concerns.

Co-authored-by: @onmax

Summary by CodeRabbit

• Bug Fixes
  • Improved streaming Markdown parsing for reference definitions, headings, and setext headings.
  • Ensured streamed and full parses produce consistent results when content is appended.
  • Corrected frontmatter handling when streamed content changes.
  • Reference definitions are now omitted from rendered output.
  • Improved parsing consistency for nested, duplicate, in-list, and CRLF-formatted headings.

[coldtea-pr-lens]

◈ PR Lens

🟢 +0 new · 🟠 ~3 changed · 🔴 -0 removed · 1 flow · 4 files · commit 5adef93

Architecture

3 components touched across 2 lanes.

Open the interactive canvas

Inside the changed components — 1 view

Component view — Streaming pipeline

The streaming parse reuse check and token processor

Data flow

Streaming parse with context validation

Open the interactive canvas

View

• Architecture lens
• Data flow lens
• Expand every detail

Tip

The diagrams are links. Click one to open it on the canvas, then press W or click play to walk through the change.

🪧 More tips

• Run npx skills add coldteadotai/pr-lens, then tell your coding agent: "Diagram the change you just made with PR Lens and attach it to the pull request."
• Run npx @coldtea/pr-lens-cli analyze --base origin/main on a branch, then npx @coldtea/pr-lens-cli render .pr-lens/graph.json. Same lenses, your own model key, before the pull request exists.
• Untick Architecture lens or Data flow lens under View to hide a diagram, or tick Expand every detail to open every section. The comment redraws in a few seconds.
• Click the link under each diagram to open it on a canvas you can zoom, pan and step through.
• Open a diagram on the canvas, then press W or click play to walk through the change one step at a time.
• The CLI's render reads .github/pr-lens.yml and applies your renames, exclusions and lane pins at draw time.
• Set github.comment.collapsed: true in .github/pr-lens.yml to fold the comment behind one View architecture and data flow row. Drawing still runs on every push.
• Add .github/workflows/pr-lens.yml with coldteadotai/pr-lens/packages/action@v0 and your model provider's key as its api-key to run PR Lens from your own CI. Any /chat/completions endpoint works.
• Push a commit and the comment redraws for the new head. A slow older run never overwrites a newer one.
• Switch GitHub to dark mode and the diagrams follow. The moving dots are this pull request's data in motion.

---

Thanks for using PR Lens! It's built by Coldtea, free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated
comark	Ready	Preview	Sep 11, 2026 11:50am UTC
comark-json-render	Ready	Preview	Sep 11, 2026 11:50am UTC
comark-nextjs	Ready	Preview	Sep 11, 2026 11:50am UTC
comark-nuxt	Ready	Preview	Sep 11, 2026 11:50am UTC
comark-svelte	Ready	Preview	Sep 11, 2026 11:50am UTC
comark-sveltekit	Ready	Preview	Sep 11, 2026 11:50am UTC
comark-twoslash	Ready	Preview	Sep 11, 2026 11:50am UTC
comark-vue	Ready	Preview	Sep 11, 2026 11:50am UTC

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Advanced

Run ID: 7d2e47dc-a762-480d-ab7c-3f96512e542b

📥 Commits

Reviewing files that changed from the base of the PR and between 91cbff7 and 5adef93.

📒 Files selected for processing (1)

• test/bundle.test.ts

Included review availability: Your plan provides up to 4 included reviews per hour; 3 remain after this review.

---

📝 Walkthrough

Walkthrough

The parser now omits reference definition nodes, forces full reparses for reference and heading-sensitive streaming input, preserves continuation frontmatter, and adds coverage for these cases.

Changes

Streaming parser correctness

Layer / File(s)	Summary
Reference token handling packages/comark/src/internal/parse/token-processor.ts, packages/comark/test/streaming.test.ts	processBlockToken skips reference tokens. Tests verify that reference definitions are omitted while reference links remain preserved.
Streaming reparse and frontmatter state packages/comark/src/parse.ts, packages/comark/test/streaming.test.ts, test/bundle.test.ts	Streaming reuse now falls back to a full reparse when reference definitions or heading-sensitive tails are present. Continuation parsing preserves prior frontmatter when no new frontmatter exists. Tests cover appended content, frontmatter replacement, and the updated bundle size snapshot.

Priority: ⬇️ Low

Estimated code review effort: 3 (Moderate) | ~20 minutes

Change: Bug fix

Suggested reviewers: atinux

Merge Risk: ⚪ Minimal · up to 5adef

The parser changes are covered by targeted tests, with no unresolved merge-blocking risk identified.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 50.00% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 2 functions across 4 files.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the main change: fixing streaming parsing to preserve heading IDs and reference links.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

• Fix all pre-merge checks with AI

✨ Finishing Touches 💡 1

📝 Generate docstrings 💡

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch fix/streaming-refs

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}

[pkg-pr-new]

Open in StackBlitz

comark

npm i https://pkg.pr.new/comark@413

@comark/angular

npm i https://pkg.pr.new/@comark/angular@413

@comark/ansi

npm i https://pkg.pr.new/@comark/ansi@413

@comark/html

npm i https://pkg.pr.new/@comark/html@413

@comark/nuxt

npm i https://pkg.pr.new/@comark/nuxt@413

@comark/react

npm i https://pkg.pr.new/@comark/react@413

@comark/svelte

npm i https://pkg.pr.new/@comark/svelte@413

@comark/vue

npm i https://pkg.pr.new/@comark/vue@413

commit: 5adef93

[github-actions]

✅ Bundle snapshot updated

Committed the new snapshot as 5adef93, verified by re-running the bundle check against it.

GitHub suppresses the events a workflow commit would normally raise, so the ci check above may
still show the earlier failure. Re-run it to refresh the status — the snapshot itself is confirmed green.

_{Produced by this run.}