Skip to content

fix(index): decode child stdout/stderr as one contiguous buffer - #605

Merged
Fdawgs merged 1 commit into
mainfrom
fix/chunk-boundary
Jul 20, 2026
Merged

fix(index): decode child stdout/stderr as one contiguous buffer#605
Fdawgs merged 1 commit into
mainfrom
fix/chunk-boundary

Conversation

@Fdawgs

@Fdawgs Fdawgs commented Jul 15, 2026

Copy link
Copy Markdown
Owner

Crazily I was just looking to reduce the transient heap size that would bloat during conversion but ended up finding a chunk boundary bug.

This PR updates convert() to collect stdout/stderr chunks into Buffer arrays and decode once on close via Buffer.concat().toString() instead of accumulating them with stdOut += data.

The previous approach implicitly called Buffer.prototype.toString() per chunk, which decoded each chunk independently. Multibyte UTF-8 sequences split across pipe chunk boundaries were replaced with U+FFFD, silently corrupting output.

Decoding once over contiguous bytes fixes the boundary bug and lowers peak heap use during conversion by ~40-50% in benchmarks.

Checklist

Copilot AI review requested due to automatic review settings July 15, 2026 10:33

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Fixes UnRTF#convert() output corruption when UTF-8 multibyte sequences are split across stdout/stderr chunk boundaries by buffering raw bytes and decoding once at process close, which also reduces transient heap pressure during conversion.

Changes:

  • Update convert() to accumulate stdout/stderr as Buffer[] and decode once via Buffer.concat(...).toString("utf8").
  • Add unit tests that simulate chunk-boundary splits for 2-, 3-, and 4-byte UTF-8 characters to prevent regression.
  • Minor test refactor to reuse EventEmitter/Readable imports for child process mocks.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File Description
src/index.js Buffers child process output as contiguous bytes and decodes once to avoid UTF-8 boundary corruption.
test/index.test.js Adds regression tests for UTF-8 chunk-boundary splits and tidies child_process mocking setup.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@Fdawgs
Fdawgs force-pushed the fix/chunk-boundary branch from 15d54e9 to aa5beb2 Compare July 18, 2026 06:29
@Fdawgs
Fdawgs merged commit 65fa151 into main Jul 20, 2026
20 checks passed
@Fdawgs
Fdawgs deleted the fix/chunk-boundary branch July 20, 2026 09:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants