Skip to content

fix: decode all members of concatenated gzip in gunzipSync#287

Open
spokodev wants to merge 1 commit into
101arrowz:masterfrom
spokodev:fix/gunzipsync-multimember
Open

fix: decode all members of concatenated gzip in gunzipSync#287
spokodev wants to merge 1 commit into
101arrowz:masterfrom
spokodev:fix/gunzipsync-multimember

Conversation

@spokodev

@spokodev spokodev commented Jun 25, 2026

Copy link
Copy Markdown

Problem

gunzipSync (and decompressSync, which routes to it) silently truncates a concatenated multi-member gzip stream to its first member, and an undersized output buffer can throw RangeError: offset is out of bounds. fflate's own streaming Gunzip class decodes the same input correctly.

const zlib = require('zlib');
const fflate = require('fflate');

const multi = Buffer.concat([
  zlib.gzipSync(Buffer.from('AAAAAAAA')),
  zlib.gzipSync(Buffer.from('BB'))
]);

zlib.gunzipSync(multi);              // "AAAAAAAABB"  (oracle)
fflate.gunzipSync(multi);            // "AA"          WRONG

const acc = [];
new fflate.Gunzip(c => acc.push(Buffer.from(c))).push(multi, true);
Buffer.concat(acc).toString();       // "AAAAAAAABB"  fflate's streaming path is already correct

CLI oracle agrees with node's zlib:

$ printf '<multi-member bytes>' | gzip -dc
AAAAAAAABB

A larger case truncates the same way:

const big = Buffer.concat([0,1,2,3,4].map(i => zlib.gzipSync(Buffer.from('member-' + i + '-payload'))));
zlib.gunzipSync(big).toString();     // "member-0-payload...member-4-payload"
fflate.gunzipSync(big).toString();   // "member-0-payload"  WRONG

Spec

RFC 1952 section 2.2: a gzip file consists of a series of members, all of which must be decoded and concatenated to recover the original data.

Root cause

Concatenated-member support was added to the streaming classes in v0.8.0 (#102), but the synchronous path was not updated. gunzipSync was:

const st = gzs(data);
if (st + 8 > data.length) err(6, 'invalid gzip data');
return inflt(data.subarray(st, -8), { i: 2 }, opts && opts.out || new u8(gzl(data)), opts && opts.dictionary);

Two issues:

  1. The output buffer is sized by gzl(data), which reads the trailing ISIZE at the end of the whole buffer, i.e. the last member's size, not the total. When the first member is larger than the last, the buffer is too small (RangeError); otherwise it just holds the first member.
  2. inflt(...) decodes a single DEFLATE stream and returns at the first member's BFINAL. There is no member-iteration loop, unlike Gunzip.push ("process concatenated GZIP").

Fix

Iterate the members in gunzipSync the same way the streaming path does: decode a member, advance past its header + DEFLATE data + 8-byte trailer (using the bit position inflt leaves in the state object), and repeat until the input is exhausted, then concatenate. Each member's output is sized by inflt itself, since the trailing ISIZE only describes the last member. The single-member fast path is preserved, and truncated/invalid input still throws invalid gzip data.

Verification

A new test (test/6-multimember.ts) covers two-member, five-member, and decompressSync multi-member streams against node zlib as the oracle, plus a single-member regression guard. It fails on master ("AAAAAAAABB" decodes to "AA"; the five-member case to "member-0-payload") and passes with this change. The full suite is green:

Total:     124
Passed:    124
Skipped:   0

Also verified manually: empty middle member, a 350 KB two-member stream, and opts.out with multi-member input all match the zlib oracle.

RFC 1952 2.2 defines a gzip stream as a series of members that must all be
decoded and concatenated. gunzipSync (and decompressSync, which routes to it)
only decoded the first member: it sized the output from the trailing ISIZE of
the whole buffer (the last member's size) and inflt returned at the first
member's BFINAL with no member-iteration loop. Concatenated streams were
silently truncated to the first member, and a large first member combined with
a small last-member ISIZE could overflow the undersized output buffer.

The streaming Gunzip class gained concatenated-member support in v0.8.0 (101arrowz#102)
but the sync path was missed. Walk the members in gunzipSync the same way:
decode a member, advance past its header, DEFLATE data, and 8-byte trailer via
the bit position inflt leaves behind, and repeat until the input is exhausted,
then concatenate. Each member's output is sized by inflt itself, since the
trailing ISIZE only describes the last member.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant