Skip to content

refactor: replace tar extraction with modern-tar #3353

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
ayuhito wants to merge 1 commit into
base: main
Choose a base branch
from
+14 −22
Open

refactor: replace tar extraction with modern-tar #3353

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 13 additions & 21 deletions lib/install.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,12 +1,13 @@
'use strict'

const { createWriteStream, promises: fs } = require('graceful-fs')
const { createReadStream, createWriteStream, promises: fs } = require('graceful-fs')
const os = require('os')
const { backOff } = require('exponential-backoff')
const tar = require('tar')
const { unpackTar } = require('modern-tar/fs')
const path = require('path')
const { Transform, promises: { pipeline } } = require('stream')
const crypto = require('crypto')
const { createGunzip } = require('zlib')
const log = require('./log')
const semver = require('semver')
const { download } = require('./download')
Expand Down Expand Up @@ -173,7 +174,6 @@ async function install (gyp, argv) {

// now download the node tarball
const tarPath = gyp.opts.tarball
let extractErrors = false
let extractCount = 0
const contentShasums = {}
const expectShasums = {}
Expand All @@ -192,10 +192,11 @@ async function install (gyp, argv) {
return isValid
}

function onwarn (code, message) {
extractErrors = true
log.error('error while extracting tarball', code, message)
}
const extract = () => unpackTar(tarExtractDir, {
strict: true,

@ayuhito ayuhito Aug 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the only section I'm somewhat unsure about if it may regress. I don't think the differences are large, but I do want to point it out.

The previous behavior was very strict, relying on the onwarn handler to throw. strict: true is the same thing, but there still may be minor differences on what our definitions of strict are. e.g. node-tar accepts an archive with a single EOF zero block, while modern-tar strict mode rejects it as truncated.

This is fine for the normal path from Node.js sources, but might affect --tarball and --dist-url? How do we feel about that?

strip: 1,
filter: header => isValid(header.name)
})

// download the tarball and extract!
// Omitted on Windows if only new node.lib is required
Expand All @@ -208,13 +209,8 @@ async function install (gyp, argv) {
try {
if (shouldDownloadTarball) {
if (tarPath) {
await tar.extract({
file: tarPath,
strip: 1,
filter: isValid,
onwarn,
cwd: tarExtractDir
})
// modern-tar recommends 256 KiB chunks for optimal extraction performance
await pipeline(createReadStream(tarPath, { highWaterMark: 256 * 1024 }), createGunzip(), extract())
} else {
try {
const res = await download(gyp, release.tarballUrl)
Expand All @@ -231,12 +227,8 @@ async function install (gyp, argv) {
contentShasums[filename] = checksum
log.verbose('content checksum', filename, checksum)
}),
tar.extract({
strip: 1,
cwd: tarExtractDir,
filter: isValid,
onwarn
})
createGunzip(),
extract()
)
} catch (err) {
// something went wrong downloading the tarball?
Expand All @@ -250,7 +242,7 @@ async function install (gyp, argv) {
}

// invoked after the tarball has finished being extracted
if (extractErrors || extractCount === 0) {
if (extractCount === 0) {
throw new Error('There was a fatal problem while downloading/extracting the tarball')
}

Expand Down
2 changes: 1 addition & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
"nopt": "^10.0.0",
"proc-log": "^7.0.0",
"semver": "^7.3.5",
"tar": "^7.5.4",
"modern-tar": "^0.8.4",
"tinyglobby": "^0.2.12",
"undici": "^8.4.1",
"which": "^7.0.0"
Expand Down