RFC: package manifest extensions#889
Conversation
|
@owlstronaut what does the team think about this? This one is essential for isolated mode to work with packages with phantom dependencies. We are already seeing blockers for our migration to isolated mode in Gutenberg. I will be more than happy to draft a PR to the CLI repo. |
owlstronaut
left a comment
There was a problem hiding this comment.
I like the shape of this. A couple comments.
Thank you for the review @owlstronaut. On a side note, I would also like to ask a broader design question before going too far down the declarative path: would the npm team be open to adopting an imperative, root-owned manifest hook API, similar to pnpm's The current RFC intentionally proposes a declarative v1 because it is easier to validate, lock, audit, explain, and keep deterministic under An imperative Would maintainers prefer that this RFC stay focused on declarative |
|
Here is the implementation PR npm/cli#9496 The reason for the above question about imperative API, is that the list in the package.json can get too long. For example, while testing the implementation in Gutenberg, here is what the list looked like 😄
|
4e4daf7 to
463cd08
Compare
|
@manzoorwanijk for a package the size of Gutenberg only having that size of a list, it makes me like the declarative even more. The package.json is ugly, but that's among the worst it'll get and is a lot safer. We can always revisit |
True. We can revisit this later. Some benefits of an imperative API are that you can:
|
|
If this is going to go in package.json, then it probably shouldn't work in packages that don't have private:true, since it'll greatly inflate the size of the packument. |
Good point. Since There is precedent from the native dependency patching work: Would it be acceptable for this RFC to require the same publishing behavior for That would let public packages use |
|
Failing seems better, since it more firmly establishes that it's only for apps. If we want packages to use it, I think it needs to go somewhere other than package.json. |
One concern with failing For example, a public library may have a single root Should this RFC intentionally make that unsupported because |
|
If that's a use case we want to support, then imo we must provide a non-package.json place to store this config. Until that time, that use case is simply not permitted. |
Then in that case, the imperative API seems more compelling. |
|
Since the imperative API needs its own RFC (IMHO), for this RFC, we can simply err in case a non-private package contains |
|
That's definitely the most futureproof approach - even if we decide to simply remove the restriction in the future. Adding features for private packages only is FAR FAR less dangerous than adding any feature for published packages. |
|
Alright, the implementation PR is also updated with the same - npm/cli@8803bd7 |
50520f9 to
1c42023
Compare
…9496) Implements package manifest extensions per [RFC #889](npm/rfcs#889): a root-only `packageExtensions` field in `package.json` that applies declarative repairs to third-party dependency manifests **before** Arborist finalizes the ideal tree. It lets a project add missing `dependencies`/`optionalDependencies`, add or correct `peerDependencies`, and mark peers optional via `peerDependenciesMeta`, without forking and republishing a package. ```json { "packageExtensions": { "broken-package@1": { "dependencies": { "missing-runtime-dep": "^2.0.0" } }, "typescript-plugin@4.3.0": { "peerDependencies": { "typescript": ">=5" }, "peerDependenciesMeta": { "typescript": { "optional": true } } } } } ``` ## Why `install-strategy=linked` gives installs strong package boundaries, which is also what makes adoption hard: a package only sees what it actually declared, so one that worked under a hoisted layout because a dependency happened to be hoisted above it can fail. A root-level dependency masks this under hoisting but does not make the package available inside the isolated boundary of the importer — the repair has to be attached to the broken package's manifest before its edges are resolved. This is the pre-resolution complement to `overrides` (which needs an existing edge to retarget) and to [native dependency patching #9439](#9439) (which edits package contents after resolution). ## The field Each key is a package selector: a name with an optional semver range (`foo`, `foo@1`, `@scope/foo@^2.3.0`). Selectors match a candidate's own manifest `name`/`version` (the underlying name for aliases) and reject dist-tag, git, file, URL, and `npm:` specs. At most one selector may match a candidate. Honored only in the root `package.json` (the workspace root); the field in dependencies and non-root workspaces, and selectors matching a workspace member, are ignored with a warning — matching the root-authority model of `overrides`. ## Merge semantics Only the four resolution-affecting fields may be extended. - `dependencies`/`optionalDependencies` add a missing name only; providing a name already declared in either field is an error (use `overrides` to change a version), which also forbids moving a name between the two. - `peerDependencies` shallow-merges by name, replacing an existing range. - `peerDependenciesMeta` merges by name then key (e.g. add `optional: true`); every meta entry must have a corresponding `peerDependencies` entry. - Deletion (`null`/`false`/`"-"`) is not supported. The extension applies to a per-tree manifest copy: the shared pacote/cache manifest is never mutated, the installed `node_modules/<pkg>/package.json` is not rewritten, and `bundleDependencies` is unchanged. `overrides` still controls the final resolution target of an extension-created edge. ## Lockfile The root entry stores a canonical `packageExtensionsHash`, and each affected entry stores minimal provenance (`packageExtensionsApplied`); effective dependency metadata is recorded as usual. Extension state forces `lockfileVersion: 4` so older npm clients abort rather than silently dropping the repaired graph. `npm install` re-resolves affected packages when the rule set changes; `npm ci` validates the hash, selector conflicts, and stale provenance before trusting the locked metadata. ## Visibility `npm explain` appends `(added by packageExtensions["foo@1"].dependencies.bar)` to the edge; `npm ls` annotates the node and `npm ls --json` includes `packageExtensionsApplied`. Publishing a non-private package containing the field warns that it does not affect consumers. ## Notes - `lockfileVersion: 4` is shared with native dependency patching ([#9439](#9439)) as a common "older npm must not silently drop this" tripwire; both bump only when their own state is present. Whichever lands second should reuse the same `maxLockfileVersion`/bump constants rather than introduce a competing version. - Opt-in and additive, so it can ship in a minor release. ## References Implements npm/rfcs#889
|
I have created an RFC for the imperative API - #903 |
## Summary Adds an RFC for a root-owned `.npm-extension.mjs` / `.npm-extension.cjs` file with a top-level `transformManifest(pkg, context)` extension point. The proposal lets a project imperatively repair third-party package manifests before Arborist reads dependency and peer edges. It builds on the accepted [Package Manifest Extensions proposal](#889) and its [npm CLI implementation](npm/cli#9496), which established the pre-resolution manifest repair phase, root-only authority model, lockfile visibility model, and publish isolation for local dependency metadata repairs. `packageExtensions` remains the safer declarative default for common metadata repairs. `.npm-extension` covers the cases where projects need comments, upstream issue links, repeated transformations, conditional logic, reading existing manifest values, deletion, dependency-range replacement, or a local policy file that does not live in publishable `package.json`. ## Motivation `install-strategy=linked` makes dependency boundaries stricter by avoiding accidental hoisting. That is useful for correctness, but it exposes packages that import dependencies or type packages they did not declare. The accepted `packageExtensions` RFC handles the most common form of this problem: small deterministic additions to dependency and peer metadata. Some repairs are harder to keep clear as declarative JSON. During the [Package Manifest Extensions proposal discussion](#889 (comment)), a Gutenberg migration example repeated the same optional `@types/react` peer repair across many React-related dependencies. The declarative form worked, but the underlying policy was really a named list or predicate: "these packages import React types but do not declare an optional `@types/react` peer." Other local repairs need conditional logic, such as adding a type package only when a matching runtime peer exists, copying an existing peer range into a type dependency, narrowing a known bad peer range, or throwing when upstream has fixed metadata and a local repair should be removed. `packageExtensions` intentionally does not support that kind of code. This RFC proposes an explicit advanced escape hatch for those cases while preserving npm's root-owned authority model, lockfile visibility, and publish isolation. ## Why a separate extension file The RFC intentionally keeps executable policy out of `package.json`. A public package may need local dependency repairs for its own tests, build, or linked-install migration, but it should not publish root-only install policy to the registry manifest or packument. `.npm-extension` is also deliberately not named `.npmfile` or shaped like pnpm's `hooks.readPackage`. The proposal borrows the useful manifest-transform idea from pnpm, but defines npm-specific semantics for trust, lockfile hashing, `npm ci`, publish exclusion, disable behavior, supported mutations, and future extension points. ## Notable semantics - Only the root project owns `.npm-extension`. - Workspace package manifests are not extension targets; non-root workspace `.npm-extension` files warn and are ignored. - Dependency package `.npm-extension` files are ignored. - The only extension point in this RFC is top-level `transformManifest(pkg, context)`. - The extension point runs synchronously before dependency and peer edges are read. - Supported output changes are limited to `dependencies`, `optionalDependencies`, `peerDependencies`, and `peerDependenciesMeta`. - Unlike `packageExtensions`, the imperative form can delete supported dependency entries and replace existing normal dependency ranges. - Unsupported field changes such as `scripts`, `bin`, `exports`, `types`, and `bundleDependencies` are rejected. - `transformManifest` runs for non-root, non-workspace dependency manifests from registry, git, remote tarball, local file, local directory, and symlinked dependency sources. - npm records an extension entry-file hash and minimal `npmExtensionApplied` provenance in `package-lock.json`. - `npm ci` verifies matching extension state without importing or executing `.npm-extension`. - Changed extension file bytes make `npm install` re-run `transformManifest` across candidate manifests rather than relying on selector-based selective re-resolution. - Extension-created, extension-changed, and extension-removed dependency metadata is visible through lockfile provenance and npm inspection output. - `.npm-extension.mjs` and `.npm-extension.cjs` are excluded from `npm pack` and `npm publish`, even when listed in `files`. - `ignore-extension=true` disables extension execution, and `ignore-scripts=true` implies `ignore-extension=true` for commands that would otherwise execute it. ## Relationship to `packageExtensions` This RFC is not trying to replace `packageExtensions`. The declarative feature should remain the first choice for small, reviewable manifest repairs. The imperative extension file is for cases where the declarative shape becomes repetitive, cannot express the needed local policy, or cannot live in a publishable package manifest. The RFC keeps the two features ordered and auditable: `transformManifest` runs before `packageExtensions`, then npm reads dependency and peer edges from the resulting effective manifest. That lets imperative repairs inspect the upstream manifest before declarative repairs are applied, while preserving the accepted `packageExtensions` validation and provenance model. ## References Follow up of #889 --- > **Disclosure**: [Codex](https://chatgpt.com/codex/) and [Claude Code](https://claude.com/claude-code) were used to draft the initial version of this RFC and to iterate on it during review.
Summary
Adds an RFC for root-owned
packageExtensions, a declarative v1 package metadata repair mechanism for npm installs.The proposal lets a project add or correct third-party package manifest fields that affect dependency graph construction before Arborist finalizes the ideal tree:
dependenciesoptionalDependenciespeerDependenciespeerDependenciesMetaMotivation
install-strategy=linkedmakes dependency boundaries stricter by avoiding accidental hoisting. That is useful for correctness, but it also exposes packages that import dependencies or type packages they did not declare. Today users often work around those issues by relying on hoisting, adding root dependencies, patching packages after extraction, or maintaining forks. Those options either do not work under linked installs or operate at the wrong phase of installation.RFC 0042: Isolated mode anticipated this exact problem space:
It also described the older workaround:
That workaround is not enough for
install-strategy=linked, because a root dependency does not become visible inside the isolated dependency boundary of the package that actually imports it.This RFC proposes a root-only, deterministic way to record small third-party manifest repairs while upstream packages catch up. It intentionally scopes v1 to declarative metadata repairs rather than arbitrary install-time manifest hooks.
Why declarative v1
An imperative hook model, similar to pnpm's
.pnpmfile.mjs, could solve a broader class of manifest transformation problems. This RFC proposes the declarative subset first because the linked-install migration cases are mostly small dependency metadata repairs, and a declarative model is easier to validate, lock, audit, explain, and remove once upstream packages are fixed.Notable semantics
packageExtensions.nameandversion, not install path.npm civalidates the canonical extension hash, selector conflicts, and minimal lockfile provenance before trusting locked effective metadata.package.jsonis not rewritten.Prior art
The RFC compares this proposal with pnpm
packageExtensions, pnpm.pnpmfile.mjshooks, YarnpackageExtensions,@yarnpkg/extensions, npmoverrides, npm isolated mode, native dependency patching, and install-script policy RFCs.Tests
Not run. This is a docs-only RFC proposal.