RFC: shared side-effects cache - #20
Conversation
pnpm's side-effects cache is local to one machine, so every CI runner and every fresh container re-runs node-gyp for the same native module with byte-identical inputs. The key, the payload format, and the read/write split already exist; what is missing is a transport and a trust model. Proposes that a registry may serve the built form of a package it already serves, fetched from the registry the package resolved from. Attaching artifacts to registries rather than to a standalone cache service means no new principal is trusted to write into node_modules, and reuses the existing per-registry auth, the registry-qualified lockfile keys for routing, and pnpr's route classification, storage split, and package rules. Two prerequisites are called out as blocking. Platform tags: the current key carries no libc, so a musl and a glibc builder compute the same key for a from-source build; nothing may cross a machine boundary until that is fixed. And the build allowlist: a cached side-effect currently bypasses the allowBuild check, which is an annoyance locally and a hole once the artifact came from elsewhere. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
Important Review skippedDraft detected. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
The endpoint sketch was per-package, which is the same flaw the REAPI section was criticising. Cache-hit determination is now one whole-install request mirroring POST /-/pnpr/v0/resolve, and should fold into an existing resolve round trip where the client is already talking to pnpr. Rewrites the REAPI alternative. The previous text claimed REAPI carries no notion of platform tags or provenance, which is wrong: Platform properties are explicitly for OS/toolchain and are part of the Action digest, ExecutedActionMetadata records the producing worker, and the ActionCache works without the Execution service. The real objection is performance. ActionCache exposes only GetActionResult/UpdateActionResult and the request carries a single action_digest, so cache-hit determination is one round trip per package with no batch form, reintroducing per-package chatter where pnpm and pnpr engineered it out. ActionResult also carries a full output tree, scaling the response with package size rather than build-output size, and the TypeScript client would need grpc-js bundled. Two non-performance objections stand: the key would come from a synthesized Command no worker can execute, and range-vs-exact platform matching is server-dependent, which a format served by many registries cannot rely on. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
Pushed 16b81b8, deciding the REAPI question on performance grounds and fixing a flaw the comparison exposed in the original design. The endpoint sketch was per-package — The REAPI decline is rewritten, and the previous version was wrong on its main point. It claimed REAPI carries no notion of platform tags or provenance. It does: The decline stands on round trips instead. Compounding: Two non-performance objections also stand: Left as revisit triggers: a batched Written by an agent (Claude Code, claude-opus-5). |
Review found the previous draft unsound in three places and factually wrong in three others. The dependency-state key is not a function of everything a build depends on, and for arbitrary npm scripts it cannot be made one: env vars, compilers, SDKs, system libraries, CPU features, network, and the clock are all reachable. libc was one omission, not the blocker. Two honest builders could publish different outputs under one key. Adds a declared builder profile and a package eligibility contract, and notes that sandboxing is what actually closes the input set rather than describing it. This is also where the Nix analogy breaks, since Nix models inputs and builds restricted. Floor-based tags cannot be hashed into an exact lookup key: a glibc 2.39 client would never request a 2.17 producer's key. Splits the input key from advertised compatibility constraints, with wheel-style selection from the client's ordered supported-tag set. The trust argument was incomplete. A frozen install pins tarball integrity, so a registry cannot substitute source bytes; nothing pins an artifact, so remote artifacts grant strictly more authority than a registry has today, and onlyBuiltDependencies does not cover accepting another builder's output. Remote reads become a separate per-registry opt-in rather than a widening of sideEffectsCacheRead, and provenance is signed over protocol version, source integrity, input key, constraints, manifest digest, and builder identity. Adds mandatory manifest validation before remote paths reach the importer; the existing sanitizeFilenames helper is a post-failure compatibility retry, not a security boundary. Batching is now per registry and parallel, since artifacts belong to the package's registry and broadcasting all candidates would leak cross-registry dependency information. Corrections: the local allowlist bypass was fixed by merged pnpm/pnpm#11039, so the RFC now requires preserving that gate rather than claiming to fix it. REAPI's ActionResult does not necessarily carry a full tree, since Command declares which outputs are captured. And concurrent gRPC RPCs are not serialized round trips, so raw round-trip count is no longer claimed as decisive; REAPI is now declined on its exact-digest lookup model being unable to express compatibility selection, with batching left as a benchmark question and a hybrid named as the route back. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
Reviewed all six points against the code and the specs. All six are valid; three correct factual errors in the draft, two of which were load-bearing for the REAPI decision. Rewritten in 4c01826. Accepted as design changes1 — build-input closure. Correct, and the deepest point. The draft said the key "proves" identical inputs; it does not, and for arbitrary npm scripts it cannot be made to. Env vars, compilers, SDKs, system libraries, CPU features, network, and the clock are all reachable from a lifecycle script, so libc was one omission rather than the blocker. Two honest builders could publish different outputs under one key. The RFC now carries a build-input closure section proposing both remedies — a declared, versioned builder profile and a package eligibility contract — and adopts the framing that this is where the Nix analogy breaks, since Nix models inputs and builds restricted. It also connects sandboxing (pnpm/pnpm#13772) as the mechanism that closes the input set rather than merely describing it. 2 — trust. Correct and sharp. In a frozen install the lockfile pins tarball integrity, so a registry cannot substitute source bytes undetected; nothing pins an artifact. Remote artifacts therefore grant strictly more authority than a registry has today, and 3 — floor semantics vs exact lookup. Correct; the draft was incoherent. It invoked manylinux while designing an exact-match lookup, so a glibc 2.39 client would never request a 2.17 producer's key. Now split into an input key (what was built, including the builder profile) and advertised compatibility constraints, with wheel-style selection from the client's ordered supported-tag set. Thanks for the PyPA reference — cited. 4 — manifest validation. Correct. Worth adding for the record: 5 — per-registry batching. Correct. Now grouped by concrete registry and issued in parallel, with the leak argument stated explicitly: broadcasting the full candidate set would expose an organization's cross-registry dependency graph to every registry it talks to. Corrections6a — pnpm/pnpm#11035 was fixed by merged pnpm/pnpm#11039. Confirmed: merged 2026-03-21, gate live at 6b — 6c — concurrent RPCs are not serialized round trips. Correct, and this one mattered most, since round-trip count was the stated basis for the decision. Raw round-trip count is no longer claimed as decisive; it moves to Unresolved Questions as something to benchmark, including at small candidate counts where one request may not pay for itself. The REAPI decision stands, on different groundsPoint 3 turns out to strengthen the case rather than weaken it. Three narrower mismatches now stated instead of the withdrawn ones: the synthesized Your hybrid suggestion — REAPI CAS and action cache plus a small batched-lookup extension — is now named in the RFC as the most likely route back to this alternative, alongside a batched Written by an agent (Claude Code, claude-opus-5). |
Second review round. Five findings, all accepted. Source visibility cannot determine artifact scope. Reusing pnpr's public/private route classification directly was wrong: a public source package built by one organization still yields an artifact that can reveal builder identity, embed organization-specific output, or differ from another organization's build of the same package. Artifacts are namespaced by tenant and signing trust domain, and publishing beyond that namespace is an explicit decision rather than a consequence of the tarball having been public. Signer trust was filed as bikeshedding and is actually load-bearing. A registry that signs its own artifacts defends against nothing it is not already trusted for, so the signing key must be independent of the registry; V1 is one organization-configured CI key per registry, held outside the server, with identifiers, rotation, and revocation. Several builders producing entries for one input key is normal, so builder identity participates in variant identity: the registry stores variants and the client selects one signed by a key it trusts. The response shape changes accordingly, to a set of variants per candidate. Blob verification is normative again — it was lost in the previous rewrite. Clients MUST recompute every blob digest before it reaches CAFS or the importer, with mismatches quarantined so a poisoned entry is not re-fetched every install. Sandboxing alone does not close the input set; only a hermetic sandbox with declared inputs does, and 13772 leaves network policy open. States network and CPU feature detection as the residual gaps, and folds in the container-image builder profile: image digest plus architecture baseline plus an environment allowlist, with the explicit caveat that it is a producer-side claim, not a verifiable property. Adds that sharing needs behavioural equivalence, not bit-identity. Prunes the REAPI decline to its one sufficient objection. Deletions are not an obstacle, since a manifest carrying its own deleted list can be one opaque ActionResult output blob, and server-dependent Platform matching governs execution placement rather than cache lookup. Wording: candidates are packages marked requiresBuild, which also covers implicit node-gyp via binding.gyp and .hooks, plus patched packages; and artifacts are regenerable derived data rather than reproducible. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
All five accepted, plus both wording points. Pushed e133aee. 1 — source visibility cannot determine artifact scope. Correct, and a real design error rather than an imprecision: I said side-effects entries "reuse 2 — signer trust and multi-builder selection. Agreed on both halves, and agreed they were misfiled as bikeshedding. A registry signing its own artifacts defends against nothing it isn't already trusted for and degrades the signature to a transport checksum — that's now stated as the load-bearing constraint in the trust section, with your V1 adopted verbatim: one organization-configured CI signing key per registry, held outside the registry server, with key IDs, rotation, and revocation defined up front. The multi-builder point had a consequence I hadn't seen: it changes the response shape. A response now carries a set of variants per candidate, and the client selects on compatibility and signer trust. That also composes with the tag model — selection was already client-side, so signer trust is a second axis on the same step rather than new machinery. 3 — blob verification. Correct, and it was a regression: v1 had "verified against their digests exactly as store content is today" and I dropped it while restructuring. Now normative — clients MUST recompute every blob digest before it reaches CAFS or the importer, since the signed manifest attests the digests but nothing attests the served bytes match them. Mismatch is a miss plus a diagnostic, with the entry quarantined so the same poisoned artifact isn't re-fetched on every install. 4 — hermetic, not merely sandboxed. Correct. Now "a hermetic sandbox with declared inputs", with the reason spelled out: 13772 is about restricting authority and leaves network policy open, and a sandbox permitting network closes nothing. Network is named as the largest residual gap ( This also let me fold in the container-profile question that prompted the round: profile = image digest + architecture baseline + environment allowlist, since 5 — REAPI objections. Both concessions correct. Deletions dissolve, since a manifest carrying its own Wording. One new unresolved question falls out of the container profile: an image digest may be too fine. A base-image security patch changes it and invalidates every artifact built under it, though almost nothing relevant to the build changed. Some notion of profile equivalence or lineage may be needed for hit rates to stay usable. Also recorded: whether pnpr ever builds, since that would make it a build service — remote execution, a declared non-goal, and REAPI's actual domain. Written by an agent (Claude Code, claude-opus-5). |
Putting the builder profile in the input key was over-engineering inherited from a scope this RFC had already declined. It fragments the cache along an axis that mostly does not affect correctness: a routine base-image security patch changes the digest and invalidates every artifact built under it, in exactly the CI scenario the feature exists to serve. It is also a producer-side claim rather than a verifiable property, so it could never have carried the weight of a correctness gate. Turborepo, Nx, and moon model the build environment not at all. They can omit it because their caches are team-scoped and their artifacts first-party, so the organization already owns correctness and nobody is defending against another organization having built something wrong. This RFC had already committed to the same scope and should have taken the same consequence. The profile becomes signed provenance, kept for debugging, incident response, and optional policy. Correctness moves entirely to the compatibility tags, which express floors and therefore stay valid when a builder's image is patched. The input key is the dependency graph and patches; platform identity leaves it for the tags. Reframes the section from closing the input set to bounding it, via three means: a trust domain that owns the residual risk, compatibility constraints as the hard gate, and package eligibility for what is inherently unshareable. Hermetic sandboxing stays as the thing that would let the trust domain widen later, not as a prerequisite. Resolves the profile-granularity question rather than answering it. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
Pushed 73cbdf0. The builder profile moves out of the cache key and becomes signed provenance; correctness moves entirely to the compatibility tags. Why. Putting the profile in the input key was over-engineering inherited from a scope this RFC had already declined. It fragments the cache along an axis that mostly does not affect correctness — a routine base-image security patch changes the digest and invalidates every artifact built under it, in exactly the CI scenario the feature exists for. And it is a producer-side claim, not a verifiable property, so it could never have carried the weight of a correctness gate anyway. The precedent is the three tools in Prior Art: none of them models the build environment at all. They can omit it because their caches are team-scoped and their artifacts first-party — the organization already owns correctness and nobody is defending against another organization having built something wrong. This RFC had already committed to that same scope in declining a public cross-organization cache, and should have taken the same consequence. What changed:
This resolves the profile-granularity question from the last round rather than answering it — that entry is removed from Unresolved Questions. The security work from rounds one and two is untouched: independent signing key, tenant and signing-trust-domain namespacing, multi-builder variant selection, normative blob verification, manifest validation, and the preserved Written by an agent (Claude Code, claude-opus-5). |
Third review round. Four findings, all accepted. A verified artifact must not become indistinguishable from a locally built one once stored. The store is global and shared across every project on a machine, so an unlabelled remote entry would be reused after its signing key was revoked, after remote artifacts were disabled, by a different project or tenant sharing the store, or for a package that now resolves through a different registry — the last defeating the tenant namespacing this design depends on. sideEffectsMaps carries only added and deleted, so remote entries cannot be fed into it and forgotten. They retain origin metadata (trust domain, registry, signer key id, builder profile, signed envelope, verification status) and trust policy is applied at every reuse rather than only at download. The implementation section no longer claims nothing downstream changes, because it does. The profile binds environment values, not names: CFLAGS=-march=x86-64 and CFLAGS=-march=native pass the same name allowlist and are not interchangeable. It is a canonical name-to-value map with undeclared variables denied, and secret-bearing variables denied rather than recorded, since provenance is served to clients. A flag like -march=native makes the artifact's real requirements unknowable from the environment, which is a compatibility-tag problem: such a build declares a narrower architecture tag or is ineligible. Patch-only packages are no longer candidates. They conflicted with the allowBuild requirement, which has nothing to approve for a package with no build; the patch hash rides the input key when a requiresBuild package is also patched, and patch overlays are cheap to apply locally. Hardening: the signature now covers the tenant/trust-domain identifier so an artifact cannot be replayed across domains, and the client caps variants per candidate and total response size, since the response is untrusted input parsed before any signature is checked. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
All four accepted, pushed in 66fe1c7. Checked each against the current draft first — the builder-profile change in 73cbdf0 landed after this round was written, and it alters one of them rather than voiding it. 1 — remote provenance must survive local caching. Fully valid, entirely unaddressed by the intervening change, and the sharpest finding across the three rounds. The store is global and shared across every project on a machine, so the fourth case listed is the worst: an unlabelled remote entry reused for a package that now resolves through a different registry defeats the tenant namespacing that the previous round established as the whole point. The RFC now has a Remote artifacts stay labelled once stored section: origin metadata persists (trust domain, registry, signer key ID, builder profile, signed envelope, verification status), and trust policy is applied at every reuse, not only at download — so a revoked key, a registry no longer opted in, or a mismatched trust domain causes the artifact to be discarded and the package built locally. Implementation choice between fields-on-entries and a separate remote index is left open, since the requirement is the invariant, not the layout. This also retires a claim I had been leaning on. The implementation section said remote artifacts feed the existing 2 — profile must bind values, not names. Valid, and 73cbdf0 changes its consequence rather than removing it. The profile is no longer in the input key, so Your point about sensitive variables gets stronger under the new design, not weaker: there's no key to hash them into, but provenance is served to clients, so a name allowlist that merely permits them would leak them. They're now denied rather than recorded. 3 — patched-only candidates. Correct, and it was my inconsistency: I added patched packages in round two from the 4 — hardening. Both taken. The signature now covers the tenant/trust-domain identifier, which closes cross-domain replay and is the natural completion of the previous round's namespacing finding. Variants per candidate and total response size are capped, with the reason stated: the response is untrusted input parsed before any signature is checked. Written by an agent (Claude Code, claude-opus-5). |
Compatibility tags were described as making an artifact safe to use. They are a signed producer assertion constraining where it may be used; safety rests on the trusted builder and the eligibility contract. A trust-policy rejection now says the artifact is not reused for this install rather than discarded. The store is global, so another project may still legitimately trust the same artifact, and deletion would be wrong. This is distinct from the digest-mismatch path, where quarantine remains correct because the served bytes did not match the signed manifest. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
Both taken, pushed in 200c1d4. Tags constrain use, they don't confer safety. Correct, and the previous phrasing quietly undercut the section it sits in — the whole point of "bounding rather than closing" is that safety rests on the trusted builder and the eligibility contract. Now: the signature makes the claim attributable; the tags are a signed producer assertion constraining where an artifact may be used. "Not reused for this install" rather than "discarded." Correct, and the more consequential of the two given the store is global — the old wording would have licensed an implementation that deletes an artifact another project still legitimately trusts, turning a per-project policy decision into machine-wide collateral damage. Worth noting the distinction this creates, which the RFC now keeps explicit: trust-policy rejection is scoped non-reuse, while the digest-mismatch path stays quarantine-and-evict. Those are different failures — one is "valid artifact, wrong context for me", the other is "the bytes served did not match the signed manifest", and only the second is evidence of something poisoned. Written by an agent (Claude Code, claude-opus-5). |
The exclusion of a public cross-organization cache was too broad. It rested on there being no party who owns correctness, which is false when the package's own publisher is the one publishing its built forms — the wheels arrangement, where PyPI serves sdists and wheels under one identity. That case is more defensible than the team cache, not less. A consumer who installs a package already executes that publisher's install script, so accepting a prebuilt artifact signed by the same identity is a smaller grant than arbitrary code execution during install. It also needs no new trust root, since the identity authenticating the artifact is the one already authenticating the tarball. And the ecosystem already does this through prebuild-install and node-pre-gyp, without integrity pinning, signatures, or offline support, which is both evidence the demand is real and an argument that a first-class mechanism would replace something worse rather than add surface. It stays the second step for two reasons that are about sequencing rather than design: it depends on registry adoption pnpm does not control, and it demands exact compatibility tags where a team cache can be approximate and self-correct. The V1 decision that keeps the door open is the signer becoming a pluggable trust root. The organizational CI key is now the only kind V1 defines rather than an assumption baked into the protocol shape, since baking it in would force a breaking change to add publisher identity later. What remains declined is the anonymous multi-builder cache, where anyone may contribute an artifact for anyone's package and the residual risk has no owner. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
Pushed 9c868db. The public-cache exclusion was too broad and is now scoped to the case it was actually about. What changed. The RFC said a public cross-organization cache is out of scope because "the residual risk has nowhere to land." That is false when the package's own publisher publishes its built forms — the publisher is precisely the owner. It is the wheels arrangement the RFC already cites for the tag design, now applied to the trust model too. Four edits:
Why it is still the second step, stated as sequencing rather than design objection: it depends on New unresolved question, which is the real open decision: how much of the publisher path to design against now. The trust root is pluggable, so identity is safe to defer — but whether the tag format and eligibility assertion should be designed against the harder publisher case from the start, or shaped by the team cache and generalized later at the risk of a format that does not stretch, is not settled. Written by an agent (Claude Code, claude-opus-5). |
Organization-signed and publisher-signed artifacts are both in scope. They are the same protocol with a different signer and a different party asserting eligibility, so the RFC now specifies both rather than treating the publisher case as a later extension. The sequencing argument for deferring it was weaker than stated. Publisher mode was said to depend on registry.npmjs.org accepting and serving artifacts, but with publisher signing the distribution channel does not have to be trusted: the signature is the authority, so a mirror, a proxying pnpr, or the publisher's existing distribution point can serve an artifact the client accepts. What actually remains unspecified is discovery — how a client learns where a publisher's artifacts live — which replaces registry adoption in the open questions. Consequences settled here rather than deferred: the pluggable trust root defines organization keys and publisher identity as peers, neither privileged; and the compatibility tag format is designed against publisher mode, since a publisher targets every consumer and must get floors right the first time, where a team cache can be approximate and corrected in an afternoon. A format shaped by the easier case would not stretch. Reframes the trust-domain argument as a named owner, which both modes supply, and separates the registry as default channel from the signature as authority. Organization mode is still expected to ship first, as a rollout order rather than a scope boundary. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
Pushed c8b84d9. Both modes are now specified, rather than one plus a future direction. Description updated to match. The sequencing argument I gave for deferring publisher mode was weaker than I wrote. I claimed it depends on So npm adoption is a convenience dependency, not a possibility one. What replaces it as the real gap is discovery — how a client learns where a publisher's artifacts live — now the main unspecified piece of publisher mode in Unresolved Questions. Two consequences are settled here rather than deferred:
Two reframings fell out, both of which I think improve the document independently:
Organization mode is still expected to ship first, since it is entirely within pnpm and pnpr and is a smaller surface on which to prove the protocol. That is now stated as a rollout order rather than a scope boundary. What remains out of scope is the anonymous, multi-builder cache, where anyone may contribute an artifact for anyone's package and the residual risk has no owner. Written by an agent (Claude Code, claude-opus-5). |
Fourth review round. The verdict was that c8b84d9 specified organization machinery plus publisher intent, which was accurate. Owner scope is now an explicit tagged union rather than a tenant identifier. An organization artifact is scoped to a trust domain and never shared across domains; a publisher artifact is scoped to a package identity and shared with every consumer that trusts the publisher. Collapsing both into one tenant field, as the previous draft did, cannot represent a publisher artifact at all. The signed envelope binds the owner scope, storage namespaces by it, and local origin metadata records it. The section defining it moves ahead of Trust, Fetching, and local storage, all of which now depend on it. Adds the immutability asymmetry, which was missing entirely. A tarball is immutable and lockfile-pinned, but an artifact can be published long afterwards, so a credential compromised in 2027 can attach new binaries to a version published in 2026 whose recorded integrity still verifies. Resolved in layers: pin the accepted manifest digest in the lockfile on first use, which restores immutability for anything a project has actually used, and acknowledge the unpinned case as mutable authority that opting in grants — the same property PyPI has for wheels. An artifact index digest in immutable release metadata would be stronger but forecloses adding a platform after release, so it is noted as the format's room to grow rather than proposed. Corrects the claim that publisher mode needs no new trust root. It reuses an existing identity anchor, but trusted publishing authenticates CI to the registry and provenance attests a package build, so artifacts need a new attestation subject and an authorization policy. Added as an unresolved question. Generalizes transport language that still assumed a registry: the opt-in attaches to the owner whose signature is trusted rather than to the host serving bytes, handshake is per channel with publisher channels needing none, and pnpr storage namespaces by owner scope and may serve publisher-owned artifacts it did not accept for publication. Two corrections. Prebuilt artifacts are not "inert data" — they are not executed during installation, but native binaries and generated JavaScript run later. And prebuildify bundles binaries inside the published tarball rather than downloading them, which makes it evidence for the opposite point: it buys correctness by shipping every platform to every consumer, which is the trade a signed artifact channel removes. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
Verdict accepted — "organization machinery plus publisher intent" was exactly right. Pushed 76099bc. Owner scope is now a tagged union, not a tenant identifier:
The two members behave oppositely on purpose: an organization artifact escaping its domain is a bug, a publisher artifact reaching every tenant is the point. The signed envelope binds owner scope, storage namespaces by it, and local origin metadata records it. I also moved the modes section ahead of Trust, Fetching, and local storage, since all three now depend on the union. Defining it after them was part of why the draft read as one mode with the other bolted on. Immutability was missing entirely, and it is the most serious of the four. A tarball is immutable and lockfile-pinned; an artifact can be published long afterwards, so a credential compromised in 2027 attaches new binaries to a version published in 2026 whose recorded integrity still verifies perfectly. Nothing in the lockfile changes and the signature is valid. Taken in layers rather than picking one of your two options:
Your artifact-index-digest option is stronger than both, and I have recorded why it is not proposed: it forecloses adding a platform after release, which publishers legitimately need. It is noted as the room the format should leave rather than as a rejected idea. "No new trust root" corrected. It reuses an identity anchor; trusted publishing authenticates CI to the registry and provenance attests a package build, so artifacts need their own attestation subject and an authorization policy for who may attest on a package's behalf. Now stated in place and added to Unresolved Questions. Stale transport language swept. The opt-in attaches to the owner whose signature is trusted rather than the host serving bytes; handshake is per channel, with publisher channels needing none since acceptance is by signature; pnpr namespaces storage by owner scope and may serve publisher-owned artifacts it did not accept for publication. Both corrections taken. "Inert data" was wrong — replaced with "not executed during installation", noting native binaries and generated JS run later. And prebuildify was misdescribed; verified against its README, it bundles binaries in the published tarball. That makes it evidence for the opposite point, so it now stands separately: it buys correctness by shipping every platform's binaries to every consumer, and that trade — bundle everything or fetch unverifiably at install time — is precisely what a signed artifact channel removes. A better argument for the design than the one it replaced. Written by an agent (Claude Code, claude-opus-5). |
Caching a workspace's own task outputs, in the manner of Turborepo, Nx, and moon, is the obvious later use of this machinery. It is not proposed and nothing here depends on it, but most of what this RFC specifies is general and a few decisions would foreclose the reuse if made carelessly. Transfers unchanged: the added/deleted manifest over CAS blobs, where a task output is the degenerate case of a file set with no base tree; the batched per-channel lookup and its folding into an existing round trip; signed provenance; owner scope, where a task cache is always organization and the publisher arm goes unused; trust policy at every reuse; blob verification; and manifest path validation, which matters more there because task outputs land in a working tree. The build-input argument is the same one Turborepo's inputs and env declarations answer. New is only the key: this RFC derives one from the lockfile, while a task key is source content, task config, environment, and upstream task keys. That is a different key producer feeding the same storage and transport. Three constraints follow, cheap now and breaking later, so they are stated as decisions: entry keys are opaque strings and must not embed package identity in the request or response shape; compatibility tags are optional, with absence meaning no constraint rather than no information; and the endpoint is named for the mechanism rather than for this application. The first two are also stated where the protocol is described, not only in the note. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
Added a Forward compatibility with a workspace task cache section in 352a8d1, plus the two constraints it implies stated inline where the protocol is described rather than only in the note. Context: caching a workspace's own task outputs — the Turborepo/Nx/moon job — is the obvious later use of this machinery. It is not proposed here and nothing depends on it, but most of what this RFC specifies is general, and a few decisions would foreclose the reuse if made carelessly. Recording them as decisions rather than leaving them to accident. What would transfer unchanged, which is most of it: the What is new is only the key. This RFC derives one from the lockfile; a task key is source content, task config, environment, and upstream task keys. That is a different key producer feeding the same storage and transport — additive, not a change to anything specified here. Three constraints, cheap now and breaking later:
One byproduct worth knowing, recorded as such rather than as a goal: because pnpm's store is content-addressed at file granularity, a task cache built on it would deduplicate identical files across tasks, packages, and dependency artifacts in a single store. Turborepo archives each task output separately, so a file present in ten outputs is stored ten times. That advantage comes from infrastructure pnpm already has, not from anything designed here. Written by an agent (Claude Code, claude-opus-5). |
The pin covered only the manifest digest, which leaves the artifact's claims about itself unprotected. Compatibility tags, owner scope, and provenance are signed alongside the manifest rather than being part of it, so a compromised signer could re-sign identical bytes with broadened compatibility — an artifact honestly tagged glibc >= 2.31 reissued as glibc >= 2.17, and thereafter selected on systems where it fails. The pin is now the digest of the complete signed envelope, keyed by input key and owner. Because a lockfile is shared across platforms, it is a set: one variant per platform in use, accepted only if its envelope digest is among them. Answers the two questions the pin left open. Pins accrue on lockfile-writing installs; a frozen install records nothing and accepts an unpinned platform on signature alone, since refusing would disable the feature exactly where it is most valuable. An unavailable pinned artifact falls back to a local build with a diagnostic rather than failing, with the caveat that a consumer who cannot build at all still fails, later and less clearly. Resolves a contradiction between owner-based trust and channel-based rejection. Reuse turns on the owner: the channel determines where a client looks and the signature determines what it accepts, so an artifact does not become untrustworthy because the registry it arrived through is no longer configured. The channel is retained for diagnostics only. Withdraws the standalone-cache rejection. Once channels are non-authoritative, a dedicated cache service is simply another channel and the design already permits arbitrary ones; the earlier objection that it introduced a new principal does not survive signature-based acceptance. What remains is a question of default, not of permission. Also drops the stale claim that the design stays inside a single trust domain, which publisher mode ended. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
The blocking issue is fixed and the three clarifications answered, in 4b0e23b. (Review was against 76099bc; the intervening commit only added the forward-compatibility section, so everything still applied.) The pin is now the digest of the complete signed envelope, keyed by input key and owner. The attack was real and I had missed it: compatibility tags, owner scope, and provenance are signed alongside the manifest rather than being part of it, so pinning the manifest digest alone leaves a compromised signer free to re-sign identical bytes with broadened claims — an artifact honestly tagged Your "multiple platform variants allowed" point is in as well, and it matters more than it first looks: a lockfile is shared across platforms, so the pin is a set — one variant per platform any contributor or runner has used, accepted only if its envelope digest is among them. A single-valued pin would have made the lockfile platform-specific. Frozen install with no pin. Pins accrue on lockfile-writing installs; a frozen install records nothing and accepts an unpinned platform on signature alone. So pins for the platforms a team develops on are committed by ordinary installs and protect CI thereafter, while a platform only ever seen in CI stays unpinned. I considered refusing unpinned artifacts on frozen installs and rejected it in the text: it would disable the feature exactly where it is most valuable. Unavailable pinned artifact. Falls back to a local build with a diagnostic rather than failing — an artifact is an optimization over building, and the package is still buildable. Stated with the honest caveat that a consumer who cannot build at all, such as a container with no toolchain, still fails, only later and with a worse message than a dedicated error would give. Owner versus channel. Real contradiction, resolved toward the owner. Reuse turns on who signed it; the channel determines where a client looks and the signature determines what it accepts, so an artifact does not become untrustworthy because the registry it arrived through is no longer configured. Channel is retained in origin metadata for diagnostics only, and that is now said explicitly where the metadata is listed. Both stale passages. The single-trust-domain claim in Prior Art is replaced with the named-owner framing that publisher mode requires. The standalone-cache rejection is withdrawn rather than updated, which I think is the right consequence of channels being non-authoritative: a dedicated cache service is simply another channel, and the design already permits arbitrary ones. The old objection — that it introduces a new principal permitted to write into Written by an agent (Claude Code, claude-opus-5). |
… union A flat set of envelope digests cannot distinguish a platform that was never pinned, where an unpinned artifact should be accepted, from a platform whose pinned artifact was replaced, where the substitute must be rejected. Both look like "this digest is not in the set". The pin is now a map from a canonical consumer platform fingerprint to an envelope digest. The fingerprint is computed from the consumer's own platform identity rather than from an artifact's tags, since tags are floors and several artifacts may match one consumer, so only the consumer's identity is a stable key. Compatibility constraints become an explicit union of universal and tagged. The previous "tags are optional, absence asserts no constraint" made missing data mean universal compatibility, which is absence of information becoming permission. Absent, malformed, or unknown-vocabulary constraints are now a miss. Opaque entry keys gain domain separation by artifact kind and version, and the signed envelope binds the kind, so a signature issued for one kind cannot be replayed as another. Without both, a party able to publish one kind could target the other's keyspace. Records that a future task cache needs declared output-root enforcement: path validation is sufficient for dependency artifacts, which land in a directory pnpm owns, but not for task outputs, which land in the user's working tree. Also drops the stale claim that V1 is scoped to one trust domain, which publisher mode ended. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
All of it applied in 9f38471, including the two blocking items. Platform-aware pin mapping. The flat set was genuinely broken and the counterexample is exact: a never-pinned macOS and a Linux whose pin was replaced both present as "this digest is not in the set", and they must resolve oppositely. The pin is now a map from a canonical consumer platform fingerprint to an envelope digest. The part worth calling out: the fingerprint is computed from the consumer's own platform identity, not from an artifact's advertised tags. Tags are floors, so several artifacts can match one consumer and the tags are not a stable key — only the consumer's identity is. Lookup then has three unambiguous outcomes: fingerprint present pins that platform and rejects any other envelope; fingerprint absent means unpinned and the unpinned rules apply. Two consumers legitimately pinning the same digest under different fingerprints costs an entry and buys exactly the distinction that was missing. The canonical form is now called out in Implementation as part of the lockfile format, stable across both stacks. Compatibility as an explicit union — and this was a bug I introduced. The forward-compatibility commit wrote "tags are optional; an artifact with none asserts no constraint", which makes missing data mean universal compatibility. That is absence of information becoming permission, in the one field that decides whether an artifact may run on a machine. Now Domain separation and kind binding. Keys are prefixed Declared output roots recorded as a task-cache constraint, with the reason: dependency artifacts land in a directory pnpm owns, task outputs land in the user's working tree, so well-formed relative paths are necessary but not sufficient there. Stale V1 single-domain claim in Prior Art replaced with the named-owner framing. The three remaining unresolved questions — attestation subject and authorization policy for publisher mode, artifact discovery, key distribution and revocation — are unchanged and still honestly scoped as needing their own rounds. Written by an agent (Claude Code, claude-opus-5). |
Three follow-ups from review. The forward-compatibility section still said compatibility tags are optional and that a missing field means no constraint, contradicting the explicit union added in the previous commit. Platform-independent output asserts universal; omission is never unconstrained. Defines the platform fingerprint as a hash over the consumer's canonically ordered supported-tag set plus the tag-schema version, rather than an informal OS/architecture tuple that would omit Node ABI, libc, CPU baseline, and any dimension added later. Defining it over the tag vocabulary means the fingerprint gains dimensions exactly when the vocabulary does, and the schema version turns a vocabulary change into a visible repin instead of a silent collision. The canonicalization is part of the lockfile format and must be byte-identical across both stacks. Adds the missing repin workflow. A pinned fingerprint rejects every other envelope, which left no way to accept a legitimately different artifact after signer rotation, cache eviction, or a corrected build. Three paths now exist: the input key changes and a fresh pin is recorded with no user action; the tag-schema version changes and repins as a visible diff; or the user asks explicitly through a lockfile-writing operation that drops the selected pins. Silent automatic repinning stays excluded, since a pin any install may rewrite protects nothing. A revoked signer needs no repin to be safe — trust policy rejects its artifacts before the pin is consulted — so repinning restores the cache rather than safety. Written by an agent (Claude Code, claude-opus-5) at the request of the repository owner.
|
All three in f605874. Thanks for the review — six rounds is what turned this from a sketch into something specifiable. The contradiction. Fixed. The forward-compatibility bullet still carried the old "optional tags" phrasing I had already replaced at the protocol definition — the exact hazard of stating a rule twice. It now says platform-independent output asserts Fingerprint definition. Taken as specified: a hash over the consumer's canonically ordered supported-tag set plus the tag-schema version. Your reasoning for it is the part worth preserving, so it is in the text — an informal OS/arch tuple silently omits Node ABI, libc, CPU baseline, and anything added later. Defining the fingerprint over the tag vocabulary means it gains dimensions exactly when the vocabulary does, and the schema version turns a vocabulary change into a visible repin rather than a silent collision. Implementation now notes the canonicalization must be byte-identical across both stacks, since divergence would have the two produce different pins for the same machine. Repinning. A real gap — the pin had no exit. Three paths now, and only the third is a repin proper:
Silent automatic repinning is excluded in the text with the reason: a pin any ordinary install may rewrite protects nothing. And one clarification that fell out while writing it — a revoked signer needs no repin to be safe, since trust policy rejects its artifacts before the pin is consulted and the package builds locally. Repinning restores the cache, not safety. Worth stating, because the opposite reading would make revocation look like it requires an urgent lockfile change. The three unresolved questions — attestation subject and authorization policy for publisher mode, artifact discovery, key distribution and revocation — are unchanged, and each needs its own round before this is implementable. Written by an agent (Claude Code, claude-opus-5). |
1 participant
Makes pnpm's side-effects cache shareable across machines: pnpm may fetch a package's built form instead of running its install scripts, signed either by the organization whose CI produced it or by the package's own publisher. pnpr is the reference implementation; the protocol is specified as an open contract so other registries can implement it.
Tracking issue: pnpm/pnpm#13771. Context: this came out of evaluating pnpm/pnpm#13639, which reaches shared build outputs by delegating materialization to an external
packageProviderexecutable — at the cost of a second install mode that voids the build allowlist, GVS, the hoisted linker, and the Rust fast path. This proposal targets the same need without a second install path.The shape of it
Two modes, one mechanism. An artifact is signed either by the organization whose CI produced it, or by the package's own publisher — the same protocol with a different signer and a different party asserting eligibility. Both are in scope. Publisher mode is the more defensible of the two: a consumer already executes that publisher's install script, so accepting a prebuilt artifact signed by the same identity is a smaller grant than arbitrary code execution at install time, and it needs no new trust root. Organization mode is expected to ship first because it is entirely within pnpm and pnpr — a rollout order, not a scope boundary.
What stays out of scope is an anonymous, multi-builder cache where anyone may contribute an artifact for anyone's package. There the residual risk has no owner.
The registry is the default channel; the signature is the authority. Lookup happens at the registry a package resolved from, which reuses per-registry auth, certs, and proxies unchanged. But acceptance turns on the signature, not on which host served the bytes — so a mirror, a proxying pnpr, or a publisher's existing distribution point can serve an artifact the client will accept. Publisher mode therefore does not depend on
registry.npmjs.orgadopting anything.Lookup is by input key; correctness is enforced by compatibility tags. The input key is the dependency graph and patches. Platform identity lives in tags the artifact advertises as floors, and the client selects from an ordered set of tags it supports — the wheels model, not exact-match hashing. An artifact built against an older libc therefore stays valid on newer systems.
Requests are batched per channel, in parallel, and fold into the existing
POST /-/pnpr/v0/resolveround trip where the client is already talking to pnpr. Candidates arerequiresBuildpackages only.What the review rounds changed
Recording these because each replaced something the first draft got wrong:
sideEffectsCacheRead), signed provenance, and a signing key independent of the registry — a registry signing its own artifacts defends against nothing it isn't already trusted for.importIndexedDir. The existingsanitizeFilenamesis a post-failure compatibility retry, not a security boundary.Alternatives
External package provider (pnpm/pnpm#13639), standalone cache URL in Turborepo's shape, and adopting the Bazel REAPI.
The REAPI decline is worth flagging because its stated grounds changed. It is not declined on round-trip count — gRPC multiplexes, and batching is now a benchmark question in Unresolved Questions — nor on payload shape or a missing platform/provenance model, all of which were withdrawn as incorrect. It is declined on one sufficient objection:
GetActionResultanswers "is there a result for precisely this digest" and structurally cannot answer "give me the best artifact compatible with this tag set, signed by a key I trust." A hybrid (REAPI CAS plus a batched lookup extension) is named as the most likely route back.Open for review
text/rather thanpnpr/text/because the substance is a pnpm client feature plus a protocol meant to be openly implementable, andpnpr/is PolyForm Shield licensed, which would work against that. Easy to move.sideEffectsMapspath so nothing downstream changed. That was false — the type carries onlyadded/deleted, so origin-aware storage and a reuse-time policy check are both real work. Still much smaller than thepackageProvideralternative, but not a thin layer over the existing cache.Draft because the endpoint shape is a sketch and the compatibility tag format deserves its own round — that format is set by publisher mode's requirements, since a publisher targets every consumer and must get tag floors right the first time.
Written by an agent (Claude Code, claude-opus-5).