From 42bb7262322dbbe9a8880d177d7020ddb13ad30e Mon Sep 17 00:00:00 2001 From: James Yong Date: Fri, 25 Sep 2026 10:35:25 -0700 Subject: [PATCH 1/2] ci(release): wait for the registries as long as they take, once per npm dependency layer Publishing 0.12.0 took a rerun, and another approval of the `release` environment, for every npm package npm was slow to serve. Nothing had failed: npm accepts an upload (`npm publish` prints `+ name@version` and exits 0) well before it records the version, and the helper gave each package about two and a half minutes to become resolvable. Measured as the registry's own `time["0.12.0"]` minus each job log's `+` line, npm took 56 s to 4m 09s per package. ghosttead-darwin-arm64 (2m 38s), ghosttea-client (2m 07s, then a view that had not caught up), ghosttea-electron (4m 09s) and ghosttea-react (4m 08s) outlasted it. - scripts/publish-npm-packages.mjs replaces publish-npm-package-if-missing.sh and the ten-line list in the workflow. The packages come from release-notes.mjs's publishedPackages(), and the order from the manifests' dependencies, optionalDependencies and peerDependencies on workspace packages. Each package publishes once npm resolves everything it depends on at the release version. The resolver still never exists before its platform packages, and now no package names a dependency npm cannot install. It waits once per layer of the graph (three today) instead of once per package: 0.12.0's delays replayed take about 8.5 minutes instead of 24. `--plan` prints the order; naming packages publishes only those (the manual first-publish path). - Twenty minutes per package (REGISTRY_VISIBILITY_TIMEOUT_MINUTES, set in the workflow so a retry tag can raise it for a release tagged after this). It polls every 10 s and logs a progress line each minute. The crate publisher shares the same budget. - A publish npm refuses because it already holds the version (E403 "cannot publish over", E409, EPUBLISHCONFLICT) is waited for, not failed: that is an earlier attempt's upload, accepted and not yet recorded, which is when an impatient rerun would otherwise collide. Any other publish failure stops the run at once, before its dependents. - The publish job's timeout goes from 45 to 120 minutes, since most of the job is now waiting. - check-first-publishes: npm no longer has a second list to reconcile, so the check instead requires the workflow to run the publisher without arguments. - PUBLISHING.md: the order, the manual publish, what a retry tag carries (the workflow file, not the release tag's scripts), and the evidence. Tests: tests/publish-npm-packages.test.mjs has 11 tests, wired into test:desktop. They replay 0.12.0's measured delays against a simulated registry on a simulated clock, covering: - dependency order and per-layer waiting - a registry three times slower than 0.12.0's worst - a version that never resolves - reruns over resolved and over accepted-but-unrecorded uploads - a hard publish failure, named-package runs, and cycles - the resolver waiting on its optional dependencies Mutation check: each of 8 mutations of the publisher fails a test. The mutations were the old budget, and removing the dependency wait, the conflict tolerance, the outside-dependency check, optionalDependencies, the timeout, the stuck guard, and the cycle check. Other checks, all local: - the real npm adapter, read-only against the registry - the crate wait, driven through stub curl/cargo: publish then resolve, never resolves, a bad budget, already published - format:check, lint, check:workspace-names, check:build-script-inputs, test:release-notes, check:first-publishes and actionlint all pass Not verifiable before the next release: a real publish through it. A retry tag cannot bring this to 0.12.0, whose publish job runs v0.12.0's own scripts. Co-Authored-By: Claude Opus 5.5 --- .github/workflows/publish-release.yml | 32 +-- PUBLISHING.md | 66 +++--- package.json | 3 +- scripts/check-first-publishes.mjs | 11 +- scripts/publish-crate-if-missing.sh | 26 ++- scripts/publish-npm-package-if-missing.sh | 36 ---- scripts/publish-npm-packages.mjs | 250 ++++++++++++++++++++++ tests/publish-npm-packages.test.mjs | 207 ++++++++++++++++++ 8 files changed, 542 insertions(+), 89 deletions(-) delete mode 100755 scripts/publish-npm-package-if-missing.sh create mode 100644 scripts/publish-npm-packages.mjs create mode 100644 tests/publish-npm-packages.test.mjs diff --git a/.github/workflows/publish-release.yml b/.github/workflows/publish-release.yml index 22627fac..ccaae796 100644 --- a/.github/workflows/publish-release.yml +++ b/.github/workflows/publish-release.yml @@ -215,10 +215,19 @@ jobs: vars.OIDC_RELEASE_ENABLED == 'true' # npm provenance requires a GitHub-hosted runner. runs-on: macos-15 - timeout-minutes: 45 + # Most of this job is waiting for the registries to serve what it uploaded: + # up to the budget below per crate and per layer of npm packages. A job + # that times out stops behind another approval, like any other failure. + timeout-minutes: 120 environment: release env: RELEASE_TAG: ${{ needs.resolve.outputs.release_tag }} + # How long each publisher waits for an upload to become resolvable before + # failing the release. npm took up to 4 minutes 9 seconds to serve a + # 0.12.0 package, when the publishers still allowed about two and a half. + # A retry tag runs this file with the release tag's scripts, so this is + # where a retry would wait longer. + REGISTRY_VISIBILITY_TIMEOUT_MINUTES: 20 permissions: contents: read id-token: write @@ -299,22 +308,13 @@ jobs: CARGO_REGISTRY_TOKEN: ${{ steps.auth-truffle.outputs.token }} run: scripts/publish-crate-if-missing.sh ghosttea-truffle - # The daemon platform packages publish before everything else and the - # resolver publishes last: each publish waits until the registry can - # resolve it, so `@vibecook/ghosttead` can never exist at a version - # whose optional dependencies do not. + # Every package the manifests publish, each once npm resolves the + # workspace packages it depends on at this version, so + # `@vibecook/ghosttead` can never exist at a version whose optional + # dependencies do not. The order comes from the manifests; `--plan` + # prints it. - name: Publish npm packages - run: | - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttead-darwin-arm64 - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttead-win32-x64 - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-native-tabs - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-protocol - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-frame - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-client - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-electron - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-react - scripts/publish-npm-package-if-missing.sh @vibecook/ghosttead + run: node scripts/publish-npm-packages.mjs # Creating the release used to be a manual step after this workflow finished, # and 0.4.0 shipped to both registries while the release page kept showing diff --git a/PUBLISHING.md b/PUBLISHING.md index 933932e6..fd7cb376 100644 --- a/PUBLISHING.md +++ b/PUBLISHING.md @@ -35,21 +35,23 @@ Publish Rust crates in dependency order: 6. `ghosttea` 7. `ghosttea-truffle` -Publish npm packages in dependency order. The binary packages go first and -the resolver goes last: every publish waits until the registry can resolve -it, so `@vibecook/ghosttead` never exists at a version whose optional -dependencies do not. - -1. `@vibecook/ghosttead-darwin-arm64` -2. `@vibecook/ghosttead-win32-x64` -3. `@vibecook/ghosttea-native-tabs` -4. `@vibecook/ghosttea-protocol` -5. `@vibecook/ghosttea-frame` -6. `@vibecook/ghosttea` -7. `@vibecook/ghosttea-client` -8. `@vibecook/ghosttea-electron` -9. `@vibecook/ghosttea-react` -10. `@vibecook/ghosttead` +Publish npm packages in dependency order. `scripts/publish-npm-packages.mjs` +derives it from the manifests: each package publishes once npm resolves every +workspace package it depends on at the release version, so +`@vibecook/ghosttead` never exists at a version whose optional dependencies do +not, and no package names a dependency npm cannot install. A package waits +only for its own dependencies, so the release waits for npm once per layer of +the graph rather than once per package. `node scripts/publish-npm-packages.mjs +--plan` prints the order without publishing anything. It is currently: + +1. `@vibecook/ghosttea-frame`, `@vibecook/ghosttea-native-tabs`, + `@vibecook/ghosttea-protocol`, `@vibecook/ghosttead-darwin-arm64`, and + `@vibecook/ghosttead-win32-x64` +2. `@vibecook/ghosttea` and `@vibecook/ghosttea-client`, after + `@vibecook/ghosttea-protocol`; `@vibecook/ghosttead`, after both binary + packages +3. `@vibecook/ghosttea-electron` and `@vibecook/ghosttea-react`, after the + packages they are built on ## Binary staging @@ -263,22 +265,16 @@ cargo publish --locked --package ghosttea-truffle ``` The npm manifests enable provenance for trusted CI publishing. Disable it only -for the first local publish, which has no CI identity: +for the first local publish, which has no CI identity. With no arguments the +publisher uploads every package npm does not already hold at this version, in +dependency order. Naming packages uploads only those, and whatever they depend +on must already resolve: ```sh export NPM_CONFIG_PROVENANCE=false export npm_config_cache=/private/tmp/ghosttea-npm-release-cache -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttead-darwin-arm64 -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttead-win32-x64 -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-native-tabs -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-protocol -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-frame -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-client -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-electron -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttea-react -scripts/publish-npm-package-if-missing.sh @vibecook/ghosttead +node scripts/publish-npm-packages.mjs unset NPM_CONFIG_PROVENANCE npm_config_cache ``` @@ -313,6 +309,12 @@ Every publish step skips artifacts a registry already holds, so only the missing remainder ships. If the retry itself exposes another workflow defect, fix it and increment the retry number; never move either tag. +Only the workflow file travels with a retry. The scripts it runs come from the +release tag's tree like everything else, so a fix to a publisher reaches the +next release, not this one. What the workflow passes to them does travel: +`REGISTRY_VISIBILITY_TIMEOUT_MINUTES` is set there so that a retry can wait +longer on a slow registry. + ## The GitHub release The workflow's `github-release` job creates it, after publishing, from the @@ -392,4 +394,14 @@ The publish helpers safely skip an exact version that already exists, making a workflow rerun resumable after partial registry success. They never overwrite or replace a published artifact. After an upload succeeds, they wait for the exact version to become publicly resolvable so ordinary registry propagation -does not produce a false release failure. +does not produce a false release failure: for up to +`REGISTRY_VISIBILITY_TIMEOUT_MINUTES`, which the workflow sets to 20. + +That margin is deliberate. npm accepts an upload well before it records the +version: during 0.12.0 each package's `time[version]` trailed the accepted +upload by 56 seconds to 4 minutes 9 seconds. The helpers then allowed about two +and a half minutes, so four of the ten packages failed after publishing +successfully, and each failure cost a rerun and another approval of the +`release` environment. A publish npm refuses because it already holds the +version — an earlier attempt's upload, accepted but not yet recorded — is +waited for the same way instead of failing. diff --git a/package.json b/package.json index 7a281fd4..813c3f66 100644 --- a/package.json +++ b/package.json @@ -125,8 +125,9 @@ "check:build-script-inputs": "node scripts/check-build-script-inputs.mjs", "check:first-publishes": "node scripts/check-first-publishes.mjs", "test:release-notes": "node --test tests/release-notes.test.mjs", + "test:publish-npm-packages": "node --test tests/publish-npm-packages.test.mjs", "check:desktop": "npm run format:check && npm run lint && npm run check:workspace-names && npm run check:build-script-inputs && npm run build:sdk && npm run check --workspaces --if-present && cargo check --workspace --all-targets --all-features --locked && cargo clippy --workspace --all-targets --all-features --locked -- -D warnings", - "test:desktop": "npm run build:sdk && npm run test --workspaces --if-present && cargo test --workspace --locked && npm run test:ghosttea-core:ffi && npm run test:release-notes && npm run test:integration:built && npm run test:bench:lib && npm run test:bench:render && npm run test:bench:truffle", + "test:desktop": "npm run build:sdk && npm run test --workspaces --if-present && cargo test --workspace --locked && npm run test:ghosttea-core:ffi && npm run test:release-notes && npm run test:publish-npm-packages && npm run test:integration:built && npm run test:bench:lib && npm run test:bench:render && npm run test:bench:truffle", "test:lifecycle:soak": "cargo build --release --package ghosttead --locked && node tests/integration/ghosttead-lifecycle-soak.mjs", "ci:desktop": "npm run check:desktop && npm run test:desktop && npm run package:check:built && npm run test:lifecycle:soak", "check": "npm run format:check && npm run lint && npm run check:workspace-names && npm run check:build-script-inputs && npm run check:ghostty-upgrade && npm run check:bundled-fonts && npm run check:swiftpm:manifests && npm run check:ios-release-bom && npm run check:ios-release-resources && npm run check:ios-app-store && npm run check:ios-beta-matrix && npm run test:ios:beta-matrix && npm run check:ios-instruments && npm run test:ios:instruments-evidence && npm run check:ios-diagnostics && npm run build:sdk && npm run check --workspaces --if-present && cargo check --workspace && cargo clippy --workspace --all-targets --all-features -- -D warnings", diff --git a/scripts/check-first-publishes.mjs b/scripts/check-first-publishes.mjs index 11067ace..f25a7d09 100644 --- a/scripts/check-first-publishes.mjs +++ b/scripts/check-first-publishes.mjs @@ -91,7 +91,16 @@ async function packageExists(name) { const crates = publishedCrates(); const packages = publishedPackages(); requireSameArtifacts("crates", crates, workflowPublishes("publish-crate-if-missing.sh")); -requireSameArtifacts("npm packages", packages, workflowPublishes("publish-npm-package-if-missing.sh")); +// The npm packages have no second list to reconcile: the workflow runs one +// publisher without arguments, and it publishes `publishedPackages()`, the +// manifests' own answer. What can still drift is the workflow calling it. +if (!/^\s*run: node scripts\/publish-npm-packages\.mjs\s*$/m.test(workflow)) { + console.error( + "publish-release.yml does not run `node scripts/publish-npm-packages.mjs` without arguments, " + + "so it would not publish every npm package the manifests declare.", + ); + process.exit(1); +} // Sequentially, and deliberately: crates.io rate-limits its API, and a gate // that trips that limit reports an outage instead of an answer. diff --git a/scripts/publish-crate-if-missing.sh b/scripts/publish-crate-if-missing.sh index fa73174d..a305391c 100755 --- a/scripts/publish-crate-if-missing.sh +++ b/scripts/publish-crate-if-missing.sh @@ -6,15 +6,25 @@ version="$(node -p "require('./package.json').version")" registry_url="https://crates.io/api/v1/crates/${crate}/${version}" registry_user_agent="ghosttea-release/${version} (https://github.com/vibecook-dev/ghosttea)" +# A registry accepts an upload before it serves it: npm took as long as 4m 09s +# to serve a 0.12.0 package (see scripts/publish-npm-packages.mjs). crates.io +# has been quicker, but a timeout here fails the release behind another +# approval, so both publishers share one generous budget. +visibility_timeout_minutes="${REGISTRY_VISIBILITY_TIMEOUT_MINUTES:-20}" +if [[ ! "$visibility_timeout_minutes" =~ ^[1-9][0-9]*$ ]]; then + echo "REGISTRY_VISIBILITY_TIMEOUT_MINUTES must be a whole number of minutes, not '${visibility_timeout_minutes}'" >&2 + exit 1 +fi + wait_until_resolvable() { - for _ in {1..60}; do - if curl --fail --silent --show-error --user-agent "$registry_user_agent" "$registry_url" >/dev/null 2>&1 && - cargo info --registry crates-io "${crate}@${version}" >/dev/null 2>&1; then - return 0 + local deadline=$((SECONDS + visibility_timeout_minutes * 60)) + until curl --fail --silent --show-error --user-agent "$registry_user_agent" "$registry_url" >/dev/null 2>&1 && + cargo info --registry crates-io "${crate}@${version}" >/dev/null 2>&1; do + if ((SECONDS >= deadline)); then + return 1 fi - sleep 2 + sleep 5 done - return 1 } if curl --fail --silent --show-error --user-agent "$registry_user_agent" "$registry_url" >/dev/null 2>&1; then @@ -22,7 +32,7 @@ if curl --fail --silent --show-error --user-agent "$registry_user_agent" "$regis echo "${crate}@${version} is already published and resolvable; skipping" exit 0 fi - echo "timed out waiting for ${crate}@${version} to become resolvable" >&2 + echo "timed out after ${visibility_timeout_minutes} minutes waiting for ${crate}@${version} to become resolvable" >&2 exit 1 fi @@ -33,5 +43,5 @@ if wait_until_resolvable; then exit 0 fi -echo "timed out waiting for ${crate}@${version} to become resolvable" >&2 +echo "timed out after ${visibility_timeout_minutes} minutes waiting for ${crate}@${version} to become resolvable" >&2 exit 1 diff --git a/scripts/publish-npm-package-if-missing.sh b/scripts/publish-npm-package-if-missing.sh deleted file mode 100755 index dfaafe98..00000000 --- a/scripts/publish-npm-package-if-missing.sh +++ /dev/null @@ -1,36 +0,0 @@ -#!/usr/bin/env bash -set -euo pipefail - -package="${1:?npm package name is required}" -version="$(node -p "require('./package.json').version")" - -wait_until_resolvable() { - for _ in {1..60}; do - if npm view "${package}@${version}" version >/dev/null 2>&1; then - return 0 - fi - sleep 2 - done - return 1 -} - -if npm view "${package}@${version}" version >/dev/null 2>&1; then - echo "${package}@${version} is already published; skipping" - exit 0 -fi - -publish_args=(publish --workspace "$package" --access public) -provenance="${NPM_CONFIG_PROVENANCE:-${npm_config_provenance:-}}" -if [[ "$provenance" == "false" || "$provenance" == "0" ]]; then - publish_args+=(--provenance=false) -fi - -npm "${publish_args[@]}" - -if wait_until_resolvable; then - echo "verified ${package}@${version} on npm" - exit 0 -fi - -echo "timed out waiting for ${package}@${version} to become resolvable" >&2 -exit 1 diff --git a/scripts/publish-npm-packages.mjs b/scripts/publish-npm-packages.mjs new file mode 100644 index 00000000..32452bfb --- /dev/null +++ b/scripts/publish-npm-packages.mjs @@ -0,0 +1,250 @@ +// Publishes the workspace's npm packages at the release version, each one once +// npm resolves every workspace package it depends on at that version, and +// returns only when npm resolves all of them. +// +// It replaced a helper that published one package at a time and gave each +// about two and a half minutes to become resolvable, which is not how long npm +// takes. npm accepts an upload — `npm publish` prints `+ name@version` and exits +// 0 — well before it records the version: during 0.12.0 the registry's own +// `time[version]` trailed the accepted upload by 56 seconds to 4 minutes 9 +// seconds. Four of the ten packages outlasted the helper's budget and failed +// after publishing successfully. Each failure stopped the job behind a rerun +// and another approval of the `release` environment. +// +// So each package now gets twenty minutes (`REGISTRY_VISIBILITY_TIMEOUT_MINUTES` +// overrides that), and the run waits once per layer of the dependency graph +// instead of once per package: a package waits only for the packages it depends +// on. The promise the old fixed order kept still holds, and now holds for every +// dependency rather than the one the order was written around: +// `@vibecook/ghosttead` never exists at a version whose optional dependencies +// do not, and no package names a dependency npm cannot install. The order comes +// from the manifests, not from a list kept beside them. +// +// It is idempotent, like the crate publisher beside it. It skips a package npm +// already resolves at this version. If npm refuses a publish because it +// already holds the version (an earlier attempt's upload that is accepted but +// not yet recorded), the run waits for that version like any other. +// +// usage: node scripts/publish-npm-packages.mjs [--plan] [package ...] +// +// With no names it publishes every package the manifests publish. Named +// packages publish alone, for "First manual publish" in PUBLISHING.md, and +// whatever they depend on must already resolve. `--plan` prints the order and +// publishes nothing. +import { spawnSync } from "node:child_process"; +import { readdirSync, readFileSync } from "node:fs"; +import { join, resolve } from "node:path"; +import { setTimeout as sleep } from "node:timers/promises"; + +import { publishedPackages } from "./release-notes.mjs"; + +const root = resolve(import.meta.dirname, ".."); +const MINUTE = 60_000; + +/** + * Every package under `packages/`, published or not, mapped to the workspace + * packages its consumers install with it. A private package stays in the map + * so that a published package depending on one is reported as a dependency + * nothing publishes, instead of passing for someone else's package. + */ +export function workspaceDependencies() { + const manifests = readdirSync(join(root, "packages"), { withFileTypes: true }) + .filter((entry) => entry.isDirectory()) + .map((entry) => JSON.parse(readFileSync(join(root, "packages", entry.name, "package.json"), "utf8"))); + const workspace = new Set(manifests.map((manifest) => manifest.name)); + return new Map( + manifests.map((manifest) => { + const names = ["dependencies", "optionalDependencies", "peerDependencies"].flatMap((field) => + Object.keys(manifest[field] ?? {}), + ); + return [manifest.name, [...new Set(names.filter((name) => workspace.has(name)))].sort()]; + }), + ); +} + +/** + * The order to publish `names` in. Each entry names the packages of this run it + * waits for (`after`) and the ones outside it that must already resolve + * (`requires`). Throws, before anything publishes, on a name the workspace does + * not have or on packages that depend on each other in a cycle. + */ +export function releasePlan(names, dependencies) { + const run = new Set(names); + for (const name of run) { + if (!dependencies.has(name)) throw new Error(`${name} is not a package in this workspace`); + } + const layers = new Map(); + const layerOf = (name, path) => { + if (layers.has(name)) return layers.get(name); + if (path.includes(name)) { + throw new Error(`these packages depend on each other in a cycle: ${[...path, name].join(" -> ")}`); + } + const after = dependencies.get(name).filter((dependency) => run.has(dependency)); + const layer = Math.max(-1, ...after.map((dependency) => layerOf(dependency, [...path, name]))) + 1; + layers.set(name, layer); + return layer; + }; + return [...run] + .map((name) => ({ name, layer: layerOf(name, []) })) + .sort((a, b) => a.layer - b.layer || (a.name < b.name ? -1 : 1)) + .map(({ name, layer }) => ({ + name, + layer, + after: dependencies.get(name).filter((dependency) => run.has(dependency)), + requires: dependencies.get(name).filter((dependency) => !run.has(dependency)), + })); +} + +/** An elapsed time the way the job log should read it: `4m 09s`. */ +function duration(ms) { + const seconds = Math.round(ms / 1000); + return `${Math.floor(seconds / 60)}m ${String(seconds % 60).padStart(2, "0")}s`; +} + +/** + * Publishes `plan` at `version`, settling once npm resolves every package in it. + * + * `registry.resolves(name, version)` answers whether a consumer could install + * that exact version now. `registry.publish(name)` uploads it and answers + * `{ ok, alreadyPublished }`. `clock` supplies `now()` and `sleep(ms)`. The + * tests drive all three with a registry that behaves the way npm did during + * 0.12.0. + */ +export async function publishPlan( + plan, + version, + { registry, clock, log = console.log, visibilityTimeout = 20 * MINUTE, pollInterval = 10_000 }, +) { + const unresolvable = [...new Set(plan.flatMap((entry) => entry.requires))].filter( + (name) => !registry.resolves(name, version), + ); + if (unresolvable.length > 0) { + throw new Error( + `npm cannot resolve ${unresolvable.map((name) => `${name}@${version}`).join(", ")}, which this run depends on ` + + `but does not publish. Publish ${unresolvable.length === 1 ? "it" : "them"} first; nothing was published.`, + ); + } + + const resolved = new Set(); + const awaiting = new Map(); // name -> when this run began waiting for it + let pending = [...plan]; + let reported = clock.now(); + const unpublished = () => + pending.length === 0 ? "" : ` Not published yet: ${pending.map((entry) => entry.name).join(", ")}.`; + + while (pending.length > 0 || awaiting.size > 0) { + let progressed = false; + + for (const entry of pending.filter((candidate) => candidate.after.every((name) => resolved.has(name)))) { + pending = pending.filter((candidate) => candidate !== entry); + progressed = true; + if (registry.resolves(entry.name, version)) { + log(`${entry.name}@${version} is already published; skipping`); + resolved.add(entry.name); + continue; + } + const result = registry.publish(entry.name); + if (result.alreadyPublished) { + log(`npm already holds ${entry.name}@${version} from an earlier attempt; waiting for it to resolve`); + } else if (!result.ok) { + throw new Error( + `npm publish failed for ${entry.name}@${version}.${unpublished()} ` + + "Every step here is idempotent; rerunning resumes where this stopped.", + ); + } + awaiting.set(entry.name, clock.now()); + } + + for (const [name, since] of awaiting) { + if (registry.resolves(name, version)) { + awaiting.delete(name); + resolved.add(name); + progressed = true; + log(`verified ${name}@${version} on npm after ${duration(clock.now() - since)}`); + } else if (clock.now() - since >= visibilityTimeout) { + throw new Error( + `npm still cannot resolve ${name}@${version} after ${duration(clock.now() - since)}.${unpublished()} ` + + "Every step here is idempotent; rerunning resumes where this stopped.", + ); + } + } + + if (progressed) continue; + // Unreachable for a plan from `releasePlan`, which refuses cycles; a plan + // that could never finish must fail rather than poll forever. + if (awaiting.size === 0) throw new Error(`nothing can publish: ${pending.map((entry) => entry.name).join(", ")}`); + if (clock.now() - reported >= MINUTE) { + reported = clock.now(); + const waits = [...awaiting].map(([name, since]) => `${name}@${version} (${duration(clock.now() - since)})`); + log(`waiting for npm to resolve ${waits.join(", ")}`); + } + await clock.sleep(pollInterval); + } +} + +// npm's answer when an upload names a version it already holds, including one +// it accepted from an earlier attempt and has not recorded yet. +const ALREADY_PUBLISHED = /\bEPUBLISHCONFLICT\b|\bE409\b|cannot publish over (?:the )?previously published version/i; + +const npm = (args, options = {}) => + spawnSync("npm", args, { cwd: root, encoding: "utf8", maxBuffer: 256 * 1024 * 1024, ...options }); + +export const npmRegistry = { + // Asked through the same registry front a consumer's install reads, which is + // the only answer that means the version is actually available. + resolves(name, version) { + const result = npm(["view", `${name}@${version}`, "version"], { timeout: MINUTE }); + return result.status === 0 && result.stdout.trim() === version; + }, + publish(name) { + const args = ["publish", "--workspace", name, "--access", "public"]; + const provenance = process.env.NPM_CONFIG_PROVENANCE || process.env.npm_config_provenance || ""; + if (provenance === "false" || provenance === "0") args.push("--provenance=false"); + const result = npm(args); + if (result.error) throw result.error; + process.stdout.write(result.stdout); + process.stderr.write(result.stderr); + const ok = result.status === 0; + return { ok, alreadyPublished: !ok && ALREADY_PUBLISHED.test(`${result.stdout}\n${result.stderr}`) }; + }, +}; + +function describePlan(plan, version) { + const width = Math.max(...plan.map((entry) => entry.name.length)); + const lines = plan.map((entry) => { + const waits = [ + entry.after.length > 0 ? `after ${entry.after.join(", ")}` : "", + entry.requires.length > 0 ? `requires ${entry.requires.join(", ")} already on npm` : "", + ].filter(Boolean); + return ` ${entry.layer + 1}. ${entry.name.padEnd(width)} ${waits.join("; ")}`.trimEnd(); + }); + return [`${plan.length} npm packages publish at ${version}, each once npm resolves what it waits for:`, ...lines, ""]; +} + +// Importable for tests; publishes when run directly. +if (process.argv[1] === import.meta.filename) { + const args = process.argv.slice(2); + const names = args.filter((arg) => arg !== "--plan"); + const version = JSON.parse(readFileSync(join(root, "package.json"), "utf8")).version; + try { + const plan = releasePlan(names.length > 0 ? names : publishedPackages(), workspaceDependencies()); + if (args.includes("--plan")) { + process.stdout.write(describePlan(plan, version).join("\n")); + } else { + const setting = process.env.REGISTRY_VISIBILITY_TIMEOUT_MINUTES || "20"; + const minutes = Number(setting); + if (!Number.isInteger(minutes) || minutes <= 0) { + throw new Error(`REGISTRY_VISIBILITY_TIMEOUT_MINUTES must be a whole number of minutes, not '${setting}'`); + } + await publishPlan(plan, version, { + registry: npmRegistry, + clock: { now: Date.now, sleep }, + visibilityTimeout: minutes * MINUTE, + }); + console.log(`verified ${plan.length} npm packages at ${version}`); + } + } catch (error) { + console.error(error.message); + process.exit(1); + } +} diff --git a/tests/publish-npm-packages.test.mjs b/tests/publish-npm-packages.test.mjs new file mode 100644 index 00000000..a723138d --- /dev/null +++ b/tests/publish-npm-packages.test.mjs @@ -0,0 +1,207 @@ +// The npm publisher runs once per release, on a tag, behind an approval of the +// `release` environment, and every failure it reports costs a rerun and another +// approval. Its order, its waiting, and its idempotence are pinned here against +// a registry that behaves the way npm did during 0.12.0: it accepts an upload +// at once and serves it minutes later. +import assert from "node:assert/strict"; +import { readFileSync } from "node:fs"; +import { test } from "node:test"; + +import { publishPlan, releasePlan, workspaceDependencies } from "../scripts/publish-npm-packages.mjs"; +import { publishedPackages } from "../scripts/release-notes.mjs"; + +const SECOND = 1000; +const MINUTE = 60 * SECOND; + +// How long npm took to record each 0.12.0 upload: the registry's +// `time["0.12.0"]` minus the job log's `+ name@version` line. The resolver +// published last, in a later run, and stands in at the slowest of the others. +const NPM_0_12_0 = { + "@vibecook/ghosttead-darwin-arm64": 158 * SECOND, + "@vibecook/ghosttead-win32-x64": 77 * SECOND, + "@vibecook/ghosttea-native-tabs": 56 * SECOND, + "@vibecook/ghosttea-protocol": 127 * SECOND, + "@vibecook/ghosttea-frame": 96 * SECOND, + "@vibecook/ghosttea": 56 * SECOND, + "@vibecook/ghosttea-client": 127 * SECOND, + "@vibecook/ghosttea-electron": 249 * SECOND, + "@vibecook/ghosttea-react": 248 * SECOND, + "@vibecook/ghosttead": 249 * SECOND, +}; + +/** + * A registry that serves each upload `delays[name]` after accepting it, on a + * clock that only moves when the publisher sleeps. `accepted` holds uploads an + * earlier attempt made, by when; `refuses` names uploads that fail outright. + */ +function simulatedNpm({ delays = {}, accepted = {}, refuses = [] } = {}) { + let now = 0; + const acceptedAt = new Map(Object.entries(accepted)); + const resolvedAt = (name) => acceptedAt.get(name) + (delays[name] ?? 0); + const uploads = []; + return { + uploads, + resolvedAt, + log: () => {}, + clock: { + now: () => now, + // A publisher that would wait forever fails here instead of hanging the + // suite, since this clock never lets a real timer fire. + sleep: async (ms) => { + now += ms; + if (now > 24 * 60 * MINUTE) throw new Error("the publisher waited a simulated day and was still waiting"); + }, + }, + registry: { + resolves: (name) => acceptedAt.has(name) && now >= resolvedAt(name), + publish(name) { + uploads.push({ name, at: now }); + if (refuses.includes(name)) return { ok: false, alreadyPublished: false }; + if (acceptedAt.has(name)) return { ok: false, alreadyPublished: true }; + acceptedAt.set(name, now); + return { ok: true, alreadyPublished: false }; + }, + }, + }; +} + +const dependencies = workspaceDependencies(); +const fullPlan = () => releasePlan(publishedPackages(), dependencies); +const uploaded = (npm) => npm.uploads.map((upload) => upload.name); + +test("publishes every package once, each only after everything it depends on resolves", async () => { + const npm = simulatedNpm({ delays: NPM_0_12_0 }); + await publishPlan(fullPlan(), "0.12.0", npm); + assert.deepEqual(uploaded(npm).sort(), publishedPackages()); + for (const upload of npm.uploads) { + for (const dependency of dependencies.get(upload.name)) { + assert.ok(npm.resolvedAt(dependency) <= upload.at, `${upload.name} was uploaded before ${dependency} resolved`); + } + } +}); + +test("waits once per layer of the dependency graph, not once per package", async () => { + // One package at a time, 0.12.0's delays add up to 24 minutes of waiting. + const npm = simulatedNpm({ delays: NPM_0_12_0 }); + await publishPlan(fullPlan(), "0.12.0", npm); + assert.ok(npm.clock.now() < 10 * MINUTE, `took ${npm.clock.now() / MINUTE} minutes`); +}); + +test("outlasts npm taking far longer than it ever did during 0.12.0", async () => { + const slow = Object.fromEntries(publishedPackages().map((name) => [name, 12 * MINUTE])); + const npm = simulatedNpm({ delays: slow }); + await publishPlan(fullPlan(), "0.12.0", npm); + assert.deepEqual(uploaded(npm).sort(), publishedPackages()); +}); + +test("fails when a version never resolves, and publishes nothing that depends on it", async () => { + const npm = simulatedNpm({ delays: { ...NPM_0_12_0, "@vibecook/ghosttea-protocol": Infinity } }); + await assert.rejects( + publishPlan(fullPlan(), "0.12.0", npm), + /npm still cannot resolve @vibecook\/ghosttea-protocol@0\.12\.0 after 20m 0\ds\..*rerunning resumes/, + ); + for (const dependent of [ + "@vibecook/ghosttea", + "@vibecook/ghosttea-client", + "@vibecook/ghosttea-electron", + "@vibecook/ghosttea-react", + ]) { + assert.ok(!uploaded(npm).includes(dependent), `${dependent} was published over a dependency npm cannot resolve`); + } +}); + +test("a rerun skips every package npm already resolves", async () => { + const everything = Object.fromEntries(publishedPackages().map((name) => [name, -30 * MINUTE])); + const npm = simulatedNpm({ delays: NPM_0_12_0, accepted: everything }); + await publishPlan(fullPlan(), "0.12.0", npm); + assert.deepEqual(npm.uploads, []); + assert.equal(npm.clock.now(), 0); +}); + +test("a rerun waits for an upload npm accepted but has not recorded, rather than failing on it", async () => { + // The earlier attempt got as far as ghosttea-client, 30 seconds before this one. + const earlier = [ + "@vibecook/ghosttead-darwin-arm64", + "@vibecook/ghosttead-win32-x64", + "@vibecook/ghosttea-native-tabs", + "@vibecook/ghosttea-protocol", + "@vibecook/ghosttea-frame", + "@vibecook/ghosttea", + ]; + const accepted = Object.fromEntries(earlier.map((name) => [name, -30 * MINUTE])); + accepted["@vibecook/ghosttea-client"] = -30 * SECOND; + const messages = []; + const npm = simulatedNpm({ delays: NPM_0_12_0, accepted }); + await publishPlan(fullPlan(), "0.12.0", { ...npm, log: (message) => messages.push(message) }); + assert.ok( + messages.includes( + "npm already holds @vibecook/ghosttea-client@0.12.0 from an earlier attempt; waiting for it to resolve", + ), + ); + const electron = npm.uploads.find((upload) => upload.name === "@vibecook/ghosttea-electron"); + assert.ok( + electron.at >= npm.resolvedAt("@vibecook/ghosttea-client"), + "electron was uploaded before its client resolved", + ); + assert.deepEqual(uploaded(npm).sort(), [ + "@vibecook/ghosttea-client", + "@vibecook/ghosttea-electron", + "@vibecook/ghosttea-react", + "@vibecook/ghosttead", + ]); +}); + +test("stops at once when npm refuses an upload for any other reason", async () => { + const npm = simulatedNpm({ delays: NPM_0_12_0, refuses: ["@vibecook/ghosttea-frame"] }); + await assert.rejects( + publishPlan(fullPlan(), "0.12.0", npm), + /npm publish failed for @vibecook\/ghosttea-frame@0\.12\.0\. Not published yet: .*@vibecook\/ghosttea-react/, + ); + assert.equal(npm.clock.now(), 0, "waited on the registry after a failure that waiting cannot fix"); + assert.ok(!uploaded(npm).includes("@vibecook/ghosttea-react")); +}); + +test("a named package publishes alone, and only once what it depends on already resolves", async () => { + const react = releasePlan(["@vibecook/ghosttea-react"], dependencies); + const missing = simulatedNpm(); + await assert.rejects( + publishPlan(react, "0.12.0", missing), + /npm cannot resolve @vibecook\/ghosttea@0\.12\.0, .*nothing was published/, + ); + assert.deepEqual(missing.uploads, []); + + const earlier = ["@vibecook/ghosttea", "@vibecook/ghosttea-frame", "@vibecook/ghosttea-protocol"]; + const present = simulatedNpm({ accepted: Object.fromEntries(earlier.map((name) => [name, 0])) }); + await publishPlan(react, "0.12.0", present); + assert.deepEqual(uploaded(present), ["@vibecook/ghosttea-react"]); +}); + +test("refuses a cycle or an unknown package before anything publishes", async () => { + const cyclic = new Map([ + ["a", ["b"]], + ["b", ["a"]], + ]); + assert.throws(() => releasePlan(["a", "b"], cyclic), /in a cycle: a -> b -> a$/); + assert.throws(() => releasePlan(["@vibecook/ghosttea-nope"], dependencies), /not a package in this workspace/); + + // A plan built by hand can still be one nothing can finish; it fails rather + // than polling forever. + const stuck = [ + { name: "a", layer: 0, after: ["b"], requires: [] }, + { name: "b", layer: 0, after: ["a"], requires: [] }, + ]; + await assert.rejects(publishPlan(stuck, "0.12.0", simulatedNpm()), /nothing can publish: a, b/); +}); + +test("the resolver waits for every platform package it names", () => { + const manifest = JSON.parse(readFileSync(new URL("../packages/ghosttead/package.json", import.meta.url), "utf8")); + const resolver = fullPlan().find((entry) => entry.name === "@vibecook/ghosttead"); + assert.deepEqual(resolver.after, Object.keys(manifest.optionalDependencies).sort()); + assert.ok(resolver.after.length > 0); +}); + +test("every workspace package a published package depends on publishes with it", () => { + for (const entry of fullPlan()) { + assert.deepEqual(entry.requires, [], `${entry.name} depends on packages that never publish`); + } +}); From ce5ba16a644e1426f280f2fb87ffcc156edee815 Mon Sep 17 00:00:00 2001 From: James Yong Date: Fri, 25 Sep 2026 10:45:57 -0700 Subject: [PATCH 2/2] test(release): the resolver's measured 0.12.0 delay replaces its stand-in The previous commit had no measurement for @vibecook/ghosttead, which was still unpublished, so its table entry stood in at 249 s, the slowest of the others. Attempt 5 of the 0.12.0 publish run uploaded it at 17:42:48.9 and npm recorded it at 17:43:45.1, 56 s later, inside the old helper's budget. With the real figure, publishing one package at a time adds up to 21 minutes of waiting, not the 24 the previous commit replayed. Waiting per layer still takes 8.5 minutes, because the resolver was never on the critical path. Co-Authored-By: Claude Opus 5.5 --- tests/publish-npm-packages.test.mjs | 7 +++---- 1 file changed, 3 insertions(+), 4 deletions(-) diff --git a/tests/publish-npm-packages.test.mjs b/tests/publish-npm-packages.test.mjs index a723138d..9e7b3a93 100644 --- a/tests/publish-npm-packages.test.mjs +++ b/tests/publish-npm-packages.test.mjs @@ -14,8 +14,7 @@ const SECOND = 1000; const MINUTE = 60 * SECOND; // How long npm took to record each 0.12.0 upload: the registry's -// `time["0.12.0"]` minus the job log's `+ name@version` line. The resolver -// published last, in a later run, and stands in at the slowest of the others. +// `time["0.12.0"]` minus the job log's `+ name@version` line. const NPM_0_12_0 = { "@vibecook/ghosttead-darwin-arm64": 158 * SECOND, "@vibecook/ghosttead-win32-x64": 77 * SECOND, @@ -26,7 +25,7 @@ const NPM_0_12_0 = { "@vibecook/ghosttea-client": 127 * SECOND, "@vibecook/ghosttea-electron": 249 * SECOND, "@vibecook/ghosttea-react": 248 * SECOND, - "@vibecook/ghosttead": 249 * SECOND, + "@vibecook/ghosttead": 56 * SECOND, }; /** @@ -81,7 +80,7 @@ test("publishes every package once, each only after everything it depends on res }); test("waits once per layer of the dependency graph, not once per package", async () => { - // One package at a time, 0.12.0's delays add up to 24 minutes of waiting. + // One package at a time, 0.12.0's delays add up to 21 minutes of waiting. const npm = simulatedNpm({ delays: NPM_0_12_0 }); await publishPlan(fullPlan(), "0.12.0", npm); assert.ok(npm.clock.now() < 10 * MINUTE, `took ${npm.clock.now() / MINUTE} minutes`);