Skip to content

ci(release): wait for the registries as long as they take, once per npm dependency layer - #64

Merged
jamesyong-42 merged 2 commits into
mainfrom
ci/npm-publish-waits
Sep 25, 2026
Merged

jamesyong-42 merged 2 commits into
mainfrom
ci/npm-publish-waits

Conversation

@jamesyong-42

@jamesyong-42 jamesyong-42 commented Sep 25, 2026 •

Copy link
Copy Markdown
Member

Publishing 0.12.0 took a rerun, and another approval of the release environment, for every npm package npm was slow to serve. Nothing had failed. npm accepts an upload well before it records the version (npm publish prints + name@version and exits 0), and publish-npm-package-if-missing.sh gave each package about 2½ minutes to become resolvable.

What npm actually took. The table measures each package as the registry's time["0.12.0"] minus the job log's + line:

package accepted → recorded old helper
ghosttea-native-tabs 0m 56s ok
ghosttea 0m 56s ok
ghosttead 0m 56s ok (published last, in attempt 5)
ghosttead-win32-x64 1m 17s ok
ghosttea-frame 1m 36s ok
ghosttea-protocol 2m 07s ok (5 s to spare)
ghosttea-client 2m 07s timed out: the view hadn't caught up
ghosttead-darwin-arm64 2m 38s timed out
ghosttea-react 4m 08s timed out
ghosttea-electron 4m 09s timed out

Once npm recorded a version, it was resolvable within seconds. The waiting is npm's processing, not CDN caching.

Change

  • scripts/publish-npm-packages.mjs replaces the per-package helper and the ten-line list in the workflow.
    • The packages come from publishedPackages(). The order comes from the manifests' dependencies, optionalDependencies and peerDependencies on workspace packages.
    • Each package publishes once npm resolves everything it depends on at the release version.
    • @vibecook/ghosttead still never exists before its platform packages, and now no package names a dependency npm can't install.
    • It waits once per layer (three today) instead of once per package. 0.12.0's delays replayed take about 8½ min instead of about 21.
    • node scripts/publish-npm-packages.mjs --plan prints the order.
  • Twenty minutes per package, polling every 10 s with a progress line each minute.
    • REGISTRY_VISIBILITY_TIMEOUT_MINUTES is set in the workflow, because a retry tag runs the new workflow file but the release tag's own scripts.
    • The crate publisher shares the budget.
  • Reruns can't collide. If npm refuses a publish because it already holds the version, the publisher waits for that version instead of failing. That covers E403 "cannot publish over", E409 and EPUBLISHCONFLICT: an earlier attempt's upload, accepted but not yet recorded. Any other publish failure stops the run at once, before its dependents.
  • The publish job's timeout goes from 45 to 120 minutes, since most of the job is now waiting.
  • check-first-publishes has no second npm list to reconcile anymore. It now requires the workflow to run the publisher without arguments.
  • PUBLISHING.md covers the order, the manual publish, and what a retry tag carries.

Verification

  • tests/publish-npm-packages.test.mjs has 11 tests and is wired into test:desktop, so both verify and windows run it. It replays the table above against a simulated registry on a simulated clock and covers:
    • dependency order and per-layer waiting
    • a registry three times slower than the worst above
    • a version that never resolves
    • reruns over resolved and over accepted-but-unrecorded uploads
    • hard failures, named-package runs, and cycles
    • the resolver's optional dependencies
  • Mutation check: 8 of 8 mutations of the publisher each fail a test. The mutations were the old budget, and removing the dependency wait, the conflict tolerance, the outside-dependency check, optionalDependencies, the timeout, the stuck guard, and the cycle check.
  • The real npm adapter was exercised read-only against the registry. The crate wait was driven through stub curl/cargo in four cases: publish then resolve, never resolves, a bad budget, already published.
  • These pass locally: format:check, lint, check:workspace-names, check:build-script-inputs, test:release-notes, check:first-publishes and actionlint.

Not verifiable before the next release: a real publish through the new path. This fix doesn't reach 0.12.0, whose publish job runs v0.12.0's scripts; 0.12.0 finishes with the current reruns.

🤖 Generated with Claude Code

jamesyong-42 and others added 2 commits September 25, 2026 10:35
…pm dependency layer

Publishing 0.12.0 took a rerun, and another approval of the `release`
environment, for every npm package npm was slow to serve. Nothing had
failed: npm accepts an upload (`npm publish` prints `+ name@version` and
exits 0) well before it records the version, and the helper gave each
package about two and a half minutes to become resolvable. Measured as
the registry's own `time["0.12.0"]` minus each job log's `+` line, npm
took 56 s to 4m 09s per package. ghosttead-darwin-arm64 (2m 38s),
ghosttea-client (2m 07s, then a view that had not caught up),
ghosttea-electron (4m 09s) and ghosttea-react (4m 08s) outlasted it.

- scripts/publish-npm-packages.mjs replaces publish-npm-package-if-missing.sh
  and the ten-line list in the workflow. The packages come from
  release-notes.mjs's publishedPackages(), and the order from the
  manifests' dependencies, optionalDependencies and peerDependencies on
  workspace packages. Each package publishes once npm resolves everything
  it depends on at the release version. The resolver still never exists
  before its platform packages, and now no package names a dependency npm
  cannot install. It waits once per layer of the graph (three today)
  instead of once per package: 0.12.0's delays replayed take about 8.5
  minutes instead of 24. `--plan` prints the order; naming packages
  publishes only those (the manual first-publish path).
- Twenty minutes per package (REGISTRY_VISIBILITY_TIMEOUT_MINUTES, set in
  the workflow so a retry tag can raise it for a release tagged after
  this). It polls every 10 s and logs a progress line each minute. The
  crate publisher shares the same budget.
- A publish npm refuses because it already holds the version (E403
  "cannot publish over", E409, EPUBLISHCONFLICT) is waited for, not
  failed: that is an earlier attempt's upload, accepted and not yet
  recorded, which is when an impatient rerun would otherwise collide.
  Any other publish failure stops the run at once, before its dependents.
- The publish job's timeout goes from 45 to 120 minutes, since most of the
  job is now waiting.
- check-first-publishes: npm no longer has a second list to reconcile, so
  the check instead requires the workflow to run the publisher without
  arguments.
- PUBLISHING.md: the order, the manual publish, what a retry tag carries
  (the workflow file, not the release tag's scripts), and the evidence.

Tests: tests/publish-npm-packages.test.mjs has 11 tests, wired into
test:desktop. They replay 0.12.0's measured delays against a simulated
registry on a simulated clock, covering:
  - dependency order and per-layer waiting
  - a registry three times slower than 0.12.0's worst
  - a version that never resolves
  - reruns over resolved and over accepted-but-unrecorded uploads
  - a hard publish failure, named-package runs, and cycles
  - the resolver waiting on its optional dependencies

Mutation check: each of 8 mutations of the publisher fails a test. The
mutations were the old budget, and removing the dependency wait, the
conflict tolerance, the outside-dependency check, optionalDependencies,
the timeout, the stuck guard, and the cycle check.

Other checks, all local:
  - the real npm adapter, read-only against the registry
  - the crate wait, driven through stub curl/cargo: publish then
    resolve, never resolves, a bad budget, already published
  - format:check, lint, check:workspace-names,
    check:build-script-inputs, test:release-notes,
    check:first-publishes and actionlint all pass

Not verifiable before the next release: a real publish through it. A
retry tag cannot bring this to 0.12.0, whose publish job runs v0.12.0's
own scripts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d-in

The previous commit had no measurement for @vibecook/ghosttead, which was
still unpublished, so its table entry stood in at 249 s, the slowest of the
others. Attempt 5 of the 0.12.0 publish run uploaded it at 17:42:48.9 and
npm recorded it at 17:43:45.1, 56 s later, inside the old helper's budget.
With the real figure, publishing one package at a time adds up to 21
minutes of waiting, not the 24 the previous commit replayed. Waiting per
layer still takes 8.5 minutes, because the resolver was never on the
critical path.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@jamesyong-42
jamesyong-42 merged commit 52bde2d into main Sep 25, 2026
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant