Repository navigation
fix(release-mirror): publish a release in one run, and on demand - #80
Merged
Merged
Conversation
A release reached the update feeds two or three runs (20-30 minutes) after it was published. The worker logs show why: - Runs were stopped at the 30 s CPU limit. The Stable feed step read back every installer the download-page step had just copied, to hash it with the manifest's SHA-512 (30.6 s of CPU for 1.3 GB on 11-19). - GitHub answered some runs with 429 at the round ten minutes. Changes: - limits.cpu_ms = 300000; a channel's release needs about 40 s of CPU. - The Stable feed runs before the download-page mirror, so the mirror step finds every installer already verified and hashes nothing again. - GitHub fetches retry 429 and gateway errors, up to three attempts, honoring Retry-After up to 30 s. - The schedule moves to three minutes past every ten. - POST /sync?channel=stable|beta (workers.dev, bearer token in the MIRROR_SYNC_TOKEN secret, 404 while unset) runs the mirror now and streams one JSON line per step, for the desktop release workflow. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Deploying scientfactory-website with
|
| Latest commit: |
5974f89
|
| Status: | ✅ Deploy successful! |
| Preview URL: | https://4fd884ed.scientfactory-website.pages.dev |
| Branch Preview URL: | https://fix-release-mirror-speed-202.scientfactory-website.pages.dev |
…ries From Codex's review of #80: - The pointer step refuses a handoff older than the pointer it read: a stale GitHub answer after the trigger promoted a newer release could otherwise swap the pointer back, since the pointer had not changed. - Retry-After is also read as an HTTP date (capped at 30 s), and cancelling a refused response's body can no longer stop the retry. - Tests: a whole Stable run reads no installer back; when the feed step could not run first, the next run reads them back once (documented). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…guard Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
From Codex's recheck of #80: a failed read of the pointer's body was treated like a malformed pointer, so the run kept its ETag and a stale GitHub handoff could still replace a newer pointer. Now only bytes read and found malformed may be replaced; a failed read rejects the run. Also qualifies when installers are read back again (until a feed run publishes the release). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Integration Readiness Review: Codex (read-only, gpt-6.2-sol high), three rounds. Round 1 raised a pointer rollback on a stale GitHub answer, overstated hash-once wording, and retry edge cases. Round 2 raised a failed pointer read still permitting rollback. All were fixed. Round 3 on |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A new desktop release reached the update feeds 20–30 minutes after it was published. With this change it should take a few minutes, and the desktop release workflow can start the mirror the moment it publishes (scient-desktop PR to follow).
Why it was slow
From the worker's own logs:
exceededCpu(32 s), 22:10 GitHub 429, 22:20 ok (30.6 s CPU)exceededCpu, 17:40 okmirror.tsverifyExisting). That re-read alone used about 28 s of the default 30 s.What changed
limits.cpu_ms: 300000(Workers Paid). Hashing one channel's release takes about 40 s.verifyExistingpasses without reading anything back. Each installer is hashed once.fetchFromGitHub. It retries 429/502/503/504 up to three attempts, honoringRetry-Afterup to 30 s, and gives each attempt its own timeout.3-59/10 * * * *instead of*/10.POST /sync?channel=stable|betaon the workers.dev address (workers_dev: true;preview_urlsstays off).MIRROR_SYNC_TOKENsecret. It answers 404 while the secret is unset, 401 for a wrong token, 405 for other methods and 400 for an unknown channel.{"message":"Release mirror finished","failures":n}.If-None-Match: *writes and compare-and-swap on the feeds and pointer).Not done: a GitHub token
The 429s come from github.com download URLs, where a token doesn't help. Authenticated downloads would mean moving to the API's asset endpoints and creating a new credential. The retries and the schedule offset address the refusals that were observed.
Tests
github.test.ts: honoursRetry-After, caps it at 30 s, backs off without it, returns the third refusal, and doesn't retry a 404.index.test.ts: step order (Stable feed, then mirror, then Beta), a failed step doesn't stop the others, single-channel runs, the scheduled run fails on a failed step, and/syncstreaming plus its 404/401/405/400 answers.mirror.test.ts: the 429 test now expects three attempts, and a new test covers recovery after one 429.bun run checkpasses locally.wrangler deploy --dry-runbundles the worker.Deploy (after merge)
bun run mirror:deploy, then set the token with README step 3. The Cron schedule changes on deploy.Documentation impact
Updated:
README.md(runbook: schedule, CPU limit, token setup, stop procedure) anddocs/architecture/desktop-downloads.md(order, retries, new "Run on publish" section).Review and merge
Quality Review is by the author, and Integration Readiness Review is by Codex (read-only), recorded below. The product owner explicitly asked for this to be merged when Codex approves. There's no user-visible page change, so no preview evidence applies.
🤖 Generated with Claude Code