feat(stack): retry transient HTTP failures in slim services downloads

[avallete]

Summary

Native artifact preparation failed on the first non-2xx response, so a single gateway error from the slim-services release host aborted a whole stack start. Today that took Test red across three unrelated PRs: github.com/supabase/slim-services/releases/download/... answered HTTP 504 for about an hour, on whichever asset was requested, and the failure cascaded from artifact preparation into Composition start had failures.

Artifact requests now retry transient failures with jittered exponential backoff, and keep deterministic failures single-attempt.

Key changes:

• transferFault classifies retryable failures: HTTP 408, 429, and 5xx, plus HttpClientError whose reason is TransportError (connect-level) or DecodeError (a transfer cut mid-body). StatusCodeError, EncodeError, InvalidUrlError, and EmptyBodyError fail on the first attempt.
• PreparationError carries an optional status, so classification reads the response code rather than parsing a message.
• Retry policy is Effect.retry({ schedule, times: TRANSFER_MAX_RETRIES, while }) — 4 retries, so 5 attempts, over a 500ms jittered exponential delay. This matches the mechanism in apps/cli/src/commands/bootstrap/bootstrap.retry.ts.
• fetchBytes and downloadToFile retry through withTransferRetry, which logs the URL and cause of each retried transfer.
• downloadToFile retries the whole transfer, so an attempt reopens the sink in truncating mode and replaces any partial download instead of appending to it. Digest comparison stays outside the retry, keeping a checksum mismatch terminal.
• The backoff is injectable on makeSlimServicesSource and slimServicesChecksum, so retry coverage runs without waiting on real delays. The attempt cap is not injectable.

Linked issue

None; this came out of a CI incident rather than a filed issue.

Checklist

• The PR title follows Conventional Commits.
• Tests added for the change: checksum recovery from a 504, archive recovery from a transfer cut after partial bytes, exactly five attempts against a persistent 504, and a single attempt on 404.
• From the repository root, pnpm check:all passes; relevant package tests pass for every touched workspace, and pnpm types:check passes for each touched TypeScript workspace.

https://claude.ai/code/session_019zn4vRJ2rkx5UXwNS1ndnj

[github-actions]

🤖 AI Review

Both independent reviews were available. After deduplication, five findings are confirmed and one is uncertain. The confirmed concerns are overly broad HttpClientError retries, missing coverage for checksum and partial-stream retries, wall-clock backoff in the integration test, invisible retry progress, and discarded Retry-After guidance. The Schedule.upTo attempt-cap concern remains uncertain because the pinned Effect source and installed type declarations are absent, so its accepted options and exact attempt semantics could not be verified.

Findings

Severity	Location	Category	Sources	Claim
🟡 MINOR	packages/stack/src/preparation/SlimServicesSource.ts:101	api-usage	claude	Schedule.upTo({ times: 4 }) may either be invalid for the pinned Effect API or permit four retries rather than four total attempts; the repository has no test establishing the intended cap.
🟡 MINOR	packages/stack/src/preparation/SlimServicesSource.ts:93	error-handling	claude	The retry predicate accepts every HttpClientError reason, so deterministic client failures can consume the full retry schedule instead of failing immediately.
🟡 MINOR	packages/stack/src/preparation/slim-services.integration.test.ts:203	test-coverage	claude+codex	The integration test does not exercise the new checksum fetch retry or recovery from an archive stream failure after partial output has been written.
🟡 MINOR	packages/stack/src/preparation/SlimServicesSource.ts:83	rate-limiting	claude+codex	429 responses are retried using the generic exponential schedule while the server's Retry-After guidance is discarded.
⚪ NIT	packages/stack/src/preparation/SlimServicesSource.ts:99	testability	claude	The hard-coded production backoff forces the new live integration test to wait on real jittered delays.
⚪ NIT	packages/stack/src/preparation/SlimServicesSource.ts:352	observability	claude	Retries provide no application-level progress or log signal, so callers remain in the downloading state throughout backoff and repeated attempts.

Findings outside the diff

• ⚪ NIT packages/stack/src/preparation/SlimServicesSource.ts:352 — Retries provide no application-level progress or log signal, so callers remain in the downloading state throughout backoff and repeated attempts.

Stats

Claude findings: 6 · Codex findings: 2 · Confirmed: 5 · Refuted: 0 · Uncertain: 1

---

Models: claude-opus-5 + gpt-5.6-sol · Trigger: auto · Workflow run

This review runs once per PR. A maintainer can request another with a /ai-review comment.

[avallete]

Run stack end-to-end tests (docker) failed on 764cf48, in whole-stack.docker.e2e.test.ts > Docker: parallel stack isolation:

Unknown Error: Function failed at http://127.0.0.1:46396/functions/v1/hello: 502: Bad Gateway

I don't believe this one belongs to this PR, and the log says why.

• The diff touches only the slim-services native download path. This job's log contains no artifact activity at all: no Unable to prepare, no Unable to download, no HTTP 5xx, and no Retrying slim-services transfer line, so the retry code under change never ran.
• The failure diagnostics show functions=running(health=healthy,wakeEnabled=true,error=undefined) at the moment of the 502. The backend was up and the proxy still answered 502.
• The next diagnostics snapshot, about four seconds later, has every service running(health=healthy). That reads as a readiness race on a lazily woken service rather than a failed start.
• The previous head of this PR, f06daa8, passed this same job, and the whole suite was green on it.

This is the same shape as the wake failures seen across several PRs earlier today, where HttpProxy discards the route/wake error (502 for requests, a destroyed socket for upgrades) and the service status keeps error=undefined, so nothing in the log names a cause. No fix for that exists to port yet; it needs the proxy to log the discarded cause and the wake failure to be recorded on the service observation, which is a change in the runtime that landed in #6684 rather than something to bolt onto this PR.

Spending the single re-run on this job to confirm. If it fails again on the same commit, I'll treat it as this PR's and dig further.

---

Generated by Claude Code