Per-PR preview deployments on Fly.io#894
Merged
Merged
Conversation
Label a PR 'preview' (or dispatch the workflow) to get a test site at restarters-pr-<N>.fly.dev running that PR's code against a copy of the live DB, restored from the latest hourly backup at cold boot. Machines suspend after ~5 min idle (sub-second resume, storage-only cost) and a scheduled sweep destroys apps once the PR closes or loses the label. - fly.pr.toml: per-PR app template (suspend requires no swap; embedded MariaDB; integrations off; cookie gate on; branch banner on) - docker/preview-startup.sh: warming page on :80 within seconds, restore latest backup, truncate restored jobs/failed_jobs (supervisord's worker is hardcoded to queue:work database), run the PR's migrations against real data, report outcome at /_preview_status - Dockerfile.fly: install MariaDB for any non-production startup script - nginx: rate-limit /_login (keyed on Fly-Client-IP); serve /_preview_status gate-exempt - FEATURE__IMAGE_UPLOAD flag gating FixometerFile::upload() and the NetworkController logo path, so previews can't write to the shared production image bucket (their Tigris creds are read-only anyway) - FEATURE__WORDPRESS_INTEGRATION flag enforced centrally in Group::eventsShouldPushToWordpress(); previously WP listeners were only safe because the secret was absent. sync:discourseusernames gains the discourse flag check it was missing - pr-preview.yml: deploys gated by the 'preview' GitHub Environment (required reviewer); workflow_dispatch verifies the PR is not from a fork; cleanup never runs on pull_request events (PR-modified workflow code must not reach the org token) - a 6-hourly sweep via the develop-restricted 'preview-cleanup' environment destroys stale apps - navbar: MAILPIT_URL override so preview banners link the shared Mailpit - Replaces the single-slot preview flow (preview-deploy.yml + fly.preview.toml), which clobbered restarters-dev and never ran migrations One-time setup (environments, org token, read-only Tigris/Drive credentials, FLY_ORG variable, label) is documented in docs/fly-deployment.md. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon
A stale branch previewed as-is runs old code against a newer live schema. Deploying refs/pull/N/merge (PR + current base, same as CI) means the code is never older than develop, and production schema always lags develop, so the restored backup can't be ahead of the code. Conflicted PRs fail fast with a clear message; the PR comment now notes how many commits (and migration-touching commits) the branch is behind. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
- Pin superfly/flyctl-actions to a full commit SHA: @master would let a compromised upstream repo ship code straight into the job that holds the org token - Move permissions from workflow level to job level (cleanup only needs read access) - Deduplicate UploadsDisabledTest setup into helpers Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon
The workflow now strips comment/blank lines before flyctl secrets import, so the filled-in template can be pasted as-is. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon
Records the concrete setup (environments, org, restore identity, label) alongside day-to-day usage, expectations, and the security model. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon
Previews currently reuse production's read-write Tigris key (owner- accepted 2026-07-13); the image-upload flag is the only upload block until it is swapped for a read-only key. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon
Closes the interim noted in the previous commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01TzY1phBjyT3HogpeS53zon
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Per-PR preview deployments on Fly.io: label any PR with
preview(or dispatch the PR preview workflow with a PR number), approve the run, and get a self-contained test site athttps://restarters-pr-<N>.fly.devrunning that PR's code against a copy of the live database. Replaces the single-slot preview flow (preview-deploy.yml+fly.preview.toml), which clobberedrestarters-devand never ran migrations.How it works
restarters-yesterdaypattern: sameDockerfile.flybuild, embedded local MariaDB, restore from the most recent hourly production backup at cold boot. Newdocker/preview-startup.shadditionally:jobs/failed_jobstables — the backup captures production's in-flight queue jobs, and the supervisord worker is hardcoded toqueue:work database, so these must never execute on a preview;php artisan migrate --force, so a PR's migrations are exercised against real data before merge, and reports the outcome at/_preview_status(no PII) — the workflow polls it and posts the result on the PR.fly.pr.tomlusesauto_stop_machines = "suspend"withmin_machines_running = 0and no swap (swap silently disables suspend): after ~5 min idle the whole VM is snapshotted with MariaDB still loaded, and the next request resumes it sub-second. Cold boots (deploys, discarded snapshots) go through the warming page instead.Security design
Previews run arbitrary PR code while holding a full live-DB copy, so they get no production-capable credentials:
APP_KEYgenerated per preview app — never production's (that key could forge production session cookies).docs/fly-deployment.md).FEATURE__IMAGE_UPLOADflag (default on) gates both upload paths (FixometerFile::upload()andNetworkController::update(), which bypassed the helper) so previews can't write to the shared production image bucket.FEATURE__WORDPRESS_INTEGRATIONflag, enforced centrally inGroup::eventsShouldPushToWordpress()— previously the WordPress listeners were only safe because the secret happened to be absent.sync:discourseusernamesalso gains the discourse feature-flag check it was missing.previewGitHub Environment (which holds the secrets), so a PR that modifies the workflow can't reach them unreviewed. Cleanup deliberately never runs onpull_requestevents — the sweep runs default-branch code via thepreview-cleanupenvironment, restricted todevelop./_login(the site gate) now has an nginx rate limit keyed onFly-Client-IP.Also in this PR
Dockerfile.fly: MariaDB is installed for any non-production startup script (the guard was hard-coded toyesterday-startup.sh).PR-<N>badge automatically (APP_SHOW_BRANCH), with a newMAILPIT_URLoverride so its mail link points at the shared Mailpit.One-time setup required before first use
Documented in
docs/fly-deployment.md→ PR Previews → One-time setup: two GitHub environments (previewwith required reviewers,preview-cleanuprestricted todevelop),FLY_ORG_TOKEN(org-scoped),FLY_PREVIEW_SECRETS(fresh gate password, read-only Tigris key, read-only Drive service account),FLY_ORGrepo variable, and thepreviewlabel. After merge + setup, the plan is to canary on PR #887.Testing
UploadsDisabledTest(5 tests) andWordpressIntegrationFlagTest(4 tests), written first, red → green; existing WordPress push and group edit tests still pass.preview-startup.sh: bash -n + shellcheck; both nginx configs validated withnginx -t;fly.pr.tomlTOML-parsed; workflow validated with actionlint.