Move to PostgreSQL 18 (supersedes #284)

[richardmhope]

Summary

Dependabot's #284 bumps postgres 17 → 18 in docker-compose.yml only, but this major bump is more involved than an image-tag change: the official postgres:18 image relocated its data volume from /var/lib/postgresql/data to /var/lib/postgresql, with PGDATA now defaulting to a versioned subdirectory (/var/lib/postgresql/18/docker) so a future in-place pg_upgrade can hold two versions side by side. With the bare bump, the named iceberg-pg-data volume would still be mounted at the old …/data path while the cluster is written under the image's new anonymous volume — i.e. silent data loss on container recreate. It also leaves the Kubernetes StatefulSet and the CI service containers on 17.

This PR does the full move and documents the operator-facing upgrade path.

Changes

• docker-compose.yml — postgres:18; the iceberg-pg-data mount moves to /var/lib/postgresql, with a comment explaining the 18 layout change and why the old path would strand the cluster on an anonymous volume.
• deploy/k8s/postgres.yaml — postgres:18; PVC mountPath moves to /var/lib/postgresql (the subPath: pgdata indirection is unchanged).
• .github/workflows/ci.yml — both service containers (postgres-smoke and the docker boot smoke) bumped to postgres:18, so CI exercises the same major version the deployments run.
• README.md — compose deployment diagram refreshed (image tag + volume path).
• website/docs/deployment.md / deploy/k8s/README.md — backup & restore sections now document that a major Postgres upgrade (17 → 18) rides the existing dump/restore path: a data directory initialised by one major version cannot be opened by the next, so a 17-era volume/PVC is not directly reusable.
• CHANGELOG.md — recorded under Unreleased → Changed.

No application-code changes are needed: the app talks to Postgres via psycopg v3 / SQLAlchemy, and nothing in the migrations or the tsvector FTS path is version-sensitive across 17 → 18.

Testing

• YAML parse-checked (docker-compose.yml, deploy/k8s/postgres.yaml, .github/workflows/ci.yml)
• Repo-wide grep confirms no remaining postgres:17 / stale /var/lib/postgresql/data mount references (the old path now appears only in upgrade-note prose)
• postgres-smoke + docker CI jobs run on push to main (they're skipped per-PR by design) and will validate migrations, tsvector search, and the boot smoke against Postgres 18

Checklist

• Docs updated (README.md, website/docs/deployment.md, deploy/k8s/README.md, CHANGELOG.md)
• Migration added — N/A, no model change
• Security-sensitive changes called out — none; deployment/CI image pins and volume paths only

---

🤖 Generated with Claude Code

https://claude.ai/code/session_01SCSCCyvyMQAF8aHzpPyFYb

---

Generated by Claude Code