feat(postgres): expose PostgreSQL tunables as SHELLHUB_POSTGRES_* variables - #6892
feat(postgres): expose PostgreSQL tunables as SHELLHUB_POSTGRES_* variables#6892otavio wants to merge 1 commit into
Conversation
5700314 to
307e51e
Compare
…iables PostgreSQL runs on stock defaults on every deployment -- 128 MB of shared_buffers regardless of the machine underneath. On the largest managed instance that is a 84.76% buffer cache hit ratio against a 5.9 GB database on a 15.6 GB host. Sizing that per host needs a seam, and overriding `command:` from a second Compose file is the wrong one: Compose replaces `command:` rather than merging it, so an override has to restate io_method, wal_compression and shared_preload_libraries, and silently drops whatever this list grows next. Parameterise the flags instead. Every default is the value the deployment already runs with, so the rendered command line is unchanged and this carries no behaviour of its own: .env documents the knobs, and .env.override -- which bin/docker-compose already loads last -- is where a host sets them. Two Docker-side knobs come along because they bound the same thing. shm_size is where parallel-query segments are allocated from, not shared_buffers, so it has to track work_mem. mem_limit stays at today's unlimited, but is worth setting once shared_buffers grows on a host with no swap. Refs: shellhub-io/team#198
ac00c50 to
a5652ac
Compare
307e51e to
9e9d9b0
Compare
|
Superseded by #6886, which now carries this work rebased onto The shape changed on the way over. Instead of thirteen The reasoning that led there: the wholesale-replacement problem this PR set out to solve is One behavioural difference worth recording: this PR defaulted No commits are lost; |
Every ShellHub deployment inherits the same untuned
postgres:18.0: 128 MB ofshared_buffersregardless of the machine underneath. On the largest managed instance that isa 84.76% buffer cache hit ratio against a 5.9 GB database on a 15.6 GB host.
This PR does not tune anything. It ships the seam, with every default equal to the value the
deployment already runs with, so the rendered command line is unchanged. The values are set
per host — for the managed fleet, in shellhub-io/shellhub-managed-deploy#400, which derives them
from
/proc/meminfo.Why env vars and not a second Compose file
The obvious alternative is for a deployment to override
command:in its own overlay. Thatbreaks quietly: Compose replaces
command:rather than merging it, so an override has torestate
io_method,wal_compressionandshared_preload_libraries— and silently dropswhatever this list grows next. #6883 and #6886 each added a flag here in the last week, so that
is not hypothetical.
bin/docker-composealready chainsCOMPOSE_ENV_FILESwith.env.overridelast, sointerpolation is a seam both community users and managed hosts already have.
:-rather than-throughout: an override that sets a key to nothing must fall back, notexpand to
-c shared_buffers=, which postgres refuses to start on.What is exposed
shared_buffers,max_wal_size,min_wal_size,max_connections128MB,1GB,80MB,100effective_cache_size,work_mem,maintenance_work_mem4GB,4MB,64MBcheckpoint_timeout,random_page_cost5min,4effective_io_concurrency16wal_compressionlz4offcheckpoint_completion_targetis not exposed: it has defaulted to0.9since PG14, sosetting it is a no-op. Verified against the running image rather than assumed.
Two Docker-side knobs come along because they bound the same thing:
shm_size— parallel-query segments are allocated from/dev/shm, which is 64 MB bydefault.
shared_buffersis not (shared_memory_typeismmap), so this trackswork_memand parallelism instead.
mem_limit— stays at today's unlimited (0, which Compose normalises away entirely), butis worth setting once
shared_buffersgrows on a host with no swap, where an OOM is a hardkill.
Testing
./bin/docker-compose configrenders and shlex-splits the folded scalar into correct argv, atdefaults and under override.
-c shared_buffers=.postgres:18.0booted with a fully tuned command line reports all twelve settingswith
source = command line— confirming they override the initdb-writtenpostgresql.conf—and
pg_stat_statementsstill loads alongside.Refs shellhub-io/team#198.