Support pg_tde Transparent Data Encryption in init scripts#64
Open
tamalsaha wants to merge 11 commits into
Open
Support pg_tde Transparent Data Encryption in init scripts#64tamalsaha wants to merge 11 commits into
tamalsaha wants to merge 11 commits into
Conversation
Compose pg_tde into shared_preload_libraries in every config-authoring path (including the PITR restore path, so encrypted clusters recover without corruption), add the pg_tde GUCs (wal_encrypt, enforce_encryption, cipher) and the default_table_access_method=tde_heap flip in the post-bootstrap rewrite, add the principal-key bootstrap block (CREATE EXTENSION, key provider, principal and server keys) on first primary init, and seed replicas with pg_tde_basebackup when TDE is enabled. All behavior is gated on TDE_ENABLED so non-TDE clusters are unaffected. Signed-off-by: Tamal Saha <tamal@appscode.com>
The file key provider is registered with pg_tde_add_database_key_provider_file, so its principal key must be set with pg_tde_set_key_using_database_key_provider. The global setter cannot see a database-scoped provider, so file-provider TDE never got a principal key (the failure was masked by || true). Branch the setter on TDE_PROVIDER_KIND: database-scoped for file, global for vault and kmip. Signed-off-by: Tamal Saha <tamal@appscode.com>
pg_tde 2.2.1 split key creation from activation: pg_tde_set_key_* errors with 'key does not exist' unless pg_tde_create_key_* was called first. The bootstrap previously called only the set function guarded by || true, which silently swallowed the missing-key error and left the principal key unset. Create the key first, use the database-scoped setter for file providers and the global setter otherwise, and fail loudly if the set does not succeed. Signed-off-by: Tamal Saha <tamal@appscode.com>
Make the pg_tde_basebackup vs pg_basebackup selection visible in standby logs so a wrong-binary seed (which silently corrupts an encrypted cluster) is greppable, per the TDE debug-logging requirement. Signed-off-by: Tamal Saha <tamal@appscode.com>
The bootstrap creates the principal key before the post-bootstrap config (which carries pg_tde.cipher) is loaded, so with cipher aes_256 the key was created at the default 16-byte aes_128 length and pg_tde warned that the key length did not match the cipher. Pass PGOPTIONS='-c pg_tde.cipher=<cipher>' on the create calls so the key is generated at the right length. Day-2 ops (RotatePrincipalKey) are unaffected: they run against the live server whose session default already carries the configured cipher. Signed-off-by: Tamal Saha <tamal@appscode.com>
Scope the configured cipher to every pg_tde key call in the bootstrap (create and set, principal and WAL) via a single exported PGOPTIONS, so both key creation and activation run at the configured length. Setting it only on create left the separate set_key session at the default cipher, which warned that the 32-byte aes_256 key mismatched the session's cipher. Unset it after the block so later connections keep the server default. Signed-off-by: Tamal Saha <tamal@appscode.com>
…ails When the principal/WAL key could not be set during first-time init (e.g. an unreachable or unauthorized key provider), the bootstrap exited non-zero but the init loop simply retried with BOOTSTRAP=false, skipped the TDE block, and booted a database with no principal key: a silent, broken-encryption boot. Since this path runs only during initial init (no user data), wipe the half-initialized data dir on failure so the pod re-runs a full bootstrap and crash-loops visibly until the key provider is reachable, rather than silently disabling TDE. Signed-off-by: Tamal Saha <tamal@appscode.com>
- standby GUC blocks (run.sh, ha_backup_job.sh, remote-replica.sh, warm_stanby.sh across all majors) now emit default_table_access_method = tde_heap when TDE_DEFAULT_AM is set, matching the primary. Without it a promoted standby kept its own postgresql.conf and defaulted new tables to plaintext heap (a silent encryption gap after failover when enforce_encryption is off). - primary start.sh fails closed with a clear message when WAL encryption is requested with the file keyring: WAL encryption is global-only, so the file (database-scoped) provider can never back it. The operator webhook already rejects this combo; this is a diagnosable backstop instead of a swallowed error that crash-loops on the misleading set-key fatal. Signed-off-by: Tamal Saha <tamal@appscode.com>
… encryption pg_tde stores the principal key per database OID and CREATE DATABASE does not copy it, so a key set in template1 is not inherited by databases cloned from it. Under a cluster-wide default_table_access_method = tde_heap, the first encrypted CREATE TABLE in a freshly created database therefore failed with 'principal key not configured'. Set a server-wide default principal key at bootstrap via pg_tde_set_default_key_using_global_key_provider (create-before-set, strict); databases without their own principal key fall back to it, so new databases can create encrypted tables out of the box. The default key is a global-provider feature, so it is set for the vault/kmip providers only (the file keyring is a standalone, database-scoped provider). Verified on k3s-188: a brand-new database created an encrypted table with no manual key setup. Signed-off-by: Tamal Saha <tamal@appscode.com>
The server-wide default principal key only matters when new tables are encrypted by default (default_table_access_method=tde_heap) or enforced cluster-wide, so gate its strict create/set on TDE_DEFAULT_AM or TDE_ENFORCE, mirroring how the WAL server-key block gates on TDE_ENCRYPT_WAL. A plain opt-in-per-table cluster no longer runs the strict set (and its crash-loop-on-failure path) for a default key it never uses. Signed-off-by: Tamal Saha <tamal@appscode.com>
…ation The operator maps spec.tde.keyProvider.vault.namespace to the TDE_VAULT_NAMESPACE env, but the bootstrap called the 5-arg pg_tde_add_global_key_provider_vault_v2 and never passed it, so a configured Vault Enterprise namespace was silently dropped. pg_tde 2.x has a 6-arg overload that takes the namespace; use it when TDE_VAULT_NAMESPACE is set, keeping the 5-arg form for the common namespace-less case. Only the primary registers the provider (standbys inherit it via pg_tde_basebackup), so this is the sole registration site. Signed-off-by: Tamal Saha <tamal@appscode.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds pg_tde (Percona Transparent Data Encryption) support to the Postgres init container scripts. Gated entirely on
TDE_ENABLED, so non-TDE clusters are unaffected.What this does
pg_tdeintoshared_preload_librariesin every config-authoring path, including the PITR restore path (an encrypted cluster must preload pg_tde during recovery or it corrupts).pg_tde.wal_encrypt,pg_tde.enforce_encryption,pg_tde.cipher) and thedefault_table_access_method=tde_heapflip in the post-bootstrap config rewrite.primary/start.sh(CREATE EXTENSION on template1 + the app db, register the key provider, set the principal and server keys) on first initialization only.pg_tde_basebackupinstead ofpg_basebackupwhen TDE is enabled.Applied identically across PostgreSQL 9-18. The operator sets the
TDE_*env contract. Requires a Percona Server for PostgreSQL 17.x image that bundles pg_tde and the CLI tools. Part of the pg_tde effort tracked in kubedb/apimachinery#1831.