Skip to content

Harden Vault observability and upgrade handling - #6

Merged
systemdbrew merged 3 commits into
mainfrom
vault-observability-hardening
Sep 15, 2026
Merged

systemdbrew merged 3 commits into
mainfrom
vault-observability-hardening

Conversation

@systemdbrew

Copy link
Copy Markdown
Owner

What changed

  • enable Vault Prometheus ServiceMonitor for all three HA members
  • add Vault-specific alerts for sealed nodes, missing/multiple active nodes, and missing telemetry targets
  • replace the generic StatefulSet rollout alert with a version that excludes only vault/vault
  • add VaultManualRolloutPending as an informational signal for Vault's intentional OnDelete upgrade workflow
  • document the safe standby-first Vault/Raft upgrade procedure
  • document the auto-unseal migration path and explicitly avoid a circular self-Transit design

Why

Vault uses OnDelete while Shamir sealing is enabled. The upstream KubeStatefulSetUpdateNotRolledOut alert therefore treats an intentional pending Vault upgrade as a failed rollout. This keeps normal StatefulSet coverage while giving Vault alerts that reflect actual service health.

Auto-unseal

This PR does not invent or commit seal credentials. Auto-unseal requires an independent KMS/HSM or separate Transit Vault; the runbook records the migration steps for when that provider is selected.

@systemdbrew
systemdbrew merged commit 38de161 into main Sep 15, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant