fix: keel-overlay-installer keeps private keys out of etckeeper's /etc/.git

.github/workflows/packages.yml

@@ -23,7 +23,9 @@
 # the overlay. The job writes the policy-rc.d that exits 101 fab installs
 # under. systemd is PID 1 in the container, so the is-active assertions are
 # made as well; the image build's case, no /run/systemd/system, is the one
-# the suite reproduces itself where it needs it. Last, with ntpsec
+# the suite reproduces itself where it needs it. etckeeper is installed
+# with them, so its postinst makes /etc/.git, and the suite checks that
+# keel-overlay-installer keeps private keys out of it (common#49). Last, with ntpsec
 # installed, tests/container-units-live.bats applies
 # conf/turnkey.d/container-units to the container itself: systemd skips
 # ntpsec and the two kernel mounts on the drop-ins' condition, and starting
@@ -124,7 +126,7 @@ jobs:
           ./dist/keel-overlay-etcd_*.deb \
           ./dist/keel-overlay-crowdsec_*.deb \
           ./dist/keel-overlay-vip_*.deb \
-          ntpsec
+          etckeeper ntpsec
         # unit states and manifests on the machine
         KEEL_OVERLAY_INSTALL_TEST=1 KEEL_DEB="$(ls /tmp/deps/keel_*.deb)" \
           bats tests/overlay-install.bats

COVERAGE.md

@@ -4,6 +4,16 @@ Measured on 2026-09-24 against upstream 19.x (b60dd23), following the
 project decision 0003 (90 percent floor per repository, 95 percent for every
 file our changes touch).
 
+## Branch fix/etckeeper-private-keys: 100 percent, one new file (2026-10-09)
+
+`packages/installer/etckeeper-ignore` is measured for the first time: 37
+of 37 lines, kcov 43 on Debian 13, from `tests/etckeeper-ignore.bats` (18
+tests). The scratch /etc is a real git repository, with the global and
+system git configuration set aside; git itself says what is ignored and
+tracked. The live tests in `tests/overlay-install.bats` (CI job install)
+run the postinst with the real etckeeper and contribute no percentage.
+Every other target is unchanged at 100.
+
 ## Branch fix/no-build-time-db-password-and-mysqltuner: 100 percent, two new files (2026-10-07)
 
 `conf/pgsql` and `conf/mysql` are measured for the first time: 13 of 13 and

packages/README.md

@@ -9,7 +9,7 @@ directory each, with its own changelog and version, released on its own
 
 | Directory | Package | Rests on | Units it owns |
 | --- | --- | --- | --- |
-| `installer/` | `keel-overlay-installer` | Keel's `inithooks` (>= 2.3.6+keel14), `confconsole` (>= 2.2.3+keel8), `keel` (>= 0.12.0) | none; its manifest names the first boot hooks the three ship |
+| `installer/` | `keel-overlay-installer` | Keel's `inithooks` (>= 2.3.6+keel14), `confconsole` (>= 2.2.3+keel8), `keel` (>= 0.12.0) | none; its manifest names the first boot hooks the three ship. Its postinst keeps private keys out of etckeeper's `/etc/.git` (below) |
 | `wireguard/` | `keel-overlay-wireguard` | trixie's `wireguard-tools` 1.0.20210914 | none; the interface is the instance spec's |
 | `etcd/` | `keel-overlay-etcd` | trixie's `etcd-server` 3.5.16 and its `etcd-client` (etcdctl, which keel asks etcd with over gRPC, keel#83), both bounded to that upstream release, `keel` (>= 0.21.0), whose gate its drop-in runs (keel 0.22.0 asks etcd with the etcdctl this brings), and `keel-overlay-wireguard`, which its manifest `requires` | `etcd.service`, and its drop-in `keel-overlay-etcd.conf`: one member restarts at a time |
 | `vip/` | `keel-overlay-vip` | `keel` (>= 0.21.0), whose `keel vip` the units run, and `keel-overlay-wireguard`, which its manifest `requires` | `keel-vip.service`, the VIP's root helper with etcd, which starts its unprivileged controller as the transient `keel-vip-control`; and `keel-vip-check.timer`, part of it |
@@ -43,6 +43,49 @@ listener, `http://[::1]:2381`, which keel renders
 (`ETCD_LISTEN_METRICS_URLS`): the client port, 2379, wants TLS and a
 client certificate signed in the mesh.
 
+## Private keys and etckeeper
+
+etckeeper commits all of `/etc` after each apt run and once a day. Keys
+that keel, the overlays and Debian write under `/etc` went into
+`/etc/.git` with it: `wireguard/wg0.key` and `mysql/keel-tls/server.key`
+were seen there on real nodes (Keel-Linux/common#49). A key that is
+rotated later stays in the history.
+
+`keel-overlay-installer`, which every Keel appliance has through
+`keel-core`, ships `/usr/lib/keel/overlays/installer/etckeeper-ignore`,
+and its postinst runs it at each configure:
+
+1. It writes this list into `/etc/.gitignore`, between its own marker
+   lines, outside the section etckeeper manages. etckeeper's `init`,
+   `update-ignore` and `uninit` keep every line outside that section, so
+   the list stays when the first boot hook `92etckeeper` makes the
+   repository again.
+
+   | Path under `/etc` | Written by |
+   | --- | --- |
+   | `wireguard/*.key` | keel, the node's WireGuard key (`keel network wireguard key`) |
+   | `mysql/keel-tls/*.key` | keel, the database's TLS key |
+   | `etcd/keel/*.key` | keel, the etcd keys of the mesh |
+   | `keel/secrets/` | the secrets of the instance spec |
+   | `ssl/private/` | turnkey-ssl and ssl-cert |
+   | `ssh/ssh_host_*_key` | openssh-server, the host keys (the `.pub` files stay tracked) |
+   | `crowdsec/local_api_credentials.yaml`, `crowdsec/online_api_credentials.yaml`, `crowdsec/bouncers/*.yaml.local` | CrowdSec's packages and keel |
+   | `anubis/keel.key`, `anubis/keel.key.*` | `keel-overlay-anubis-key.service` |
+
+2. Where `/etc/.git` exists, it removes the tracked files of the list from
+   the index (`git rm --cached`) and commits that removal with the new
+   `.gitignore`, and with no other change. The files stay on disk. The
+   history is not rewritten, so each key that was tracked must be rotated:
+   the postinst lists the files. keel has no command that rotates a key
+   yet; keel makes each key once, and a renewal of a certificate keeps
+   the key.
+
+A failure (for example, a lock on the index) does not fail the
+installation: the postinst says so, and the script can be run again as
+root. `tests/etckeeper-ignore.bats` tests the script on a scratch
+repository; `tests/overlay-install.bats` tests it with the real
+etckeeper on a booted trixie container.
+
 ## Building
 
 ```

packages/installer/debian/changelog

@@ -1,3 +1,31 @@
+keel-overlay-installer (0.2.0) trixie; urgency=medium
+
+  * etckeeper does not commit private keys and secrets into /etc/.git
+    (Keel-Linux/common#49). Seen on real nodes with keel 0.23.3:
+    wireguard/wg0.key at the keel installation and
+    mysql/keel-tls/server.key at a reinstallation.
+  * The postinst runs /usr/lib/keel/overlays/installer/etckeeper-ignore at
+    each configure. It writes this list into /etc/.gitignore, outside the
+    section etckeeper manages, so etckeeper's init, update-ignore and
+    uninit keep it: wireguard/*.key, mysql/keel-tls/*.key,
+    etcd/keel/*.key, keel/secrets/, ssl/private/, ssh/ssh_host_*_key,
+    crowdsec/local_api_credentials.yaml,
+    crowdsec/online_api_credentials.yaml, crowdsec/bouncers/*.yaml.local,
+    anubis/keel.key and anubis/keel.key.*.
+  * Where /etc/.git exists, it removes the tracked files of the list from
+    the index (git rm --cached) and commits that removal with the new
+    .gitignore, and with no other change. The files stay on disk. The
+    history is not rewritten.
+  * Rotate every key that was tracked: the old commits of /etc/.git still
+    hold it, and so does each copy or backup of /etc/.git. The postinst
+    lists the files it removed from the index. keel 0.23.3 has no command
+    that rotates a key: keel makes each key once, and a renewal of a
+    certificate (etcd, the database's TLS) keeps the same key. Rotate the
+    SSH host keys, the TLS keys of ssl/private, CrowdSec's credentials and
+    Anubis's signing key with the tools of their own packages.
+
+ -- KeelLinux maintainers <admin@keellinux.org>  Fri, 09 Oct 2026 12:00:00 +0000
+
 keel-overlay-installer (0.1.0) trixie; urgency=medium
 
   * First release (handbook decision 0041, first implementation, step 2).

packages/installer/debian/control

@@ -26,3 +26,8 @@ Description: Keel overlay: the installer
  .
  The manifest names the first boot hooks the three packages ship. The
  overlay declares no process: it is enabled in every installation mode.
+ .
+ It also keeps private keys and secrets out of etckeeper's /etc/.git:
+ the postinst writes a list of their paths into /etc/.gitignore and
+ removes the keys that are already tracked from the index, without a
+ change to the history.

packages/installer/debian/postinst

@@ -0,0 +1,19 @@
+#!/bin/sh
+# keel-overlay-installer: keep private keys and secrets out of etckeeper's
+# /etc/.git (Keel-Linux/common#49). At each configure, the first
+# installation and every upgrade or reinstall, etckeeper-ignore writes its
+# list into /etc/.gitignore and, where /etc/.git exists, removes the
+# tracked keys from the index in one commit, without a change to the
+# history (packages/README.md, "Private keys and etckeeper"). A failure
+# does not fail the installation: the operator is told how to run it again.
+set -e
+
+if [ "$1" = configure ]; then
+    if ! /usr/lib/keel/overlays/installer/etckeeper-ignore; then
+        echo "keel-overlay-installer: warning: private keys can still be tracked in /etc/.git; run /usr/lib/keel/overlays/installer/etckeeper-ignore as root when the cause is corrected" >&2
+    fi
+fi
+
+#DEBHELPER#
+
+exit 0

packages/installer/debian/rules

@@ -1,11 +1,13 @@
 #!/usr/bin/make -f
-# The overlay manifest sits beside debian/ (docs/manifest-v1.md, "Kinds,
-# and where each file lives") and is installed under the name keel looks
-# it up by.
+# The overlay manifest and etckeeper-ignore sit beside debian/
+# (docs/manifest-v1.md, "Kinds, and where each file lives") and are
+# installed under the names keel and the postinst look them up by.
+
+PKG = debian/keel-overlay-installer
 
 %:
 	dh $@
 
 execute_after_dh_auto_install:
-	install -D -m 0644 manifest.yaml \
-		debian/keel-overlay-installer/usr/share/keel/overlays/installer.yaml
+	install -D -m 0644 manifest.yaml $(PKG)/usr/share/keel/overlays/installer.yaml
+	install -D -m 0755 etckeeper-ignore $(PKG)/usr/lib/keel/overlays/installer/etckeeper-ignore

packages/installer/etckeeper-ignore

@@ -0,0 +1,83 @@
+#!/bin/bash
+# /usr/lib/keel/overlays/installer/etckeeper-ignore
+#
+# Keeps private keys and secrets out of etckeeper's /etc/.git
+# (Keel-Linux/common#49). etckeeper commits all of /etc after each apt run
+# and once a day, so a key that keel, an overlay or a package writes under
+# /etc goes into the history: wireguard/wg0.key and
+# mysql/keel-tls/server.key were seen there on real nodes.
+#
+# 1. Writes the list below into /etc/.gitignore, between its own two marker
+#    lines, after a removal of the list an earlier run wrote. etckeeper
+#    keeps every line outside its "managed by etckeeper" section through
+#    init, update-ignore and uninit, so the list stays after a first boot
+#    makes the repository again.
+# 2. Where /etc/.git exists, removes from the index the files the list
+#    matches that are tracked, and commits that removal with the new
+#    .gitignore, and with no other change. The files stay on disk. The
+#    history is not rewritten: the old commits still hold the keys, so
+#    each key that was tracked must be rotated.
+#
+# keel-overlay-installer's postinst runs it at each configure. Exit codes:
+# 0 done, 1 the removal or the commit failed (nothing is committed then).
+#
+# KEEL_ETC_DIR exists for tests/etckeeper-ignore.bats; on a machine it is
+# /etc.
+set -euo pipefail
+
+ETC="${KEEL_ETC_DIR:-/etc}"
+IGNORE="$ETC/.gitignore"
+BEGIN="# begin keel-overlay-installer: private keys and secrets, never committed (Keel-Linux/common#49)"
+END="# end keel-overlay-installer"
+MESSAGE="keel-overlay-installer: private keys and secrets removed from the index (Keel-Linux/common#49)"
+
+# Relative to /etc. Keys keel writes: the WireGuard key beside the
+# interface's configuration, the database's TLS key, the etcd keys of the
+# mesh, the secrets of the instance spec. Keys the overlays and Debian
+# write: Anubis's signing key, CrowdSec's credentials, the TLS keys of
+# turnkey-ssl and ssl-cert, the SSH host keys. Each statement is kept on
+# one line, so that kcov counts every line it runs.
+PATTERNS=("/wireguard/*.key" "/mysql/keel-tls/*.key" "/etcd/keel/*.key" "/keel/secrets/" "/ssl/private/" "/ssh/ssh_host_*_key" "/crowdsec/local_api_credentials.yaml" "/crowdsec/online_api_credentials.yaml" "/crowdsec/bouncers/*.yaml.local" "/anubis/keel.key" "/anubis/keel.key.*")
+
+write_list() {
+    local staged
+    staged="$(mktemp "$ETC/.gitignore.keel.XXXXXX")"
+    trap 'rm -f -- "$staged"' EXIT
+    chmod 0644 "$staged"
+    if [ -f "$IGNORE" ]; then
+        awk -v end="$END" 'index($0, "# begin keel-overlay-installer") == 1 { skip = 1; next } skip && $0 == end { skip = 0; next } !skip' "$IGNORE" > "$staged"
+        chmod --reference="$IGNORE" "$staged"
+    fi
+    { echo "$BEGIN"; printf '%s\n' "${PATTERNS[@]}"; echo "$END"; } >> "$staged"
+    mv -f -- "$staged" "$IGNORE"
+    trap - EXIT
+}
+
+untrack() {
+    local excludes=() pattern tracked
+    for pattern in "${PATTERNS[@]}"; do
+        excludes+=(-x "$pattern")
+    done
+    tracked="$(git -C "$ETC" -c core.quotePath=false ls-files --cached --ignored "${excludes[@]}")"
+    if [ -z "$tracked" ]; then
+        return 0
+    fi
+    if ! printf '%s\n' "$tracked" | GIT_LITERAL_PATHSPECS=1 git -C "$ETC" rm --cached --quiet --pathspec-from-file=- || ! git -C "$ETC" add -- .gitignore; then
+        echo "etckeeper-ignore: could not remove the private files from the index of $ETC/.git; nothing was committed" >&2
+        return 1
+    fi
+    export GIT_AUTHOR_NAME="${GIT_AUTHOR_NAME:-keel-overlay-installer}" GIT_AUTHOR_EMAIL="${GIT_AUTHOR_EMAIL:-root@localhost}"
+    export GIT_COMMITTER_NAME="${GIT_COMMITTER_NAME:-$GIT_AUTHOR_NAME}" GIT_COMMITTER_EMAIL="${GIT_COMMITTER_EMAIL:-$GIT_AUTHOR_EMAIL}"
+    if ! git -C "$ETC" commit --quiet -m "$MESSAGE"; then
+        echo "etckeeper-ignore: could not commit in $ETC/.git" >&2
+        return 1
+    fi
+    echo "etckeeper-ignore: these files are no longer tracked in $ETC/.git:"
+    echo "  ${tracked//$'\n'/$'\n'  }"
+    echo "etckeeper-ignore: the history of $ETC/.git still holds them: rotate each key and secret listed (Keel-Linux/common#49)"
+}
+
+write_list
+if [ -d "$ETC/.git" ]; then
+    untrack
+fi

tests/coverage.sh

@@ -41,6 +41,7 @@ targets=(
     "overlays/turnkey.d/resolvconf-ifupdown-ng/etc/network/if-down.d/resolvconf-ifupdown-ng:tests/resolvconf-ifupdown-ng.bats"
     "packages/coraza/state:tests/coraza-state.bats"
     "packages/anubis/signing-key:tests/anubis-signing-key.bats"
+    "packages/installer/etckeeper-ignore:tests/etckeeper-ignore.bats"
     "lib/version-files.sh:tests/version-files.bats"
     "bin/keel-version-files:tests/version-files.bats"
 )