From 1d8d208f7528fa7b5b14dedb123f1f86be778ca7 Mon Sep 17 00:00:00 2001 From: navigator Date: Fri, 2 Oct 2026 15:03:25 +0000 Subject: [PATCH 1/3] fix: drop the overlay's Keel source and 1001 pin; 990 everywhere The overlay shipped /etc/apt/sources.list.d/keel.sources and /etc/apt/preferences.d/keel at 1001. At 1001 apt downgrades every package newer than the archive's (tracker#23), and at those paths both files override the source and the 990 pin Keel-Linux/common#30 ships. conf.d/zzz-keel-archive verifies common's two files and changes nothing, or writes the same ones on a bootstrap from before common shipped them, and refuses any pin but 990; it no longer turns every Enabled: no into yes, which would enable the testing track. The build time archive is pinned by its Label for the build only, removed by zz-project-packages, which fails if any apt file still names it. The boot test expects 990, reads the stable stanza only, and accepts a newer installed version apt keeps instead of downgrading. --- COVERAGE.md | 24 ++-- README.rst | 9 +- changelog | 18 +++ conf.d/main | 10 ++ conf.d/zz-project-packages | 22 ++-- conf.d/zzz-keel-archive | 86 ++++++++++---- overlay/etc/apt/preferences.d/keel | 5 - overlay/etc/apt/sources.list.d/keel.sources | 24 ---- tests/apt-files.bats | 46 ++++++++ tests/boot-test.bats | 88 +++++++++++--- tests/keel-archive.bats | 121 ++++++++++++++++---- tests/lib/boot-test-lib.sh | 33 +++++- tests/lib/candidate-source.awk | 2 +- tests/project-packages.bats | 36 +++++- 14 files changed, 405 insertions(+), 119 deletions(-) delete mode 100644 overlay/etc/apt/preferences.d/keel delete mode 100644 overlay/etc/apt/sources.list.d/keel.sources create mode 100644 tests/apt-files.bats diff --git a/COVERAGE.md b/COVERAGE.md index ab96a3b..43316bc 100644 --- a/COVERAGE.md +++ b/COVERAGE.md @@ -12,16 +12,16 @@ acceptance test of a recipe, docs/org-plan.md section 1). | `overlay/usr/lib/inithooks/firstboot.d/40wordpress` | `tests/hook.bats` (29 tests) | 97.73 percent (43/44) under kcov | the hook itself, run for real | | `overlay/usr/local/bin/keel-wp` | `tests/wrappers.bats` (25 tests) | 100 percent (8/8) under kcov | both cache branches, the quoting, the exit code it hands back, `DEBUG`, and the `turnkey-wp` link run for real | | `overlay/usr/local/sbin/keel-wordpress-update` | `tests/wrappers.bats` (the same 25) | 100 percent (21/21) under kcov | both guards refused and satisfied, each wp-cli call made to fail, the whole ownership boundary, and the two names it must not take from the environment | -| `tests/lib/boot-test-lib.sh` | `tests/boot-test.bats` (73 tests) | 98.95 percent (282/285) under kcov | parsing, addresses, deadlines, the container marks, every verdict, and the image carrying none of the build time archive files | -| `conf.d/zzz-keel-archive` | `tests/keel-archive.bats` (13 tests) | 100 percent (26/26) under kcov | every way it enables and every way it refuses, including a staging keyring left in the image | -| `conf.d/zz-project-packages` | `tests/project-packages.bats` (14 tests) | 100 percent (31/31) under kcov | shared with keel-nodebb, where the pattern is maintained | +| `tests/lib/boot-test-lib.sh` | `tests/boot-test.bats` (79 tests) | 99.32 percent (292/294) under kcov | parsing, addresses, deadlines, the container marks, every verdict, and the image carrying none of the build time archive files | +| `conf.d/zzz-keel-archive` | `tests/keel-archive.bats` (17 tests) | 100 percent (37/37) under kcov | common's source and 990 pin verified and left alone, both written where common did not ship them, a 1001 pin refused, testing never enabled, and every way it refuses, including a staging keyring left in the image | +| `conf.d/zz-project-packages` | `tests/project-packages.bats` (16 tests) | 100 percent (35/35) under kcov | shared with keel-nodebb, where the pattern is maintained | | `bin/keel-archive-check` | `tests/archive-check.bats` (27 tests) | 100 percent (54/54) under kcov | the build time check of tracker#7: the copy is the live archive, the entry names the keyring through signed-by, nothing says trusted=yes, and the copied InRelease verifies against the staging key | | `overlay/usr/lib/inithooks/bin/wordpress.py` | `tests/dialog.bats` (3 tests) | not measured (kcov measures the shell) | dialog wrapper, run as the hook runs it inside a pseudo terminal: the answers reach the hook and the boxes are drawn on the terminal | | `overlay/usr/lib/inithooks/lib/*.php` | the boot test | integration only | two PHP files `wp eval-file` runs; `conf.d/main` has PHP lint them | | `conf.d/main` | the build | integration only | build time script, 0004 pragmatic limits | | `tests/boot-test.sh` | itself | integration only | the thin main of the acceptance test: keel and LXC as root | -Total over the eight measured shell files: **99.12 percent (564/569)**, 221 +Total over the eight measured shell files: **99.33 percent (589/593)**, 239 bats tests, none failing. `tests/coverage.sh` fails below `COVERAGE_THRESHOLD`, which the workflow sets to **97**, the lowest measured file. It is only ever raised (decision 0006). @@ -30,11 +30,11 @@ sets to **97**, the lowest measured file. It is only ever raised (decision kcov line coverage (threshold 97 percent): 100.00 21/21 keel-wordpress-update 100.00 8/8 keel-wp - 100.00 31/31 zz-project-packages - 100.00 26/26 zzz-keel-archive + 100.00 35/35 zz-project-packages + 100.00 37/37 zzz-keel-archive 99.00 99/100 wordpress.sh 97.73 43/44 40wordpress - 98.95 282/285 boot-test-lib.sh + 99.32 292/294 boot-test-lib.sh 100.00 54/54 keel-archive-check ### The two operator commands, and the link beside each @@ -143,7 +143,7 @@ boot-test: /wp-admin/ answered 200 with the dashboard for the logged in admin boot-test: a wrong password was refused boot-test: https://archive.keellinux.org trixie is enabled and verified with /usr/share/keyrings/keel-archive-keyring.gpg boot-test: apt-get update read https://archive.keellinux.org trixie and verified its signature -boot-test: apt takes inithooks from https://archive.keellinux.org at priority 1001, candidate 2.3.6+keel5 +boot-test: apt takes inithooks from https://archive.keellinux.org at priority 990, candidate 2.3.6+keel5 boot-test: keel-transition is not in the image, which is what makes the next step a proof boot-test: the keel-transition archive is 13836 bytes boot-test: apt fetched keel-transition from https://archive.keellinux.org, against the digest of the signed index @@ -181,8 +181,12 @@ The following packages will be DOWNGRADED: and `keel-archive-keyring 0.1.0` ships only the **revoked** signing subkey `694DE5E8`, so the appliance would have lost the ability to verify the archive -at all. The rule that follows is the one the policy check now guards: **an -image must not carry a project package the signed archive has not got.** +at all. The rule that followed was: **an image must not carry a project +package the signed archive has not got.** The pin is now 990 (tracker#23, +Keel-Linux/common#30), below 1000, so apt keeps a newer installed version +instead of downgrading it, and the policy check accepts exactly that case: the +candidate is the installed version alone, newer than the archive's, from no +other source. So the test asserts the path instead of the increment, in four steps that are all true today: `apt-get update` verifies the archive's signature; diff --git a/README.rst b/README.rst index 4fe3370..8dfe33f 100644 --- a/README.rst +++ b/README.rst @@ -133,7 +133,11 @@ The appliance ships ``/etc/apt/sources.list.d/keel.sources`` **enabled** for the signed ``trixie`` distribution of ``https://archive.keellinux.org``, with ``Signed-By`` naming ``/usr/share/keyrings/keel-archive-keyring.gpg`` from the ``keel-archive-keyring`` package, and ``/etc/apt/preferences.d/keel`` pins that -origin at 1001. So on a booted appliance:: +origin at 990. Both come from common (``overlays/turnkey.d/keel-apt``, +Keel-Linux/common#30); on a bootstrap from before that, ``conf.d/zzz-keel-archive`` +writes the same two files. 990 makes our build of a package the candidate over +any other archive and never replaces a newer installed version; the 1001 this +recipe used to ship downgraded (tracker#23). So on a booted appliance:: apt-get update # reads our archive and verifies its signature apt-get upgrade # takes newer Debian and newer project packages @@ -149,7 +153,8 @@ The boot test proves that path rather than proving that a newer version happens to exist on the day it runs. It asserts that ``apt-get update`` verifies our archive's signature; that ``apt-cache policy`` shows a project package the image carries with our archive as the source of its candidate at -that pin priority; that ``keel-transition``, which the image does not carry, +that pin priority, or the installed version kept because it is newer than +our archive's, which is what 990 promises; that ``keel-transition``, which the image does not carry, comes down from our archive against the digest the signed index holds; and that a project package the image does carry is downloaded again and put through dpkg. apt refuses an archive it cannot verify before it asks for a diff --git a/changelog b/changelog index d511a65..fa9101d 100644 --- a/changelog +++ b/changelog @@ -1,5 +1,23 @@ turnkey-wordpress-19.0 (5) turnkey; urgency=medium + * The overlay no longer ships /etc/apt/sources.list.d/keel.sources or + /etc/apt/preferences.d/keel (the Keel origin at 1001). At 1001 apt + downgraded every package newer than the archive's (tracker#23; COVERAGE.md + measured keel-archive-keyring 0.1.1 going back to 0.1.0), and both files, + at the paths common uses, would override the source and the 990 pin + Keel-Linux/common#30 ships. conf.d/zzz-keel-archive now verifies + common's source and pin and changes nothing, writes the same two files + (stable enabled, testing disabled, pin 990) on a bootstrap from before + common shipped them, refuses any other pin priority, and no longer + enables every Enabled: no line, which would have turned on the testing + track. The build time archive still wins over TurnKey's 999 pin during + the build: conf.d/main pins it by its Label, l=Keel Linux staging, at + 1001, and conf.d/zz-project-packages removes that pin with the build time + source and fails if any apt file of the image still names that archive. + The boot test expects the archive at 990, reads only the stable stanza + of keel.sources, and accepts an installed project package newer than the + archive's that apt keeps, which is the no downgrade case. + * bin/wordpress.py draws its password boxes on the terminal. firstboot.d/40wordpress reads the script's standard output for APP_PASS= and DB_PASS=, and dialog draws on standard output, so an diff --git a/conf.d/main b/conf.d/main index f55cb11..d2de31a 100755 --- a/conf.d/main +++ b/conf.d/main @@ -166,6 +166,16 @@ apache2ctl configtest # do with it. The conf script runs with stdin closed, dpkg reads end of file at # the prompt and leaves confconsole "install ok unpacked", which fails this # script. Keep the overlay's file without asking. +# +# The build time archive has to win over every other source here: a +# bootstrap from before Keel-Linux/common#30 pins TurnKey's archive at 999. +# It used to win through the overlay's /etc/apt/preferences.d/keel at 1001, +# which then shipped in the image and downgraded every package newer than the +# archive's (tracker#23). This pin names the staging distribution by its +# Label, exists only during the build and goes with the build time source +# (conf.d/zz-project-packages). +printf 'Package: *\nPin: release l=Keel Linux staging\nPin-Priority: 1001\n' \ + > /etc/apt/preferences.d/keel-staging export DEBIAN_FRONTEND=noninteractive apt-get install -y --only-upgrade \ -o Dpkg::Options::=--force-confdef \ diff --git a/conf.d/zz-project-packages b/conf.d/zz-project-packages index 2e65dfd..38b2936 100755 --- a/conf.d/zz-project-packages +++ b/conf.d/zz-project-packages @@ -27,7 +27,8 @@ KEEL_APT_ROOT="${KEEL_APT_ROOT:-/srv/keel-apt}" KEEL_APT_REPO="$KEEL_APT_ROOT/repo" KEEL_STAGING_LIST="${KEEL_STAGING_LIST:-/etc/apt/sources.list.d/keel-staging.list}" KEEL_STAGING_KEYRING="${KEEL_STAGING_KEYRING:-/etc/apt/keyrings/keel-staging-keyring.asc}" -KEEL_SOURCES="${KEEL_SOURCES:-/etc/apt/sources.list.d/keel.sources}" +KEEL_STAGING_PIN="${KEEL_STAGING_PIN:-/etc/apt/preferences.d/keel-staging}" +KEEL_APT_ETC="${KEEL_APT_ETC:-/etc/apt}" KEEL_APT_LISTS="${KEEL_APT_LISTS:-/var/lib/apt/lists}" KEEL_PROJECT_PACKAGES="${KEEL_PROJECT_PACKAGES:-inithooks confconsole keel}" @@ -74,14 +75,19 @@ for package in $KEEL_PROJECT_PACKAGES; do done # the build time package source is not for appliances: remove the source entry, -# the copy of the archive it names and the keyring the build verified that -# archive with, and keep the documented, disabled entry for the future signed -# repository (overlay). The staging key signs whatever the build host produced, -# so an image that kept it would carry trust in a nightly (tracker#7). -# common/removelists-final/turnkey removes the same three paths at the end of -# the build, whatever a recipe does. +# the pin conf.d/main gave it for the build, the copy of the archive it names +# and the keyring the build verified that archive with. The staging key signs +# whatever the build host produced, so an image that kept it would carry trust +# in a nightly (tracker#7). common/removelists-final/turnkey removes the same +# paths at the end of the build, whatever a recipe does. The appliance's own +# Keel source and its 990 pin are common's, or conf.d/zzz-keel-archive's on a +# bootstrap from before common shipped them, so nothing the image keeps may +# still name the build time archive. rm -f "$KEEL_STAGING_LIST" +rm -f "$KEEL_STAGING_PIN" rm -f "$KEEL_STAGING_KEYRING" rm -rf "$KEEL_APT_ROOT" rm -rf "${KEEL_APT_LISTS:?}"/* -grep -q '^Enabled: no' "$KEEL_SOURCES" +apt_files=("$KEEL_APT_ETC/sources.list" "$KEEL_APT_ETC/sources.list.d" "$KEEL_APT_ETC/preferences" "$KEEL_APT_ETC/preferences.d") +leftover=$(grep -rlsE 'trixie-staging|/srv/keel-apt|l=Keel Linux staging' "${apt_files[@]}" || true) +[ -z "$leftover" ] || fatal "these apt files still name the build time archive: $(tr '\n' ' ' <<< "$leftover")" diff --git a/conf.d/zzz-keel-archive b/conf.d/zzz-keel-archive index 24697a4..8de56c5 100755 --- a/conf.d/zzz-keel-archive +++ b/conf.d/zzz-keel-archive @@ -1,13 +1,21 @@ #!/bin/bash -e -# Turn on the project's signed APT archive, which is the appliance's update -# path, and prove that the key it will be verified with is in the image. +# The project's signed APT archive, which is the appliance's update path: +# its source enabled, its pin at 990, and proof that the key it will be +# verified with is in the image. +# +# Keel-Linux/common#30 ships the source and the pin to every image +# (overlays/turnkey.d/keel-apt); this script then checks them and changes +# nothing. On a bootstrap from before that change it writes the same two +# files. The recipe's overlay used to ship them, the pin at 1001, which made +# apt downgrade every package newer than the archive's (tracker#23); at 990 +# the archive's build is still the candidate over any other archive, and a +# newer installed version is never replaced. # # Runs last, after conf.d/zz-project-packages has removed the build time -# file: source and emptied /var/lib/apt/lists. The order matters: with this -# source enabled during the build, apt would resolve the project packages over -# the network instead of from the copy of the archive inside the build tree, -# which is the one thing zz-project-packages exists to prove, and the layer -# would need a name to be reachable to build at all. +# file: source and its pin and emptied /var/lib/apt/lists. The order matters: +# with this source enabled during the build, apt would resolve the project +# packages over the network instead of from the copy of the archive inside +# the build tree, which is the one thing zz-project-packages exists to prove. # # It does not run "apt-get update". A conf script has no guarantee of a # network, an index fetched here would be stale in the image anyway, and @@ -19,9 +27,11 @@ # scratch tree. KEEL_SOURCES="${KEEL_SOURCES:-/etc/apt/sources.list.d/keel.sources}" +KEEL_PREFS="${KEEL_PREFS:-/etc/apt/preferences.d/keel}" KEEL_KEYRING="${KEEL_KEYRING:-/usr/share/keyrings/keel-archive-keyring.gpg}" KEEL_ARCHIVE_URI="${KEEL_ARCHIVE_URI:-https://archive.keellinux.org}" KEEL_ARCHIVE_SUITE="${KEEL_ARCHIVE_SUITE:-trixie}" +KEEL_PIN_PRIORITY=990 KEEL_STAGING_LIST="${KEEL_STAGING_LIST:-/etc/apt/sources.list.d/keel-staging.list}" KEEL_STAGING_KEYRING="${KEEL_STAGING_KEYRING:-/etc/apt/keyrings/keel-staging-keyring.asc}" @@ -30,12 +40,17 @@ fatal() { exit 1 } +# the stanza of the stable track: the first that names KEEL_ARCHIVE_SUITE, +# or the first of the file when none does, so a wrong suite is reported +stable_stanza() { + awk -v suite="$KEEL_ARCHIVE_SUITE" 'BEGIN { RS = "" } { if (first == "") first = $0 } $0 ~ "(^|\n)Suites:[ \t]*" suite "[ \t]*(\n|$)" { print; found = 1; exit } END { if (!found) print first }' "$KEEL_SOURCES" +} + +# field NAME: the value of a deb822 field of the stable stanza field() { - # field NAME FILE: the value of a deb822 field, first occurrence - awk -v name="$1:" '$1 == name { $1 = ""; sub(/^ /, ""); print; exit }' "$2" + awk -v name="$1:" '$1 == name { $1 = ""; sub(/^ /, ""); print; exit }' <<< "$stanza" } -[ -f "$KEEL_SOURCES" ] || fatal "$KEEL_SOURCES is not in the image" [ -s "$KEEL_KEYRING" ] || fatal "$KEEL_KEYRING is missing or empty: the plan asks for keel-archive-keyring" # The build time source must be gone before this one is enabled, and so must @@ -47,30 +62,57 @@ field() { [ ! -e "$KEEL_STAGING_KEYRING" ] \ || fatal "$KEEL_STAGING_KEYRING is still in the image: the staging key must not reach an appliance" +# A keyring with no key in it verifies nothing and apt says so only at run +# time, so the key is counted here. gpg is in the image (core), and reading a +# keyring needs no network and no agent. +keys=$(gpg --show-keys --with-colons "$KEEL_KEYRING" 2>/dev/null | grep -c '^pub:' || true) +[ "$keys" -ge 1 ] || fatal "$KEEL_KEYRING carries no public key" + +# Written only when common did not ship them: the same text as common's. +if [ ! -e "$KEEL_SOURCES" ]; then + mkdir -p "$(dirname "$KEEL_SOURCES")" + printf '%s\n' \ + "# Keel Linux packages, from the Keel repository (handbook decision 0039)." \ + "# trixie is the stable track; trixie-testing is off unless the operator" \ + "# turns it on. Written by keel-wordpress's conf.d/zzz-keel-archive where" \ + "# common did not ship it (Keel-Linux/common#30)." \ + "Types: deb" "URIs: $KEEL_ARCHIVE_URI" "Suites: $KEEL_ARCHIVE_SUITE" \ + "Components: main" "Enabled: yes" "Signed-By: $KEEL_KEYRING" "" \ + "Types: deb" "URIs: $KEEL_ARCHIVE_URI" "Suites: $KEEL_ARCHIVE_SUITE-testing" \ + "Components: main" "Enabled: no" "Signed-By: $KEEL_KEYRING" > "$KEEL_SOURCES" +fi +if [ ! -e "$KEEL_PREFS" ]; then + mkdir -p "$(dirname "$KEEL_PREFS")" + printf '%s\n' \ + "# Keel Linux: prefer the project's packages over Debian's, never by" \ + "# going backwards (tracker#23)." \ + "Package: *" "Pin: release o=Keel Linux" "Pin-Priority: $KEEL_PIN_PRIORITY" > "$KEEL_PREFS" +fi + # Never the unsigned staging distribution: it exists only inside a build. -suite=$(field Suites "$KEEL_SOURCES") +stanza=$(stable_stanza) +suite=$(field Suites) [ "$suite" = "$KEEL_ARCHIVE_SUITE" ] \ || fatal "$KEEL_SOURCES names the suite '$suite', not '$KEEL_ARCHIVE_SUITE'" case "$suite" in *staging*) fatal "$KEEL_SOURCES names an unsigned staging distribution" ;; esac -uri=$(field URIs "$KEEL_SOURCES") +uri=$(field URIs) [ "$uri" = "$KEEL_ARCHIVE_URI" ] \ || fatal "$KEEL_SOURCES names '$uri', not '$KEEL_ARCHIVE_URI'" -signed_by=$(field Signed-By "$KEEL_SOURCES") +signed_by=$(field Signed-By) [ "$signed_by" = "$KEEL_KEYRING" ] \ || fatal "$KEEL_SOURCES is signed by '$signed_by', not '$KEEL_KEYRING'" -# A keyring with no key in it verifies nothing and apt says so only at run -# time, so the key is counted here. gpg is in the image (core), and reading a -# keyring needs no network and no agent. -keys=$(gpg --show-keys --with-colons "$KEEL_KEYRING" 2>/dev/null | grep -c '^pub:' || true) -[ "$keys" -ge 1 ] || fatal "$KEEL_KEYRING carries no public key" +[ "$(field Enabled)" != no ] || fatal "$KEEL_SOURCES: $KEEL_ARCHIVE_SUITE is not enabled" -sed -i 's/^Enabled: no$/Enabled: yes/' "$KEEL_SOURCES" -grep -qx 'Enabled: yes' "$KEEL_SOURCES" \ - || fatal "$KEEL_SOURCES is still not enabled" +# every priority the pin file gives, which must be the one 990 +priorities=$(awk '$1 == "Pin-Priority:" { print $2 }' "$KEEL_PREFS" | sort -u) +[ "$priorities" = "$KEEL_PIN_PRIORITY" ] \ + || fatal "$KEEL_PREFS pins at '$(tr '\n' ' ' <<< "$priorities")', not $KEEL_PIN_PRIORITY (tracker#23)" +grep -qx 'Pin: release o=Keel Linux' "$KEEL_PREFS" \ + || fatal "$KEEL_PREFS does not pin o=Keel Linux at $KEEL_PIN_PRIORITY" -echo "[zzz-keel-archive] $uri $suite enabled, verified with $KEEL_KEYRING ($keys key)" +echo "[zzz-keel-archive] $uri $suite enabled, verified with $KEEL_KEYRING ($keys key), pinned at $KEEL_PIN_PRIORITY" diff --git a/overlay/etc/apt/preferences.d/keel b/overlay/etc/apt/preferences.d/keel deleted file mode 100644 index dcfdba2..0000000 --- a/overlay/etc/apt/preferences.d/keel +++ /dev/null @@ -1,5 +0,0 @@ -# Keel Linux: prefer the project's packages over every other archive. -# Generated by bin/pin-file of the apt tooling; the same file keel-transition installs. -Package: * -Pin: release o=Keel Linux -Pin-Priority: 1001 diff --git a/overlay/etc/apt/sources.list.d/keel.sources b/overlay/etc/apt/sources.list.d/keel.sources deleted file mode 100644 index 0d8acfd..0000000 --- a/overlay/etc/apt/sources.list.d/keel.sources +++ /dev/null @@ -1,24 +0,0 @@ -# Keel Linux package repository (brief section 5.4). -# -# This appliance is the first one that ships this source enabled: it carries -# keel-archive-keyring, so it holds the key the archive is signed with and -# apt can verify what it downloads. The distribution named here is the signed -# one at the root of archive.keellinux.org, never trixie-staging, which is -# unsigned by design and exists only inside a build. -# -# The file is shipped disabled and enabled by conf.d/zzz-keel-archive, the -# last conf script of the build, after the build time file: source has been -# removed. A build that had this source enabled would resolve the project -# packages over the network instead of from the archive copy inside the build -# tree, which is the thing conf.d/zz-project-packages exists to prove, and -# would make the layer depend on a name being reachable at build time. -# -# What it updates and what it does not is in README.rst: the system packages -# and the project's own packages come from here; WordPress core does not, -# because nobody packages it. -Types: deb -URIs: https://archive.keellinux.org -Suites: trixie -Components: main -Enabled: no -Signed-By: /usr/share/keyrings/keel-archive-keyring.gpg diff --git a/tests/apt-files.bats b/tests/apt-files.bats new file mode 100644 index 0000000..12dc91c --- /dev/null +++ b/tests/apt-files.bats @@ -0,0 +1,46 @@ +#!/usr/bin/env bats +# The apt files this recipe leaves in the image (Keel-Linux/common#30, +# tracker#23). Common ships the appliance's Keel source and its pin at 990 +# (overlays/turnkey.d/keel-apt); a recipe overlay at the same paths would win +# over them, so this recipe ships neither. The build time archive still has +# to win over TurnKey's 999 pin while the recipe upgrades the project +# packages, so conf.d/main pins it by its Label for the build only, and +# conf.d/zz-project-packages removes that pin with the build time source +# (tests/project-packages.bats). +# +# conf.d/main runs inside a chroot during the build; what it leaves is proved +# on the booted machine by the boot test. These check the recipe itself. + +bats_require_minimum_version 1.5.0 + +setup() { + REPO="$(cd "$BATS_TEST_DIRNAME/.." && pwd)" + MAIN="$REPO/conf.d/main" +} + +# line LITERAL: the line number of the first line of conf.d/main equal to it +line() { + grep -nxF -- "$1" "$MAIN" | head -n 1 | cut -d: -f1 +} + +@test "the overlay ships no Keel source and no Keel pin" { + [ ! -e "$REPO/overlay/etc/apt/sources.list.d/keel.sources" ] + [ ! -e "$REPO/overlay/etc/apt/preferences.d/keel" ] + run ! grep -rlsE 'Pin-Priority: *1001' "$REPO/overlay" +} + +@test "the build time pin names the staging Label, and is written before the upgrade" { + local pin upgrade + pin="$(line "printf 'Package: *\nPin: release l=Keel Linux staging\nPin-Priority: 1001\n' \\")" + # the backslash is the script's line continuation, matched literally + # shellcheck disable=SC1003 + upgrade="$(line 'apt-get install -y --only-upgrade \')" + [ -n "$pin" ] + [ -n "$upgrade" ] + [ "$pin" -lt "$upgrade" ] + grep -qxF ' > /etc/apt/preferences.d/keel-staging' "$MAIN" +} + +@test "conf.d/main parses" { + bash -n "$MAIN" +} diff --git a/tests/boot-test.bats b/tests/boot-test.bats index 3ac33e4..0c5b1ad 100644 --- a/tests/boot-test.bats +++ b/tests/boot-test.bats @@ -493,6 +493,15 @@ EOF [[ "$output" == *"is enabled and verified with"* ]] } +@test "sources_verdict passes on common's file, the testing track beside it disabled" { + _sources + printf '\nTypes: deb\nURIs: https://archive.keellinux.org\nSuites: trixie-testing\nComponents: main\nEnabled: no\nSigned-By: /usr/share/keyrings/keel-archive-keyring.gpg\n' \ + >> "$S/keel.sources" + run bt_sources_verdict "$S/keel.sources" + [ "$status" -eq 0 ] + [[ "$output" == *"https://archive.keellinux.org trixie is enabled"* ]] +} + @test "build_leftovers_verdict passes on an image that kept none of them" { run bt_build_leftovers_verdict "$S/rootfs" [ "$status" -eq 0 ] @@ -584,8 +593,8 @@ inithooks: Installed: ${1:-2.3.6+keel4} Candidate: ${2:-2.3.6+keel5} Version table: - ${2:-2.3.6+keel5} ${3:-1001} - ${3:-1001} ${4:-https://archive.keellinux.org} trixie/main amd64 Packages + ${2:-2.3.6+keel5} ${3:-990} + ${3:-990} ${4:-https://archive.keellinux.org} trixie/main amd64 Packages *** ${1:-2.3.6+keel4} 100 100 /var/lib/dpkg/status EOF @@ -595,7 +604,7 @@ EOF _policy run bt_policy_verdict inithooks "$S/policy" [ "$status" -eq 0 ] - [[ "$output" == *"takes inithooks from https://archive.keellinux.org at priority 1001"* ]] + [[ "$output" == *"takes inithooks from https://archive.keellinux.org at priority 990"* ]] [[ "$output" == *"candidate 2.3.6+keel5"* ]] } @@ -615,8 +624,8 @@ inithooks: Installed: 2.3.6+keel5 Candidate: 2.3.6+keel5 Version table: - *** 2.3.6+keel5 1001 - 1001 https://archive.keellinux.org trixie/main amd64 Packages + *** 2.3.6+keel5 990 + 990 https://archive.keellinux.org trixie/main amd64 Packages 100 /var/lib/dpkg/status EOF run bt_policy_verdict inithooks "$S/policy" @@ -632,11 +641,62 @@ inithooks: Installed: 2.3.6+evil Candidate: 2.3.6+evil Version table: - *** 2.3.6+evil 1001 - 1001 http://mirror.example.org trixie/main amd64 Packages + *** 2.3.6+evil 990 + 990 http://mirror.example.org trixie/main amd64 Packages + 100 /var/lib/dpkg/status + 2.3.6+keel5 990 + 990 https://archive.keellinux.org trixie/main amd64 Packages +EOF + run bt_policy_verdict inithooks "$S/policy" + [ "$status" -eq 1 ] + [[ "$output" == *"does not come from"* ]] +} + +@test "policy_verdict passes when apt keeps an installed version newer than the archive's" { + # at 990 apt never goes backwards (tracker#23): an image built from a + # newer project package than the archive publishes keeps it + cat > "$S/policy" < "$S/policy" < "$S/policy" < "$KEEL_SOURCES" < "$KEEL_PREFS" +} + +# _sources [KEY=VALUE ...]: a one stanza source with any field a test changes _sources() { - local enabled=no uri=https://archive.keellinux.org suite=trixie + local enabled=yes uri=https://archive.keellinux.org suite=trixie local signed=$KEEL_KEYRING pair for pair in "$@"; do case "$pair" in @@ -44,7 +70,6 @@ _sources() { esac done cat > "$KEEL_SOURCES" < "$KEEL_KEYRING" } -@test "the shipped source is enabled and the script says what it enabled" { +# _without_common: a tree from a bootstrap before common shipped the files +_without_common() { + rm -f "$KEEL_SOURCES" "$KEEL_PREFS" +} + +@test "common's source and pin are verified and left byte for byte" { + cp "$KEEL_SOURCES" "$S/sources.before" + cp "$KEEL_PREFS" "$S/prefs.before" run bash "$SCRIPT" [ "$status" -eq 0 ] [[ "$output" == *"https://archive.keellinux.org trixie enabled"* ]] - grep -qx "Enabled: yes" "$KEEL_SOURCES" + [[ "$output" == *"pinned at 990"* ]] + cmp "$S/sources.before" "$KEEL_SOURCES" + cmp "$S/prefs.before" "$KEEL_PREFS" } -@test "running it twice leaves the source enabled" { +@test "without common's files it writes the source, enabled, and the pin at 990" { + _without_common run bash "$SCRIPT" [ "$status" -eq 0 ] + [[ "$output" == *"https://archive.keellinux.org trixie enabled"* ]] + awk 'BEGIN { RS = "" } /Suites: trixie\n/ && /Enabled: yes/ { f = 1 } END { exit !f }' "$KEEL_SOURCES" + awk 'BEGIN { RS = "" } /Suites: trixie-testing/ && /Enabled: no/ { f = 1 } END { exit !f }' "$KEEL_SOURCES" + grep -qx "Signed-By: $KEEL_KEYRING" "$KEEL_SOURCES" + grep -qx 'Pin: release o=Keel Linux' "$KEEL_PREFS" + grep -qx 'Pin-Priority: 990' "$KEEL_PREFS" +} + +@test "running it twice leaves the same bytes" { + _without_common + run bash "$SCRIPT" + [ "$status" -eq 0 ] + cp "$KEEL_SOURCES" "$S/sources.first" + cp "$KEEL_PREFS" "$S/prefs.first" + run bash "$SCRIPT" + [ "$status" -eq 0 ] + cmp "$S/sources.first" "$KEEL_SOURCES" + cmp "$S/prefs.first" "$KEEL_PREFS" +} + +@test "it never turns on the testing track" { run bash "$SCRIPT" [ "$status" -eq 0 ] - [ "$(grep -c '^Enabled: yes$' "$KEEL_SOURCES")" -eq 1 ] + awk 'BEGIN { RS = "" } /Suites: trixie-testing/ && /Enabled: no/ { f = 1 } END { exit !f }' "$KEEL_SOURCES" } -@test "a missing source file is fatal" { - rm -f "$KEEL_SOURCES" +@test "a Keel pin at 1001 is refused (tracker#23)" { + printf 'Package: *\nPin: release o=Keel Linux\nPin-Priority: 1001\n' > "$KEEL_PREFS" run bash "$SCRIPT" [ "$status" -ne 0 ] - [[ "$output" == *"is not in the image"* ]] + [[ "$output" == *"1001"* ]] + [[ "$output" == *"990"* ]] } -@test "a missing keyring is fatal and names the package that carries it" { +@test "a pin file with no Keel pin at 990 in it is refused" { + printf 'Package: *\nPin: release o=Debian\nPin-Priority: 500\n' > "$KEEL_PREFS" + run bash "$SCRIPT" + [ "$status" -ne 0 ] + [[ "$output" == *"990"* ]] +} + +@test "a disabled stable source is refused" { + _sources Enabled=no + run bash "$SCRIPT" + [ "$status" -ne 0 ] + [[ "$output" == *"is not enabled"* ]] +} + +@test "a missing keyring is fatal, names the package that carries it, and writes nothing" { + _without_common rm -f "$KEEL_KEYRING" run bash "$SCRIPT" [ "$status" -ne 0 ] [[ "$output" == *"keel-archive-keyring"* ]] + [ ! -e "$KEEL_SOURCES" ] + [ ! -e "$KEEL_PREFS" ] } @test "an empty keyring is fatal" { @@ -94,21 +168,23 @@ _keyring() { [[ "$output" == *"missing or empty"* ]] } -@test "a keyring with no public key in it is fatal" { +@test "a keyring with no public key in it is fatal, and writes nothing" { + _without_common export GPG_TEST_KEYS=0 run bash "$SCRIPT" [ "$status" -ne 0 ] [[ "$output" == *"carries no public key"* ]] - grep -qx "Enabled: no" "$KEEL_SOURCES" + [ ! -e "$KEEL_SOURCES" ] } -@test "a build time source still in the image is fatal" { +@test "a build time source still in the image is fatal, and writes nothing" { + _without_common printf 'deb [signed-by=/etc/apt/keyrings/keel-staging-keyring.asc] file:///srv/keel-apt/repo trixie-staging main\n' \ > "$KEEL_STAGING_LIST" run bash "$SCRIPT" [ "$status" -ne 0 ] [[ "$output" == *"zz-project-packages did not run"* ]] - grep -qx "Enabled: no" "$KEEL_SOURCES" + [ ! -e "$KEEL_SOURCES" ] } @test "the build time staging keyring still in the image is fatal" { @@ -118,7 +194,6 @@ _keyring() { run bash "$SCRIPT" [ "$status" -ne 0 ] [[ "$output" == *"must not reach an appliance"* ]] - grep -qx "Enabled: no" "$KEEL_SOURCES" } @test "a staging suite is refused by name" { @@ -156,7 +231,7 @@ Types: deb URIs: https://archive.keellinux.org Suites: trixie Components: main -Enabled: no +Enabled: yes EOF run bash "$SCRIPT" [ "$status" -ne 0 ] diff --git a/tests/lib/boot-test-lib.sh b/tests/lib/boot-test-lib.sh index cdd60f1..ce7d4c1 100644 --- a/tests/lib/boot-test-lib.sh +++ b/tests/lib/boot-test-lib.sh @@ -59,9 +59,10 @@ etc/apt/keyrings/keel-staging-keyring.asc" # A project package the image already carries, to show that apt would take its # next version from our archive rather than from anywhere else: the candidate # has to come from our archive, at the priority the appliance's own pin file -# sets. +# sets, unless the image carries a newer one, which 990 never replaces +# (tracker#23). BT_POLICY_PACKAGE="inithooks" -BT_ARCHIVE_PIN=1001 +BT_ARCHIVE_PIN=990 # Where this library and the awk programs beside it live. BT_LIB_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" # And the package path itself, in two steps, neither of which needs a version @@ -550,7 +551,11 @@ bt_build_leftovers_verdict() { bt_sources_verdict() { # bt_sources_verdict FILE: the appliance's own APT source, as it ships: # enabled, the signed distribution, our keyring, and never a staging one. - local sources=$1 field value + # Only the stanza of the stable track is read: common's file carries the + # testing track beside it, disabled. With no such stanza the first one is + # read, so the suite it names is what the refusal reports. + local sources=$1 field value stanza + stanza=$(awk -v suite="$BT_ARCHIVE_SUITE" 'BEGIN { RS = "" } { if (first == "") first = $0 } $0 ~ "(^|\n)Suites:[ \t]*" suite "[ \t]*(\n|$)" { print; found = 1; exit } END { if (!found) print first }' "$sources") while read -r field value; do case "$field" in Enabled:) [ "$value" = yes ] || { @@ -562,7 +567,7 @@ bt_sources_verdict() { Signed-By:) [ "$value" = "$BT_ARCHIVE_KEYRING" ] || { echo "boot-test: $sources is signed by '$value', not $BT_ARCHIVE_KEYRING" >&2; return 1; } ;; esac - done < "$sources" + done <<< "$stanza" echo "boot-test: $BT_ARCHIVE_URI $BT_ARCHIVE_SUITE is enabled and verified with $BT_ARCHIVE_KEYRING" } @@ -619,6 +624,15 @@ bt_apt_update_verdict() { echo "boot-test: apt-get update read $BT_ARCHIVE_URI $BT_ARCHIVE_SUITE and verified its signature" } +bt_policy_block() { + # bt_policy_block FILE: the version table of "apt-cache policy" in FILE as + # "VERSION PRIORITY SOURCE", one line per source of each version + # the $ are awk's, not the shell's + # shellcheck disable=SC2016 + local prog='{ if ($1 == "***") { $1 = ""; $0 = $0 } } $1 == "Version" && $2 == "table:" { table = 1; next } !table { next } NF == 2 && $2 ~ /^-?[0-9]+$/ && $1 !~ /^-?[0-9]+$/ { cur = $1; next } $1 ~ /^-?[0-9]+$/ { print cur, $1, $2 }' + awk "$prog" "$1" +} + bt_policy_verdict() { # bt_policy_verdict PACKAGE FILE: FILE is "apt-cache policy PACKAGE" from # inside the appliance. Our archive has to be a source apt knows, at the @@ -638,10 +652,19 @@ bt_policy_verdict() { return 1 fi # the candidate's own block of the version table has to list our archive - # at our pin (candidate-source.awk says why) + # at our pin (candidate-source.awk says why), or be the installed version + # alone, newer than what our archive offers: below 1000 apt never goes + # backwards, which is the point of 990 (tracker#23) if ! awk -v version="$candidate" -v pin="$BT_ARCHIVE_PIN" \ -v uri="$BT_ARCHIVE_URI" -f "$BT_LIB_DIR/candidate-source.awk" \ "$file"; then + local offered sources + offered=$(bt_policy_block "$file" | awk -v pin="$BT_ARCHIVE_PIN" -v uri="$BT_ARCHIVE_URI" '$2 == pin && $3 == uri { print $1; exit }') + sources=$(bt_policy_block "$file" | awk -v version="$candidate" '$1 == version { print $3 }') + if [ "$sources" = /var/lib/dpkg/status ] && dpkg --compare-versions "$candidate" gt "$offered"; then + echo "boot-test: apt keeps the installed $package $candidate, newer than $offered from $BT_ARCHIVE_URI at priority $BT_ARCHIVE_PIN: never downgraded" + return 0 + fi echo "boot-test: the candidate $package $candidate does not come from" \ "$BT_ARCHIVE_URI at priority $BT_ARCHIVE_PIN" >&2 return 1 diff --git a/tests/lib/candidate-source.awk b/tests/lib/candidate-source.awk index a86723e..3bc8839 100644 --- a/tests/lib/candidate-source.awk +++ b/tests/lib/candidate-source.awk @@ -4,7 +4,7 @@ # The number on a version line is only the highest priority of that version's # sources, so it does not say which source the version comes from; the source # lines under the version do. apt marks the installed version with *** in front -# of it (a current appliance: " *** 2.3.6+keel5 1001"); the marker is dropped. +# of it (a current appliance: " *** 2.3.6+keel5 990"); the marker is dropped. # # awk -v version=V -v pin=P -v uri=U -f candidate-source.awk POLICY_FILE { if ($1 == "***") { $1 = ""; $0 = $0 } } diff --git a/tests/project-packages.bats b/tests/project-packages.bats index b09ca7c..a0e69f4 100644 --- a/tests/project-packages.bats +++ b/tests/project-packages.bats @@ -14,7 +14,8 @@ setup() { export KEEL_APT_ROOT="$scratch/srv/keel-apt" export KEEL_STAGING_LIST="$scratch/apt/sources.list.d/keel-staging.list" export KEEL_STAGING_KEYRING="$scratch/apt/keyrings/keel-staging-keyring.asc" - export KEEL_SOURCES="$scratch/apt/sources.list.d/keel.sources" + export KEEL_APT_ETC="$scratch/apt" + export KEEL_STAGING_PIN="$scratch/apt/preferences.d/keel-staging" export KEEL_APT_LISTS="$scratch/apt/lists" export FIXTURES="$scratch/fixtures" @@ -27,7 +28,10 @@ setup() { echo "deb [signed-by=$KEEL_STAGING_KEYRING] file://$KEEL_APT_ROOT/repo $DIST main" \ > "$KEEL_STAGING_LIST" printf 'not a key, and this script never reads one\n' > "$KEEL_STAGING_KEYRING" - printf 'Types: deb\nURIs: https://apt.keellinux.org\nEnabled: no\n' > "$KEEL_SOURCES" + # the build time pin conf.d/main writes before its upgrade + mkdir -p "$(dirname "$KEEL_STAGING_PIN")" + printf 'Package: *\nPin: release l=Keel Linux staging\nPin-Priority: 1001\n' \ + > "$KEEL_STAGING_PIN" offer inithooks 2.3.6+keel4 offer confconsole 2.2.3+keel2 @@ -93,7 +97,7 @@ installed() { printf 'version=%s\nstatus=%s\n' "$2" "$3" > "$FIXTURES/installed. [ ! -e "$KEEL_APT_ROOT" ] [ ! -e "$KEEL_STAGING_LIST" ] [ -z "$(ls -A "$KEEL_APT_LISTS")" ] - [ -f "$KEEL_SOURCES" ] + [ ! -e "$KEEL_STAGING_PIN" ] } @test "the keyring that verified the staging archive is gone too" { @@ -190,8 +194,30 @@ installed() { printf 'version=%s\nstatus=%s\n' "$2" "$3" > "$FIXTURES/installed. [[ "$output" == *"trixie-nowhere"* ]] } -@test "the future signed repository has to stay in place, disabled" { - printf 'Types: deb\nURIs: https://apt.keellinux.org\nEnabled: yes\n' > "$KEEL_SOURCES" +@test "an apt source that still names the build time archive fails the build" { + printf 'Types: deb\nURIs: file:///srv/keel-apt/repo\nSuites: trixie-staging\n' \ + > "$KEEL_APT_ETC/sources.list.d/leftover.sources" run "$SCRIPT" [ "$status" -ne 0 ] + [[ "$output" == *leftover.sources* ]] +} + +@test "a pin that still names the staging Label fails the build" { + printf 'Package: *\nPin: release l=Keel Linux staging\nPin-Priority: 1001\n' \ + > "$KEEL_APT_ETC/preferences.d/other" + run "$SCRIPT" + [ "$status" -ne 0 ] + [[ "$output" == *preferences.d/other* ]] +} + +@test "common's Keel source and its 990 pin are left in place" { + # what Keel-Linux/common#30 ships (overlays/turnkey.d/keel-apt) + printf 'Types: deb\nURIs: https://archive.keellinux.org\nSuites: trixie\nComponents: main\nEnabled: yes\n' \ + > "$KEEL_APT_ETC/sources.list.d/keel.sources" + printf 'Package: *\nPin: release o=Keel Linux\nPin-Priority: 990\n' \ + > "$KEEL_APT_ETC/preferences.d/keel" + run "$SCRIPT" + [ "$status" -eq 0 ] + [ -f "$KEEL_APT_ETC/sources.list.d/keel.sources" ] + [ -f "$KEEL_APT_ETC/preferences.d/keel" ] } From b250a5369b9ae6438cd5ff679776cae725015ce6 Mon Sep 17 00:00:00 2001 From: navigator Date: Fri, 2 Oct 2026 15:03:41 +0000 Subject: [PATCH 2/3] docs: COVERAGE.md says the 1001 pin in the past tense --- COVERAGE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/COVERAGE.md b/COVERAGE.md index 43316bc..304befc 100644 --- a/COVERAGE.md +++ b/COVERAGE.md @@ -168,7 +168,7 @@ the image deliberately stale so the archive is always ahead. Both would be lies told to make a test pass. The second is also dangerous here, and measuring it is what settled the -argument. `/etc/apt/preferences.d/keel` pins our origin at **1001**, the +argument. `/etc/apt/preferences.d/keel` pinned our origin at **1001**, the priority that downgrades as well as upgrades. With the image one release ahead of the archive: From 98e210dc7249d421051d2c1a9b6a1d1e8e6ab70f Mon Sep 17 00:00:00 2001 From: navigator Date: Fri, 2 Oct 2026 15:50:41 +0000 Subject: [PATCH 3/3] fix: the overlay change gets its own changelog entry require-changelog wants a new top entry, not a bullet in the base branch's. --- changelog | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/changelog b/changelog index fa9101d..0578af4 100644 --- a/changelog +++ b/changelog @@ -1,4 +1,4 @@ -turnkey-wordpress-19.0 (5) turnkey; urgency=medium +turnkey-wordpress-19.0 (6) turnkey; urgency=medium * The overlay no longer ships /etc/apt/sources.list.d/keel.sources or /etc/apt/preferences.d/keel (the Keel origin at 1001). At 1001 apt @@ -18,6 +18,10 @@ turnkey-wordpress-19.0 (5) turnkey; urgency=medium of keel.sources, and accepts an installed project package newer than the archive's that apt keeps, which is the no downgrade case. + -- Marcos Mendez Fri, 02 Oct 2026 17:00:00 +0000 + +turnkey-wordpress-19.0 (5) turnkey; urgency=medium + * bin/wordpress.py draws its password boxes on the terminal. firstboot.d/40wordpress reads the script's standard output for APP_PASS= and DB_PASS=, and dialog draws on standard output, so an