From 5cf506e41a0588710ab30fcbb3510d55109c9ae1 Mon Sep 17 00:00:00 2001 From: navigator Date: Fri, 2 Oct 2026 14:54:27 +0000 Subject: [PATCH 1/2] fix: drop the overlay's Keel source and 1001 pin; pin staging for the build only The overlay shipped /etc/apt/sources.list.d/keel.sources (apt.keellinux.org, disabled) and /etc/apt/preferences.d/keel at 1001. At 1001 apt downgrades every package newer than the archive's (tracker#23), and at those paths both files override the source and the 990 pin Keel-Linux/common#30 ships. The 1001 pin was also what let the build time archive beat TurnKey's 999 pin during the upgrade in conf.d/main. That is now a build-only pin on the staging Label; conf.d/zz-project-packages removes it with the build time source and fails if any apt file of the image still names that archive. --- COVERAGE.md | 8 ++-- Makefile | 6 +-- README.rst | 9 ++-- changelog | 13 ++++++ conf.d/main | 10 +++++ conf.d/zz-project-packages | 21 ++++++---- overlay/etc/apt/preferences.d/keel | 5 --- overlay/etc/apt/sources.list.d/keel.sources | 17 -------- tests/apt-files.bats | 46 +++++++++++++++++++++ tests/project-packages.bats | 36 +++++++++++++--- 10 files changed, 126 insertions(+), 45 deletions(-) delete mode 100644 overlay/etc/apt/preferences.d/keel delete mode 100644 overlay/etc/apt/sources.list.d/keel.sources create mode 100644 tests/apt-files.bats diff --git a/COVERAGE.md b/COVERAGE.md index 7c8bf2f..be5efd0 100644 --- a/COVERAGE.md +++ b/COVERAGE.md @@ -13,7 +13,8 @@ acceptance test of an appliance recipe, docs/org-plan.md section 1). | overlay/etc/nginx/* | tests/nginx.bats (12 tests) | not executable | asserted as content: the geo variable, the map, the listeners, the proxy headers | | tests/lib/boot-test-lib.sh | tests/boot-test.bats (41 tests) | 100 percent (137/137) under kcov | the logic of the boot test: argument parsing, address discovery, deadlines, the HTTP and diff verdicts | | bin/keel-archive-check | tests/archive-check.bats (27 tests) | 100 percent (54/54) under kcov | the build time check: the archive copy in the build tree is the live archive, the source entry names the keyring through signed-by, nothing says trusted=yes, and the signature on the copied InRelease verifies against the staging key (tracker#7) | -| conf.d/zz-project-packages | tests/project-packages.bats (14 tests) | 100 percent (31/31) under kcov | the build time check that each project package is the candidate of the archive and a project build, and that the archive copy, its source entry and the staging keyring leave the image | +| conf.d/zz-project-packages | tests/project-packages.bats (16 tests) | 100 percent (35/35) under kcov | the build time check that each project package is the candidate of the archive and a project build, and that the archive copy, its source entry, its build-only pin and the staging keyring leave the image, with no apt file still naming that archive | +| overlay, conf.d/main | tests/apt-files.bats (3 tests) | static | no Keel source or 1001 pin in the overlay; the build time pin on the staging Label is written before the upgrade | | overlay/usr/lib/inithooks/bin/nodebb.py | none | 0 | dialog wrapper, only reached with a terminal attached | | conf.d/main | tests/boot-test.sh (build step) | integration only | build time script, 0004 pragmatic limits | | tests/boot-test.sh | itself | integration only | the thin main of the acceptance test: keel and LXC as root | @@ -116,8 +117,9 @@ archive, a candidate that is not what the archive offers, an upstream build of the same version, a package the archive does not offer, an archive that offers two of them, a package that is not installed, a half configured one, a build with no project archive in its source list, a distribution the archive has not -got, and the removal of the build time source with the disabled -`apt.keellinux.org` entry left in place. +got, the removal of the build time source and its build-only pin, a source +or a pin that still names the build time archive, and common's Keel source +and 990 pin left in place. ## Plan diff --git a/Makefile b/Makefile index 35a346d..641b47b 100644 --- a/Makefile +++ b/Makefile @@ -16,9 +16,9 @@ include $(FAB_PATH)/common/mk/turnkey.mk # nothing in the tree says trusted=yes, and apt runs with --error-on=any, so a # signature that cannot be checked fails the build instead of warning about it # and carrying on. conf.d/zz-project-packages checks what was installed -# against that copy, then removes the copy, the source entry and the keyring -# from the image and leaves the future apt.keellinux.org entry in place, -# disabled. +# against that copy, then removes the copy, the source entry, the build-only +# pin conf.d/main gave it and the keyring from the image. The appliance's own +# Keel source and pin are common's (overlays/turnkey.d/keel-apt). # # None of the three build time files is for an installed appliance, because # the staging key signs whatever the build host produced. The removelist at diff --git a/README.rst b/README.rst index 3784f6b..9138563 100644 --- a/README.rst +++ b/README.rst @@ -83,10 +83,11 @@ version, that the version comes from the project archive rather than an upstream source, and that the installed package is that candidate and configured. No version is written down anywhere, so a rebuild made after a publication either carries the new versions or fails. It then removes the -build time source from the image and leaves -``/etc/apt/sources.list.d/keel.sources`` pointing at the future -``apt.keellinux.org``, disabled, with the origin pin in -``/etc/apt/preferences.d/keel``. +build time source and the build-only pin ``conf.d/main`` gave it (the +staging Label at 1001, so it wins over TurnKey's 999 pin during the +upgrade), and fails if any apt file of the image still names that archive. +The appliance's own Keel source and its pin at 990 come from common +(``overlays/turnkey.d/keel-apt``); this recipe ships neither. First boot ---------- diff --git a/changelog b/changelog index cf5ebd4..4cf4b8f 100644 --- a/changelog +++ b/changelog @@ -1,5 +1,18 @@ turnkey-nodebb-19.0 (4) turnkey; urgency=low + * The overlay no longer ships /etc/apt/sources.list.d/keel.sources + (apt.keellinux.org, disabled) or /etc/apt/preferences.d/keel (the Keel + origin at 1001). At 1001 apt downgraded every package newer than the + archive's (tracker#23), and both files, at the paths common uses, would + override the source and the 990 pin Keel-Linux/common#30 ships. The + build time archive still wins over TurnKey's 999 pin during the build: + conf.d/main pins it by its Label, l=Keel Linux staging, at 1001 before + the upgrade, and conf.d/zz-project-packages removes that pin with the + build time source, then fails if any apt source or pin of the image + still names the build time archive, in place of the check that + keel.sources was disabled. tests/project-packages.bats (16 tests, 100 + percent) and tests/apt-files.bats check it. + * bin/keel-archive-check refuses a trusted=yes on the project archive rather than on every apt source in the build tree. The captured pool of decision 0012 sets Trusted: yes on purpose, for a file: index generated on this diff --git a/conf.d/main b/conf.d/main index 74b555b..b30474e 100755 --- a/conf.d/main +++ b/conf.d/main @@ -95,6 +95,16 @@ systemctl enable nodebb # what to do with it. The conf script runs with stdin closed, dpkg reads end # of file at the prompt and leaves confconsole "install ok unpacked", which # fails this script. Keep the overlay's file without asking. +# +# The build time archive has to win over every other source here: a +# bootstrap from before Keel-Linux/common#30 pins TurnKey's archive at 999. +# It used to win through the overlay's /etc/apt/preferences.d/keel at 1001, +# which then shipped in the image and downgraded every package newer than the +# archive's (tracker#23). This pin names the staging distribution by its +# Label, exists only during the build and goes with the build time source +# (conf.d/zz-project-packages). +printf 'Package: *\nPin: release l=Keel Linux staging\nPin-Priority: 1001\n' \ + > /etc/apt/preferences.d/keel-staging export DEBIAN_FRONTEND=noninteractive apt-get install -y --only-upgrade \ -o Dpkg::Options::=--force-confdef \ diff --git a/conf.d/zz-project-packages b/conf.d/zz-project-packages index 2e65dfd..53472c1 100755 --- a/conf.d/zz-project-packages +++ b/conf.d/zz-project-packages @@ -27,7 +27,8 @@ KEEL_APT_ROOT="${KEEL_APT_ROOT:-/srv/keel-apt}" KEEL_APT_REPO="$KEEL_APT_ROOT/repo" KEEL_STAGING_LIST="${KEEL_STAGING_LIST:-/etc/apt/sources.list.d/keel-staging.list}" KEEL_STAGING_KEYRING="${KEEL_STAGING_KEYRING:-/etc/apt/keyrings/keel-staging-keyring.asc}" -KEEL_SOURCES="${KEEL_SOURCES:-/etc/apt/sources.list.d/keel.sources}" +KEEL_STAGING_PIN="${KEEL_STAGING_PIN:-/etc/apt/preferences.d/keel-staging}" +KEEL_APT_ETC="${KEEL_APT_ETC:-/etc/apt}" KEEL_APT_LISTS="${KEEL_APT_LISTS:-/var/lib/apt/lists}" KEEL_PROJECT_PACKAGES="${KEEL_PROJECT_PACKAGES:-inithooks confconsole keel}" @@ -74,14 +75,18 @@ for package in $KEEL_PROJECT_PACKAGES; do done # the build time package source is not for appliances: remove the source entry, -# the copy of the archive it names and the keyring the build verified that -# archive with, and keep the documented, disabled entry for the future signed -# repository (overlay). The staging key signs whatever the build host produced, -# so an image that kept it would carry trust in a nightly (tracker#7). -# common/removelists-final/turnkey removes the same three paths at the end of -# the build, whatever a recipe does. +# the pin conf.d/main gave it for the build, the copy of the archive it names +# and the keyring the build verified that archive with. The staging key signs +# whatever the build host produced, so an image that kept it would carry trust +# in a nightly (tracker#7). common/removelists-final/turnkey removes the same +# paths at the end of the build, whatever a recipe does. The appliance's own +# Keel source and its 990 pin are common's (overlays/turnkey.d/keel-apt), so +# nothing the image keeps may still name the build time archive. rm -f "$KEEL_STAGING_LIST" +rm -f "$KEEL_STAGING_PIN" rm -f "$KEEL_STAGING_KEYRING" rm -rf "$KEEL_APT_ROOT" rm -rf "${KEEL_APT_LISTS:?}"/* -grep -q '^Enabled: no' "$KEEL_SOURCES" +apt_files=("$KEEL_APT_ETC/sources.list" "$KEEL_APT_ETC/sources.list.d" "$KEEL_APT_ETC/preferences" "$KEEL_APT_ETC/preferences.d") +leftover=$(grep -rlsE 'trixie-staging|/srv/keel-apt|l=Keel Linux staging' "${apt_files[@]}" || true) +[ -z "$leftover" ] || fatal "these apt files still name the build time archive: $(tr '\n' ' ' <<< "$leftover")" diff --git a/overlay/etc/apt/preferences.d/keel b/overlay/etc/apt/preferences.d/keel deleted file mode 100644 index dcfdba2..0000000 --- a/overlay/etc/apt/preferences.d/keel +++ /dev/null @@ -1,5 +0,0 @@ -# Keel Linux: prefer the project's packages over every other archive. -# Generated by bin/pin-file of the apt tooling; the same file keel-transition installs. -Package: * -Pin: release o=Keel Linux -Pin-Priority: 1001 diff --git a/overlay/etc/apt/sources.list.d/keel.sources b/overlay/etc/apt/sources.list.d/keel.sources deleted file mode 100644 index 75032bc..0000000 --- a/overlay/etc/apt/sources.list.d/keel.sources +++ /dev/null @@ -1,17 +0,0 @@ -# Keel Linux package repository (brief section 5.4). -# -# Disabled until the repository is published and signed: the signing subkey -# is being rotated and apt.keellinux.org is not serving yet. The evaluation -# build took inithooks, confconsole and keel from the build host's unsigned -# staging distribution through a source that existed only during the build -# and was removed from this image (conf.d/main). Never point this file at -# trixie-staging: it is unsigned by design. -# -# To enable once the key is installed as /usr/share/keyrings/keel-archive-keyring.gpg: -# change "Enabled: no" to "Enabled: yes" and run apt update. -Types: deb -URIs: https://apt.keellinux.org -Suites: trixie -Components: main -Enabled: no -Signed-By: /usr/share/keyrings/keel-archive-keyring.gpg diff --git a/tests/apt-files.bats b/tests/apt-files.bats new file mode 100644 index 0000000..12dc91c --- /dev/null +++ b/tests/apt-files.bats @@ -0,0 +1,46 @@ +#!/usr/bin/env bats +# The apt files this recipe leaves in the image (Keel-Linux/common#30, +# tracker#23). Common ships the appliance's Keel source and its pin at 990 +# (overlays/turnkey.d/keel-apt); a recipe overlay at the same paths would win +# over them, so this recipe ships neither. The build time archive still has +# to win over TurnKey's 999 pin while the recipe upgrades the project +# packages, so conf.d/main pins it by its Label for the build only, and +# conf.d/zz-project-packages removes that pin with the build time source +# (tests/project-packages.bats). +# +# conf.d/main runs inside a chroot during the build; what it leaves is proved +# on the booted machine by the boot test. These check the recipe itself. + +bats_require_minimum_version 1.5.0 + +setup() { + REPO="$(cd "$BATS_TEST_DIRNAME/.." && pwd)" + MAIN="$REPO/conf.d/main" +} + +# line LITERAL: the line number of the first line of conf.d/main equal to it +line() { + grep -nxF -- "$1" "$MAIN" | head -n 1 | cut -d: -f1 +} + +@test "the overlay ships no Keel source and no Keel pin" { + [ ! -e "$REPO/overlay/etc/apt/sources.list.d/keel.sources" ] + [ ! -e "$REPO/overlay/etc/apt/preferences.d/keel" ] + run ! grep -rlsE 'Pin-Priority: *1001' "$REPO/overlay" +} + +@test "the build time pin names the staging Label, and is written before the upgrade" { + local pin upgrade + pin="$(line "printf 'Package: *\nPin: release l=Keel Linux staging\nPin-Priority: 1001\n' \\")" + # the backslash is the script's line continuation, matched literally + # shellcheck disable=SC1003 + upgrade="$(line 'apt-get install -y --only-upgrade \')" + [ -n "$pin" ] + [ -n "$upgrade" ] + [ "$pin" -lt "$upgrade" ] + grep -qxF ' > /etc/apt/preferences.d/keel-staging' "$MAIN" +} + +@test "conf.d/main parses" { + bash -n "$MAIN" +} diff --git a/tests/project-packages.bats b/tests/project-packages.bats index b09ca7c..a0e69f4 100644 --- a/tests/project-packages.bats +++ b/tests/project-packages.bats @@ -14,7 +14,8 @@ setup() { export KEEL_APT_ROOT="$scratch/srv/keel-apt" export KEEL_STAGING_LIST="$scratch/apt/sources.list.d/keel-staging.list" export KEEL_STAGING_KEYRING="$scratch/apt/keyrings/keel-staging-keyring.asc" - export KEEL_SOURCES="$scratch/apt/sources.list.d/keel.sources" + export KEEL_APT_ETC="$scratch/apt" + export KEEL_STAGING_PIN="$scratch/apt/preferences.d/keel-staging" export KEEL_APT_LISTS="$scratch/apt/lists" export FIXTURES="$scratch/fixtures" @@ -27,7 +28,10 @@ setup() { echo "deb [signed-by=$KEEL_STAGING_KEYRING] file://$KEEL_APT_ROOT/repo $DIST main" \ > "$KEEL_STAGING_LIST" printf 'not a key, and this script never reads one\n' > "$KEEL_STAGING_KEYRING" - printf 'Types: deb\nURIs: https://apt.keellinux.org\nEnabled: no\n' > "$KEEL_SOURCES" + # the build time pin conf.d/main writes before its upgrade + mkdir -p "$(dirname "$KEEL_STAGING_PIN")" + printf 'Package: *\nPin: release l=Keel Linux staging\nPin-Priority: 1001\n' \ + > "$KEEL_STAGING_PIN" offer inithooks 2.3.6+keel4 offer confconsole 2.2.3+keel2 @@ -93,7 +97,7 @@ installed() { printf 'version=%s\nstatus=%s\n' "$2" "$3" > "$FIXTURES/installed. [ ! -e "$KEEL_APT_ROOT" ] [ ! -e "$KEEL_STAGING_LIST" ] [ -z "$(ls -A "$KEEL_APT_LISTS")" ] - [ -f "$KEEL_SOURCES" ] + [ ! -e "$KEEL_STAGING_PIN" ] } @test "the keyring that verified the staging archive is gone too" { @@ -190,8 +194,30 @@ installed() { printf 'version=%s\nstatus=%s\n' "$2" "$3" > "$FIXTURES/installed. [[ "$output" == *"trixie-nowhere"* ]] } -@test "the future signed repository has to stay in place, disabled" { - printf 'Types: deb\nURIs: https://apt.keellinux.org\nEnabled: yes\n' > "$KEEL_SOURCES" +@test "an apt source that still names the build time archive fails the build" { + printf 'Types: deb\nURIs: file:///srv/keel-apt/repo\nSuites: trixie-staging\n' \ + > "$KEEL_APT_ETC/sources.list.d/leftover.sources" run "$SCRIPT" [ "$status" -ne 0 ] + [[ "$output" == *leftover.sources* ]] +} + +@test "a pin that still names the staging Label fails the build" { + printf 'Package: *\nPin: release l=Keel Linux staging\nPin-Priority: 1001\n' \ + > "$KEEL_APT_ETC/preferences.d/other" + run "$SCRIPT" + [ "$status" -ne 0 ] + [[ "$output" == *preferences.d/other* ]] +} + +@test "common's Keel source and its 990 pin are left in place" { + # what Keel-Linux/common#30 ships (overlays/turnkey.d/keel-apt) + printf 'Types: deb\nURIs: https://archive.keellinux.org\nSuites: trixie\nComponents: main\nEnabled: yes\n' \ + > "$KEEL_APT_ETC/sources.list.d/keel.sources" + printf 'Package: *\nPin: release o=Keel Linux\nPin-Priority: 990\n' \ + > "$KEEL_APT_ETC/preferences.d/keel" + run "$SCRIPT" + [ "$status" -eq 0 ] + [ -f "$KEEL_APT_ETC/sources.list.d/keel.sources" ] + [ -f "$KEEL_APT_ETC/preferences.d/keel" ] } From 70d974d0503123715d36b44002371c2e1aa246cd Mon Sep 17 00:00:00 2001 From: navigator Date: Fri, 2 Oct 2026 15:50:51 +0000 Subject: [PATCH 2/2] fix: the overlay change gets its own changelog entry require-changelog wants a new top entry, not a bullet in the base branch's. --- changelog | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/changelog b/changelog index 4cf4b8f..b786347 100644 --- a/changelog +++ b/changelog @@ -1,4 +1,4 @@ -turnkey-nodebb-19.0 (4) turnkey; urgency=low +turnkey-nodebb-19.0 (5) turnkey; urgency=low * The overlay no longer ships /etc/apt/sources.list.d/keel.sources (apt.keellinux.org, disabled) or /etc/apt/preferences.d/keel (the Keel @@ -13,6 +13,10 @@ turnkey-nodebb-19.0 (4) turnkey; urgency=low keel.sources was disabled. tests/project-packages.bats (16 tests, 100 percent) and tests/apt-files.bats check it. + -- Marcos Mendez Fri, 02 Oct 2026 17:00:00 +0000 + +turnkey-nodebb-19.0 (4) turnkey; urgency=low + * bin/keel-archive-check refuses a trusted=yes on the project archive rather than on every apt source in the build tree. The captured pool of decision 0012 sets Trusted: yes on purpose, for a file: index generated on this