From 0631b11f9c8d9113af5aeae61f1705e092aed2d8 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Sat, 7 Nov 2026 18:29:09 -0400 Subject: [PATCH 1/3] pack-proof: the release installs every package it publishes, or it does not release (Refs #1279) #1279 item 4 asked for proof per family: installs in a clean container of that distro, `1bit-engine --version` and the shim run, and it uninstalls cleanly. That proof existed once, by hand, in #1292 and #1301 - and after those merged, the weekly's package stage shipped the four packages with nothing verifying them. A `.rpm` that cannot install (the /usr/bin-at-0775 conflict in #1292 is exactly that class), a tarball that changes system ownership (the #1301 review), or a package that carries a different payload than its siblings would all reach a user and fail there. scripts/pack-proof.sh is those rows, committed and run by the release. Six of them, one PASS/FAIL line each, non-zero exit if any fails: deb ubuntu:26.04 apt-get install, version, shim, remove, nothing left rpm fedora:latest dnf install, same checks rpm opensuse/tumbleweed zypper install of the same file, same checks arch archlinux:latest pacman -U, same checks tarball ubuntu:26.04 the tarball's members are root's, the documented extraction as root leaves /, /usr, /usr/bin and /opt alone, the payload runs from it same ubuntu:26.04 the four unpacked as root: paths, modes, owner, sizes, sha256 It needs docker and nothing else on the host (every install is in a throwaway container), which the package stage already needs: the .rpm and the Arch package are built in fedora and archlinux containers. scripts/weekly.sh runs it right after the four are built and stops the release on a failing row, docs/releases.md says so in the package stage and in what the box must have, and docs/install-any-distro.md carries the measured row. Measured 2026-11-07 on the rc-2026-11-06-c31 artifacts: 6/6 rows PASS, 3m54s with the images already pulled the shim's version banner is on stderr (it is llama.cpp underneath), which the harness captures with 2>&1 - the first version of this script read an empty `$shim` because of it handed a tarball rebuilt the pre-#1301 way (uid 1000, a "./" member, the filesystem directories), the same harness reports 4/6: "151 tarball members are not owned by root" for the tarball row and the payload comparison for the identity row. That is the row that would have caught the escalation the review found, so the check bites rather than merely passing. its own failure paths: no arguments -> rc 2 with the usage; a directory without the four packages -> rc 2 naming the missing file --- docs/install-any-distro.md | 1 + docs/releases.md | 14 +++- scripts/pack-proof.sh | 161 +++++++++++++++++++++++++++++++++++++ scripts/weekly.sh | 8 ++ 4 files changed, 181 insertions(+), 3 deletions(-) create mode 100755 scripts/pack-proof.sh diff --git a/docs/install-any-distro.md b/docs/install-any-distro.md index 5aa4f43f8..c1ad63250 100644 --- a/docs/install-any-distro.md +++ b/docs/install-any-distro.md @@ -350,6 +350,7 @@ registry model, since this Lemonade version has no local-file registration): | the tarball cannot hand a system directory to uid 1000 | clean `ubuntu:26.04`, the documented `sudo tar -C / --no-same-owner --no-overwrite-dir -xf …` | `/`, `/usr`, `/usr/bin`, `/opt` read `0:0 755` before and after; the payload lands `0:0`, `0` of 75 files not owned by root, and the shim runs (`version: 1 (32559003)`). Before the fix the same extraction as root left uid 1000 owning `/`, `/usr`, `/usr/bin` and `/opt` — a local root escalation on every "any other distribution" install (#1292 review) | | the glibc floor is enforced, not just documented | clean `ubuntu:24.04` container (glibc 2.39), `apt-get install ./1bit-engine__amd64.deb` | refused: `1bit-engine : Depends: libc6 (>= 2.43) but 2.39-0ubuntu8.9 is to be installed`. Before the floor was declared, the same deb installed, `ldd` was happy and the binary then failed to load | | **the package's payload serves on the GPU** | strixhalo, the Fedora `.rpm` unpacked with `rpm2cpio` (so exactly the files the package installs), `1bit serve -m Qwen3-0.6B-Q4_K_M.gguf --device hrx --no-strict-device`, two chats | `1bit serve: HRX: HSA runtime …/_rocm_sdk_core/lib/libhsa-runtime64.so.1 sha256 21a66824…` — the runtime the package ships, not the box's — then `Qwen3-0.6B-Q4_K_M on hrx`, health `200`, and `request /v1/chat/completions -> hrx status 200 in 202.5 ms | prompt 16 completion 16 79.0 tok/s`; a second chat with `max_tokens: 200` came back `content: "5"` (109 completion tokens, 69.1 tok/s). The first one's content was empty because Qwen3 spends the opening tokens on its thinking trace, so a 16-token budget never reaches the answer. Both this model and `Qwen3-4B-Q4_K_M` are refused first by strict placement (`"embd" is GET_ROWS f32 … 1 of 986 nodes off HRX`), which is why the runs pass `--no-strict-device` | +| **the release proves its own packages** | `scripts/pack-proof.sh `, wired into `scripts/weekly.sh`'s package stage, 2026-11-07, on the `rc-2026-11-06-c31` artifacts | 6/6 rows PASS: the `.deb` installed in `ubuntu:26.04`, the `.rpm` in both `fedora:latest` and `opensuse/tumbleweed`, the pacman package in `archlinux:latest` (each asked `1bit-engine --version` = `1bit 0.0.1`, made to run the shim = `version: 1 (32559003)`, then removed with nothing left), the tarball unpacked as root the way this page says, and the four compared as root (paths, modes, owner `0:0`, sizes, sha256). Handed the pre-fix tarball — uid 1000, a `./` member, the filesystem directories — the same harness fails **4/6**, naming the 151 members that are not root's and the payload difference, so the row is a check and not a claim | | a stock Lemonade takes a backend binary from `hrx_bin` / `LEMONADE_HRX_HRX_BIN` | extracted PPA deb `2026.44.1~26.04` on the dev box, isolated cache/config | `llamacpp-hrx` backend `installable` → `installed`, both forms, with no fork and no patch | | the same recipe, both packages side by side | clean `ubuntu:26.04` container: PPA `lemonade-server` + this `.deb` | `lemond 2026.44.1` starts, `1bit-engine --version` and the shim run; the recipe's backend state is hardware-gated there (`unsupported: Unsupported GPU`), so the end-to-end chat is the hardware row below | | the shim is the engine's own wrapper | `1bit/lemonade-hrx` from this repo's package (`scripts/package-linux.sh`) | it runs `1bit serve --print-backend-env` and then `exec`s the engine with serve's environment and defaults; `lemonade-bin/llama-server` is a symlink to it | diff --git a/docs/releases.md b/docs/releases.md index c3e071156..7737973d9 100644 --- a/docs/releases.md +++ b/docs/releases.md @@ -482,8 +482,14 @@ WEEKLY_DRY_RUN=1 WEEKLY_TAG=rehearsal-private scripts/weekly.sh ~/.cache/1bit-we 2. **build** and **test**: `main` at the merged pins, the same way. 3. **package**: the two packages above, plus the engine's own packages for every family (`scripts/mk-plugin-package.sh`: the `.deb`, `.rpm`, Arch package and tarball, all built from one - payload tree by `scripts/pack-distro.sh`). `scripts/package-linux.sh` packs the engine first; its - tarball goes into the Lemonade + onebit package as `1bit/`, not into the release on its own. + payload tree by `scripts/pack-distro.sh`), and then the proof of them: + `scripts/pack-proof.sh` installs each one in a throwaway container of its distribution + (`ubuntu:26.04`, `fedora:latest`, `opensuse/tumbleweed`, `archlinux:latest`), asks the engine its + version, runs the shim, removes it and checks nothing is left; unpacks the tarball as root the way + the guide says; and compares the four as root so a package carrying a different payload fails the + release instead of a user. A failing row stops the release. `scripts/package-linux.sh` packs the + engine first; its tarball goes into the Lemonade + onebit package as `1bit/`, not into the release + on its own. 4. **release**: `tools/weekly_changes.py` lists, for every pin that moved since the last release, the upstream commits (authors, PRs) and releases in between; that becomes the release notes and `changes.json`, and `gh release create` publishes it all. @@ -500,7 +506,9 @@ rehearsal found these only after the build) and stops with a message naming what or `ROCM_PATH`. (The optional DwarfStar build that also wanted it is gone since 2026-11-02.) - **package:** `patchelf` (`scripts/package-linux.sh`); `dpkg-deb` and `docker` (`scripts/pack-distro.sh` builds the `.rpm` with rpmbuild in a `fedora` container and the Arch - package with makepkg in an `archlinux` one, since the release box has neither toolchain); `node` + package with makepkg in an `archlinux` one, since the release box has neither toolchain, and + `scripts/pack-proof.sh` installs all four in `ubuntu:26.04`, `fedora:latest`, + `opensuse/tumbleweed` and `archlinux:latest`, so the network has to reach those four images); `node` and `npm` for Lemonade's web app (`LEMONADE_WEB_APP=1 scripts/build-lemonade.sh`; its `npm ci` needs the network); a readable kernel image for the `os/mini` image, `/boot/vmlinuz-$(uname -r)` or `VMLINUZ=` (Ubuntu's `/boot/vmlinuz-*` is root-only). diff --git a/scripts/pack-proof.sh b/scripts/pack-proof.sh new file mode 100755 index 000000000..d239a64e1 --- /dev/null +++ b/scripts/pack-proof.sh @@ -0,0 +1,161 @@ +#!/usr/bin/env bash +# Copyright 2026 bong-water-water-bong +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# +# pack-proof.sh +# +# The proof for the engine's own packages (engine #1279 item 4): every family installs it, the engine +# answers, the shim runs, and it uninstalls cleanly; the tarball is unpacked as root the way +# docs/install-any-distro.md says and must not touch a directory that is not ours; and the four +# packages are unpacked side by side to check they carry the same thing. One PASS/FAIL row per check, +# non-zero exit if any row fails. +# +# Needs docker and nothing else on the host: every install happens inside a throwaway container of the +# family it is for, and the host is left with one temporary directory. Run from the release's package +# stage (scripts/weekly.sh) or by hand: +# +# scripts/pack-proof.sh 2026.45 +# +# The rows and what each one means: +# deb ubuntu:26.04 apt-get install ./1bit-engine__amd64.deb +# rpm fedora:latest dnf install ./1bit-engine--1.x86_64.rpm +# rpm opensuse/tumbleweed zypper install ./... (the same file: one rpm for both families) +# arch archlinux:latest pacman -U ./1bit-engine--1-x86_64.pkg.tar.zst +# tarball ubuntu:26.04 tar -C / --zstd --no-same-owner --no-overwrite-dir -xf ... +# same ubuntu:26.04 the four unpacked as root: paths, modes, owner, sizes, sha256 +# +# The deb/rpm/arch rows check removal too (nothing of the payload may be left); the tarball row does +# not, because a tarball has nothing to uninstall. The images are the distributions the release says +# it supports, named once here so a release cannot quietly prove itself on something else. +set -uo pipefail + +[ $# -eq 2 ] || { sed -n '16,43p' "$0"; exit 2; } +dir=$(cd "$1" && pwd) || exit 2 +ver=${2#v} +deb=1bit-engine_${ver}_amd64.deb +rpm=1bit-engine-${ver}-1.x86_64.rpm +arch_pkg=1bit-engine-${ver}-1-x86_64.pkg.tar.zst +tarball=1bit-engine-${ver}-linux-x86_64.tar.zst +for f in "$deb" "$rpm" "$arch_pkg" "$tarball"; do + [ -f "$dir/$f" ] || { echo "pack-proof: no $f in $dir (the release builds all four: scripts/mk-plugin-package.sh)" >&2; exit 2; } +done +command -v docker >/dev/null || { echo "pack-proof: no docker on PATH; the rows are container installs" >&2; exit 2; } + +rows=0 failed=0 + +# run_row