diff --git a/docs/install-any-distro.md b/docs/install-any-distro.md index 36879d9e9..40e1a655d 100644 --- a/docs/install-any-distro.md +++ b/docs/install-any-distro.md @@ -203,9 +203,14 @@ that stays on its distribution's kernel only needs the second and third: | `ec-su-axb35-dkms` | the AXB35-02 board's embedded-controller driver (fans, EC temperature, power mode), GPL-2.0, upstream `cmetz/ec-su_axb35-linux` at `f62c2c22`, plus a DMI gate so it attaches only on an AXB35 | the fans on a Bosgame AXB35; inert on any other board | | `1bit-npu-config` | `limits.d/99-amdnpu-memlock.conf` (unlimited memlock), `udev/rules.d/90-1bit-npu-power.rules` (NPU keep-awake), `modules-load.d/ec_su_axb35.conf`, and the public MOK certificate at `/usr/share/1bit/mok/MOK-Kernel.der` | the NPU runtime's locked command buffers, and engine#1051's suspend/resume hang | -The three driver/config packages are the same set in all three families. They install to +The three driver/config packages are the same set in all three families, and that set alone is a +complete install for a machine that keeps its distribution's kernel — proved as its own path +(`mk-kernel-packages.sh --skip-kernel`, then `pkg-proof.sh`): 22 rows PASS with the kernel rows marked +skipped, ending with the distribution's own `amdxdna` restored. They install to `/lib/modules//updates/` (Debian, Arch) or `/extra/` (Fedora) through dkms, which is in front of -the in-tree `amdxdna`, and uninstalling the package puts the in-tree driver back. +the in-tree `amdxdna`, and uninstalling the package puts the in-tree driver back — measured, not assumed: +dkms archives the distribution's own module to `/var/lib/dkms//original_module/` when ours goes in, +and on removal the `kernel/` file and what `modprobe` resolves to are both back. ```bash # Debian / Ubuntu: the kernel comes from 1bit OS's own build, the drivers build against your headers. @@ -390,11 +395,15 @@ registry model, since this Lemonade version has no local-file registration): | the shim is the engine's own wrapper | `1bit/lemonade-hrx` from this repo's package (`scripts/package-linux.sh`) | it runs `1bit serve --print-backend-env` and then `exec`s the engine with serve's environment and defaults; `lemonade-bin/llama-server` is a symlink to it | | the recipe's env for its backend is one variable | this tree, `src/cpp/server/backends/hrx/hrx_server.cpp` `build_server_environment()` | Lemonade passes its backend `GGML_DISABLE_VULKAN=1` only — which is why the wrapper, not our binary, has to be what the recipe starts | | `amdxdna-dkms` from the PPA drags kernel bits | clean container, `apt-get -s install amdxdna-dkms` | `dkms`, `build-essential`, `linux-headers-7.0.0-42-generic`, `linux-headers-generic`, `amdxdna-dkms` | -| the driver packages install, build and uninstall on Debian/Ubuntu | clean `ubuntu:26.04` container, `scripts/pkg-proof.sh deb` | all 24 rows PASS: dkms builds both modules against `7.0.0-43-generic`, `modprobe --show-depends amdxdna` resolves to `/lib/modules/7.0.0-43-generic/updates/dkms/amdxdna.ko.zst` (in front of the in-tree one), the kernel debs' `vmlinuz` is `sbverify`-clean against the certificate in `1bit-npu-config`, purge removes the /etc files | -| the same on Fedora | clean `fedora:latest` (fc45) container, `scripts/pkg-proof.sh rpm` | all 22 rows PASS: modules under `/lib/modules/7.2.15-300.fc45.x86_64/extra/`, kernel rpms install `1bit-kernel` + `1bit-kernel-devel` with the build symlink at `/usr/src/kernels/7.4.0-rc1-1bit`, uninstall leaves nothing | -| the same on Arch | clean `archlinux:latest` container, `scripts/pkg-proof.sh arch` | all 22 rows PASS: the pacman dkms hook builds both modules at install time, `linux-1bit` + `linux-1bit-headers` install with the build tree at `/usr/lib/modules/7.4.0-rc1-1bit/build` and `/etc/mkinitcpio.d/1bit.preset` | +| the driver packages install, build and uninstall on Debian/Ubuntu | clean `ubuntu:26.04` container, `scripts/pkg-proof.sh deb` | all 31 rows PASS: dkms builds both modules against `7.0.0-43-generic`, `modprobe --show-depends amdxdna` resolves to `/lib/modules/7.0.0-43-generic/updates/dkms/amdxdna.ko.zst` (in front of the in-tree one), the kernel debs' `vmlinuz` is `sbverify`-clean against the certificate in `1bit-npu-config`, purge removes the /etc files | +| the same on Fedora | clean `fedora:latest` (fc45) container, `scripts/pkg-proof.sh rpm` | all 29 rows PASS: modules under `/lib/modules/7.2.15-300.fc45.x86_64/extra/`, kernel rpms install `1bit-kernel` + `1bit-kernel-devel` with the build symlink at `/usr/src/kernels/7.4.0-rc1-1bit`, uninstall leaves nothing | +| the same on Arch | clean `archlinux:latest` container, `scripts/pkg-proof.sh arch` | all 29 rows PASS: the pacman dkms hook builds both modules at install time, `linux-1bit` + `linux-1bit-headers` install with the build tree at `/usr/lib/modules/7.4.0-rc1-1bit/build` and `/etc/mkinitcpio.d/1bit.preset` | | the dkms source reproduces the driver this box runs | `scripts/mk-kernel-packages.sh` payload built in a container against `7.0.0-43-generic` | `srcversion EE1F36A2B9BA294CE11AD80`, the srcversion of the `amdxdna.ko` installed at `/lib/modules/7.4.0-rc1-1bit/updates/` — the package's source *is* the fleet's series | | the kernel the packages ship is the kernel this box runs | `linux-image-7.4.0-rc1-1bit_7.4.0~rc1-1_amd64.deb` payload vs `/boot/vmlinuz-7.4.0-rc1-1bit` | `sha256 6f3c6764…` in both, and `sbverify --cert MOK-Kernel.der` reports `Signature verification OK` | | the amdxdna floor is really 6.10 | clean `ubuntu:24.04` container, headers `6.8.0-149-generic`, this page's packages | `dkms build` refuses with the declared exclusive rule (`rc=77`, "should not be built"), and with the floor bypassed the driver does not compile: `amdxdna.h:32: macro "__assign_str" requires 2 arguments, but only 1 given` — the 6.10 trace-event API. `ec-su-axb35-dkms` builds on the same headers (`380 440 B`), so only the NPU driver has the floor | | a weekly release tag works in all three families, and upgrades | `scripts/mk-kernel-packages.sh 2026.45.0 --skip-kernel`, each family's own tool, in `fedora:latest` / `ubuntu:26.04` / `archlinux:latest` | deb `0.18.0-1 < 0.18.0-2026.45.0` (dpkg), rpm `0.18.0-1 < 0.18.0-2026.45.0` (rpmdev-vercmp), arch `vercmp` `-1`; each installs over the previous package (`rc=0`) and reports `0.18.0-2026.45.0` afterwards, with one `/usr/src/amdxdna-1bit-0.18.0` left on deb — no stale duplicate source tree | +| uninstalling really restores the in-tree driver, in all three families | `scripts/pkg-proof.sh`, each family in its container, after the purge | the in-tree module is gone from the tree while ours is installed (dkms archives it to `/var/lib/dkms//original_module/`), and after removal both `modprobe --show-depends amdxdna` and the file are back on the `kernel/` path: `amdxdna.ko.zst` on Ubuntu and Arch, `amdxdna.ko.xz` on Fedora. Same three containers as the install rows, `PROOF-RC=0` | +| uninstalling the kernel package leaves the machine as it found it | `scripts/pkg-proof.sh`, each family in its container, after removing the kernel package | deb `dpkg -P linux-image` rc=0, rpm `rpm -e 1bit-kernel-devel 1bit-kernel` rc=0, arch `pacman -R linux-1bit-headers linux-1bit` rc=0; `/boot/vmlinuz-7.4.0-rc1-1bit` (or `/boot/vmlinuz-1bit` on Arch) is gone and the module tree under `modules/7.4.0-rc1-1bit` has **0 files** left. deb **29/29**, rpm **27/27**, arch **27/27** rows PASS, `PROOF-RC=0` | +| the memlock file is honoured, not just present | `scripts/pkg-proof.sh`, all three families, running the container with `--cap-add=sys_resource` | a probe login with the file rewritten to `4096` came back with exactly `4096` (lowering always works, so that value can only come from the file), and with the shipped file back the same login reports `ulimit -l: unlimited`. Without `CAP_SYS_RESOURCE` — which a container's default set excludes — raising the hard limit fails and the session keeps the container's `8192`, which is the container, not the package | +| the distribution-kernel path, on its own | clean `ubuntu:26.04` container, packages built with `--skip-kernel`, `scripts/pkg-proof.sh deb` | 22 rows PASS and one `INFO no kernel packages in this dir` (the kernel rows skip themselves instead of failing): install, config, memlock reaching a login session, dkms building both modules against `7.0.0-43-generic`, `amdxdna` resolving to `updates/dkms`, uninstall, and the distribution's in-tree `amdxdna` back in place. This is the path a user on their own kernel takes | | the shipped driver answers a known answer on the NPU | this box, `npu-hold`, `1bit serve --device npu` on Qwen3-0.6B, `/v1/completions` greedy, twice | load 2.2 s, `The capital of France is` → **` Paris. The capital of Italy is Rome. The capital of Spain is Madrid.`** identically on both runs (56.9 and 50.7 tok/s), 0 `amdxdna` lines in the two windows, and the loaded module is `srcversion EE1F36A2B9BA294CE11AD80` — the build this package's source produces. No install, no `modprobe`, no udev reload | diff --git a/scripts/pkg-proof.sh b/scripts/pkg-proof.sh index adee6744b..6b79e4abf 100755 --- a/scripts/pkg-proof.sh +++ b/scripts/pkg-proof.sh @@ -23,16 +23,21 @@ # then uninstalls everything and checks nothing is left behind. Prints one PASS/FAIL row per check and # exits non-zero if any row fails. # -# deb docker run --rm -v :/pkg:ro ubuntu:26.04 bash -c \ +# deb docker run --rm --cap-add=sys_resource -v :/pkg:ro ubuntu:26.04 bash -c \ # 'apt-get update -qq && apt-get install -y -qq dkms udev kmod sbsigntool linux-headers-generic \ # && apt-get install -y -qq linux-modules-$(ls /lib/modules | sort -V | tail -1) \ # && bash /pkg/pkg-proof.sh deb /pkg' -# rpm docker run --rm -v :/pkg:ro fedora:latest bash -c \ +# rpm docker run --rm --cap-add=sys_resource -v :/pkg:ro fedora:latest bash -c \ # 'dnf -y install dkms kernel-devel kernel-core kmod binutils xz; bash /pkg/pkg-proof.sh rpm /pkg' -# arch docker run --rm -v :/pkg:ro archlinux:latest bash -c \ +# arch docker run --rm --cap-add=sys_resource -v :/pkg:ro archlinux:latest bash -c \ # 'pacman -Sy --noconfirm && pacman -S --noconfirm dkms linux linux-headers kmod binutils zstd; \ # bash /pkg/pkg-proof.sh arch /pkg' # +# `--cap-add=sys_resource` is for the two memlock rows: a container's default capability set excludes +# CAP_SYS_RESOURCE, so raising the hard memlock limit to `unlimited` fails there and the login session +# keeps the container's inherited 8192 KB. A real machine's root has the capability; the flag makes the +# container behave like one. +# # `linux-modules-` / `kernel-core` / `linux` are the container's way of having the kernel's own # module tree, which a real machine always has. Without it there is no in-tree amdxdna for the DKMS # module to take precedence over, and the precedence row below cannot be measured. @@ -50,6 +55,7 @@ fam=${1:?usage: pkg-proof.sh } pkgdir=${2:?usage: pkg-proof.sh } pkgdir=$(cd "$pkgdir" && pwd) fail=0 +krel= # set by the kernel-package block; empty on the distribution-kernel path (no kernel packages) row() { # printf '%-4s %-46s %s\n' "$1" "$2" "$3" [ "$1" = PASS ] || fail=1 @@ -64,6 +70,18 @@ case $fam in *) echo "unknown family: $fam" >&2; exit 2 ;; esac [ -n "$kv" ] || { echo "pkg-proof: no kernel headers in this container; install them first (see the header)" >&2; exit 2; } +# Does this package dir carry the kernel packages? `mk-kernel-packages.sh --skip-kernel` does not, and that +# is the most common user path — a machine that stays on its distribution's kernel takes only the drivers +# and the config. Those rows are then skipped instead of failing; the rest of the proof runs unchanged. +case $fam in + deb) have_kernel=$(ls "$pkgdir"/linux-image-*.deb 2>/dev/null | wc -l) ;; + rpm) have_kernel=$(ls "$pkgdir"/1bit-kernel-[0-9]*.rpm 2>/dev/null | wc -l) ;; + arch) have_kernel=$(ls "$pkgdir"/linux-1bit-[0-9]*.pkg.tar.zst 2>/dev/null | wc -l) ;; + *) have_kernel=1 ;; +esac +if [ "$have_kernel" = 0 ] && [ "$fam" != coexist ]; then + printf 'INFO %-46s %s\n' "no kernel packages in this dir" "distribution-kernel path: kernel rows skipped" +fi echo "== $fam in $(grep -h '^PRETTY_NAME' /etc/os-release | cut -d= -f2- | tr -d '\"') | headers $kv | running $(uname -r)" # the in-tree module this must displace; without the kernel's own module tree there is nothing to @@ -97,6 +115,34 @@ check "public MOK certificate (kernel key)" \ check "config files are 0644" "not group-writable" \ bash -c '! find /etc/security/limits.d/99-amdnpu-memlock.conf /etc/udev/rules.d/90-1bit-npu-power.rules -perm /022 | grep -q .' +# Is the memlock file actually READ by the login path, or only present? The NPU runtime locks command +# buffers and fails without it, so a right-looking file at a path pam_limits does not read would be a +# confusing user-facing failure. Proof: write a distinctive finite limit into the installed file, log in +# as a fresh user, and see that value come back; then restore the shipped content and read it again. +if id kpkgprobe >/dev/null 2>&1 || useradd -m -s /bin/bash kpkgprobe >/dev/null 2>&1; then + lim=/etc/security/limits.d/99-amdnpu-memlock.conf + keep=$(mktemp); cp "$lim" "$keep" + # 4096, not 8192: LOWERING a limit is always permitted, so the value can only come from the file. + # (The first version of this row wrote 8192, which happened to equal the container's inherited + # memlock and so passed without proving anything.) + printf '* soft memlock 4096\n* hard memlock 4096\n' > "$lim" + probe=$(su - kpkgprobe -c 'ulimit -l' 2>/dev/null | tr -d ' ') + if [ "$probe" = 4096 ]; then + row PASS "the memlock file is what pam_limits reads" "a probe login came back with 4096 KB" + else + row FAIL "the memlock file is what pam_limits reads" "probe login reports '${probe:-nothing}'" + fi + cp "$keep" "$lim"; rm -f "$keep" + shipped=$(su - kpkgprobe -c 'ulimit -l' 2>/dev/null | tr -d ' ') + case $shipped in + unlimited) row PASS "the shipped limit reaches a login session" "ulimit -l: unlimited" ;; + *) row FAIL "the shipped limit reaches a login session" \ + "ulimit -l: '${shipped:-nothing}' — raising a hard limit needs CAP_SYS_RESOURCE: run the container with --cap-add=sys_resource (a real machine's root has it)" ;; + esac +else + row FAIL "the memlock file is what pam_limits reads" "could not create a probe user" +fi + # ---- the DKMS modules ----------------------------------------------------------------------- dkmsstate=$(dkms status 2>/dev/null || :) case $dkmsstate in *amdxdna-1bit/0.18.0*) amdx=1 ;; *) amdx=0 ;; esac @@ -147,6 +193,7 @@ else fi # ---- the kernel package --------------------------------------------------------------------- +if [ "$have_kernel" = 1 ]; then case $fam in deb) dpkg -i "$pkgdir"/linux-image-*.deb "$pkgdir"/linux-headers-*.deb >/tmp/k.log 2>&1 \ @@ -188,6 +235,7 @@ if [ -r "$img" ]; then else row FAIL "vmlinuz in the package" "$img missing" fi +fi # have_kernel # ---- uninstall ------------------------------------------------------------------------------ case $fam in @@ -206,6 +254,49 @@ case $leftstate in *amdxdna-1bit*|*ec-su-axb35*) row FAIL "dkms forgot the modul *) row PASS "dkms forgot the module" "dkms status clean" ;; esac check "the config files are gone" "/etc rules removed" \ bash -c "! test -e /etc/udev/rules.d/90-1bit-npu-power.rules" +if [ "$fam" != coexist ]; then + # The other half of "uninstalling puts the in-tree driver back": dkms archives the in-tree module + # when it installs ours (into /var/lib/dkms//original_module/), so removal has to restore it — + # both the file and what a modprobe resolves to, because a restored file that nothing resolves to is + # not a restore. Verified by hand first: before our package the resolution is + # /lib/modules//kernel/drivers/accel/amdxdna/amdxdna.ko.zst, with it .../updates/dkms/amdxdna.ko.zst, + # and after the purge back to the kernel/ path. + depmod -a "$kv" 2>/dev/null || : + back=$(modprobe -S "$kv" --show-depends amdxdna 2>/dev/null | sed -n 's/^insmod //p' | tail -1) + case $back in + */kernel/drivers/accel/amdxdna/*) + row PASS "the in-tree amdxdna resolves again" "$(basename "$back")" ;; + *) row FAIL "the in-tree amdxdna resolves again" "${back:-nothing resolves amdxdna}" ;; + esac + intree_file=$(find -L /lib/modules/"$kv"/kernel -name 'amdxdna.ko*' 2>/dev/null | head -1) + if [ -n "$intree_file" ]; then + row PASS "the in-tree module file is restored" "$(basename "$intree_file")" + else + row FAIL "the in-tree module file is restored" "not under /lib/modules/$kv/kernel" + fi +fi + +# ---- removing the kernel package again ------------------------------------------------------- +# The kernel rows above install it; nothing removed it, so "and uninstall cleanly" had no evidence for +# the kernel half. It also has to leave the filesystem as it found it: no /boot entry, no module tree. +if [ -n "$krel" ] && [ "$fam" != coexist ] && [ "$have_kernel" = 1 ]; then + case $fam in + deb) dpkg -P "linux-image-$krel" "linux-headers-$krel" >/tmp/kr.log 2>&1 ;; + rpm) rpm -e 1bit-kernel-devel 1bit-kernel >/tmp/kr.log 2>&1 ;; + arch) pacman -R --noconfirm linux-1bit-headers linux-1bit >/tmp/kr.log 2>&1 ;; + esac + if [ $? -eq 0 ]; then row PASS "the kernel package removes" "rc=0" + else row FAIL "the kernel package removes" "$(tail -2 /tmp/kr.log)"; fi + case $fam in + deb|rpm) leftover=/boot/vmlinuz-$krel ;; + arch) leftover=/boot/vmlinuz-1bit ;; + esac + [ -e "$leftover" ] && row FAIL "the kernel image is gone" "$leftover still there" \ + || row PASS "the kernel image is gone" "$leftover removed" + mods_left=$(find /lib/modules/"$krel" /usr/lib/modules/"$krel" -type f 2>/dev/null | wc -l) + [ "$mods_left" = 0 ] && row PASS "the module tree is gone" "0 files under modules/$krel" \ + || row FAIL "the module tree is gone" "$mods_left files left" +fi echo if [ "$fail" = 0 ]; then echo "== $fam: ALL ROWS PASS"; else echo "== $fam: FAILURES ABOVE"; fi