Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -249,6 +249,11 @@ add_test(NAME seal_manifest COMMAND ${CMAKE_SOURCE_DIR}/tests/seal_manifest_test
# release, and the ordering a candidate needs: previous release < candidate < its release. The
# ordering is the row dpkg, rpm and pacman had to agree on; no container, no build, dpkg only.
add_test(NAME pack_version COMMAND ${CMAKE_SOURCE_DIR}/tests/pack_version_test.sh)
# engine #1461/#1411: the two release rows that read a PACKED tree (scripts/pack-check.sh). Synthetic trees
# built with the C compiler: a build-machine path in an ELF, an absolute RUNPATH, a relative rpath that
# resolves outside the payload, and a runtime set shipped without the aqlprofile agent libhsa opens. Each
# arm asserts the row's message, and a clean tree is the control. No package build, no GPU, runs in CI.
add_test(NAME pack_check COMMAND ${CMAKE_SOURCE_DIR}/tests/pack_check_test.sh)
# engine #713: the pinned HRX toolchain (config/hrx-toolchain.config, scripts/fetch-rocm-sdk.sh) -- the
# pin file's refusals, the completeness check against a fabricated tree, and that cmake and
# weekly.sh both use the pin instead of the box's /opt/rocm-therock. No network, no SDK, no GPU.
Expand Down
34 changes: 34 additions & 0 deletions NOTICE-BINARY
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,40 @@ NPU add-on npu-lax (from the private npu-kernels repository, addons/npu-lax)
(https://github.com/vegah/LLMNpuTest). Apache License, Version 2.0
(http://www.apache.org/licenses/LICENSE-2.0).

-------------------------------------------------------------------------------------
AMD HSA runtime set (libhsa-runtime64.so.1, libhsa-amd-aqlprofile64.so.1,
librocprofiler-register.so.0 and the rocm_sysdeps libraries, packed beside the HRX llama.cpp libraries in
hrx/llama/bin)

HRX reaches the GPU through this set. The engine's backend opens the runtime BY NAME at run time
(ggml-hrx-dmabuf.cpp:29, dlopen("libhsa-runtime64.so.1")), and the runtime in turn opens the aqlprofile
agent by name, so ldd never sees either: the whole set ships in the payload, beside the backend, and the
payload's RUNPATH is $ORIGIN - nothing resolves out of the install (engine #1411). Shipped unmodified, as
built by TheRock; the release refuses a payload that carries the set incompletely.

- libhsa-runtime64.so.1 (ROCr). Upstream https://github.com/ROCm/rocr-runtime, built by TheRock
(https://github.com/ROCm/TheRock). Copyright (c) 2014-2025 Advanced Micro Devices, Inc.
University of Illinois/NCSA Open Source License (NCSA).

- libhsa-amd-aqlprofile64.so.1, the profiling agent the runtime opens by name. Upstream
https://github.com/ROCm/aqlprofile. Copyright (c) Advanced Micro Devices, Inc. MIT License.

- librocprofiler-register.so.0, the profiling registration shim. Upstream
https://github.com/ROCm/rocprofiler-register. Copyright (c) Advanced Micro Devices, Inc. MIT License.

- librocm_sysdeps_*.so* - TheRock's rocm_sysdeps build of the system libraries the runtime links, so the
payload does not depend on which version a distribution ships. Each is unmodified from TheRock and keeps
its upstream licence:

librocm_sysdeps_bz2.so libbzip2 https://sourceware.org/bzip2/ bzip2-1.0.6
librocm_sysdeps_drm.so.2 libdrm https://gitlab.freedesktop.org/mesa/drm MIT
librocm_sysdeps_drm_amdgpu.so.1 libdrm https://gitlab.freedesktop.org/mesa/drm MIT
librocm_sysdeps_elf.so.1 libelf https://sourceware.org/elfutils/ GPL-3.0-or-later OR LGPL-3.0-or-later
librocm_sysdeps_liblzma.so.5 liblzma https://tukaani.org/xz/ public domain (0BSD)
librocm_sysdeps_numa.so.1 libnuma https://github.com/numactl/numactl LGPL-2.1
librocm_sysdeps_z.so.1 zlib https://zlib.net/ Zlib
librocm_sysdeps_zstd.so.1 zstd https://github.com/facebook/zstd BSD-3-Clause OR GPL-2.0

-------------------------------------------------------------------------------------
LLVM OpenMP runtime (libomp.so, packed beside the HRX llama.cpp libraries in hrx/llama/bin)

Expand Down
11 changes: 11 additions & 0 deletions scripts/mk-plugin-package.sh
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,17 @@ for so in $(LD_LIBRARY_PATH=$libdirs ldd "$libhsa" | awk '/=> \// {print $3}' |
install -m 0644 "$so" "$hsa_dir/$(basename "$so")"
copied=$((copied + 1))
done
# The ldd closure above is not the whole set: libhsa OPENS the aqlprofile agent by name at run time
# (libhsa-amd-aqlprofile64.so.1, the profiling interface), so ldd cannot see it and a payload built from
# ldd alone ships a runtime that reaches for it wherever the machine happens to offer it - which is how
# engine #1411's c22 install ended up mapping 64 files out of a builder's toolchain. It ships beside the
# runtime with the rest, and the build refuses if it is not where the runtime was found.
for pat in libhsa-amd-aqlprofile64.so*; do
aq=$(ls "$srcdir"/$pat 2>/dev/null | head -1)
[ -n "$aq" ] || { echo "no $pat beside $libhsa: the HSA runtime opens the aqlprofile agent by name at run time, so a payload without it resolves it from wherever the machine has one (engine #1411)" >&2; exit 1; }
install -m 0644 "$aq" "$hsa_dir/$(basename "$aq")"
copied=$((copied + 1))
done
missing=$(LD_LIBRARY_PATH=$hsa_dir ldd "$hsa_dir/libhsa-runtime64.so.1" | awk '/not found/ {print $1}' | tr '\n' ' ')
[ -z "${missing// /}" ] || {
echo "libhsa needs $missing and this build could not stage it beside the runtime;
Expand Down
110 changes: 110 additions & 0 deletions scripts/pack-check.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,110 @@
#!/usr/bin/env bash
# Copyright 2026 bong-water-water-bong
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# pack-check.sh: the release rows that look at a PACKED tree rather than a build tree. Sourced by
# scripts/weekly.sh beside scripts/pack-elf.sh, and by tests/pack_check_test.sh, which is how these rows run
# in CI without a package build:
#
# build_path_strings <unpacked tree> no packed ELF names a build-machine path (engine #1461)
# runtime_selfcontained <unpacked tree> nothing resolves outside the payload, and the HSA runtime set
# that libhsa opens at run time ships beside it (engine #1411)

# What the engine tarball must and must not hold (docs/releases.md, "Private components"):
# It is packed by scripts/package-linux.sh and folded into the Lemonade + onebit package as 1bit/;
# runtime-deps no program or library needs or names an external accelerator-runtime library
# (EXT_RUNTIME_LIBS below: readelf NEEDED, ldd, strings), and no such library is packed;
# the NPU runtime is compiled into 1bit
# kernels the HRX ggml library carries the private HIP kernels (it exports the add-on's
# ggml_hrx_hip_addon_register), compiled in as code objects
# no source no source file (.hip .c .cc .cpp .cxx .h .hpp .inc .cu .cmake) and no file named
# like one of the private checkouts' files -- except the three GGUF-on-NPU helper scripts
# (scripts/repack_gguf.py, scripts/check_repack_config.py, tools/gguf_to_q4nx.py), which
# are the engine's own Apache-2.0 source, published in this repository, and without
# which a packed install cannot serve a GGUF on the NPU
# licensed the closed binaries' LICENSE-BINARY and NOTICE-BINARY are packed beside LICENSE and NOTICE
# npu (with WEEKLY_NPU_KERNELS) every file under npu/kernels but layout.json is sealed, never a
# plaintext ELF; every ELF in the tarball is for the host (x86-64); the packed 1bit has no
# symbol table and its strings name no private source file and no build-machine home
# Checks the build's 1bit and every file in the tarball.
# engine #1461: no packed ELF may name a build-machine path. One row, used for BOTH packages - the engine
# tarball and the Lemonade package that carries it - over every ELF rather than the packed `1bit` alone: that
# narrow scope is why 146 build paths shipped in c32's payload (lemond 126, lemonade 13, libloomc.so.0 7)
# while the row printed "no build-machine paths", exactly as #1460's `/home/`-only predicate had been too
# narrow a pattern. OCCURRENCES are counted, not lines: a strings run glues paths to their neighbours, so one
# of lemonade's 13 does not begin a line and `grep -c` reports 12 for a file carrying 13 (measured
# 2026-11-08 on c32's own bytes). /home/ covers any builder's home; the source tree, the workdir and the
# build tree cover a build that ran elsewhere on this machine.
build_path_strings() { # build_path_strings <unpacked tree> -> one FAIL line per offending file, rc 1
local t=$1 f n pats rc=0
pats=$(printf '%s\n' /home/ "${SRC:-/nonexistent-build-src}" "${W:-/nonexistent-build-w}" \
"${BUILD:-/nonexistent-build-build}")
while IFS= read -r -d '' f; do
pack_is_elf "$f" || continue
n=$(strings -a "$f" | grep -o -F -f <(printf '%s\n' "$pats") | wc -l || true)
[ "$n" = 0 ] || {
printf 'FAIL %s names a build-machine path (%s occurrences): %s\n' "$(basename "$f")" "$n" \
"$(strings -a "$f" | grep -o -F -f <(printf '%s\n' "$pats") | head -2 | tr '\n' ' ')"
rc=1
}
done < <(find "$t" -type f -print0)
[ "$rc" = 0 ] && echo "ok no packed ELF names a build-machine path (/home/, the source tree, the workdir, the build tree)"
return $rc
}

# engine #1411: a packed ELF must resolve its libraries inside the payload or from the system loader, and it
# must carry no absolute RUNPATH at all. ldd alone does not decide this: the HSA runtime is dlopened by
# name, so a payload can look clean to ldd while the running process maps a builder's toolchain - measured
# on the c22 install, where /proc/<llama-server>/maps held 64 entries under /work-c22 that arrive through
# RUNPATH entries naming a toolchain nobody would keep. Both halves are checked: the statically declared
# paths (readelf) and what the loader actually does (ldd), plus the runtime set the dlopen needs present.
runtime_selfcontained() { # runtime_selfcontained <unpacked tree> -> FAIL lines, rc 1
local t=$1 tcanon f n abs rc=0
tcanon=$(readlink -f "$t" 2>/dev/null || printf '%s' "$t")
while IFS= read -r -d '' f; do
pack_is_elf "$f" || continue
abs=$(readelf -d "$f" 2>/dev/null | grep -E '\((RUNPATH|RPATH)\)' | tr ':[]' '\n\n\n' |
grep -E '^/' | head -3 | tr '\n' ' ')
[ -z "$abs" ] || {
printf 'FAIL %s declares an absolute RUNPATH/RPATH entry: %s\n' "$(basename "$f")" "$abs"
rc=1
}
# what the loader resolves, with nothing inherited: each resolved path is either in the payload or a
# system directory. A library it cannot find at all is fine here (a package may depend on the distro
# for libc etc.); one it finds outside both is the leak.
while IFS= read -r so; do
# ldd prints the path it was given, unnormalised ("<tree>/bin/../../outside/libx.so"), so the
# prefix test below has to run on the real path or a library that leaves the payload through a
# relative rpath reads as "inside" it (found by tests/pack_check_test.sh's escaping arm)
so=$(readlink -f "$so" 2>/dev/null || printf '%s' "$so")
case $so in
/lib/*|/lib64/*|/usr/lib/*|/usr/lib64/*|"$tcanon"/*) ;;
*) printf 'FAIL %s resolves %s, which is neither in the payload nor a system directory\n' \
"$(basename "$f")" "$so"; rc=1 ;;
esac
done < <(LD_LIBRARY_PATH='' ldd "$f" 2>/dev/null | awk '/=> \// {print $3}' | sort -u)
done < <(find "$t" -type f -print0)
# and the set the dlopen needs has to be there, not merely resolvable from somewhere else
local hsa; hsa=$(find "$t" -type d -path '*/hrx/llama/bin' | head -1)
if [ -n "$hsa" ]; then
for n in libhsa-runtime64.so.1 libhsa-amd-aqlprofile64.so.1 librocprofiler-register.so.0; do
[ -f "$hsa/$n" ] || { printf 'FAIL the HSA runtime set ships without %s (libhsa opens it at run time; engine #1411)\n' "$n"; rc=1; }
done
n=$(find "$t" -path '*/hrx/llama/bin/librocm_sysdeps_*.so*' -type f | wc -l)
[ "$n" -ge 8 ] || { printf 'FAIL the HSA runtime set ships %s rocm_sysdeps entries, fewer than the 8 the runtime needs\n' "$n"; rc=1; }
fi
[ "$rc" = 0 ] && echo "ok every packed ELF resolves inside the payload or the system loader, declares no absolute RUNPATH, and the HSA runtime set it dlopens ships beside it"
return $rc
}
38 changes: 38 additions & 0 deletions scripts/package-linux.sh
Original file line number Diff line number Diff line change
Expand Up @@ -143,6 +143,44 @@ bad=$(pack_build_rpaths "$stage" "$build")
hygiene=$(pack_artifact_hygiene "$stage")
[ -z "$hygiene" ] || { echo "artifact hygiene:"; echo "$hygiene"; exit 1; }

# engine #1411: the HSA runtime set ships beside the backend, because the backend opens it BY NAME at run
# time (ggml-hrx-dmabuf.cpp:29, dlopen("libhsa-runtime64.so.1")) and the runtime opens the aqlprofile agent
# the same way - so ldd sees none of it and a payload without the set resolves it from wherever the machine
# happens to have one. That is what the c22 install does: 64 files mapped out of a builder's /work-c22
# toolchain, which nobody would keep. The set is TheRock's, taken from the toolchain THIS build used, and
# staged last so it lands as built (scripts/pack-elf.sh has already localized and stripped ours by now, and
# NOTICE-BINARY promises these unmodified).
# Staging it here rather than in mk-plugin-package.sh is what makes both payloads self-contained: the engine
# package does its own overlay, and the Lemonade + onebit package carries this tarball as 1bit/.
if [ -f "$stage/hrx/llama/bin/llama-server" ]; then
tc=$(sed -n 's/^ONEBIT_HRX_TOOLCHAIN[^=]*=//p' "$build/CMakeCache.txt" 2>/dev/null | head -1)
[ -n "$tc" ] || { echo "no ONEBIT_HRX_TOOLCHAIN in $build/CMakeCache.txt: the HSA runtime set cannot be staged beside the backend (engine #1411)" >&2; exit 1; }
core=$(ls -d "$tc"/lib/python3*/site-packages/_rocm_sdk_core/lib 2>/dev/null | head -1)
[ -n "$core" ] || { echo "no _rocm_sdk_core/lib under $tc (ONEBIT_HRX_TOOLCHAIN): the HSA runtime set cannot be staged (engine #1411)" >&2; exit 1; }
# The set is the transitive NEEDED closure of the three roots, not every file in rocm_sysdeps/lib: the
# runtime links what it needs (elf, drm, drm_amdgpu, numa, z, zstd, bz2, liblzma, the profiling shim),
# and the toolchain's directory also holds libraries nothing here opens. Copying the closure keeps the
# payload to the set the owner named and keeps it self-maintaining - the same rule mk-plugin-package.sh
# applies to the engine package, and ldd is the arbiter in both.
rt=0
search=$core:$core/rocm_sysdeps/lib
for root in libhsa-runtime64.so.1 libhsa-amd-aqlprofile64.so.1 librocprofiler-register.so.0; do
f=$(ls "$core"/$root 2>/dev/null | head -1)
[ -n "$f" ] || { echo "the toolchain at $tc has no $root: the HRX backend opens it by name at run time, so the payload would resolve one from outside the install (engine #1411)" >&2; exit 1; }
install -m 0644 "$f" "$stage/hrx/llama/bin/$root"
rt=$((rt + 1))
while IFS= read -r so; do
case "$so" in /lib/*|/lib64/*|/usr/lib/*|/usr/lib64/*) continue ;; esac # the distro's own
[ -f "$stage/hrx/llama/bin/$(basename "$so")" ] && continue
install -m 0644 "$so" "$stage/hrx/llama/bin/$(basename "$so")"
rt=$((rt + 1))
done < <(LD_LIBRARY_PATH=$search ldd "$f" 2>/dev/null | awk '/=> \// {print $3}' | sort -u)
done
missing_set=$(LD_LIBRARY_PATH=$stage/hrx/llama/bin ldd "$stage/hrx/llama/bin/libhsa-runtime64.so.1" 2>/dev/null | awk '/not found/ {print $1}' | tr '\n' ' ')
[ -z "${missing_set// /}" ] || { echo "the staged HSA runtime still needs $missing_set beside it (engine #1411)" >&2; exit 1; }
echo " hrx/llama/bin: + the HSA runtime set ($rt files: the runtime, its aqlprofile agent, the profiling shim and rocm_sysdeps)"
fi

cp "$src/LICENSE" "$src/NOTICE" "$src/LICENSE-BINARY" "$src/NOTICE-BINARY" "$stage/"
# the stock Lemonade PPA's lemond (Ubuntu 26.04) runs our HRX llama-server through lemonade-hrx
# (docs/lemonade.md, "Ubuntu 26.04: the PPA's lemond"); install-lemonade-ppa.sh wires its drop-in
Expand Down
Loading
Loading