From 4bb2df21bfbab888d6d2903951a11c630e6f4860 Mon Sep 17 00:00:00 2001 From: agent Date: Sun, 8 Nov 2026 03:50:56 -0400 Subject: [PATCH] package: the binary drop is the third packing site, and it inherits its inputs' modes Follow-up to #1352 (closed by #1361), found by enumerating the packing sites instead of reasoning about the two already fixed: three scripts source scripts/pack-elf.sh and write an archive, and only two of them applied the mode rule. make-binary-drop.sh sources pack-elf.sh, writes a PUBLISHED tarball (docs/releases.md, "The public binary drop") -- neither helper called package-linux.sh fixed in #1361 weekly.sh the lemonade payload fold, fixed in #1360 pack-distro.sh normalises per payload pass; its own comment explains why it does not assert (it stages on the formatter's own copy, so the chmod cannot fail) Measured, on a real tree rather than argued: /home/bcloud/wt/rlk-build/hrx/llama/bin is 13 of 13 files at 0775 (-rwxrwxr-x), which is exactly what the drop stages with `cp -a` when built with --build; with --from-package it inherits whatever an already-published tarball had, including one packed before #1361/#1360 (c31's payload measured 111x0664 + 23x0775). So the drop now runs pack_normalise_modes and asserts with pack_writable_files before writing its archive, and docs/releases.md says so in the drop's own section. tests/pack_modes_test.sh grows the check whose absence let this reopen: it ENUMERATES the packing sites from the tree (sources pack-elf.sh + writes an archive) instead of listing them, requires a real invocation of pack_normalise_modes at every one, and requires pack_writable_files at the sites that pack a flat payload -- with pack-distro.sh the single documented exemption, recorded in the test so a second site that wants it has to edit the list. Three negative controls, run and restored: normalise removed from the drop -> rc 1 naming it; only the assert removed -> rc 1 naming it; the exempt site losing its normalise calls -> rc 1. The first version of this check MATCHED A COMMENTED-OUT call and passed with the guard disabled -- caught by the negative control, which is why the pattern now anchors at the start of a line. tools/copyright.py --check rc 0. 0 s GPU, 0 s NPU. --- docs/releases.md | 7 +++++++ scripts/make-binary-drop.sh | 11 +++++++++++ tests/pack_modes_test.sh | 39 +++++++++++++++++++++++++++++++++++++ 3 files changed, 57 insertions(+) diff --git a/docs/releases.md b/docs/releases.md index 367c91937..39efafd4d 100644 --- a/docs/releases.md +++ b/docs/releases.md @@ -421,6 +421,13 @@ checks the release package runs: the NPU runtime compiled in, the kernels really nothing named like the private sources, both binary licenses present, every library found. It publishes nothing; the orchestrator publishes the tarball and adds its line to `config/binary-drop.config`. +Its file modes are its **own** decision, not its inputs': the drop stages with `cp -a`, so built with `--build` it would +carry that build tree's `0775` (measured: 13 of 13 files in a real `hrx/llama/bin` are `-rwxrwxr-x`), and built +`--from-package` from an older tarball it would carry whatever that one had. It therefore runs the same +`pack_normalise_modes` / `pack_writable_files` pair as the engine package before writing its archive, and +`tests/pack_modes_test.sh` asserts that **every** script which stages with `scripts/pack-elf.sh` and writes an archive +does the same. + To build against a drop, point CMake at it — a directory or the tarball's URL: ``` diff --git a/scripts/make-binary-drop.sh b/scripts/make-binary-drop.sh index 2cded6e34..3a634c08d 100755 --- a/scripts/make-binary-drop.sh +++ b/scripts/make-binary-drop.sh @@ -211,6 +211,17 @@ echo " MANIFEST: $(grep -cE '^[0-9a-f]{64} ' "$stage/MANIFEST") files, pins en [ -f "$stage/MANIFEST" ] || fail "MANIFEST was not written" [ $bad = 0 ] && echo " checks: ok" || die "checks failed, nothing written" +# ── the modes, then the tarball and its SHA256SUMS ────────────────────────────────────────────── +# A file mode in the drop is THIS script's decision, not its inputs' (engine#1352). That matters more here than in +# the two sites already fixed: --build stages straight out of a build tree, and measured on a real one +# (rlk-build/hrx/llama/bin) all 13 files are 0775, so a drop built that way ships 0775 binaries; --from-package +# inherits whatever an already-published tarball had, including one packed before #1361/#1360. Measured, not assumed: +# the same class shipped 134 of 195 files writable in c31 and c32. +pack_normalise_modes "$stage" +modes_bad=$(pack_writable_files "$stage") +[ -z "$modes_bad" ] || die "files still group/ or other-writable after normalising: +$modes_bad" + # ── the tarball and its SHA256SUMS ────────────────────────────────────────────────────────────── name=binary-drop-$version # Rebuild cleanly: the target is this script's own output, and zstd refuses an existing one. diff --git a/tests/pack_modes_test.sh b/tests/pack_modes_test.sh index 73f59a7d9..0be3f3f23 100755 --- a/tests/pack_modes_test.sh +++ b/tests/pack_modes_test.sh @@ -91,6 +91,45 @@ chmod 0755 "$pristine" check "a file staged below the normalisation is caught" "$(pack_writable_files "$stage" | cut -d' ' -f1)" "staged-after.json" check "and reports the mode a user would see" "$(pack_writable_files "$stage" | cut -d' ' -f2)" "664" +# The rule has to hold at EVERY packing site, not at the one being fixed. This is the check whose absence let the gap +# reopen: #1361 fixed package-linux.sh, #1360 fixed weekly.sh's lemonade fold, and make-binary-drop.sh -- a third +# script that sources this same library and writes a PUBLISHED tarball -- was missed by both until it was looked for. +# Enumerated from the tree rather than listed here, so a fourth site cannot be forgotten either. +# +# Two tiers, because the sites are not the same job: +# * every site that stages files must call pack_normalise_modes -- no exceptions, it cannot change behaviour; +# * the sites that pack a FLAT payload must also assert with pack_writable_files. +# pack-distro.sh is the one exemption and it is documented in that script (lines 63-74): it stages on the +# formatter's own copy, so the chmod cannot fail there, and it decided the reporter would be "decoration". This test +# records that decision instead of silently widening the rule; if a second site wants the exemption, someone has to +# edit the list below, which is the point. +flat="package-linux.sh weekly.sh make-binary-drop.sh" +echo "packing sites:" +for f in "$here"/../scripts/*.sh; do + grep -q 'pack-elf.sh' "$f" || continue # only scripts that stage files with this library + grep -qE '\btar -C' "$f" || continue # and actually write an archive + b=$(basename "$f") + printf ' %-28s ' "$b" + # A real invocation, not a mention: the first version of this check matched a COMMENTED-OUT call and passed on a + # site with the guard disabled -- the same "a check that cannot fail" defect the fixture half already caught once. + if ! grep -qE '^[[:space:]]*pack_normalise_modes' "$f"; then + echo "MISSING pack_normalise_modes"; fails=$((fails+1)); continue + fi + case " $flat " in + *" $b "*) + if grep -qE '^[[:space:]]*[a-z_]+=\$\(pack_writable_files' "$f"; then + echo "normalises and asserts" + else + echo "MISSING pack_writable_files (a flat payload must assert)"; fails=$((fails+1)) + fi ;; + *) echo "normalises (flat-payload assertion not required: see its own comment)" ;; + esac +done +# A floor, not an exact count: a fifth packing site that does carry the guard is a good change, and failing on it +# would teach the next person to edit the test instead of the packer. The per-site loop above is the real assertion. +sites=$(for f in "$here"/../scripts/*.sh; do grep -q 'pack-elf.sh' "$f" && grep -qE '\btar -C' "$f" && echo x; done | wc -l) +check "the enumeration found every known packing site (>= 3)" "$([ "$sites" -ge 3 ] && echo yes || echo no)" "yes" + rm -rf "$(dirname "$stage")" if [ "$fails" -ne 0 ]; then printf '%d check(s) failed\n' "$fails"; exit 1; fi echo "pack_modes: ok"