update-aqua-checksum: build the commit payload on disk, never on a command line - #84
Merged
Merged
Conversation
…mmand line
An aqua-bump branch in ossein-kernel came out of the workflow with a bumped
limen pin, no checksum for it, and a red pipeline that never reached its
first recipe:
▶ lint: limen
aqua failed … package_version=v0.2.1 error="checksum is required"
The checksum was missing because the job that writes it had already died:
/usr/bin/jq: Argument list too long
Converging the baseline across a limen release rewrites the whole .limen/
tree. In that repository it is ~120 kB of configuration, ~160 kB once
base64-encoded, and it was passed to jq as a single `--argjson` argument.
Linux caps one argument at 128 kB (MAX_ARG_STRLEN), so the payload was 1.2x
over and the mutation was never built. The same data then went through argv
a second time as `curl -d "${payload}"`, so fixing only the first site would
have moved the failure a line further down.
Three sites, all now reading from disk: the per-file accumulation takes the
base64 through `--rawfile` instead of `--arg "$(base64 …)"`, the mutation is
assembled with `--slurpfile` instead of `--argjson`, and the request body is
uploaded with `--data-binary @file` instead of `-d`. Only short scalars —
a path, a branch, a commit oid — remain on a command line, so the payload no
longer has a size ceiling at all rather than a larger one.
This is why the same bump was green in limen and red everywhere else. The
convergence step is skipped in this repository by design (the working tree
IS the next baseline), so limen's own aqua-bump commits one small file and
never approaches the limit. The failure needs a repository that both pins
limen and has drifted far enough for the convergence to be large — which is
every governed repository, the moment its limen gap is wide enough.
Verified against a simulated 173 kB dirty tree, 21 entries: 20 additions
round-trip byte-identically through base64 and the deletion is carried,
with the array that used to go into one argument measuring 232 kB. The
overflow itself cannot be reproduced on darwin, which has no per-argument
limit; on Linux it is the difference between 232 kB and 128 kB.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Signed-off-by: closer-claudio <claudio@farcloser.world>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The failure
ossein-kernel#43 came out of this workflow with a bumped limen pin, no checksum for it, and a pipeline that died before its first recipe:
Because the job that writes the checksum had already died:
Why
Converging the baseline across a limen release rewrites the whole
.limen/tree. Measured in ossein-kernel:.limen/**+aqua-checksums.json--argjsonargumentMAX_ARG_STRLENThe same data then went through argv a second time as
curl -d "${payload}", so fixing only the first site would have moved the failure one line down.The fix — three sites, all now reading from disk
--arg contents "$(base64 -w0 …)"--rawfile contents--argjson additions "${additions}"--slurpfile additionscurl -d "${payload}"curl --data-binary @fileOnly short scalars stay on a command line — a path, a branch, a commit oid. The payload no longer has a size ceiling at all, rather than a larger one.
Why limen never saw it
The convergence step is skipped in this repository by design:
So limen's own aqua-bump commits one small file (
aqua-checksums.json, ~18 kB of base64) and never approaches the limit — #79 is green for that reason, not because the mechanism works. The failure needs a repository that both pins limen and has drifted far enough for the convergence to be large, which is every governed repository once its limen gap is wide enough.Verification
Simulated a 173 kB dirty tree of 21 entries (20 modified/added, 1 deleted):
just lintandjust testgreenThe overflow itself cannot be reproduced on darwin, which has no per-argument limit — the old logic passes locally at 232 kB. On Linux it is the difference between 232 kB and 128 kB.