Why
./run.sh drift-checks the state currently being built. That is the right check
for a branch — it catches a figure moving after the freeze, inside the same pull
request — but it deliberately stops guarding a state once the work moves past it.
Re-checking Minimal from a commit that is building Secure would be
meaningless: the code has legitimately moved on.
So nothing currently guards the numbers this repository publishes. Every figure
in the README cost table is a claim about a tagged commit, and each of those
commits is immutable — but the environment they are measured in is not. The
build pulls a digest-pinned cross image and a digest-pinned collector, and
FetchContent pins a SolidSyslog commit. Any of those moving, deliberately or
by a future bump, silently invalidates every published number with nothing
failing.
The guarantee worth having is narrow and checkable:
checking out tag T and running still reproduces measurements/<T>.csv
Scope
A CI job that enumerates the tags, checks each one out, runs it, and asserts its
frozen CSV still reproduces. A matrix over git tag -l keeps it automatic as
tags are added.
It does not need to run on every pull request — the tags do not change when a
branch does. Scheduled, plus on-demand, is enough, and that also makes it a
canary for the pinned images drifting.
Worth deciding as part of this: whether a mismatch should fail or report. A
toolchain bump that moves every figure by a few bytes is not a defect, it is
news — and the honest response is to re-freeze and say so in the table, not to
pretend the old number still holds.
Notes
Two things already make this cheap. run.sh takes TAG=<state>, so verifying
one tag is a single invocation. And the figures are reproducible across hosts:
the same build measured 348,216 / 2,584 byte-identically on a Windows
workstation and a Linux CI runner, so a mismatch means something really did
change rather than the environment being noisy.
Raised from the Minimal work — see #10, where the branch self-check was reworked
to follow the state under construction.
Why
./run.shdrift-checks the state currently being built. That is the right checkfor a branch — it catches a figure moving after the freeze, inside the same pull
request — but it deliberately stops guarding a state once the work moves past it.
Re-checking
Minimalfrom a commit that is buildingSecurewould bemeaningless: the code has legitimately moved on.
So nothing currently guards the numbers this repository publishes. Every figure
in the README cost table is a claim about a tagged commit, and each of those
commits is immutable — but the environment they are measured in is not. The
build pulls a digest-pinned cross image and a digest-pinned collector, and
FetchContentpins a SolidSyslog commit. Any of those moving, deliberately orby a future bump, silently invalidates every published number with nothing
failing.
The guarantee worth having is narrow and checkable:
Scope
A CI job that enumerates the tags, checks each one out, runs it, and asserts its
frozen CSV still reproduces. A matrix over
git tag -lkeeps it automatic astags are added.
It does not need to run on every pull request — the tags do not change when a
branch does. Scheduled, plus on-demand, is enough, and that also makes it a
canary for the pinned images drifting.
Worth deciding as part of this: whether a mismatch should fail or report. A
toolchain bump that moves every figure by a few bytes is not a defect, it is
news — and the honest response is to re-freeze and say so in the table, not to
pretend the old number still holds.
Notes
Two things already make this cheap.
run.shtakesTAG=<state>, so verifyingone tag is a single invocation. And the figures are reproducible across hosts:
the same build measured 348,216 / 2,584 byte-identically on a Windows
workstation and a Linux CI runner, so a mismatch means something really did
change rather than the environment being noisy.
Raised from the Minimal work — see #10, where the branch self-check was reworked
to follow the state under construction.