Skip to content

Name the command decision 0010 step 3 asks for, and what a recount of the two extractions found - #8

Merged
marcos-mendez merged 5 commits into
mainfrom
docs/0010-name-the-comparison
Sep 29, 2026
Merged

marcos-mendez merged 5 commits into
mainfrom
docs/0010-name-the-comparison

Conversation

@marcos-mendez

@marcos-mendez marcos-mendez commented Sep 28, 2026 •

Copy link
Copy Markdown
Contributor

Merges second. Keel-Linux/buildtasks#11 has to land first. This note
amends 0010 step 3 to name bt-layer-measure and describes the repeatable
--control-again and the union floor it subtracts; none of that is true of
anything on a default branch until that pull request merges, and a decision
naming a command that does not exist yet is the shape of defect this very
amendment is about.

Closes #5. Paired with Keel-Linux/buildtasks#11, which adds the command.

Decision 0010 step 3 required a three-build comparison for every component
taken out of the shared tree and pointed at "this three-build comparison",
meaning the narrative of the experiment earlier in the note. Nothing was
runnable: the capture was /root/measure/build.sh on the build host, a scratch
file in root's home directory on one machine, and the comparison was ad hoc
shell that was never committed. Step 3 now names bt-layer-measure and states
what a pull request extracting a component has to quote from it.

0010 is amended in place. The decision is unchanged, only its step 3 was
unexecutable, and 0014 and 0015 are being written by others.

The part that is not just naming a command

Both extractions' captures are still on the build host. Recounting them on the
tab separated path<TAB>hash key the tool uses, rather than on sha256sum's
whitespace-separated default, gives different numbers from the ones the note and
the two pull requests reported:

pair mariadb of which ./var/lib/mysql/** postgresql of which ./var/lib/postgresql/**
control vs unit 182 173 17 6
control vs control again 179 173 13 6
control again vs a third control 182 173 16 6

Three findings, all reproducible from those captures.

One control pair is not a noise floor. Against control vs control again
alone, mariadb leaves three files unaccounted for:
./usr/share/doc/turnkey-mariadb-19.0/changelog,
./var/lib/dpkg/info/turnkey-mariadb-19.0.md5sums and ./var/lib/dpkg/status.
All three sit inside the other control pair's floor, so they are install-time
noise and the conclusion of zero survives, but against the union of both pairs,
which is an argument the pull request did not make. Two builds of one recipe do
not differ by a fixed set: 179 in one pair, 182 in another.

The postgresql extraction was not at zero. One difference survives either
control pair and their union, ./etc/postgresql/17/main/postgresql.conf:

-listen_addresses = '::1,127.0.0.1'   # set by conf.d/main of keel-postgresql: see the comment there
+listen_addresses = '::1,127.0.0.1'   # set by the postgresql component: see the comment in its conf script

The value is identical and only the trailing comment moved, because the
component's conf script writes a different comment than conf.d/main did. It is
harmless, and it is precisely the class of thing the measurement exists to
surface: a file the component writes, differing because the component wrote it.
It was reported as zero.

A path inside the noise floor was never evidence of noise. 173 of the
mariadb differences are ./var/lib/mysql/**, subtracted because of where they
are. That directory holds mysql/global_priv and mysql/user.*, and all four
of global_priv.MAD, global_priv.MAI, global_priv.frm and user.frm differ
in the control-unit pair. The component's build-time job includes deleting
accounts. So the measurement subtracted, as noise, every file the thing being
measured would have shown up in.

Neither extraction is reopened. Both layers are built, published and
boot-tested, and the postgresql residue is a comment. What changes is the
standard for the next component: the block bt-layer-measure attribute prints
is what an extraction is reviewed on, and a residue is named and explained in
the pull request rather than absent from it.

Changes

  • docs/decisions/0010-composable-units.md: step 3 names the command; two new
    sections, "The command, and what a pull request has to quote" and "What a
    recount found, 2026-09-28"; the original measurement table is marked
    superseded in place rather than rewritten, because what the note is now about
    is that nobody could check it.
  • docs/traps.md: two entries beside "A path with a space in it", which is the
    same shape of defect. "One control pair is not a noise floor" and "A differing
    path inside the noise floor is not a noise difference".

Kept in step with the command

Keel-Linux/buildtasks#11 changed after this was approved: the first version
cancelled a state path's differences against the controls by line and byte
position, which review showed is itself the defect. 0010 and the trap entry now
describe the command as it is on that pull request's final head: three
controls in the worked example, the preconditions attribute refuses on
(including the anchor to the committed state path list), the verdicts
noise, overlapping, real and not sampled and what clears each, and the
first real run, 0 attributable and FAIL on 32 real Aria header clock fields
open in Keel-Linux/buildtasks#13. main is merged in; docs/traps.md
conflicted only because both sides added an entry at the same place, and both
are kept.

Test plan

  • The recount is reproducible from /root/tree-{m,p}-*.sha256 on the build
    host; the commands are in the commit message and the note
  • No em dashes, per the brief's typography rule
  • Every behaviour of bt-layer-measure the note states checked against
    buildtasks#11 at its final head
  • Nothing else to gate: this repository has no CI

Step 3 of the order of work required a three-build comparison per extracted
component and pointed at the narrative of the experiment, not at anything
runnable. Until now nothing was: the capture was a scratch file in root's home
directory on the build host and the comparison was never committed, so two
extractions were approved on a number only their author could produce. The
command is now bt-layer-measure in buildtasks; step 3 names it and says what a
pull request has to quote from its output.

A recount of the captures still on the build host, on the tab separated
path/hash key the tool uses, also corrects what those numbers were worth:

- The mariadb pairs are 182, 179 and 182 differing files, not 173 three times.
  Against the single control pair the measurement used, three files are left
  unaccounted for; they are inside the other control pair's floor, so the
  conclusion of zero survives against the union of both pairs, reached by an
  argument the pull request did not make.
- The postgresql extraction was not at zero. One difference survives either
  control pair and their union: the trailing comment on the listen_addresses
  line of /etc/postgresql/17/main/postgresql.conf, written by the component's
  conf script instead of conf.d/main. Harmless, and exactly what the
  measurement exists to surface.
- 173 of the mariadb differences are ./var/lib/mysql/**, subtracted by path.
  That directory holds mysql/global_priv and mysql/user.*, all four of which
  differ in the control-unit pair, and the component's build-time job includes
  deleting accounts. The measurement was blind where the accounts live.

Neither extraction is reopened: both layers are built, published and
boot-tested. What changes is the standard for the next one.

Amends 0010 in place rather than adding a note, since the decision is
unchanged and only its step 3 was unexecutable.

Closes #5
Both were hit recounting the mariadb and postgresql extractions on 2026-09-28,
and both are the same shape as the entry already here: a measurement that
quietly loses a record rather than failing.

One control pair is not a noise floor. The mariadb control pairs differ by 179
files and by 182 depending on which two builds are compared, so subtracting one
pair answers whether a difference is in that pair, not whether it is noise.
Three mariadb files and one postgresql file survived the pair the measurements
used.

A differing path inside the noise floor is not a noise difference. 173 of the
179 are /var/lib/mysql/**, which holds mysql/global_priv and mysql/user.*, and
all four of those differ in the control-unit pair of a component whose
build-time job includes deleting accounts.
@marcos-mendez
marcos-mendez force-pushed the docs/0010-name-the-comparison branch from b13bcbb to f5a7fb7 Compare September 28, 2026 04:18
@marcos-mendez

Copy link
Copy Markdown
Contributor Author

Review

I reproduced the recount independently from /root/tree-{m,p}-*.sha256 on the build host, with my own script keyed on path<TAB>hash and not with bt-layer-measure, then again through bt-layer-measure attribute. Every number in the new table and all three findings hold exactly:

pair mariadb of which ./var/lib/mysql/** postgresql of which ./var/lib/postgresql/**
control vs unit 182 173 17 6
control vs control again 179 173 13 6
control again vs a third control 182 173 16 6

The mariadb residue against the first control pair is exactly ./usr/share/doc/turnkey-mariadb-19.0/changelog, ./var/lib/dpkg/info/turnkey-mariadb-19.0.md5sums and ./var/lib/dpkg/status, and it is zero against the union. ./etc/postgresql/17/main/postgresql.conf survives all three control pairs and the union. The quoted diff of that line is right byte for byte: keel-postgresql@b9565ea:conf.d/main:61 wrote # set by conf.d/main of keel-postgresql: see the comment there and unit-postgresql/conf:52 writes # set by the postgresql component: see the comment in its conf script. All four of global_priv.MAD, .MAI, .frm and user.frm differ in the control-unit pair and sit inside every control floor.

The history is clean. Both commits sit on 3483b5a, the current tip of main, with nothing from any other branch; only the two files the description claims; both pass the repository's commit-msg and pre-commit hooks when replayed against them. The author/committer date skew is consistent with the disclosed reset and recommit, and nothing bypassed a check silently.

Three things.


HIGH 1. The new trap entry prescribes a procedure the command it names cannot perform

docs/traps.md, "One control pair is not a noise floor", the Fix paragraph: "bt-layer-measure attribute in buildtasks, and when a residue survives, capture a third control and subtract the union of both control pairs."

bt-layer-measure attribute takes exactly one --control-again and offers no way to pass a second (bin/layer-measure-lib:542 on Keel-Linux/buildtasks#11). The union cannot be asked for. On the real captures the attributable count swings 3 → 0 for mariadb and 4 → 1 for postgresql purely on which pair is named as the floor, so the first run of the command this note now makes mandatory will print FAIL (3 attributable) for the very component this note concludes is at zero.

So the record here states a lesson and points at an instrument that has not learned it. Either say plainly in the Fix that the union is not yet mechanised and has to be done by hand, or land the repeatable --control-again in buildtasks#11 first. As written a reader follows the entry and cannot do what it says.

MEDIUM 2. The superseded marker does not cover the subsection that carries the number

docs/decisions/0010-composable-units.md: the blockquote is placed immediately above ### The 173, named and explained, and its own wording, "the counts in this table do not", reads as pointing at the table above it. The subsection below is titled with the number the recount contradicts, and its first row reads 167 | /var/lib/mysql/** where the recount gives 173 for the control-unit pair. That subsection is the one anyone will cite later for "here are the noise files and why they are noise", and it is reachable directly from a heading link with the marker out of view.

Marking the table you replaced and leaving the numbers standing is the right call, and the reason given for it is the right reason. It just needs one more line, under that heading, saying the breakdown is superseded too.

MEDIUM 3. The recount does not carry the command that produced it

The note's new rule is that a claim of zero which does not carry the block that produced it is not reviewable. The recount table is the first claim made under that rule and it carries no command: the test plan says "the commands are in the commit message and the note", and neither contains one; both describe the key (path<TAB>hash) and stop there.

This is not pedantry. bt-layer-measure cannot read those captures: they are bare sha256sum files, not captures in its format. I had to convert them into capture directories before attribute would run on them. So the recount was necessarily produced by another piece of uncommitted ad hoc shell, which is the exact thing this note exists to end. Paste the command, or say in the note that the historical captures predate the format and name the conversion.


LOW 4. The union is offered as a rescue without saying it is the weaker test

Both here and in the trap entry, "the residue is zero against the union of both control pairs" is presented as recovering the conclusion. It does, but a union floor subtracts strictly more, so zero against the union proves less than zero against one pair would have: mariadb's floors are 179 and 182 and their union is 182. The note is honest that this is "an argument the pull request did not make"; one clause saying the union is more permissive, not more rigorous, would stop a future reader reading it the other way.


I verified by running: the full recount for both families across all pairs, the three residues against each floor and the union, the four blind mariadb files, the quoted postgresql.conf diff against keel-postgresql history, and the branch and hook checks above. I inferred, without running: how a reader navigates to the superseded subsection.

Warning: no blocking issue. The recount is correct, the record is honest and the history is clean. Finding 1 is worth settling before this lands, because it is the one line in the two pull requests where the written standard and the shipped tool disagree.

The trap entry prescribed the union of both control pairs and named a command
that could not be asked for it. bt-layer-measure now takes --control-again as
many times as there are control captures and subtracts the union, so the entry
names that and stops describing a procedure the instrument cannot perform.

The superseded marker covered the table and not the subsection below it, whose
own heading is one of the wrong numbers and whose first row describes 167 files
of /var/lib/mysql that nobody opened. That subsection now carries its own
marker, saying which of it survives: the explanations for the paths it names,
and not the counts.

The recount was the first claim made under this note's own rule and it carried
no command. The historical captures are bare sha256sum output, two spaces
between hash and path, so bt-layer-measure cannot read them and the recount had
to convert them first; that conversion was more uncommitted shell, which is the
thing this note exists to end. The command is now in the note in full,
including the key function that moves the path into field 1, and the note says
that captures taken from here on are in the tool's format and need none of it.

Both the note and the trap entry now say that a union floor is more permissive
than a single pair rather than more rigorous. It subtracts strictly more, so
zero against the union proves less than zero against one pair would have; it is
the right test because its answer does not depend on which two builds were
named first.
@marcos-mendez

Copy link
Copy Markdown
Contributor Author

All four settled, in 814e007.

1. The entry prescribed what the command could not do. Fixed in the
command rather than in the prose: bt-layer-measure attribute now takes
--control-again as many times as there are control captures and subtracts the
union, and prints what each single pair would have said. The Fix paragraph
names that instead of describing a procedure by hand.

2. The marker did not cover the subsection. ### The 173, named and explained now carries its own, saying which of it survives: the explanations
for the paths it names, not the counts, and not the 167 | /var/lib/mysql/**
row, which describes files nobody opened.

3. The recount carried no command. You are right that this was the first
claim made under the note's own rule and that it failed it, and right about why
it could not simply cite the tool: those captures are bare sha256sum output,
so bt-layer-measure cannot read them and the recount was another piece of
uncommitted shell. The note now carries the whole of it, including the key
function that moves the path into field 1 and separates it with a tab, and says
that captures taken from here on are in the tool's format and need none of it.

4. The union is the weaker test. Said in both places: it subtracts strictly
more, so zero against the union proves less than zero against one pair would
have, and it is the right test because its answer does not depend on which two
builds were named first, not because it is the harder one.

Since you reviewed this, the tool has been run for real: four captures of
mariadb, three of them controls. 0 attributable against every single pair and
against the union, and 167 state paths that differ, of which 32 come out real
and 33 overlapping. The 32 are almost all *.MAI, and in 30 of them the
differing value increases strictly in capture order, so they are the Aria index
header clock whose carry three controls did not span. Details in
Keel-Linux/buildtasks#11. Nothing in that changes this note, which claims zero
attributable and gets it; it does mean the sentence about /var/lib/mysql/**
being install-time state of the same kind as the allow-list entries is now
supported by bytes somebody read rather than by the path's name.

@marcos-mendez

Copy link
Copy Markdown
Contributor Author

Re-review

All four findings are settled, and I checked each one rather than taking the summary.

1. The trap entry no longer prescribes what the tool cannot do. The Fix now says to capture at least three controls and pass every one, because --control-again is repeatable and the floor is the union over every pair. I verified that against the implementation: attribute takes the controls as an array, measure_floor_positions unions every unordered pair, and the block prints what each single pair on its own would have said. The entry and the instrument now agree.

2. The ### The 173, named and explained subsection carries its own marker, and it does the harder thing of saying which parts survive: the explanations for the paths named, not the counts, and it calls out that the row for /var/lib/mysql/** describes files nobody opened. That is the right split, and a reader arriving at that heading from a link now sees it.

3. The recount carries its command, and the command works. I ran the published key and differing functions verbatim against /root/tree-m-*.sha256 and /root/tree-p-*.sha256, changing only the directory:

m-control vs m-unit      182     (note says 182)
m-control vs m-control2  179     (note says 179)
m-control2 vs m-control3 182     (note says 182)
p-control vs p-unit       17     (note says 17)
residue against the first pair:
  ./usr/share/doc/turnkey-mariadb-19.0/changelog
  ./var/lib/dpkg/info/turnkey-mariadb-19.0.md5sums
  ./var/lib/dpkg/status

Every number and the exact residue. Saying plainly that the historical captures predate the tool's format, that the conversion was itself uncommitted shell, and that captures from here on need none of it, is the honest version of this and it applies the note's own rule to the note.

4. The union correction is right, and it inverts the argument correctly in both places. "A union floor is more permissive, not more rigorous: it subtracts strictly more than either pair, so zero against the union proves less than zero against one pair would have. It is the right test because it is the one whose answer does not depend on which two builds were named first, not because it is the harder one." That last clause is the part that matters: it gives the union a reason that survives the concession, rather than leaving the reader to wonder why a weaker test is being preferred.

I re-confirmed the underlying facts as well: the recount table, the three mariadb files inside the other control pair's floor, ./etc/postgresql/17/main/postgresql.conf surviving every pair and the union, and the four global_priv and user.frm files differing in the control-unit pair and sitting inside every floor.

One thing worth knowing, which is not a change request here because it belongs to the other pull request: I have blocked Keel-Linux/buildtasks#11 on two points, one of which is that a noise verdict can still admit a planted value wherever the controls vary at random. It does not affect anything this note asserts, and it does not affect the recount, which is a hash comparison. But this note now makes the block that command prints the standard a component extraction is reviewed on, so the standard is only as good as that command. If the fix there changes what the block looks like, this note's description of it may need a line.

History is clean: three commits on the tip of main, only the two files, all three passing the repository's commit-msg and pre-commit hooks when replayed, and no forbidden strings.

Verified by running: the published recount command verbatim against both capture families, the multi-control and union behaviour in bt-layer-measure, and the branch and hook checks. Inferred, without running: nothing material.

Approve.

marcos-mendez and others added 2 commits September 29, 2026 03:22
0010 and the trap entry described the first version of the command: one
--control-again in the worked example, and a floor subtracted per line and per
byte offset in which a clock "touches the same line or offset in both pairs
and cancels". Review of Keel-Linux/buildtasks#11 showed that scheme is the
defect, not the fix: position cancels against position with the content
thrown away, so an account added on the line a clock moves on came back as
noise. The command no longer works that way, and a note that makes its block
the standard has to describe the block it actually prints.

- The worked example takes three controls and passes every one, since the
  floor is the union over every pair.
- The preconditions attribute enforces are named, including the anchor to
  the committed state path list and --state-paths-override.
- The block's contents are listed as they now are, and it carries the
  sentence that a PASS does not mean the layer ships no secret it should not.
- The verdicts are described as they are: noise only where the controls pin
  the varying field, overlapping for anything else at a varying position and
  for every binary, real where the controls do not vary, and waivers that
  clear only overlapping.
- The first real run is recorded: 0 attributable, FAIL on 32 real state
  paths that are the Aria header wall clock, open in Keel-Linux/buildtasks#13.

The trap entry's Fix now says not to cancel by position, instead of
prescribing it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SfWQScmDZ94KCS5DMYrfe6
…mparison

Conflict in docs/traps.md: both sides added an entry after "A path with a space in it". Both kept, this branch's two first.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01SfWQScmDZ94KCS5DMYrfe6
@marcos-mendez

Copy link
Copy Markdown
Contributor Author

bb0484d brings 0010 and the trap entry in line with buildtasks#11's final head: three controls in the example, the enforced preconditions, and the pinned-field verdicts in place of the position cancelling the first version did. 52d1a86 merges main; the traps.md conflict was two independent additions, both kept. Still merges after buildtasks#11.

@marcos-mendez
marcos-mendez merged commit 7c1f9b3 into main Sep 29, 2026
marcos-mendez added a commit that referenced this pull request Sep 29, 2026
Name the command decision 0010 step 3 asks for, and what a recount of the two extractions found
@marcos-mendez
marcos-mendez deleted the docs/0010-name-the-comparison branch September 29, 2026 06:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Decision 0010 step 3 asks for a three-build comparison and names no command

1 participant