Filed unassigned and ungraded by the #12782 dev (session session_01PfaSTikked61BkcsB5Rn69) while repairing the arrival-depth heading one line below this one. Not fixed there: #12782 was ruled SERIAL against its own neighbour (#12783), so this is recorded rather than folded in.
Measured
mode_report builds all three distributions with one loop, and the filter is shared:
awk -v f="$field=" '{ ... if (v ~ /^[0-9]+$/ && v > 0) print v; ... }'
v > 0 is then described differently by each of the three headings it feeds:
| bucket |
heading |
does the heading describe the filter? |
waited |
acquisition wait, over runs that waited at all (seconds) |
yes — "waited at all" IS waited > 0 |
held |
lock hold, over runs that acquired (seconds) |
no — a run that acquired and held under a second did acquire |
depth |
(repaired on #12782) |
not applicable — the recorded depth cannot be 0 |
Read on this container 2026-08-28, the live ledger at /tmp/os-heavy-verify.lock.ledger: 69 records, every one of them carrying an acquiring outcome (62 command-exit, 7 batch-last-exit, zero queue-timeout, zero lock-unusable). --report prints lock hold, over runs that acquired (seconds): n=66. The three missing rows are real and they are not degenerate:
outcome=command-exit waited=0 held=0 depth=1 rc=0 label=cd /home/user/objectstac…
outcome=command-exit waited=0 held=0 depth=1 rc=0 label=node scripts/check-bash32…
outcome=command-exit waited=78 held=0 depth=1 rc=0 label=node scripts/check-bash…
The third one waited 78 seconds for the lock and then held it for under a second. It is a run that acquired by any reading, and the bucket headed "over runs that acquired" does not contain it.
Why it matters
Same family as #12782 and one line away from it: a heading that misdescribes the population its filter actually admits. The direction is the awkward one — the excluded rows are the FASTEST holds, so p50 and p90 are biased upward, and the bias grows exactly as the fleet gets better at holding the lock briefly. It also makes the two n values in the report non-comparable in a way nothing says out loud: records: 69 in the header, n=66 in the hold bucket, and no line explains the gap.
The choice this needs
It is a choice, not a mechanical repair, which is why it is filed rather than fixed:
- Fix the heading — say "over runs that held it for at least a second", matching what
waited already does honestly. Presentation only; every existing number stays comparable.
- Fix the filter — admit
held = 0 for this bucket. Reads better, but it MOVES n, p50 and p90 for every ledger, past and future, so a figure quoted from an older report stops matching a re-run over the same rows.
⛔ Suggestions, not a ruling. Note that whichever is chosen, waited's heading is already correct and should not be changed to match.
Refs
Filed unassigned and ungraded by the #12782 dev (session
session_01PfaSTikked61BkcsB5Rn69) while repairing the arrival-depth heading one line below this one. Not fixed there: #12782 was ruled SERIAL against its own neighbour (#12783), so this is recorded rather than folded in.Measured
mode_reportbuilds all three distributions with one loop, and the filter is shared:v > 0is then described differently by each of the three headings it feeds:waitedacquisition wait, over runs that waited at all (seconds)waited > 0heldlock hold, over runs that acquired (seconds)depthRead on this container 2026-08-28, the live ledger at
/tmp/os-heavy-verify.lock.ledger: 69 records, every one of them carrying an acquiring outcome (62command-exit, 7batch-last-exit, zeroqueue-timeout, zerolock-unusable).--reportprintslock hold, over runs that acquired (seconds): n=66. The three missing rows are real and they are not degenerate:The third one waited 78 seconds for the lock and then held it for under a second. It is a run that acquired by any reading, and the bucket headed "over runs that acquired" does not contain it.
Why it matters
Same family as #12782 and one line away from it: a heading that misdescribes the population its filter actually admits. The direction is the awkward one — the excluded rows are the FASTEST holds, so
p50andp90are biased upward, and the bias grows exactly as the fleet gets better at holding the lock briefly. It also makes the twonvalues in the report non-comparable in a way nothing says out loud:records: 69in the header,n=66in the hold bucket, and no line explains the gap.The choice this needs
It is a choice, not a mechanical repair, which is why it is filed rather than fixed:
waitedalready does honestly. Presentation only; every existing number stays comparable.held = 0for this bucket. Reads better, but it MOVESn,p50andp90for every ledger, past and future, so a figure quoted from an older report stops matching a re-run over the same rows.⛔ Suggestions, not a ruling. Note that whichever is chosen,
waited's heading is already correct and should not be changed to match.Refs
os-verify-lock --reportlabels its arrival-depth column "waiters already ahead" while the recorded value counts the caller itself — an uncontended fleet reads as one-deep #12782 — the arrival-depth heading, same function, same defect classos-verify-lockledger lives in the container's /tmp, so--reportcan never answer the question it was built for — "where does hold time go" is scoped to one shift, and says so nowhere #12783 — the ledger's population is scoped to one container's uptime, which also bears on how much any of these percentiles can claim