fix: clean-room verification found a VACUOUS negative control in a merged safety artifact — and five more (AFD-048) - #195
Merged
Conversation
…rged safety artifact — and five more (AFD-048)
A fresh-context verifier was run over the three PRs merged today, per the standing every-~2-features
rule. It found six real defects. The headline is the worst kind: the campaign's own anti-vacuity
discipline failed inside the artifact that most loudly advertised it.
F1 CRITICAL — AFD-044's negative control (i) was VACUOUS. It perturbed wx by +0.05, re-ticked, and
reported the changed fold (1348 -> 2000) as proof "the fold tracks its input". The second tick ran
ON THE SAME INSTANCE. The rate loop carries an integrator, so the second call differs REGARDLESS of
input — ticking the IDENTICAL input twice also gives 1348 -> 2000. Independently reproduced. And on
a FRESH instance the documented perturbation moves the fold by ZERO, because this operating point is
mixer-SATURATED (m1=m2=0, m4=1). A build that ignored its input entirely would have passed.
FIXED: fresh instance, and perturb wy, which demonstrably moves the fold.
MEASURED SCOPE, now printed rather than assumed: only 4 of 18 input scalars move the fold on a
single tick. The whole quaternion, position and velocity — 10 scalars — are INERT even at ±5.0.
F3 — the "COMPLETELY DIFFERENT ROUTE" claim was FALSE. Both legs execute the same compiled falcon
code; the rate#tick bodies are instruction-identical apart from a uniform data rebase. The oracle
catches composition/lowering/ABI faults and CANNOT catch a falcon arithmetic bug. The header claimed
exactly the property it lacked.
F4 — the oracle's window was 0.07%. int(sum*1000) meant composed=1348 accepted any reference in
[1.348,1.349); the verifier falsified the thrust setpoint to 0.75 and it still passed. FIXED: the
fold is now raw f32 BITS. Both escapes now FAIL — the thrust falsification differs by a SINGLE LSB
(…774 vs …773), which the old fold rounded away.
F5 — build-and-verify.sh EXITED 1 as shipped: it preflighted four tools and four artifacts but not
the wasmtime PYTHON module its oracle imports, failing at step 5 with a bare ModuleNotFoundError
AFTER composition succeeded — so it read as an oracle mismatch, not a missing dependency.
FIXED: preflight + requirements.txt.
F6 — check-drift.sh had two vacuities of its own, in the tool written to prevent this: all-absent
printed "no drift" and exited 0 (green on a toolchain never inspected; now exit 2), and a
single-source tool scored "ok" (now "single-source (not compared)"). Both negative-controlled, and
the ok path proven still REACHABLE. Third issue RECORDED NOT FIXED: tools lacking --version (spar,
ordeal) read as absent though the binary IS on PATH — they drop silently out of the comparison.
F10 — documentation overclaims, one of them mine THIS SESSION:
- AFD-042's "ABSENT on every loop tick … has never found it" is FALSE. The board WAS attached
2026-08-25 and jess captured 77,034 bytes of live telemetry (AFD-037, 4ee2a5f), re-parsed
independently at 1,021 CRC-valid frames. I repeated that false absolute verbally today while
correcting a different error. The defensible claim is scoped: no falcon code has EXECUTED on the
RT1176.
- hardware/silicon/README.md said "this is the actual chip" while everything under it ran on Renode.
- README.md labelled the Renode node "HIL emulation". No hardware is in that loop.
- AFD-037's TITLE still carried "ZERO CRC failures", withdrawn as vacuous in the same PR's second
commit; the withdrawal never reached the title.
Also: AFD-046 said func_4, it is func_3; and "DISCHARGED" now says plainly that the discharge is a
successful LINK, not an execution.
WHAT THE VERIFIER CONFIRMED: m4-matrix reproduced BYTE-IDENTICALLY, its "0 undefined" is not vacuous,
its zero-skips claim is substantiated, and the CI gate is genuinely non-empty. The composition, the
run and the agreement are all real — the CONTROL around them was weak.
FOR THE USER (F2, not fixable here): main has NO required status checks. deletion,
non_fast_forward, pull_request and required_signatures are set; required_status_checks is not, and
required_approving_review_count is 0. The five green checks are ADVISORY.
All three oracles re-run green after the fixes. rivet validate PASS.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…control # Conflicts: # artifacts/findings.yaml
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Clean-room verification found a vacuous negative control inside a merged safety artifact
A fresh-context verifier was run over the three PRs merged today, per the standing every-~2-features rule. It found six real defects. The headline is the worst kind: the campaign's own anti-vacuity discipline failed inside the artifact that most loudly advertised it.
F1 (CRITICAL) — AFD-044's negative control measured statefulness, not input sensitivity
It perturbed
wxby +0.05, re-ticked, and reported the changed fold (1348 → 2000) as proof "the fold tracks its input". The second tick ran on the same instance. The rate loop carries an integrator, so the second call differs regardless of input:Row D is the proof. Row B is worse: the documented perturbation moves the fold by zero on a fresh instance, because this operating point is mixer-saturated (
m1=m2=0, m4=1) and absorbs small moves. A build that ignored its input entirely would have passed this control.Fixed: fresh instance, and perturb
wy, which demonstrably moves the fold.And the oracle now prints its own measured scope instead of implying generality: only 4 of 18 input scalars move the fold on a single tick. The entire quaternion, position and velocity — 10 scalars — are inert even at ±5.0. This differential cannot detect a miscompile confined to attitude/position/velocity handling.
F3 — the "completely different route" claim was false
Both legs execute the same compiled falcon code; disassembly shows the
rate#tickbodies instruction-identical apart from a uniform data rebase. The oracle catches composition, lowering and ABI-marshalling faults and cannot catch a falcon arithmetic bug — both sides would share it. The header claimed exactly the property it lacked.F4 — the oracle's window was 0.07%
int(sum * 1000.0)meantcomposed=1348accepted any reference in[1.348, 1.349). The verifier falsified the reference's thrust setpoint to 0.75 and the gate passed.Fixed: the fold is now the raw f32 bits. Both escapes now fail — and the thrust falsification differs by a single LSB (
…774vs…773), which the old fold rounded away.F5 — the script exited 1 as shipped
It preflighted four tools and four supplier artifacts, but not the wasmtime Python module its oracle imports — failing at step 5 with a bare
ModuleNotFoundErrorafter the composition succeeded, so it read as an oracle mismatch rather than a missing dependency. Fixed: preflight +requirements.txt.F6 — the drift checker had two vacuities of its own
In the tool written to prevent exactly this: all-absent printed "no drift" and exited 0 — a green verdict on a toolchain it never inspected (now exit 2); and a single-source tool scored "ok" — one value compared against nothing (now "single-source (not compared)"). Both negative-controlled, and the
okpath proven still reachable.Recorded but not fixed:
verreads versions via--version, so tools lacking that flag (spar,ordeal) read as absent though the binary is on PATH — they drop silently out of the comparison. That's the inverse and more dangerous direction.F10 — documentation overclaims, one of them mine today
4ee2a5f), which the verifier independently re-parsed at 1,021 CRC-valid MAVLink v1 frames, single sysid. I repeated that false absolute verbally today while correcting a different error. The defensible claim is the scoped one: no falcon code has ever executed on the RT1176.hardware/silicon/README.mdasserted "this is the actual chip" while everything under it ran on Renode and the board is marked "(ordered)".README.mdlabelled the Renode node "HIL emulation". There is no hardware in that loop.Also: AFD-046 said
func_4; it isfunc_3. And "the obligation was DISCHARGED" now says plainly that the discharge is a successful link, not an execution.What the verifier confirmed
Not all bad news. The m4-matrix result reproduced byte-identically from the recorded input, its "0 undefined" is not vacuous (an empty object fails the post-link 5/5 recount), its zero-skips claim is substantiated, and the CI gate is genuinely non-empty. The composition, the run and the agreement are all real — it was the control around them that was weak.
For you — F2, not fixable in a PR
mainhas no required status checks. The ruleset carriesdeletion,non_fast_forward,pull_requestandrequired_signatures; there is norequired_status_checks, andrequired_approving_review_countis 0. The five green checks are advisory — a red board would not block a merge. jess has been asserting "confirm CI success before merging" as a process rule and honouring it by hand, while the repo would not have enforced it.All three oracles re-run green after the fixes.
rivet validatePASS.🤖 Generated with Claude Code
https://claude.ai/code/session_017EKh2onUHpWSUiAEqWxfy2