fix(hardware): let a flashed board actually run, and stop refusing big twin results - #43
Conversation
…g twin results Two defects stood between the twin-desk differential and its first live measurement. Both were found by running it against a real NUCLEO-H563ZI. `probe-rs download` leaves the core HALTED. Nothing reset it, so the board held the exact bytes we had just written and executed none of them. Every observation that followed read a silent port and recorded "serial marker was not observed" — a false negative indistinguishable from firmware that does not work. Measured: serial-capture read 0 bytes after download and matched immediately after an explicit reset. Fixed in lib/probe.sh and lib/probe-flash.ps1 together, because leaving Windows behind would keep the bug where it is hardest to see. Reading the simulator's result.json and PERSISTING it shared one 128 KB budget. A 10M-step H563 run publishes 280 KB, of which 106 KB is the `inspect` register dump — a blob no behavior decision reads. The run was refused at READ time, so the twin lane failed and there was no bundle to diff against a board. The read budget is now separate and generous (the simulator is our own tool writing to a path we identity-check); what we KEEP is still bounded. The persisted projection drops members by SIZE and names every one in `evidence_omitted`. An allowlist was the first attempt and was wrong twice: it discarded `diagnostics`, whose redaction the suite proves, and it would discard every field the simulator adds next. A bundle that quietly loses part of its source is the thing this format exists to prevent. End to end on real silicon: hardware run PASS (receipt dbfad84d), twin-only run PASS (receipt 16a22498), hardware diff verdict "agree", exit 0 — both sides authenticity "verified", twin model_observed, desk hardware_observed, one paired behavior, same artifact digest 3beaf379.
A banner printed once at startup could never be observed. The flash stage reset the board, the board printed, and only afterwards did serial-capture open the port — so `LABWIRED_OK` went to nobody and came back as "marker was not observed", which reads exactly like firmware that does not work. A physical serial observation now starts the target itself, from inside the capture, after the port is open and flushed. This is not an arbitrary command hook: serial-capture takes `--reset-chip` and `--reset-probe` and the only thing it will run is `probe-rs reset` for that explicit chip and probe, resolved in the shell rather than passed in. A failed start is reported as a failed start. `started_target` is false and `start_error` carries the reason, and the status becomes `blocked` rather than `failed` — otherwise a launch we broke is indistinguishable from a board that had nothing to say, which is the same false negative one level up. The flags are attached only when the profile has a flash stage and a probe: if nothing of ours put the firmware there, nothing of ours resets it. Both cases are asserted, so their absence is a property and not an accident. Measured on a NUCLEO-H563ZI before it was unplugged: capturing `LABWIRED_OK`, previously unobservable, returned matched=true, 1219 bytes, started_target=true, with `H563 blink start / LABWIRED_OK` in the excerpt.
|
Third defect fixed in The PR body listed this as still open. It is now closed. A banner printed once at startup could never be observed: the flash stage reset the board, the board printed, and only afterwards did A physical serial observation now starts the target itself, from inside the capture, after the port is open and flushed. Not an arbitrary command hook: A failed start is reported as a failed start — The flags attach only when the profile has a flash stage and a probe: if nothing of ours put the firmware there, nothing of ours resets it. Both cases are asserted, so the absence is a property rather than an accident. Measured on the NUCLEO-H563ZI, capturing the previously-unobservable boot marker: Gates: Not re-run end to end. The H5 was unplugged partway through — the ST-LINK V3 is gone and a J-Link OB is now present, answering as a locked nRF52840 ( |
|
End to end with the boot-only marker, on the reconnected H5 — closed. The PR body noted the boot marker was proven at the capture level but had not gone through Same probe and port as before (
Both sides The desk evidence records the fix being exercised — the captured argv is with Live failure modes also checked against real bundles, not fixtures: no desk bundle → exit 4; good bundles → exit 0. One design question, not a change. A desk bundle supplied with a wrong receipt also exits 4, identical to no bundle at all. I started to separate them — a refused bundle arguably being Three existing tests pin the current behaviour deliberately, one of them named "an edited desk record breaks bundle authentication instead of changing the verdict". That intent is clear — unauthenticated evidence must never influence a verdict, so it degrades uniformly regardless of cause. I reverted my change rather than override a decision that is tested on purpose. Worth deciding explicitly: should absence and failed authentication share an exit code? If the answer is no, the reason codes already differ ( Suite unchanged at 293 pass / 0 fail / 0 skipped. |
The differential from #41 had never consumed a live desk bundle. Running it against a real NUCLEO-H563ZI found two defects standing in the way. Both are fixed here, and the differential now produces a real verdict on real silicon.
1. A flashed board was never started
probe-rs downloadleaves the core halted. Nothing reset it, so the board held the exact bytes we had just written and executed none of them. Every observation that followed read a silent port and recordedserial marker was not observed— a false negative indistinguishable from firmware that does not work.Measured on the bench:
serial-captureread 0 bytes afterdownload, and matched immediately after an explicitprobe-rs reset.Fixed in
lib/probe.shandlib/probe-flash.ps1together — leaving Windows behind would keep the bug where it is hardest to notice.2. A legitimate twin run was refused before it could be read
Reading the simulator's
result.jsonand persisting it shared one 128 KB budget. A 10M-step H563 run publishes 280 KB, of which 106 KB is theinspectregister dump — a blob no behavior decision reads. The run was refused at read time, so the twin lane failed and there was no bundle to diff against a board.The read budget is now separate and generous (the simulator is our own tool, writing to a path we already identity-check). What we keep is still bounded.
The persisted projection drops members by size, and names every one in
evidence_omitted. An allowlist was the first attempt and was wrong twice: it discardeddiagnostics, whose redaction the suite proves, and it would discard every field the simulator adds next. A bundle that quietly loses part of its source is the thing this format exists to prevent.End to end, on real silicon
hardware run(twin + flash + serial)dbfad84d…hardware run(twin only)16a22498…hardware diffagree, exit 0Both sides
authenticity: verified; twinmodel_observed, deskhardware_observed; one paired behavior; same artifact digest3beaf379…. Target pinned by register read —DBGMCU_IDCODE 0x10016484, the same IDCODEcore/configs/chips/stm32h563.yamlcites from its June silicon capture.Verification
hardware-*293 pass / 0 fail / 0 skipped.public-install,hardware-cli,hardware-public-docs,skills-verify-all,release-evidence-contract,harness,skills-inventory,prefix-unitall exit 0.The over-drop in my first projection attempt was caught by the existing suite (
surrogate adapter output normalizes into the real evidence contract), which is exactly what it is for.Two notes for whoever runs this next
target.probeSerialmust hold the full probe-rs id (0483:374e:0021…), not the bare serial. With the bare serial the resolver correctly returns nothing andhardware planblocks — the field name invites the wrong value.planandrunmust use the same--out.