Skip to content

fix(hardware): let a flashed board actually run, and stop refusing big twin results - #43

Merged
w1ne merged 2 commits into
mainfrom
fix/twin-evidence-projection
Aug 21, 2026
Merged

fix(hardware): let a flashed board actually run, and stop refusing big twin results#43
w1ne merged 2 commits into
mainfrom
fix/twin-evidence-projection

Conversation

@w1ne

@w1ne w1ne commented Aug 20, 2026

Copy link
Copy Markdown
Contributor

The differential from #41 had never consumed a live desk bundle. Running it against a real NUCLEO-H563ZI found two defects standing in the way. Both are fixed here, and the differential now produces a real verdict on real silicon.

1. A flashed board was never started

probe-rs download leaves the core halted. Nothing reset it, so the board held the exact bytes we had just written and executed none of them. Every observation that followed read a silent port and recorded serial marker was not observed — a false negative indistinguishable from firmware that does not work.

Measured on the bench: serial-capture read 0 bytes after download, and matched immediately after an explicit probe-rs reset.

Fixed in lib/probe.sh and lib/probe-flash.ps1 together — leaving Windows behind would keep the bug where it is hardest to notice.

2. A legitimate twin run was refused before it could be read

Reading the simulator's result.json and persisting it shared one 128 KB budget. A 10M-step H563 run publishes 280 KB, of which 106 KB is the inspect register dump — a blob no behavior decision reads. The run was refused at read time, so the twin lane failed and there was no bundle to diff against a board.

The read budget is now separate and generous (the simulator is our own tool, writing to a path we already identity-check). What we keep is still bounded.

The persisted projection drops members by size, and names every one in evidence_omitted. An allowlist was the first attempt and was wrong twice: it discarded diagnostics, whose redaction the suite proves, and it would discard every field the simulator adds next. A bundle that quietly loses part of its source is the thing this format exists to prevent.

End to end, on real silicon

step result
hardware run (twin + flash + serial) PASS, receipt dbfad84d…
hardware run (twin only) PASS, receipt 16a22498…
hardware diff agree, exit 0

Both sides authenticity: verified; twin model_observed, desk hardware_observed; one paired behavior; same artifact digest 3beaf379…. Target pinned by register read — DBGMCU_IDCODE 0x10016484, the same IDCODE core/configs/chips/stm32h563.yaml cites from its June silicon capture.

Verification

hardware-* 293 pass / 0 fail / 0 skipped. public-install, hardware-cli, hardware-public-docs, skills-verify-all, release-evidence-contract, harness, skills-inventory, prefix-unit all exit 0.

The over-drop in my first projection attempt was caught by the existing suite (surrogate adapter output normalizes into the real evidence contract), which is exactly what it is for.

Two notes for whoever runs this next

  • target.probeSerial must hold the full probe-rs id (0483:374e:0021…), not the bare serial. With the bare serial the resolver correctly returns nothing and hardware plan blocks — the field name invites the wrong value.
  • The plan digest binds the evidence directory, so plan and run must use the same --out.

w1ne added 2 commits August 20, 2026 23:51
…g twin results

Two defects stood between the twin-desk differential and its first live
measurement. Both were found by running it against a real NUCLEO-H563ZI.

`probe-rs download` leaves the core HALTED. Nothing reset it, so the board held
the exact bytes we had just written and executed none of them. Every observation
that followed read a silent port and recorded "serial marker was not observed" —
a false negative indistinguishable from firmware that does not work. Measured:
serial-capture read 0 bytes after download and matched immediately after an
explicit reset. Fixed in lib/probe.sh and lib/probe-flash.ps1 together, because
leaving Windows behind would keep the bug where it is hardest to see.

Reading the simulator's result.json and PERSISTING it shared one 128 KB budget.
A 10M-step H563 run publishes 280 KB, of which 106 KB is the `inspect` register
dump — a blob no behavior decision reads. The run was refused at READ time, so
the twin lane failed and there was no bundle to diff against a board. The read
budget is now separate and generous (the simulator is our own tool writing to a
path we identity-check); what we KEEP is still bounded.

The persisted projection drops members by SIZE and names every one in
`evidence_omitted`. An allowlist was the first attempt and was wrong twice: it
discarded `diagnostics`, whose redaction the suite proves, and it would discard
every field the simulator adds next. A bundle that quietly loses part of its
source is the thing this format exists to prevent.

End to end on real silicon: hardware run PASS (receipt dbfad84d), twin-only run
PASS (receipt 16a22498), hardware diff verdict "agree", exit 0 — both sides
authenticity "verified", twin model_observed, desk hardware_observed, one paired
behavior, same artifact digest 3beaf379.
A banner printed once at startup could never be observed. The flash stage reset
the board, the board printed, and only afterwards did serial-capture open the
port — so `LABWIRED_OK` went to nobody and came back as "marker was not
observed", which reads exactly like firmware that does not work.

A physical serial observation now starts the target itself, from inside the
capture, after the port is open and flushed. This is not an arbitrary command
hook: serial-capture takes `--reset-chip` and `--reset-probe` and the only thing
it will run is `probe-rs reset` for that explicit chip and probe, resolved in
the shell rather than passed in.

A failed start is reported as a failed start. `started_target` is false and
`start_error` carries the reason, and the status becomes `blocked` rather than
`failed` — otherwise a launch we broke is indistinguishable from a board that
had nothing to say, which is the same false negative one level up.

The flags are attached only when the profile has a flash stage and a probe: if
nothing of ours put the firmware there, nothing of ours resets it. Both cases
are asserted, so their absence is a property and not an accident.

Measured on a NUCLEO-H563ZI before it was unplugged: capturing `LABWIRED_OK`,
previously unobservable, returned matched=true, 1219 bytes, started_target=true,
with `H563 blink start / LABWIRED_OK` in the excerpt.
@w1ne

w1ne commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

Third defect fixed in 3a124b9: the port was opened after the target booted.

The PR body listed this as still open. It is now closed.

A banner printed once at startup could never be observed: the flash stage reset the board, the board printed, and only afterwards did serial-capture open the port. LABWIRED_OK went to nobody and came back as marker was not observed — indistinguishable from firmware that does not work.

A physical serial observation now starts the target itself, from inside the capture, after the port is open and flushed. Not an arbitrary command hook: serial-capture takes --reset-chip and --reset-probe, and the only thing it will run is probe-rs reset for that explicit chip and probe, resolved in the shell rather than passed in.

A failed start is reported as a failed start — started_target: false, start_error carrying the reason, and status blocked rather than failed. Otherwise a launch we broke is indistinguishable from a board with nothing to say, which is the same false negative one level up.

The flags attach only when the profile has a flash stage and a probe: if nothing of ours put the firmware there, nothing of ours resets it. Both cases are asserted, so the absence is a property rather than an accident.

Measured on the NUCLEO-H563ZI, capturing the previously-unobservable boot marker:

{"matched":true,"marker":"LABWIRED_OK","bytes_captured":1219,
 "excerpt":" OFF\r\nLED ON\r\nLED ON\r\nH563 blink start\r\nLABWIRED_OK\r",
 "status":"hardware_observed","started_target":true}

Gates: hardware-* 293 pass / 0 fail / 0 skipped; public-install, hardware-cli, hardware-public-docs, skills-verify-all, harness, skills-inventory, prefix-unit, release-evidence-contract all exit 0.

Not re-run end to end. The H5 was unplugged partway through — the ST-LINK V3 is gone and a J-Link OB is now present, answering as a locked nRF52840 (erase_all permission denied). The resolver blocked the run correctly, since the pinned probe is absent. The agree verdict in the PR body stands as measured with a continuously-emitted marker; the boot-only marker is proven at the capture level above but has not yet gone through plan → run → diff in one pass. That needs the H5 reconnected.

@w1ne

w1ne commented Aug 20, 2026

Copy link
Copy Markdown
Contributor Author

End to end with the boot-only marker, on the reconnected H5 — closed.

The PR body noted the boot marker was proven at the capture level but had not gone through plan → run → diff in one pass. It has now.

Same probe and port as before (0483:374e:0021…/dev/cu.usbmodem11103), identity re-confirmed by register read: DBGMCU_IDCODE 0x10016484.

step result
hardware run desk, marker LABWIRED_OK PASS, receipt 0c50f71c…
hardware run twin PASS, receipt ebfcc6a5…
hardware diff agree, exit 0

Both sides authenticity: verified, twin model_observed, desk hardware_observed, one paired behaviour, artifact 3beaf379….

The desk evidence records the fix being exercised — the captured argv is

serial-capture /dev/cu.usbmodem11103 115200 LABWIRED_OK 15
  --reset-chip STM32H563ZITx --reset-probe 0483:374e:002100174741500220383733

with matched: true in the captured stdout. The boot banner that was previously emitted to nobody is now inside the authenticated bundle.

Live failure modes also checked against real bundles, not fixtures: no desk bundle → exit 4; good bundles → exit 0.


One design question, not a change.

A desk bundle supplied with a wrong receipt also exits 4, identical to no bundle at all. I started to separate them — a refused bundle arguably being invalid (exit 2), since a lane that reads desk-unavailable as "no board on this runner today, skip" would also skip a bundle whose contents do not match its receipt.

Three existing tests pin the current behaviour deliberately, one of them named "an edited desk record breaks bundle authentication instead of changing the verdict". That intent is clear — unauthenticated evidence must never influence a verdict, so it degrades uniformly regardless of cause. I reverted my change rather than override a decision that is tested on purpose.

Worth deciding explicitly: should absence and failed authentication share an exit code? If the answer is no, the reason codes already differ (evidence_absent vs evidence_unverified) and only the verdict mapping needs to move.

Suite unchanged at 293 pass / 0 fail / 0 skipped.

@w1ne
w1ne merged commit 3bd99a3 into main Aug 21, 2026
14 checks passed
@w1ne
w1ne deleted the fix/twin-evidence-projection branch August 21, 2026 10:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant