SYN-45 - Add a reference-board qualification matrix to CI by JeffMboya · Pull Request #75 · absmach/synth

JeffMboya · 2026-10-05T12:49:12Z

What type of PR is this?

This is a feature

What does this do?

Gates eight reference boards against qualification/matrix.toml, and lets check pick the router its stages use.

Which issue(s) does this PR fix/relate to?

Resolves #45. Records #98 and #51 as the causes four boards fail.

Have you included tests for your changes?

Yes: qualification_matrix.rs gates the corpus, and a second test keeps matrix.toml well formed.

Did you document any new/modified features?

Yes: docs/qualification-handoff.md covers statuses, the router requirement and what review automation cannot replace.

Notes

Rebuilt on current main; its boards and parts now live in #195, #196, #103 and #110. Measurements in a comment.

SammyOina · 2026-10-05T19:46:48Z

the actual boards required for qualification defined in issues need to be defined first

The manufacturing job failed on PR absmach#75 because its expectations had never been established — I could not run that stage without a KiCad install, which is why the job was continue-on-error and labelled a baseline. CI has now run it on KiCad 10.0.6 and the uploaded artifact answers the question: hello mfg=pass placement_and_diff_pair mfg=pass env_logger mfg=fail sensor_logger mfg=fail the four reference boards mfg=unknown (their source gate fails first) env_logger and sensor_logger pass source and DRC and still cannot produce a fab package, so one `expect` cannot carry both answers. The matrix gains `expect_fab`, consulted only on a --fab run, and those two now record `pass` for the gate and `fail` for manufacturing. Boards whose fab baseline is not yet known carry `expect_fab = "unestablished"`: the observed value is written to the artifact so the next run says what to record, but it is not compared, because asserting a guess is how a gate teaches people to ignore it. That makes the job safe to promote, so continue-on-error is gone and it blocks. The runner also could not say *why* manufacturing failed — `check` reports the stage status but not the export's stderr, and an export that bails early never reaches the evidence report. On a failure it now re-runs the export directly and records the refusal lines, which is the difference between a result a reviewer can act on and one that only says "fail". Both failing boards uniquely share bme680_env and spst_tactile, whose footprints are the likely cause, but that is inference rather than evidence and the next artifact will name it.

Two RP2350s on six layers, each on its own native USB, each pair at 90 ohm with `max_skew 0.15mm` and `couple tight`, and `match_length 0.5mm` on the USB class holding all four nets to one length. Two independent pairs on one board is what case 3 asks about. Adapted from the board in absmach#75, whose header carried a STATED LIMITATION saying SynthSpec had no length-matching or skew syntax so these budgets could not be declared. They can now, and these are the budgets that limitation stood in for. Two substitutions, both forced. The second endpoint is an RP2350 rather than the ESP32-C3 the design was written around: that part names a symbol and footprint absent from KiCad 10 and declares USB on two pins the module does not have (absmach#146), so a board using it cannot export. And the two keepouts are gone — a declared keepout reaches no exported board (absmach#142), and even when it does a KiCad keepout is unconditional and would forbid the pair it protects. What the README says plainly is where the budgets stop. Neither reaches KiCad (absmach#147) and neither is measured in this build, so the board states them and nothing yet holds it to them.

JeffMboya · 2026-10-18T05:48:53Z

Rebuilt on current main. The previous 46 commits sat 166 behind it, and
almost all of their content has landed or moved: the Class A boards are in
#196, the part pin maps in #195, the rotated-pad fix merged as #109 and #200,
and Class B is being done in the per-board shape by #103 and #110. The old
qualification/boards/class-b/*.synth used the flat layout that per-board
directories replaced. What survives is the matrix itself, and it is here.

The old branch is preserved at 28c7e84 if anything needs recovering.

Every expectation is measured

KiCad 10.0.6, KiCadRoutingTools v0.23.0 (grid_router 9399bfb), release build,
synth check <board> --json [--fab] --router kicad-routing-tools:

board	class	status	gate	manufacturing
`hello`	A	experimental	fail (drc `unknown`)	fail
`env_logger`	A	experimental	fail (drc fail)	fail
`sensor_logger`	A	experimental	fail (drc fail)	fail
`placement_and_diff_pair`	C	experimental	fail (drc fail)	fail
`ref_ldo_sensor`	A	unsupported	fail (source fail)	unknown
`ref_crystal_mcu`	A	unsupported	fail (source fail)	unknown
`ref_usb_cdc`	B	unsupported	fail (source fail)	unknown
`secure_tracker`	D	unsupported	fail (source fail)	unknown

Nothing clears the gate, for four distinct reasons

1. The declared design rules never reach the engine. All three real boards
route completely — zero open nets, 206/143/60 segments, 25/17/11 vias — and
then fail kicad-cli pcb drc on their own board's rules:

env_logger               6 clearance errors, actual 0.1652-0.1944 mm vs 0.2 mm declared
sensor_logger            5 clearance errors, actual 0.1652-0.1950 mm, plus 1 undersized via
placement_and_diff_pair  5 vias at 0.56 mm vs 0.6 mm minimum, plus 4 dangling-via warnings

The exported .kicad_dru contains three rules, all of them manufacturer
geometry:

(rule "manufacturer copper to board edge"  (constraint edge_clearance (min 0.3mm)))
(rule "manufacturer copper to NPTH hole"   (constraint edge_clearance (min 0.2mm)) ...)
(rule "manufacturer copper to hole"        (constraint hole_clearance (min 0.2mm)))

The per-netclass clearance and the minimum via diameter live in the
.kicad_pcb board setup, which the external engine does not read, so it
routes to the global clearance_min 0.127 floor instead. This looks like the
same boundary as #147.

2. The engine exits 1 on an empty board. hello is
board "hello" { layers 2 }. The route stage crashes, so DRC reports
unknown, which is not a pass. That is #98, still open.

3. Manufacturing refuses on unreviewed parts. Every board whose source
passes reports production_status: "untrusted" with reviewed: false against
each part. That is #51.

4. The four imported designs fail the source gate — which is what they are
in the corpus to prove. Undefined pins, required pins unconnected and
valueless passives, each named in expect_source_codes.

One test change worth calling out

the_matrix_is_well_formed asserted that some row declares expect = "pass",
which contradicted its own doctrine three assertions earlier: "a positive
board's gate can fail at DRC with the source stage clean". Since nothing
passes today, that assertion would have forced the matrix to claim a pass
nobody observed. Positive and negative are now read off status, which is the
claim, rather than off expect, which is the measurement.

One regression caught before this went up

The commit that adds --router originally gave it
default_value_t = ExternalRouter::FreeRouting and forwarded it on every
child invocation. That default was right when this branch was written; main
has since moved every command's default to kicad-routing-tools. The effect
was that check quietly told its children to use freerouting, overriding the
engine selected through KICAD_ROUTING_TOOLS_REPO — so a stub router
installed through the environment came back as "no Java runtime found", and
native_verification_gate::release_gate::a_clean_tool_lets_the_gate_pass
failed on this branch while passing on main.

--router now has no default. Unset means "do not mention it", so the stages
resolve their own engine. Both affected targets now match clean main
exactly: native_verification_gate 26/0, and release_overrides 6 passed /
5 failed with the same five names as main.

The remaining local failures — release_overrides (5) and golden_svg (1) —
were measured on untouched upstream/main in the same tree and fail
identically there.

Two RP2350s on six layers, each on its own native USB, each pair at 90 ohm with `max_skew 0.15mm` and `couple tight`, and `match_length 0.5mm` on the USB class holding all four nets to one length. Two independent pairs on one board is what case 3 asks about. Adapted from the board in absmach#75, whose header carried a STATED LIMITATION saying SynthSpec had no length-matching or skew syntax so these budgets could not be declared. They can now, and these are the budgets that limitation stood in for. Two substitutions, both forced. The second endpoint is an RP2350 rather than the ESP32-C3 the design was written around: that part names a symbol and footprint absent from KiCad 10 and declares USB on two pins the module does not have (absmach#146), so a board using it cannot export. And the two keepouts are gone — a declared keepout reaches no exported board (absmach#142), and even when it does a KiCad keepout is unconditional and would forbid the pair it protects. What the README says plainly is where the budgets stop. Neither reaches KiCad (absmach#147) and neither is measured in this build, so the board states them and nothing yet holds it to them.

SammyOina · 2026-10-20T20:35:06Z

I found three issues that should be addressed before merging:

[P1] The qualification workflow path filter is incomplete. .github/workflows/qualification.yml does not trigger for changes to root Cargo.toml, Cargo.lock, or rust-toolchain.toml. Those files can change the release output without running the qualification matrix. Please include the relevant root/toolchain files in qualification_paths.
[P1] Artifact hashes are recorded but never validated. qualification_matrix.rs records schematic, PCB, and BOM hashes and uploads them, but does not compare them against a committed baseline. An accidental fixture/output change can therefore pass as long as the status and diagnostics remain unchanged. Please add versioned expected hashes (or a committed manifest comparison), or remove the claim that these hashes detect drift.
[P2] Diagnostic matching permits unrelated failures. The test only checks that every expected source diagnostic is present; it allows additional codes. An unsupported board could acquire a new regression and still pass while one expected error remains. Please compare the observed and expected code sets exactly, or document that the list is intentionally a minimum contract.

Two RP2350s on six layers, each on its own native USB, each pair at 90 ohm with `max_skew 0.15mm` and `couple tight`, and `match_length 0.5mm` on the USB class holding all four nets to one length. Two independent pairs on one board is what case 3 asks about. Adapted from the board in absmach#75, whose header carried a STATED LIMITATION saying SynthSpec had no length-matching or skew syntax so these budgets could not be declared. They can now, and these are the budgets that limitation stood in for. Two substitutions, both forced. The second endpoint is an RP2350 rather than the ESP32-C3 the design was written around: that part names a symbol and footprint absent from KiCad 10 and declares USB on two pins the module does not have (absmach#146), so a board using it cannot export. And the two keepouts are gone — a declared keepout reaches no exported board (absmach#142), and even when it does a KiCad keepout is unconditional and would forbid the pair it protects. What the README says plainly is where the budgets stop. Neither reaches KiCad (absmach#147) and neither is measured in this build, so the board states them and nothing yet holds it to them.

rodneyosodo

LGTM. The router forwarding is the right shape: --router is Option (unset means “don't mention it”, so an env/default selection survives), and child_args takes clap's own value name rather than RouterEngine::as_str — which is the freerouting vs free-routing trap. Stage budgets via SYNTH_CHECK_*_TIMEOUT_SECS with the old defaults preserved is a clean fix, and the qualification test is genuinely #[ignore]d so the --ignored job is not vacuous.

Only red is Lint and Build, and that is the repo-wide rustls-webpki 0.103.15 advisory (RUSTSEC-2026-0381/0382) — main is red for the same reason and #248 fixes it. Nothing to change here; rebase after #248 and the check clears.

Two RP2350s on six layers, each on its own native USB, each pair at 90 ohm with `max_skew 0.15mm` and `couple tight`, and `match_length 0.5mm` on the USB class holding all four nets to one length. Two independent pairs on one board is what case 3 asks about. Adapted from the board in absmach#75, whose header carried a STATED LIMITATION saying SynthSpec had no length-matching or skew syntax so these budgets could not be declared. They can now, and these are the budgets that limitation stood in for. Two substitutions, both forced. The second endpoint is an RP2350 rather than the ESP32-C3 the design was written around: that part names a symbol and footprint absent from KiCad 10 and declares USB on two pins the module does not have (absmach#146), so a board using it cannot export. And the two keepouts are gone — a declared keepout reaches no exported board (absmach#142), and even when it does a KiCad keepout is unconditional and would forbid the pair it protects. What the README says plainly is where the budgets stop. Neither reaches KiCad (absmach#147) and neither is measured in this build, so the board states them and nothing yet holds it to them.

JeffMboya · 2026-10-21T08:35:33Z

Path filter takes Cargo.lock and rust-toolchain.toml; diagnostics now an exact set; hash claim removed. 8b0b692, e56df8e, 3b41fe9.

…using it Completing the W5500's pin map renamed its digital supply from `vcc` to `vdd`, pin 28, and three fixtures kept connecting `vcc`. The pin no longer exists, so each of them reported `E-SYNTH-COMP-002` for an undefined pin and `E-SYNTH-CONNECT-002` for the supply that never arrived. CI stayed green throughout, which is the part worth noting. The ERC fixture harness asserts that a `<CODE>__name.synth` file emits the code it is named for and allows any others, so `E-SYNTH-RESET-001` and `E-SYNTH-SPI-002` kept firing and nothing failed -- while the boards they fire on had become different boards, with an unpowered Ethernet controller and two extra errors each. A fixture that no longer isolates its own case stops being evidence for it. Both codes are gone now and the named ones remain. The two `U1.vcc` left in the SPI fixture are a `ch340g`, which does declare `vcc`. The harness weakness is the same one the qualification matrix had on absmach#75, where `expect_source_codes` was a subset check: a declared list that is a floor rather than a set lets a case decay quietly. Worth fixing there too, and out of scope here.

Two RP2350s on six layers, each on its own native USB, each pair at 90 ohm with `max_skew 0.15mm` and `couple tight`, and `match_length 0.5mm` on the USB class holding all four nets to one length. Two independent pairs on one board is what case 3 asks about. Adapted from the board in absmach#75, whose header carried a STATED LIMITATION saying SynthSpec had no length-matching or skew syntax so these budgets could not be declared. They can now, and these are the budgets that limitation stood in for. Two substitutions, both forced. The second endpoint is an RP2350 rather than the ESP32-C3 the design was written around: that part names a symbol and footprint absent from KiCad 10 and declares USB on two pins the module does not have (absmach#146), so a board using it cannot export. And the two keepouts are gone — a declared keepout reaches no exported board (absmach#142), and even when it does a KiCad keepout is unconditional and would forbid the pair it protects. What the README says plainly is where the budgets stop. Neither reaches KiCad (absmach#147) and neither is measured in this build, so the board states them and nothing yet holds it to them.

SammyOina · 2026-10-21T09:38:55Z

[P1] Accidental fixture/result changes are still not automatically detected. The matrix records registry and artifact hashes, but compares them against no versioned baseline. A board’s schematic, PCB, BOM, or registry data can change while preserving the same statuses and diagnostics, and CI will still pass. Issue #45 explicitly requires detection of accidental fixture/result changes. Please add a committed manifest/golden comparison for stable inputs or outputs, or explicitly narrow the issue acceptance criteria.

…using it Completing the W5500's pin map renamed its digital supply from `vcc` to `vdd`, pin 28, and three fixtures kept connecting `vcc`. The pin no longer exists, so each of them reported `E-SYNTH-COMP-002` for an undefined pin and `E-SYNTH-CONNECT-002` for the supply that never arrived. CI stayed green throughout, which is the part worth noting. The ERC fixture harness asserts that a `<CODE>__name.synth` file emits the code it is named for and allows any others, so `E-SYNTH-RESET-001` and `E-SYNTH-SPI-002` kept firing and nothing failed -- while the boards they fire on had become different boards, with an unpowered Ethernet controller and two extra errors each. A fixture that no longer isolates its own case stops being evidence for it. Both codes are gone now and the named ones remain. The two `U1.vcc` left in the SPI fixture are a `ch340g`, which does declare `vcc`. The harness weakness is the same one the qualification matrix had on absmach#75, where `expect_source_codes` was a subset check: a declared list that is a floor rather than a set lets a case decay quietly. Worth fixing there too, and out of scope here.

Two RP2350s on six layers, each on its own native USB, each pair at 90 ohm with `max_skew 0.15mm` and `couple tight`, and `match_length 0.5mm` on the USB class holding all four nets to one length. Two independent pairs on one board is what case 3 asks about. Adapted from the board in absmach#75, whose header carried a STATED LIMITATION saying SynthSpec had no length-matching or skew syntax so these budgets could not be declared. They can now, and these are the budgets that limitation stood in for. Two substitutions, both forced. The second endpoint is an RP2350 rather than the ESP32-C3 the design was written around: that part names a symbol and footprint absent from KiCad 10 and declares USB on two pins the module does not have (absmach#146), so a board using it cannot export. And the two keepouts are gone — a declared keepout reaches no exported board (absmach#142), and even when it does a KiCad keepout is unconditional and would forbid the pair it protects. What the README says plainly is where the budgets stop. Neither reaches KiCad (absmach#147) and neither is measured in this build, so the board states them and nothing yet holds it to them.

…using it Completing the W5500's pin map renamed its digital supply from `vcc` to `vdd`, pin 28, and three fixtures kept connecting `vcc`. The pin no longer exists, so each of them reported `E-SYNTH-COMP-002` for an undefined pin and `E-SYNTH-CONNECT-002` for the supply that never arrived. CI stayed green throughout, which is the part worth noting. The ERC fixture harness asserts that a `<CODE>__name.synth` file emits the code it is named for and allows any others, so `E-SYNTH-RESET-001` and `E-SYNTH-SPI-002` kept firing and nothing failed -- while the boards they fire on had become different boards, with an unpowered Ethernet controller and two extra errors each. A fixture that no longer isolates its own case stops being evidence for it. Both codes are gone now and the named ones remain. The two `U1.vcc` left in the SPI fixture are a `ch340g`, which does declare `vcc`. The harness weakness is the same one the qualification matrix had on absmach#75, where `expect_source_codes` was a subset check: a declared list that is a floor rather than a set lets a case decay quietly. Worth fixing there too, and out of scope here.

Two RP2350s on six layers, each on its own native USB, each pair at 90 ohm with `max_skew 0.15mm` and `couple tight`, and `match_length 0.5mm` on the USB class holding all four nets to one length. Two independent pairs on one board is what case 3 asks about. Adapted from the board in absmach#75, whose header carried a STATED LIMITATION saying SynthSpec had no length-matching or skew syntax so these budgets could not be declared. They can now, and these are the budgets that limitation stood in for. Two substitutions, both forced. The second endpoint is an RP2350 rather than the ESP32-C3 the design was written around: that part names a symbol and footprint absent from KiCad 10 and declares USB on two pins the module does not have (absmach#146), so a board using it cannot export. And the two keepouts are gone — a declared keepout reaches no exported board (absmach#142), and even when it does a KiCad keepout is unconditional and would forbid the pair it protects. What the README says plainly is where the budgets stop. Neither reaches KiCad (absmach#147) and neither is measured in this build, so the board states them and nothing yet holds it to them.

…using it Completing the W5500's pin map renamed its digital supply from `vcc` to `vdd`, pin 28, and three fixtures kept connecting `vcc`. The pin no longer exists, so each of them reported `E-SYNTH-COMP-002` for an undefined pin and `E-SYNTH-CONNECT-002` for the supply that never arrived. CI stayed green throughout, which is the part worth noting. The ERC fixture harness asserts that a `<CODE>__name.synth` file emits the code it is named for and allows any others, so `E-SYNTH-RESET-001` and `E-SYNTH-SPI-002` kept firing and nothing failed -- while the boards they fire on had become different boards, with an unpowered Ethernet controller and two extra errors each. A fixture that no longer isolates its own case stops being evidence for it. Both codes are gone now and the named ones remain. The two `U1.vcc` left in the SPI fixture are a `ch340g`, which does declare `vcc`. The harness weakness is the same one the qualification matrix had on absmach#75, where `expect_source_codes` was a subset check: a declared list that is a floor rather than a set lets a case decay quietly. Worth fixing there too, and out of scope here.

The DRC stage had a hardcoded 30s budget. A real board's DRC runs a full place-and-route: env_logger takes 18s on a release build but 205s on a debug one, so `synth check` reported drc=unknown — correctly, since no evidence was produced, but uselessly, because nothing was actually wrong. SYNTH_CHECK_SOURCE_TIMEOUT_SECS, SYNTH_CHECK_DRC_TIMEOUT_SECS and SYNTH_CHECK_FAB_TIMEOUT_SECS let a release gate buy the time its boards need. The defaults are unchanged and are adequate for the corpus on a release build. Mirrors SYNTH_KICAD_CLI_TIMEOUT_SECS on the native stages.

`synth check` does not route; it spawns `synth drc` and `synth export-kicad`, and those own the engine. There was no way to tell it which one to use, so a release gate could not select one. With no engine available `synth drc` refuses to route, emits no DRC counts, and the stage reports `unknown` — not a pass. The gate then fails for every board, the empty one included, for a reason that is not about any design. `--router` and `--kicad-routing-tools-repo` now travel to both children. `--router` is an Option with no default, and unset means "do not mention it" rather than "use the default". The stages already have a default and read the environment, and an explicit argument outranks both, so forwarding one unasked replaces a selection made another way. Giving this flag a default value is not the harmless thing it looks like: it made a KiCadRoutingTools checkout installed through `KICAD_ROUTING_TOOLS_REPO` come back as a missing Java runtime, because `check` was quietly telling its children to use freerouting instead. The engine's name in the child's argument list is clap's value for the variant, not `RouterEngine::as_str`. Those two disagree — `as_str` says `freerouting` where the CLI accepts `free-routing` — and a child invoked with the wrong spelling is rejected by its own parser.

qualification/matrix.toml declares, per board, a capability class, why the board is in the corpus, and the result the deterministic release command must produce. The test runs `synth check <board> --json` over the corpus and fails when any board stops matching its declaration — in either direction, so a launch claim cannot grow without someone declaring it. Eight boards, classes A through D, four positive and four negative. Every expectation was measured on KiCad 10.0.6 with KiCadRoutingTools v0.23.0 on a release build, and the rationale for each records what was observed rather than what was hoped for. No board clears the gate today, and four distinct causes are named: the per-netclass clearance and minimum via diameter are not expressed in the exported .kicad_dru, so the engine routes to the global floor and kicad-cli pcb drc rejects its own board's rules; the engine exits 1 on a board with nothing to route; no shipped part carries a reviewer, so manufacturing refuses; the four imported reference designs fail the source gate, which is what they are in the corpus to prove. Positive and negative are read off `status`, not off `expect`. `expect` is a measurement: a board whose copper is complete can still fail at DRC, and requiring some row to declare `expect = "pass"` would force the matrix to claim a pass nobody had observed. The release gate itself is `#[ignore]`d — each board runs a full place-and-route — and runs in its own job. A second test keeps the corpus well formed on every shard: paths resolve, ids are unique, every negative names the diagnostics it expects, and every class is represented.

Two jobs, both blocking. `Qualification matrix` runs the release command over the corpus; `Qualification matrix (manufacturing)` runs it again with the export gate, which needs KiCad present to produce any evidence at all. Both upload target/qualification/results.json whether they pass or fail, so a result is attributable to a compiler change, a registry edit or a different KiCad rather than being a mystery. Three things the jobs cannot do without: KiCad itself, from the 10.0.6 PPA, plus the stock symbol and footprint libraries. Without them most registry parts come back `unknown` instead of pass or fail. An engine. Synth does not route copper; it drives one and validates what it wrote. With none available `synth drc` refuses to route, emits no counts, and the DRC stage reports `unknown` — not a pass — so every board fails for a reason that has nothing to do with any design. KiCadRoutingTools, pinned to v0.23.0 because "whatever main is today" is not a reproducible run. A release build. A debug DRC on a real board takes roughly ten times as long and overruns the stage budget, which reports `unknown` correctly but uselessly. The toolchain is pinned to 1.98.1 and the cache keyed on rust-toolchain.toml, matching ci.yml; `@stable` disagrees with the toolchain file this repo pins. Path-filtered to the compiler crates, the registry and the corpus, so it does not run on documentation-only changes. GitHub reports a filtered-out workflow as missing rather than passing, which the handoff doc warns about for anyone making this a required check.

A board that passes the matrix has passed the compiler, the rule checks and the exported-artifact checks. That is not the same as being fabricable, and the table here names the six areas — visual, assembly, signal integrity, power integrity, thermal, fabrication — that need a human with PCB experience looking at the artifacts, because no amount of green CI substitutes for them. Promotion from `experimental` to `qualified` is a review with a recorded verdict per area, not a code change. Also records where the corpus stands: every row reads `fail`, and the four causes are distinct and named. A matrix whose rows all read `fail` is doing its job, and the day one cause is fixed the gate will say so by failing in the other direction.

The path filter listed every crate that can change a board's release output and then stopped at the workspace boundary, so `Cargo.toml`, `Cargo.lock` and `rust-toolchain.toml` could all change what the compiler emits for all seventeen boards without the gate running once. This is not hypothetical. The webpki advisory fix is a lockfile-only commit that touches no path the filter named, and GitHub reports a filtered-out workflow as missing rather than as passing, so nothing would have said so.

The check was a subset: every code in `expect_source_codes` had to be reported, and anything else the board said was ignored. So an `unsupported` board could acquire a second, unrelated defect and still pass the gate, as long as the code already written down kept firing. That is the direction this matrix exists to catch -- a board's behaviour changing without anyone declaring it -- and it was the one direction the diagnostics were not checked in. Both differences are now reported. A code that appears and a code that vanishes each mean the compiler says something new about a board nobody re-declared, and the second is the easier one to miss: a rule that stops firing looks like silence, not like a failure. This tightens nothing today. Against CI's own `results.json` for all seventeen boards, every declared list already equals the observed set exactly, so the change is free and only the hole closes. Only error-level codes are compared, which `source_codes` already filters to, so a new warning still does not fail the gate.

…gate them `deterministic_artifacts` called the schematic, PCB and BOM hashes "the drift detector", and the handoff document said the matrix "compares" them. Neither is true: the hashes are collected into `results.json` and nothing ever reads them back. A fixture or output change could therefore pass unnoticed, as long as the status and the diagnostics held -- while the comment claimed it could not. Pinning them is the wrong fix, for a reason worth writing down rather than rediscovering. The `.kicad_pcb` carries the copper an external routing engine laid, so its hash moves when that engine is upgraded even though no board, no part and no compiler changed. A gate that fails for a reason no reviewer can act on is a gate people learn to ignore, which is the same argument the Gerber timestamps already lost. So the claim goes, and both places now say what the hashes are for: a human comparing two runs' artifacts, and attribution when a result moves. What the matrix actually gates on is the declared verdict -- `expect`, `expect_fab` and `expect_source_codes`, the last of them now an exact set.

Two RP2350s on six layers, each on its own native USB, each pair at 90 ohm with `max_skew 0.15mm` and `couple tight`, and `match_length 0.5mm` on the USB class holding all four nets to one length. Two independent pairs on one board is what case 3 asks about. Adapted from the board in absmach#75, whose header carried a STATED LIMITATION saying SynthSpec had no length-matching or skew syntax so these budgets could not be declared. They can now, and these are the budgets that limitation stood in for. Two substitutions, both forced. The second endpoint is an RP2350 rather than the ESP32-C3 the design was written around: that part names a symbol and footprint absent from KiCad 10 and declares USB on two pins the module does not have (absmach#146), so a board using it cannot export. And the two keepouts are gone — a declared keepout reaches no exported board (absmach#142), and even when it does a KiCad keepout is unconditional and would forbid the pair it protects. What the README says plainly is where the budgets stop. Neither reaches KiCad (absmach#147) and neither is measured in this build, so the board states them and nothing yet holds it to them.

…using it Completing the W5500's pin map renamed its digital supply from `vcc` to `vdd`, pin 28, and three fixtures kept connecting `vcc`. The pin no longer exists, so each of them reported `E-SYNTH-COMP-002` for an undefined pin and `E-SYNTH-CONNECT-002` for the supply that never arrived. CI stayed green throughout, which is the part worth noting. The ERC fixture harness asserts that a `<CODE>__name.synth` file emits the code it is named for and allows any others, so `E-SYNTH-RESET-001` and `E-SYNTH-SPI-002` kept firing and nothing failed -- while the boards they fire on had become different boards, with an unpowered Ethernet controller and two extra errors each. A fixture that no longer isolates its own case stops being evidence for it. Both codes are gone now and the named ones remain. The two `U1.vcc` left in the SPI fixture are a `ch340g`, which does declare `vcc`. The harness weakness is the same one the qualification matrix had on absmach#75, where `expect_source_codes` was a subset check: a declared list that is a floor rather than a set lets a case decay quietly. Worth fixing there too, and out of scope here.

JeffMboya force-pushed the SYN-45 branch from cfd4027 to 612d555 Compare October 6, 2026 05:14

JeffMboya force-pushed the SYN-45 branch from 8dcdf68 to 1d54081 Compare October 9, 2026 04:49

JeffMboya force-pushed the SYN-45 branch from 824db18 to d1b6485 Compare October 12, 2026 06:10

JeffMboya mentioned this pull request Oct 13, 2026

SYN-40 - Two independent USB pairs with declared length and skew budgets #148

Open

JeffMboya force-pushed the SYN-45 branch from 28c7e84 to 3a0feab Compare October 18, 2026 05:47

JeffMboya force-pushed the SYN-45 branch from 3a0feab to a7c082b Compare October 20, 2026 06:57

JeffMboya force-pushed the SYN-45 branch from a7c082b to aac66fd Compare October 20, 2026 07:02

This was referenced Oct 20, 2026

Bug: synth check deadlocks when a stage's output exceeds the pipe buffer #245

Closed

SYN-40 - Declare the nine qualification boards in the reference matrix #247

Open

JeffMboya force-pushed the SYN-45 branch from aac66fd to 0651fe7 Compare October 21, 2026 02:58

rodneyosodo approved these changes Oct 21, 2026

View reviewed changes

JeffMboya force-pushed the SYN-45 branch from b517e6f to 3b41fe9 Compare October 21, 2026 08:23

JeffMboya force-pushed the SYN-45 branch from 3b41fe9 to 709bc9f Compare October 21, 2026 09:38

JeffMboya force-pushed the SYN-45 branch from 709bc9f to 366dad6 Compare October 21, 2026 10:14

JeffMboya force-pushed the SYN-45 branch from 366dad6 to 44c847c Compare October 21, 2026 11:26

JeffMboya added 8 commits October 21, 2026 19:21

JeffMboya force-pushed the SYN-45 branch from 44c847c to 900338a Compare October 21, 2026 16:25

JeffMboya mentioned this pull request Oct 22, 2026

NOISSUE - Refuse to continue when apt falls back to the archive's KiCad 7 #251

Merged

nyagamunene mentioned this pull request Oct 28, 2026

SYN-67 - Add a 24 V buck board with power-good and thermal copper to the Class B matrix #110

Open

Navigation Menu

Repository navigation

SYN-45 - Add a reference-board qualification matrix to CI - #75

SYN-45 - Add a reference-board qualification matrix to CI#75
JeffMboya wants to merge 8 commits into
absmach:mainfrom
JeffMboya:SYN-45

JeffMboya commented Oct 5, 2026 •

edited

Loading

Uh oh!

SammyOina commented Oct 5, 2026

Uh oh!

JeffMboya commented Oct 18, 2026

Uh oh!

SammyOina commented Oct 20, 2026 •

edited

Loading

Uh oh!

rodneyosodo left a comment

Uh oh!

JeffMboya commented Oct 21, 2026

Uh oh!

SammyOina commented Oct 21, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

JeffMboya commented Oct 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What type of PR is this?

What does this do?

Which issue(s) does this PR fix/relate to?

Have you included tests for your changes?

Did you document any new/modified features?

Notes

Uh oh!

SammyOina commented Oct 5, 2026

Uh oh!

JeffMboya commented Oct 18, 2026

Every expectation is measured

Nothing clears the gate, for four distinct reasons

One test change worth calling out

One regression caught before this went up

Uh oh!

SammyOina commented Oct 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

rodneyosodo left a comment

Choose a reason for hiding this comment

Uh oh!

JeffMboya commented Oct 21, 2026

Uh oh!

SammyOina commented Oct 21, 2026

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

JeffMboya commented Oct 5, 2026 •

edited

Loading

SammyOina commented Oct 20, 2026 •

edited

Loading