Repository navigation
Resolve cloned ModelRepo parameter layouts - #45
tony-xmelon wants to merge 17 commits into
Conversation
|
Synced with upstream �2331cc at head �44bd54 and moved back to draft pending the required full hardware suite on this exact commit. Mypy is clean. The full Windows run reports 2700 passed, 3 skipped, 3 failed; the three failures are the existing Windows portability cases fixed in dependent PR #42, rather than failures in this PR's cloned-layout change. |
jonathanstokes
left a comment
There was a problem hiding this comment.
I want this in. It is correct, and it fixes more than your summary says.
I ran your branch on my own unit, on CorOS 4.0.1. The wire agrees with you everywhere I could measure. Three readings are worth keeping:
- A
Plini Cab (M)carries 32 parameters on the wire. It holds strings at 1, 9, 29 and 30. A cab that clones 12000 carries 22, with strings at 1, 9, 19 and 20. So the 31-parameter PCOM layout is real, and this library has been reading those cabs against the wrong layout. - On a freshly placed
Plini Cab (M), wire index 5 reads 0.5. Only your reading predicts that. The borrowed 12000 layout predicts 0.4. The bare 12100 parent predicts 0.3. So areplacesparameter does carry the child's own span to the parent's index. Gojira REVcarries 30 parameters, and all 29 of its defaults match your order. That block is not a cab, so nothing was borrowing a layout for it. It meansPRE 1 FREQsits at wire index 8, and this library says 6. Naming that parameter wroteHIGH PASS TYPEinstead. You found a live bug.
The next part is on me. When you wrote this, the library was built around one unit, and the multi-profile design was not written down. So you had no way to follow it. I wrote it down since, in ADR-0020 and in contributing.md under "Working with hardware", "Before you mark a pull request ready" and "Adding a device profile". Please read those two before the next round. Your Hardware section already does most of what the new rules ask. It names the final commit, it names the CorOS version of your unit, and it says you have no 4.0.1 unit. That is what I want to see.
Three notes on how this fits the profile system.
The parser does not belong to a profile. ADR-0020 puts the catalog snapshot, the announce string, the hardware facts and the differing operations on the profile class. A parser is none of those, and parse_model_repo is not an operation. So nothing moves to QuadCortex41. Leave the code where it is.
Put your 4.1.0 readings in docs/protocol.md, next to the 4.0.1 entry, with the date and the CorOS version. I will add mine beside yours.
You do not move the 4.0.1 snapshot, and I checked. I regenerated the constants from my own catalog with your branch applied, and the file is identical.
Most of my comments are on the lines. Three of them are in files this PR does not touch, so I could not attach them:
- critical: the
Cabsimconstants inpyquadcortex/protocol/catalogs/coros_4_0_1/params.py:163are wrong in two ways now. The docstring says the wire carries 22 parameters whichever cab is loaded. That is false for a PCOM cab, which carries 32. The set also stops at index 20, so it cannot reach a PCOM cab's extra controls. Please fix the docstring, and say there that the set covers the 21-parameter layouts. - critical:
MIC_1_PANneeds a line in that same file. The catalog calls index 3BALANCEon a stereo cab, over -1 to +1, where the monoPANruns 0 to 10. That matters, becauseReal(0.0)is hard left on one span and center on the other. I have not read a stereo cab's screen yet, so write down what the catalog says and leave the reading to me. - critical: the table in
docs/domain-model.md:1398lists a parameter'sreplacesas something we can see and cannot explain, and it guesses a different meaning. Your PR explains it, so please take it out of that table. Record the other meaning on its own, because it is also real. Model 7029MX Vibehas a parameter withreplaces="INTENSITY", which is a name and not an index, and that model supersedes another model. Then addclones, which is not written down anywhere. Last, the two counts in that section need to say they are CorOS 4.0.1 numbers.
One thing in the description. It says two IR selectors against 21 controls. A PCOM cab has 31.
I will run pytest tests/hardware --hardware on my 4.0.1 unit before this merges, and I will write what I see next to your record, with the date and version. Your run on 4.1.0 stays as your evidence.
If this is more than you signed up for, tell me. I will split it up myself and land your parser as its own commit, with your authorship. Or drop the PR if you would rather not carry it. Either is fine.
Stokes
jonathanstokes
left a comment
There was a problem hiding this comment.
This is close now. Every point from my last review is handled, and on two of them you went further than I asked: replacements are applied before the extensions, and a repeated index raises instead of overwriting. The per-model fallback is the right shape too, since one bad model no longer costs the whole catalog. Removing the layout borrowing and its two constants was the correct call, and the docs work is thorough.
I ran the hardware suite on my own unit. Three tests fail, and all three come from this change. I ran the same file against main on the same unit as a control, and it passed 4 of 4.
commit: 0490350
CorOS: 4.0.1 (Quad Cortex, firmware d14e)
============== operations on QuadCortex (CorOS 4.0.1, MAINTAINER) ==============
tests: 92 passed, 3 failed, 0 errored, 4 skipped; operations measured: 16 of 16 claimed
passed: ['active_scene', 'clear_expression', 'read_current_preset', 'set_chain_input', 'set_expression', 'set_global_eq', 'set_hold_timing', 'set_input_port', 'set_param', 'set_scene_color', 'set_scene_label', 'set_tempo_mode', 'switch_scene', 'tempo_mode', 'update_settings'] (15)
failed or skipped: ['set_bypass'] (1)
passed, not VERIFIED: [] (0) <- candidates to add
VERIFIED and claimed by a test, failed: [] (0) <- regressions by name
3 failed, 92 passed, 4 skipped in 189.22s (0:03:09)
The offline suite passes and mypy passes, so CI will look green. Both of the checks below only run with a unit attached, which is why nothing caught them earlier.
critical: tests/hardware/test_generated_constants.py::test_the_cab_layout_claim_still_holds fails. It asserts that every factory cab publishes exactly two parameters. After your change they publish 21 or 31, so it fails, and it is right to. Its own failure message predicts this moment: "the catalog may have started describing cabs properly, which would make the shared params.Cabsim layout unnecessary". An earlier change left it as a tripwire, so please give it a new claim to check rather than deleting it.
critical: test_the_committed_snapshot_matches_this_unit[options] fails. The committed options.py header says 110 enums cover 267 parameters. Regenerated from my unit it says 295. Clone resolution means inherited parameters now carry stepNames, so more parameters have a fixed list. Please run python scripts/generate_options.py --snapshot coros_4_0_1, then read the diff before you commit it, because it may add option enums.
The third failure is on a line I commented on.
Two smaller things in files this PR does not touch, so I could not attach them:
- critical: docs/STEERING.md still carries the old account. Its 2026-08-27 entry says a cab's own entry is a local list whose numbers do not address the wire. Your change makes that false. It is a change log entry, so I would add a short note pointing at the new account rather than rewriting the history.
- nit: the docstring on
test_parallax_carries_the_cab_law_itselfintests/test_scales.pystill explains itself through layout borrowing, which is gone.
What I measured for you
I read the screen on my unit, so two of your claims are now evidence rather than inference.
Your PCOM span is right. On a Plini Cab (M), wire 1.0 at index 4 and 5 showed 1.00, and wire 0.0 showed 0.00. So DISTANCE and POSITION really do run 0 to 1 on a PCOM cab, and the old 0..10 borrowing was wrong for them. This was the part of your change I was least sure about.
Your per-layout naming is right. The mono cab shows PAN and the stereo cab shows BALANCE, at the same wire index. The catalog declares that per layout and the unit agrees.
I also found a problem that is not yours. The numbers the catalog gives that control are not the numbers the unit displays, on either layout. I opened issue #55 for it. It is older than this PR, and this PR neither causes it nor fixes it, so please do not take it on here. It does mean one sentence in your docstring needs a small change, which I left inline.
On my side
I will run the suite again on your final commit once these are in, and write what I see beside your record with the date and version.
If the two snapshot regenerations are more than you want to carry, say so and I will do those two commits myself on top of your branch. Your parser is the part that matters and it is correct.
Stokes
|
Thank you for the swift, detailed review and for the CorOS 4.0.1 measurements. I pushed One item deliberately remains: the regenerated CorOS 4.0.1 Validation on
I have not requested re-review yet because the 4.0.1 options artifact and your final 4.0.1 hardware rerun are still outstanding. |
…ne-inheritance # Conflicts: # changelog.md # docs/domain-model.md # tests/test_catalog.py
|
Synced with upstream main at 640a8e9 (including #63–#65). New head: 1b5523d. Offline suite excluding the three #42-owned Windows portability checks: |
…ne-inheritance # Conflicts: # changelog.md # docs/STEERING.md # docs/domain-model.md # docs/protocol.md
…ne-inheritance # Conflicts: # changelog.md
|
The CorOS 4.0.1 catalog payload is captured, and I have run the generators The unit
Only
|
| artifact | result |
|---|---|
models.py |
byte-identical |
params.py |
byte-identical |
options.py |
9 lines changed |
So your committed params.py is confirmed correct. It reproduces exactly from a
real 4.0.1 payload, which also confirms the new render() text. Nothing to redo
there.
What changed in options.py
Counts only. No new enum, no changed member, no changed wire index.
- 527 to 611 parameters carrying a fixed list
Small,Med,Large12 to 40, which moves it to the top of the unread rankingOff,On222 to 278- unread 190 to 218, still across 94 lists
- the five biggest unread cover 54 to 82
That is 28 models times (2 Off/On plus 1 Size) = 84, which is what clone
inheritance predicts. I checked it per model rather than inferring it from the
totals.
The counts do not depend on what a unit owns
Worth stating, because the numbers come from one unit. All 28 models behind the
delta are non-factory, which at first looks like it makes the snapshot personal.
It does not. Model.is_factory is false for anything carrying a sku or a
plugin_id, and that marks a model purchasable, not purchased. This unit has no
plugin purchases and its catalog still lists the Gojira, Cory Wong, Nolly and
Plini families in full. Neural Capture slots are the only genuinely user-specific
content, and they carry no option lists at all, so they cannot move a count.
The catalog is a firmware fact. As a check, the payload read on 2026-07-26 and
the one read on 2026-09-22 are identical once the 351 per-model blob tokens
are blanked, and those rotate on every read. Compare parsed catalogs, never
payload hashes.
The parser change is bigger than the artifact suggests
Worth recording, because the generated file hides it: 173 of 533 models gain a
layout (169 cabs, 4 reverbs), and the catalog goes from 3,809 to 7,319
parameters. 240 existing wire indexes now name a different parameter. On every
ordinary cab, index 0 was ir selector and is now bypass, because the two
published selectors carry replaces of 1 and 9. That is the silent-wrong-row
case, and this fixes it.
The generated file alone turns the branch red
With the regenerated options.py in place on ff77dac7:
python -m pytest: 2970 passed, 7 skipped, 4 failedpython -m mypy: clean across 46 source files
All four failures are in tests/test_option_audit.py, and all four are caused by
the regeneration. The committed options.py passes all 21 on the same head.
They are the tests that hold the documents to the snapshot's own numbers, so
two documents need the new counts in the same commit:
docs/domain-model.md
- 527 to 611 parameters carrying a fixed list
- the read lists cover 301 to 357
- the 94 unread cover 190 to 218
- the biggest five cover 54 to 82
- the worklist table reorders:
Small,Med,Largebecomes the top row at 40
parameters over 3 positions, ahead of the 17-entrySYNC NOTEat 14
changelog.md
- the ranking recipe publishes its own answer on line 66:
# 14 parameters - the biggest unread listbecomes# 40 parameters
Also stale and not covered by a test: docs/domain-model.md says the two Off/On
lists and the metronome list are "260 parameters between them". That is now 316.
The payload
Committed, so you can regenerate here and nobody needs my unit for this again:
#80 adds it as tests/fixtures/catalog/model_repo_coros_4_0_1.bin.
python scripts/generate_options.py --snapshot coros_4_0_1 \
--payload tests/fixtures/catalog/model_repo_coros_4_0_1.bin
I have not pushed anything to your branch. The prose above is yours to write, and
changelog.md is where this branch currently conflicts with main, so it is
better done in one pass by you than patched by me.
…ne-inheritance # Conflicts: # changelog.md
|
Thank you for capturing and documenting the 4.0.1 payload. I merged current main and used the #80 branch payload to rerun all three generators at �c9df0d: models.py and params.py reproduced byte-for-byte, while options.py produced exactly the nine count-line changes you reported. The generated output, docs, changelog recipe, and stale option-audit assertions are committed together. Counts now read 611 total, 357 read, 218 unread, top five 82, Small,Med,Large 40, Off,On 278, and the combined vocabulary figure 316. Exact-head validation: 2977 passed, 7 skipped; mypy clean across 46 files; build and twine check passed; diff check clean. |
The three generators in `scripts/` accept `--payload`, and until now the only payload was on the maintainer's disk. Regenerating the 4.0.1 snapshot therefore needed the baseline unit, which is the one thing a contributor cannot borrow. stokes-audio#45 stalled on exactly that: its parser change makes `options.py` stale and its author runs CorOS 4.1.0, where generating into the 4.0.1 snapshot would mislabel firmware data. Read from the baseline unit on 2026-09-22: `zenos_git_hash` 4.0.1, `app_fw_version` d14e, `device_type` QC. The payload is a firmware fact, not a fact about this unit. It describes every model the firmware knows, including plugin content the unit holds no licence for: `is_factory` is false for anything carrying a `sku` or a `plugin_id`, which marks a model purchasable rather than purchased. This unit has no plugin purchases and its catalog still lists the Gojira, Cory Wong, Nolly and Plini families. Neural Capture slots are user content and carry no option lists, so they cannot move a generated count either. Each read rotates a per-model `blob` token, so payloads never match byte for byte. With the 351 `blob` attributes blanked, the catalog read on 2026-07-26 is identical to this one. Compare parsed catalogs, never payload hashes. `docs/capture.md` gains the section that says all of this and gives the command. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
jonathanstokes
left a comment
There was a problem hiding this comment.
Picking this up again. The 4.0.1 payload is on main now, so the parser change is finally reviewable on its merits.
The generated files reproduce
I ran all three generators on this head against tests/fixtures/catalog/model_repo_coros_4_0_1.bin (46,695 bytes, sha256 fdc30861953b...) and diffed the result against what you committed under pyquadcortex/protocol/catalogs/coros_4_0_1/:
models.py 0 differing lines
params.py 0 differing lines
options.py 0 differing lines
That was the thing blocking this pull request and it is settled. An independent run at ff77dac7 had options.py off by 9 lines; your regeneration at bc9df0d7 moved exactly those 9. Offline suite on this head: 2977 passed, 7 skipped. mypy: clean across 46 source files.
The measured facts fall out of the payload
Every row of tests/fixtures/catalog_clones.json reproduces from the payload: parent names, the 21/31/21/31/29 layout sizes, child names, replacement indexes and defaults, all five families. PRE 1 FREQ is at index 8 of Default Reverb (8015) and of all four of its clones.
The clone declarations in the 4.0.1 catalog are 12000 x71, 32000 x70, 12100 x14, 32100 x14, 8015 x4 and 8010 x2. The one nonnumeric replacement is MX Vibe with DEPTH replaces="INTENSITY", exactly as you documented. The 8010 Shimmer family is a second cloned family the fixture does not cover. Not a problem, just noting it is there.
I also checked that nothing still leans on the heuristic you removed from targets.py. CABSIM_LAYOUT and CABSIM_CATEGORIES survive only in scripts/generate_params.py, where they are a deliberate grouping choice and not the borrowing rule.
One blocker
The cab tripwire in tests/hardware/test_generated_constants.py cannot pass on any unit. Detail inline, with a replacement I have run against the payload.
Smaller things
Five inline, on lines this pull request touches. One more that is not on a changed line:
scripts/generate_options.py still carries the pre-clone counts in its own module docstring. Line 21 says "527 parameters", line 26 says "247 of those 527", and line 545 says "the Off/On pair alone is 247 parameters". Clone resolution moved those to 611 and 303. You updated the same numbers in changelog.md and docs/domain-model.md; these three are the ones no test reads.
Not yet
This branch is 25 commits behind main and the ruleset uses strict status checks, so BEHIND will block the merge. Do not sync now. It is the last step, after the hardware run. The part of the sync that could have bitten you is already clear: main's tests/test_catalog_payload.py regenerates from the payload and compares, and that is the check I ran above.
|
Pushed |
|
I checked all six remaining conversations against current head 43cfbe4 and the passing exact-head CI run 36282242452. The paid PCOM hardware assumption now skips when no factory cab exists; the option-audit buckets are 357/36/218 with a derived coverage assertion; the protocol note names both observers, firmware and dates; the clone fallback logs and tests missing-parent diagnostics; |
|
Synced current upstream Validation on pushed head
This does not replace the requested CorOS 4.0.1 verification of clone inheritance; the connected unit is 4.1.0. CI is triggered by the push, and the PR remains draft pending the maintainer's 4.0.1 check/review. |
|
The branch is now synced to the current |
|
Addressed the remaining docstring-count note from the 23 September review in 0d2bbe8: the generator now says 611 fixed-list parameters and 303 Off/On pairs in all three stale references. Validation: full offline suite 2,983 passed, 7 skipped; mypy clean (46 source files); git diff --check clean. The full CorOS 4.0.1 hardware rerun is still yours as requested; I have kept the PR draft and have not synced it past that gate. |
Summary
clonesinheritance before building each model's wire-indexed parameter tuplereplacesoverrides before appending child-only parameters, so XML order cannot discard a parameterVisible child entries may publish only replacement parameters. Resolving the declared parent layout also fixes non-cab families such as Gojira REV.
Related issue
Follow-up contribution discussed in #42.
Testing
Final head:
66c399688e77e24760e957893d06af54b03aa77f821 passed, 2 skippedpython -m mypy: success across 46 source files2704 passed, 3 skipped, 3 failed; the three failures are the existing Windows-only path-separator/CRLF fixture checksgit diff --check: passedCoverage includes inheritance order independence, malformed-model isolation, recursion, missing parents, replacement gaps/collisions, fallback diagnostics, literal measured expectations independent of fixture input, cab layout tripwires, generator parity, and the corrected display observations.
Hardware and generated artifact
The branch is deliberately draft. The remaining generated CorOS 4.0.1
options.pymust come from a 4.0.1 ModelRepo payload. The attached unit here runs CorOS 4.1.0, and generating into the 4.0.1 snapshot from it would mislabel firmware data. The maintainer has offered to regenerate this artifact from the baseline unit (or can provide the saved payload for generation here).The complete hardware suite must then run on the resulting final commit against the maintainer's CorOS 4.0.1 profile. The earlier read-only 4.1 catalog inspection remains supporting cross-profile evidence, not a substitute for that baseline run.
Checklist
options.pyfrom CorOS 4.0.1 dataUpstream sync — 2026-09-12
Merged current upstream main at 640a8e9 (including merged PRs #63–#65) into this branch. Current head:
1b5523dOffline suite excluding the three #42-owned Windows portability checks:
2602 passed, 3 skipped; full run:2729 passed, 3 skipped, 3 known portability failures; mypy: clean across 46 source files;git diff --check: clean.The three excluded Windows checks are unrelated path-separator and CRLF-fixture assertions fixed within #42; they remain isolated there rather than being copied into this PR.
Upstream sync - 2026-09-19
Merged upstream main at
cfe39cc6592148d4196fa3028c2776f6618c41b6(including maintainer PRs #70-#75) and resolved overlaps in favor of the current upstream API and evidence model.Current head:
66c399688e77e24760e957893d06af54b03aa77fFocused validation: 12 cloned-layout tests passed; writing-policy checks passed (18 passed, 1 skipped).
GitHub CI on this exact head is green: build + twine check, mypy, and tests on Python 3.11, 3.12, and 3.13 all pass.
No new maintainer comments were posted during this sync. This PR remains draft under its stated hardware/evidence policy; the displayed CHANGES_REQUESTED decision is the earlier review and has not yet been refreshed.
Latest maintainer follow-up — 2026-09-27
Current head:
43cfbe4a5c6720076760086d8cd4401e82c67bd7.Validation on this head: full offline suite 2,977 passed, 7 skipped (38.77s); focused catalog/audit/target tests 178 passed; mypy clean across 46 source files; hardware test module collects 4 tests (not executed);
git diff --checkclean. The maintainer’s 4.0.1 hardware run remains pending.Current synced status — 2026-09-27 (head
0d3c3df)This supersedes the earlier head and validation above. Merged current upstream
main(d8d40f3) in0d3c3df; the 4.0.1 ModelRepo payload and byte-for-byte regeneration test are now in this branch.2,983 passed, 7 skipped; exact-head CI run36289331406is green across build/twine, mypy, and Python 3.11–3.13.1 passed, 3 deselected). The other three snapshot comparisons cannot run againstQuadCortex41, which has no committed models/params/options snapshot; they are not evidence for the 4.0.1 claim.