Skip to content

Expose ModelRepo editor metadata - #46

Open
tony-xmelon wants to merge 15 commits into
stokes-audio:mainfrom
tony-xmelon:codex/catalog-ui-metadata
Open

tony-xmelon wants to merge 15 commits into
stokes-audio:mainfrom
tony-xmelon:codex/catalog-ui-metadata

Conversation

@tony-xmelon

@tony-xmelon tony-xmelon commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Expose the three remaining shared ModelRepo conditional-editor fields without inventing UI semantics:

  • toggle_on
  • toggle_off
  • toggle_steps

mid_label and display_pos are already on main. The API here is additive with neutral defaults. Documentation records that the conditional-visibility reading is inferred from catalog shape and that three Mono Synth self-references remain unexplained. The CorOS 4.1-only linkedSceneMode field remains excluded from shared code and recorded as a firmware-labelled observation.

Evidence

The fixture is now generated from the real CorOS 4.0.1 ModelRepo payload supplied in #80 by the committed scripts/extract_editor_metadata_fixture.py; it is no longer a hand-written aggregate echo. It pins distilled raw rows for:

  • 36 midpoint-label carriers: 35 C, one A/B; all carry min/max labels and parse to skew 1.0
  • 1,446 display-position carriers and five collision models
  • 21 gap models on the explicit all-parameter 0..max(displayPos) basis
  • the Analog Delay (ST) isplayPos typo
  • 132 / 83 / 13 toggleOn / toggleOff / toggleStep carriers
  • all 13 toggleStep carriers also carrying toggleOn or toggleOff
  • the three real Mono Synth self-references at indexes 15, 24, and 48

The earlier figure of 18 gap models was not reproducible and is removed.

Review corrections

  • restored the conditional-visibility reading in ADR-0015 and docs/protocol.md
  • corrected the catalog-attribute overview from 17 parsed / 7 unexplained to 20 / 4
  • added the Splitter citation and rendered documentation for toggle_off
  • changed the changelog to public API names with the CorOS 4.0.1 counts
  • made the synthetic self-reference test use real index 15 and restored test spacing

Validation

Final head: fe83a5042347264f627f54e5282623b28fa8cdd0

  • full offline suite: 2972 passed, 7 skipped
  • focused catalog/docs suite: 336 passed, 1 skipped
  • mypy: success across 46 source files
  • build: sdist and wheel built successfully
  • twine check: both artifacts passed
  • git diff --check: passed

Hardware

The complete hardware suite is for the maintainer's CorOS 4.0.1 unit after review, as requested. The device-derived catalog evidence above is reproducible offline from the captured 4.0.1 payload and does not claim a new hardware run.

Checklist

  • Current upstream main merged
  • Real-row extractor and fixture committed
  • All requested documentation corrections restored
  • Full offline, typing, build, and package validation passed
  • No unrelated changes bundled in

Hardware: The owner waived the hardware run on 2026-09-23; this catalog-metadata change parses fixtures only and sends no device operation.

Offline validation on 200864c: 2,972 passed, 7 skipped; mypy clean across 46 source files; git diff --check clean. The payload-regeneration assertion is now part of the branch after the one-time upstream sync.

Upstream reconciliation — 2026-09-27

Merged upstream main at d8d40f37d141215213284b3ba45050ff1efac237 in commit 466587c. The current main already removed the duplicate _parse_indexes helper; this PR now reuses main's _parse_replaces integer-list parser for toggleOn, toggleOff, and toggleStep as well, without carrying a second parser.

Current head: 3aec9f5b6687e9c98ad05aec802cd716635ecf6d

Validation: full offline suite 2,979 passed, 7 skipped; mypy clean across 46 source files; git diff --check clean. Exact-head CI run 36285068348 is green: build + twine, mypy, and Python 3.11/3.12/3.13. The maintainer waived a hardware run for this metadata-only change. All review threads are addressed; re-review is requested.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Updated onto the merged #52 device-profile architecture in cac6220. The editor-metadata change remains isolated. Focused model/catalog validation: 531 passed; mypy clean; git diff --check clean.

@tony-xmelon
tony-xmelon marked this pull request as draft September 8, 2026 08:41
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Synced with upstream �2331cc at head 4a05091 and moved back to draft pending the required full hardware suite on this exact commit. Focused catalog/model/hardware-report validation: 567 passed; mypy clean.

@tony-xmelon
tony-xmelon marked this pull request as ready for review September 8, 2026 09:35

@jonathanstokes jonathanstokes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. The parser work is clean and I want the change in.

I checked three of your claims rather than trust them. All three hold. The rename of _parse_replaces to _parse_indexes leaves the function body identical, and nothing calls the old name, so replaces parses as before. The six new fields go after the last existing field, so no position moves. The offline suite and mypy are both green on your current head.

I also checked your reading of toggleOn and toggleStep against the catalog. You are right, and the repo currently says something different. I left the details at the line.

Now the part that is on me. After you opened this, I merged a design that changes where a change like yours belongs. None of it was written down when you started, so you could not have followed it. ADR-0020 is the record. One baseline became a registry of device profiles. A Version reply now picks a profile at connect. Anything that differs by firmware or model lives on that profile. contributing.md has the two sections that follow from it: "Before you mark a pull request ready" and "Adding a device profile". Please read both before the next round.

The good news is that the profile system asks almost nothing of this PR. Parameter and parse_model_repo are not methods on QuadCortex, so this adds no operation. That means no VERIFIED entry and no hardware marker. The parser reads whatever the connected unit publishes, so it works for any profile. Leave it where it is. The one exception is linkedSceneMode, and I left a comment on it at the line.

Two points that do not belong at any single line.

critical: This repo keeps a list of catalog attributes we can see but cannot explain. It is the appendix of docs/domain-model.md. Five of your six are on that list today. A field leaves the list when someone writes down the evidence. That list is what stops us reading an attribute as whatever its name suggests. So please edit the appendix in this PR. Its header says the device puts 24 attributes on a <Parameter> and fourteen are parsed. After this change nineteen are parsed, so that sentence needs fixing, and the five rows need to move out with your evidence. Two other files carry the same question and will now contradict your docstrings: docs/protocol.md:3394 and docs/ADR.md:251.

critical: Every number in the description is from CorOS 4.1.0. None of them reproduces on 4.0.1. I get 36 midpoint labels, not 49. I get 1,446 declared positions, not 1,965. I get 132 / 83 / 13 toggles, not 168 / 124 / 16. I get zero linkedSceneMode, not 44. Measuring on 4.1.0 is fine. The problem is that the description does not say which firmware. Under ADR-0020 a 4.1.0 observation goes beside the 4.0.1 record and names itself. Please label them. Where a docstring carries a count, use the 4.0.1 number, so a reader can check it.

nit: A pull request now opens as a draft and carries a Hardware section before it is marked ready. That rule landed after you opened this, so it is not a complaint. This change cannot reach the wire, so I am happy to waive the suite run and will write that in the section myself.

On my side, I will run pytest tests/hardware --hardware on my own 4.0.1 unit before merging. I do that for every PR. If what I see differs from your 4.1.0 audit, I will record my numbers beside yours, not in place of them.

Last thing. This is more work than you signed up for, and most of it is documentation. If it is more than you want, say so. I will split the doc half out and do it myself, and keep your commit for the parser. Or just drop the PR if you would rather not have it. Either is fine, and the parser work stands either way.

Stokes

Comment thread pyquadcortex/protocol/catalog.py Resolved
Comment thread pyquadcortex/protocol/catalog.py Resolved
Comment thread pyquadcortex/protocol/catalog.py Resolved
Comment thread pyquadcortex/protocol/catalog.py Resolved
Comment thread pyquadcortex/protocol/catalog.py Resolved
Comment thread tests/test_catalog.py Resolved
@tony-xmelon
tony-xmelon marked this pull request as draft September 8, 2026 18:37
@tony-xmelon
tony-xmelon marked this pull request as ready for review September 8, 2026 18:39
@tony-xmelon
tony-xmelon marked this pull request as draft September 8, 2026 19:24
…tadata

# Conflicts:
#	changelog.md
#	docs/domain-model.md
#	pyquadcortex/protocol/catalog.py
#	tests/test_catalog.py
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Synced with upstream main at 640a8e9 (including #63–#65). New head: ef95034. Offline suite excluding the three #42-owned Windows portability checks: 2597 passed, 3 skipped; full run: 2724 passed, 3 skipped, 3 known portability failures; mypy: clean across 46 source files; git diff --check: clean. The three shared Windows-only baseline checks remain isolated in #42.

…tadata

# Conflicts:
#	changelog.md
#	docs/ADR.md
#	docs/domain-model.md
#	docs/protocol.md
#	pyquadcortex/protocol/catalog.py

@jonathanstokes jonathanstokes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The payload you asked for now exists, and it closes this thread.

PR #80 proposes tests/fixtures/catalog/model_repo_coros_4_0_1.bin (46,695 bytes,
sha256 fdc3086...77ac14) captured from a unit reporting zenos_git_hash 4.0.1,
app_fw_version d14e, device_type QC. It is not merged yet, so take it from the
branch catalog/model-repo-payload-4-0-1. All three generators in scripts/ accept
--payload.

I ran your own parser at 579a244 against that payload. It reproduces every number
this thread asked to be pinned, offline, with no unit attached:

mid_label carriers    36   (35 "C", 1 "A/B" = Micro Processor (ST) "A/B PITCH MIX")
                           all 36 carry min_string and max_string; all parse to skew 1.0
display_pos carriers  1446
displayPos collisions 5 models
toggle_on / off / step  132 / 83 / 13
toggleStep carriers lacking both toggleOn and toggleOff: 0
self-references       3   (Mono Synth OSC2 WAVE@15, SLOPE@24, FREE RATE@48)
Analog Delay (ST)[20] "FEEDBACK DEPTH" display_pos = None   <- the isplayPos casualty

So the evidence gap is closable without a 4.0.1 unit. A ~35 line extractor patterned
after scripts/extract_scale_fixture.py produced a 12.5 KB real-row fixture here in
one run.

Three things to settle before this goes ready.

1. The pinning test does not test the device.
test_real_catalog_editor_metadata_evidence_is_pinned loads editor_metadata.json
and asserts the literals typed into the same pull request. Change a number in both
places and CI stays green. editor_metadata.json is read by that one test and
nothing else; scales.json and option_readings.json are read by generators, by
the documents, and by hardware drift tests. Regenerate the fixture from the payload
with a committed extractor, and the assertions become assertions about the unit.

2. gap_models: 18 does not reproduce.
Against the real 4.0.1 payload I get 21 gap models counting a gap as any missing
position in 0..max, or 17 counting from 1. That holds on the all-parameter basis
and on the visible-only basis. Excluding the 5 collision models gives 20 and 16.
None of the eight combinations gives 18. collision_models: 5 is right, but it is on
the all-parameter basis while catalog.py quotes 1 for the same thing on the visible
basis, and the file does not say which basis it uses. An extractor settles both.

3. The ADR and protocol.md corrections were lost in a merge.
You made them in 45195dc. The 19 Sep merge a8d026c, resolved in favour of upstream,
reverted them. At 579a244, docs/ADR.md:253 and docs/protocol.md:2185 still say
the open question is whether these attributes "describe a footswitch toggle's two
values" - the reading the Splitter evidence disproves - while docs/domain-model.md:1207
in the same commit says the visibility reading holds except for the three self-references.
Two files in one commit, contradicting each other. Please restore the 45195dc text.

While you are in docs/domain-model.md: the section header at 1194 still reads "The
unit puts 24 distinct attributes on its <Parameter> elements. Seventeen are parsed
... and the seven that are not yet explained." There are exactly 24, and this branch
parses three of the seven, so it is 20 and 4 now.

Smaller things, none blocking:

  • The changelog entry names toggleOn/toggleOff/toggleStep rather than the
    toggle_on/toggle_off/toggle_steps a caller types, and carries no firmware tag
    where both neighbouring entries do.
  • test_self_referencing_toggle_metadata_is_preserved_not_interpreted builds its
    parameter at index 0 with toggleOn="0", where a self-reference is indistinguishable
    from ordinary parsing. The real cases are at 15, 24 and 48.
  • toggle_off gets no #: comment, so it renders undocumented next to its two siblings.
  • Two blank lines are missing before
    def test_a_labelled_end_control_carries_the_span_the_unit_draws.
  • The description still sells five fields. mid_label and display_pos reached main
    during the upstream merges, so this branch now adds three.

I have not run the hardware suite. That is mine to run, not yours to waive, and it
comes after the above.

Comment thread pyquadcortex/protocol/catalog.py Resolved
Comment thread changelog.md Resolved
Comment thread tests/test_catalog.py Resolved
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Thank you for supplying the payload and the complete second review. Current head e83a50 addresses every listed point.

The fixture is now generated by a committed extractor from #80's real 4.0.1 payload and carries distilled raw rows, not hand-written aggregate assertions. The unreproducible gap count is corrected to 21 on the explicit all-parameter 0..max(displayPos) basis. ADR-0015 and protocol.md have the conditional-visibility reading restored; the attribute overview is corrected to 20 parsed / 4 unexplained; the synthetic self-reference uses index 15; the PR description now says this branch adds three fields; and the smaller inline items are all applied and resolved.

Exact-head validation: 2972 passed, 7 skipped; focused catalog/docs 336 passed, 1 skipped; mypy clean across 46 files; build and twine check passed; diff check clean. The full 4.0.1 hardware suite remains yours to run as requested.

Stokes-Quorum

This comment was marked as duplicate.

@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 23, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 23, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 23, 2026
@jonathanstokes
jonathanstokes dismissed Stokes-Quorum’s stale review September 23, 2026 16:52

Posted from the wrong account. Reposting the same review as jonathanstokes; please read that one.

@jonathanstokes jonathanstokes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I reproduced the fixture instead of reading it. scripts/extract_editor_metadata_fixture.py, run against tests/fixtures/catalog/model_repo_coros_4_0_1.bin on main, writes tests/fixtures/catalog/editor_metadata.json byte for byte, same sha256. Every number in the new text falls out of the payload on an independent parse: 3,809 parameters, 36 midpoint carriers at 35 C and one A/B, 1,446 display positions, 5 collision models, 21 gap models, the single isplayPos on Analog Delay (ST) index 20, and 132 / 83 / 13 toggle carriers across 212 parameters. The three Mono Synth self-references are at 15, 24 and 48. The offline suite and mypy are clean on your head and on a local merge with current main. Thank you for making it reproducible, that is exactly what I wanted.

Your reading is better evidenced than the text claims. All 216 toggleOn and toggleOff values are parameter indexes inside their own model, and none is out of range. 214 of the 216 point at a switch, comboBox or rotarySwitch. Every toggleStep value is a valid option index of the parameter that toggleOn names. Beyond the Splitter, Pitch Correction hides ROOT when MODE is Chrom, which is the inverse case toggle_off covers, and Phase Doctor carries both on one parameter to combine two conditions. You could cite either of those if you want a second model behind the claim.

Two things I need changed, then four small ones.

1. The linkedSceneMode record is gone. The description says the field "remains excluded from shared code and recorded as a firmware-labelled observation". The first half is true. The second is not: grep -r linkedSceneMode over fe83a50 returns nothing. It was in docs/domain-model.md at ef95034 and the merge at 579a244 dropped it. That is the whole record of a 4.1 observation nobody else has, and docs/writing.md routes a catalog attribute we cannot explain to that file. Please put it back. Here is the text as it stood:

linkedSceneMode appears 44 times in the contributed CorOS 4.1.0 catalog and
zero times in 4.0.1. Its meaning and relationship, if any, to preset
scene_mode are unknown, so it is recorded here and not exposed as shared API.

2. Please drop the docs/ADR.md hunk. ADR-0015 is Decided (2026-08-26), and that file's own Lifecycle section says a decided record is append-only, with one in-place edit allowed: resolving an open question as Resolved (date): .... The hunk rewrites the question's wording instead, and the original text is deleted. Your evidence already lives in docs/protocol.md and docs/domain-model.md, which is where docs/writing.md sends it. I will append the Resolved line to ADR-0015 myself. There is a suggestion below that restores the original line.

3. _parse_indexes lost its docstring somewhere in the merges. At 45195dc it said the silence is deliberate and why, which is what I asked for, and it went through _as_int. The version on your head is a one-line summary and a hand-rolled int(), sitting next to _parse_replaces, which does the same job the same way. Suggestion below.

4. The gap-model comment names the origin basis but not the model basis. It counts all 533 models, two of which are hidden (PCOM Tape Delay (ST) and Chief DC2W PCOM). The display_pos docstring above it counts over non-hidden models, which is where its 163, 165, 23 and 43 come from. On that basis the gap count is 19, not 21. Both numbers are right, they just answer different questions, and one clause stops the next reader from trying to reconcile them. Suggestion below.

5. docs/STEERING.md section 4 lists the contents of scripts/ by name, and extract_editor_metadata_fixture.py is not in it. Please add it.

6. Nothing holds editor_metadata.json to the payload it comes from. scales.json has tests/hardware/test_scales_on_unit.py behind it, and tests/test_catalog_payload.py explains the failure for the generated snapshots: a saved input nothing reads is a file that rots silently. Your fixture is derived entirely from the payload, so an offline test can regenerate and compare, the way test_the_payload_still_generates_the_committed_snapshot does. That needs the payload, which arrives when you sync with main, so it is the one item here that has to wait for the sync.

On hardware: I am waiving the run for this one. It parses three catalog attributes, changes no operation and writes nothing to the unit, and the evidence is now reproducible offline from the committed payload, so a run would exercise nothing this diff touches. When you push the fixes, please add a line to the description saying the owner waived the hardware run on 2026-09-23. contributing.md wants that recorded there before I can mark it ready.

One last thing, and please do not do it yet. This branch is behind main and the ruleset uses strict status checks, so it needs a sync right before I merge. I will ask for it then, so you only do it once.

Comment thread docs/ADR.md Resolved
Comment thread pyquadcortex/protocol/catalog.py Resolved
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

@jonathanstokes I pushed 200864c addressing the requested items: restored the original append-only ADR-0015 question, restored the 4.1-only linkedSceneMode observation, documented the model basis for the 21/19 gap count (including the two hidden models), improved _parse_indexes using _as_int with the deliberate-invalid-token rationale, and added the fixture generator to the owned-paths inventory. I added the requested hardware waiver dated 2026-09-23 to the PR description. Validation: 2,972 passed, 7 skipped; mypy clean across 46 files; git diff --check clean. I have not synced main, per your request; I will add the payload-regeneration test when you request that single sync. The PR remains draft pending that test and your review.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

I followed the cleanup guidance in your inline note against the current upstream main: after sync at d8d40f3, main no longer carries the duplicate _parse_indexes parser, so this PR now reuses its _parse_replaces integer-list parser for the three toggle index lists too. This removes the duplication without reintroducing the helper you asked to leave to main. The other two threads are also reflected on the current head: the ADR-0015 wording matches main, and the editor metadata fixture regenerates from the captured payload. Full suite 2,979 passed / 7 skipped; mypy clean (46 files); diff check clean. Exact-head CI is pending.

@tony-xmelon
tony-xmelon marked this pull request as ready for review September 27, 2026 01:25
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

All review threads are now resolved against the current head, exact-head CI run 36285068348 is green, and the owner-waived hardware condition does not apply to this fixture-backed metadata change. I marked the PR ready and request re-review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants