Skip to content

Scope native setlist duplication to CorOS 4.1 - #61

Draft
tony-xmelon wants to merge 12 commits into
stokes-audio:mainfrom
tony-xmelon:codex/native-duplicate-setlist
Draft

tony-xmelon wants to merge 12 commits into
stokes-audio:mainfrom
tony-xmelon:codex/native-duplicate-setlist

Conversation

@tony-xmelon

@tony-xmelon tony-xmelon commented Sep 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • expose firmware-native setlist COPY only on the measured QuadCortex41 profile
  • make the CorOS 4.0.1 base refuse immediately with ControlNotDrivable and a copy_preset() workaround
  • stabilize two folder generations before trusting source/destination identity
  • require two identical complete 256-slot preset generations and identify occupancy by name
  • send persistent COPY exactly once; polling and timeout paths never replay it
  • remove the unmeasured 12-setlist ceiling

Hardware evidence

Exact predecessor 2d70432 was exercised on Quad Cortex / CorOS 4.1.0 on 2026-09-11. One sparse COPY created the firmware-named destination; two complete matching inventories confirmed it after 104.587 seconds. Source and destination were then deleted and independently confirmed absent. This correction does not change the measured wire message; it scopes and verifies it safely.

Validation

Current head: 3e74f1e3bbd6c701e5f44520a73679b0818665ec

  • list/duplication focus: 14 passed, 323 deselected
  • profile/docs/coverage/hardware-marker scope: 273 passed, 1 skipped
  • python -m mypy pyquadcortex: clean (46 source files)
  • git diff --check: clean

The PR remains draft because its profile is intentionally completed by the separate CorOS 4.1 catalog/profile PR #44.

Upstream sync — 2026-09-12

Merged upstream main at 640a8e9 (maintainer PRs #63–#65: measured pan spans and Global EQ scale/offset corrections). Post-merge validation: 14 listing/duplication tests plus 273 profile/docs tests passed, 1 skipped; mypy clean (46 files); git diff --check clean.

Upstream sync - 2026-09-19

Merged upstream main at cfe39cc6592148d4196fa3028c2776f6618c41b6 (including maintainer PRs #70-#75) and resolved overlaps in favor of the current upstream API and evidence model.

Current head: 3e74f1e3bbd6c701e5f44520a73679b0818665ec

Focused validation: 10 duplicate-setlist tests and 26 session tests passed; writing-policy checks passed (18 passed, 1 skipped).

GitHub CI on this exact head is green: build + twine check, mypy, and tests on Python 3.11, 3.12, and 3.13 all pass.

No new maintainer comments were posted during this sync. This PR remains draft under its stated hardware/evidence policy; the displayed CHANGES_REQUESTED decision is the earlier review and has not yet been refreshed.

Current-head correction and hardware coverage — 2026-09-27

The older upstream-sync entry above records an intermediate 3e74f1e snapshot. The actual current PR head is 5eed351ee74871643f159b46d248e9db8d9cd708.

Added tests/hardware/test_duplicate_setlist.py, marked for duplicate_setlist. It creates a unique empty source setlist, invokes native COPY once, verifies the device-selected destination, then removes only the exact source and a returned/token-identifiable destination after confirming each is empty. It never replays an uncertain COPY or DELETE; an unidentifiable destination is reported and left untouched. The operation is no longer listed as lacking a hardware test.

Validation on this head: full offline suite 2,979 passed / 7 skipped; marker, hardware-gate, report and writing checks 134 passed / 3 skipped; mypy clean across 46 source files; the focused hardware test collects (1 test); git diff --check clean. The hardware test has not been run on this head. The 2026-09-26 device attempt stopped before COPY because the source listing read back empty, and confirmed cleanup of the temporary source. No duplicate was sent. A CorOS 4.1.0 hardware run of the newly named test remains necessary.

Final follow-up — 2026-09-27

Current head: c7961c2e4bd1a6c5a3c0120f0ceec0db8471dd95 (supersedes the 5eed351 head recorded above). The hardware test now bases preflight, source/destination cleanup, and deletion confirmation on two identical folder generations, matching the library's convergence rule and avoiding decisions from partial listings.

On this head: marker/hardware-gate/report checks 57 passed; hardware test collection 1 test; py_compile, mypy (46 source files), and git diff --check clean. GitHub CI run 36284075440 passed all five jobs: build/twine, mypy, Python 3.11, 3.12, and 3.13. Hardware execution is still outstanding; no operation is claimed from collection or CI.

Exact-head hardware confirmation — 2026-09-27

Ran tests/hardware/test_duplicate_setlist.py --hardware --verifies duplicate_setlist on exact head c7961c2e4bd1a6c5a3c0120f0ceec0db8471dd95, Quad Cortex / CorOS 4.1.0:

1 passed, 0 failed, 0 errored, 0 skipped; operations measured: 1 of 1 claimed
passed: ['duplicate_setlist']
1 passed in 344.55s

The UUID-named empty source was created, native COPY was sent once, the device-selected empty destination was read back, and both test-owned folders were deleted and confirmed absent. The test did not touch any preset. The shared Rust broker is restored; fresh state reads show the device still on clean empty slot 52. The PR remains draft pending the profile dependency in #44 and maintainer re-review.

Current synced status — 2026-09-27 (head 9feead8)

This supersedes the earlier head and validation above. Merged current upstream main (d8d40f3) in 9feead8, reconciling the current profile/test API.

  • Full offline suite: 2,985 passed, 7 skipped; exact-head CI run 36288982490 is green (build/twine, mypy, Python 3.11–3.13).
  • Exact-head CorOS 4.1.0 direct-HID duplicate test: 1 passed in 344.24s. It copied a uniquely created empty source once, verified the destination, and removed only test-owned empty folders; stable read-back confirmed cleanup.
  • QC Remote is restored and synchronized on the original clean preset at 1A; no disposable test folders remain.

The full hardware suite has not run on this exact head. The PR remains draft pending that gate, its #44 profile dependency, and maintainer review.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Same-commit hardware preflight passed and the read path returned two complete 588-folder generations. It sent zero persistent File frames because the loaded preset is dirty and 11 of 12 custom setlist slots are occupied; a disposable source plus firmware-created duplicate need two simultaneous free slots. Keeping this draft until the owner cleans the loaded preset and frees one existing custom setlist slot. No existing setlist will be repurposed or deleted.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Hardware follow-up and listing-race correction are now pushed at final branch head \2d70432.

A same-combined-commit CorOS 4.1.0 run sent the sparse firmware COPY exactly once. The device-created destination matched the disposable two-preset source by position/name/instrument after 49.878 seconds, and cleanup removed both source and destination. No per-preset writes or COPY replay occurred.

That run also disproved the prior assumption that every matching File broadcast is complete: mutation polling saw one-slot and transient blank generations before later 256-slot listings. The implementation now refuses to baseline or confirm from those partial generations. Source and destination each require two identical generations containing every slot index 0..255; COPY remains exactly-once. Offline validation on this head: duplicate-setlist tests 4 passed; docs/marker tests 242 passed, 1 skipped; mypy clean; diff check clean.

The broader offline run produced 2698 passed, 3 skipped, 4 failures. Those four are existing branch/Windows fixture failures outside this change (manual-coverage tally, two path-separator hardware-gate assertions, and the pre-flip snapshot hash). This PR remains draft because the new final-head read-only stabilization has not itself had the required final-commit hardware-suite run.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Exact hardened head \2d70432\ now has isolated CorOS 4.1.0 physical corroboration. One COPY (request_id 2) created the firmware-named destination and completed in 104.587s; two identical complete 256-slot destination generations matched the stabilized source, and both disposable folders were deleted and confirmed absent. The PR description now carries the exact payload, branch boundary, validation, and cleanup evidence. I am leaving the PR draft because \duplicate_setlist\ still has no state-neutral test in the formal pytest hardware suite; the repository rule requires that gate or a maintainer waiver before ready-for-review.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Formal final-head hardware gate completed on exact \2d70432: \5 failed, 91 passed, 3 skipped in 217.16s (0:03:37). The operations report passed all 16/16 claimed operations with zero operation regressions. The five failures are three missing QuadCortex41 generated snapshots, the newly observed three-message 4.1.0 Version sequence, and one lazy Recall/delayed-cache timeout. The exact-head duplicate run remains successful and clean. I have added the complete gate result to the description and left this draft pending those unrelated profile/state-suite fixes or a maintainer waiver.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

The five formal-gate failures are now fully accounted for without expanding this PR's scope: the three CorOS 4.1 snapshots and corrected three-message Version observation are updated in #44 at f25caf1; the lazy RecallPreset retry and two-arrival/cache handling are isolated in draft #62 at 25a7126. The combined exact-hardware tree carrying both fixes passed 96 tests with 3 fixture-dependent skips and all 16/16 claimed operations green. This duplicate-setlist PR remains draft until those dependencies land, then it can be rebased and its exact-head gate repeated without bundling either fix here.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Dependency status refreshed: #44 is now 5bfec83 and #62 is b73d38c. Their clean integration commit 99a5cd5 passed the physical hardware suite 95 passed / 4 skipped / 0 failed in 204.66s, with all 16 operations claimed and no VERIFIED regression. The earlier combined run passed 96 / 3 with all 16 green. This duplicate-setlist PR can be rebased and its exact gate repeated after those two isolated prerequisites land; no profile/cache code has been bundled here.

@jonathanstokes jonathanstokes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for this. Replacing N recall-and-save round trips with one firmware-side copy is the right direction, and the wire message is minimal and decodes exactly as you describe. The COPY really is sent once and every verification path really is read-only, which is the property that matters most here. Two things need to change before this can merge, plus a handful of smaller ones I have left on the lines.

The main ask: this belongs on the 4.1 profile, not on the base class.

QuadCortex in client.py is not a generic client. Under ADR-0020 it is the profile for a Quad Cortex on CorOS 4.0.1, which is the unit I have here, and every other profile inherits from it. So writing the new duplication into that class is a statement that a 4.0.1 unit does this, and nothing in the PR measures a 4.0.1 unit. Your evidence is a 4.1.0 run, which is good evidence about 4.1.0.

The result is backwards from what either of us wants. My 4.0.1 unit would start sending a message it has never been seen to accept, while a real 4.1 unit connecting as QuadCortex41 gets the operation refused, because that profile's verified list is still empty. The profile that actually measured the thing is the one that would refuse it.

There are two ways out and both are fine by me:

  1. Put the override on QuadCortex41 and list it in that class's VERIFIED, and have the base refuse with ControlNotDrivable(control, evidence, workaround) whose evidence says it has not been measured on 4.0.1. That is the shape ADR-0020 describes and it needs nothing from me.
  2. Leave it on the base and I measure it on 4.0.1 first. Say the word and I will run it here. If it works on both, the base is the right home and your 4.1 observation gets recorded beside the 4.0.1 one.

Second: I cannot check the message shape against anything.

The whole change rests on Cortex Control 4.1.0 building File{COPY, type: 0, folder{key, is_factory: false}}. I went looking for corroboration and found none available to me. The three raw Cortex Control 4.0.1 captures in my lab repo carry host-sent File READ, CREATE, DELETE and MOVE, and no COPY at all, and I have no Cortex Control source to read. That does not make the shape wrong. It does mean your report is the only evidence in existence, so it is worth saying explicitly in the PR description how the builder was observed, and it is another reason the operation should sit on the profile that saw it.

One more thing to add before merge: the operation coverage table in docs/protocol.md, the one carrying the create_setlist and delete_setlist rows near line 2830, gains no row for this. That table is the index of what the library can do and how, and this change turns duplication from a client-side composition into a real device operation with its own message, which is exactly the case the table exists for.

On marking it ready: you have already noted the suite is not green on the final head and kept this as a draft, which is the right call, and the three snapshot failures and the Version-count failure are all known 4.1 profile gaps rather than anything you broke. Worth saying plainly that the run you did have is not a substitute for pytest tests/hardware --hardware naming this operation: no hardware test names duplicate_setlist, so the suite never measured the thing this PR adds, whatever the pass count says.

The rest of my comments are on the lines. The first one is a green-to-red change in the offline suite and takes one line to fix.

Comment thread docs/manual-coverage.md Resolved
Comment thread pyquadcortex/protocol/client.py Resolved
Comment thread pyquadcortex/protocol/client.py Resolved
Comment thread docs/protocol.md Resolved
Comment thread pyquadcortex/protocol/client.py Resolved
Comment thread pyquadcortex/protocol/client.py Resolved
Comment thread pyquadcortex/protocol/client.py Resolved
Comment thread tests/test_client.py Resolved
Comment thread tests/test_client.py Resolved
Comment thread tests/test_hardware_markers.py Resolved
@tony-xmelon tony-xmelon changed the title Use firmware-native setlist duplication Scope native setlist duplication to CorOS 4.1 Sep 11, 2026
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Thank you — addressed in ed52166. Native COPY now exists only on QuadCortex41; 4.0.1 refuses immediately with an auditable copy_preset workaround. The unmeasured setlist ceiling is gone, occupancy uses name, folders and full 256-slot inventories stabilize, deadline/error/no-replay paths are tested, and the older 4.0.1 record is restored beside 4.1.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Synchronized with upstream main at 640a8e9 (merged maintainer PRs #63–#65). Current head is a8f5400; post-merge validation is recorded in the PR body.

…ate-setlist

# Conflicts:
#	changelog.md
#	docs/api.md
#	docs/manual-coverage.md
#	docs/protocol.md
#	pyquadcortex/protocol/profiles.py
#	tests/test_profiles.py
@jonathanstokes

Copy link
Copy Markdown
Contributor

#42 has merged, and it brought pyquadcortex/protocol/profiles.py into main in a new shape. This branch no longer merges cleanly. The conflicts are in:

  • changelog.md
  • pyquadcortex/protocol/profiles.py
  • tests/test_profiles.py

Could you merge current main in? A review before that reads code that cannot land.

…ate-setlist

# Conflicts:
#	changelog.md
#	pyquadcortex/protocol/profiles.py
#	tests/test_profiles.py
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Merged current upstream main and resolved the profile overlap by retaining the measured native duplicate operation while inheriting main's set_device_name evidence. Current head: d346757.

Exact-head validation: 2979 passed, 7 skipped; mypy clean; build and twine check passed; diff check clean.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Attempted a safe focused CorOS 4.1.0 validation on exact PR #61 head d346757. I created a unique temporary source setlist, saved the currently loaded clean preset into two temporary slots, and required listing read-back before sending COPY. Although both save calls returned their requested names, a fresh preset listing returned zero entries, so I stopped before sending duplicate_setlist. I deleted the exact temporary source and confirmed it was absent. No duplicate operation was sent and no existing setlist was touched. This is inconclusive, not a hardware pass; the device's inconsistent preset-list read-back needs resolving before a controlled duplicate test.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

I added the hardware test you asked for at 5eed351. It exercises native COPY against a uniquely created empty setlist, confirms the destination, and cleans up only exact test-owned empty folders. It does not retry uncertain writes; an unidentifiable device-named folder is reported rather than deleted. The full offline suite passes (2,979 passed, 7 skipped), and the focused hardware test collects (1 test), but I have not run it on hardware. The prior 2026-09-26 attempt stopped before COPY after the source listing returned empty, then removed and verified absence of the temporary source. Could you run pytest tests/hardware/test_duplicate_setlist.py --hardware --profile QuadCortex41 on a 4.1.0 unit, or let me know when the shared device is available for a direct-HID run?

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Follow-up: I tightened cleanup/readback to require two identical folder generations before acting, so partial File listings cannot falsely confirm absence or trigger a wrong delete. The final head is c7961c2. CI run 36284075440 is green on build/twine, mypy, and Python 3.11–3.13; focused hardware/report/gate tests 57 passed; the hardware test collects one test but has not been run on device.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Exact-head CorOS 4.1 hardware evidence is now recorded in the description: the marked duplicate_setlist test passed (1/1) in 344.55s, including one native COPY and confirmed cleanup of both test-owned empty setlists. The device remains on clean empty slot 52, and the shared Rust broker is restored. I’m keeping this draft until the #44 profile dependency is settled; please re-review the now-tested operation when convenient.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Following your 2026-09-22 request, I merged current upstream main (d8d40f3) into this branch in 9feead8; the updated profile architecture and test expectations are now in the PR branch.

Validation on pushed head 9feead8:

  • full offline suite: 2,985 passed, 7 skipped
  • exact-head CorOS 4.1.0 direct-HID test: 1 passed; native duplicate_setlist measured in 344.24s
  • the test created a unique empty source, verified the returned duplicate, then removed only the two test-owned empty folders; stable directory read-back confirmed cleanup
  • shared QC broker is restored and synchronized; original Fender Deluxe 37 at 1A is clean, with no QCdup-* or disposable test folders remaining

CI is pending from the push. I am keeping the PR draft pending the profile dependency/review sequence.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants