Skip to content

Allow deferring the initial file listing - #48

Merged
jonathanstokes merged 7 commits into
stokes-audio:mainfrom
tony-xmelon:codex/optional-initial-file-listing
Sep 14, 2026
Merged

jonathanstokes merged 7 commits into
stokes-audio:mainfrom
tony-xmelon:codex/optional-initial-file-listing

Conversation

@tony-xmelon

@tony-xmelon tony-xmelon commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • add initial_file_listing to protocol and model-level connect()
  • preserve eager enumeration at every public/internal default
  • when disabled, omit exactly the handshake's initial File READ and leave every other READ unchanged
  • document that explicit listing still works on verified profiles, while unverified profiles require Support.EXPERIMENTAL
  • leave the effect on later file-change broadcasts explicitly unmeasured

Testing

Final head: 68f2a35

  • python -m pytest tests/test_client.py tests/test_session.py tests/test_device.py tests/test_docs.py -q: 619 passed, 1 skipped
  • full Windows run: 2697 passed, 3 skipped, 3 failed; the three failures are the known Windows path/line-ending fixtures fixed in Expose protocol operations and make the test suite portable #42
  • python -m mypy: passed
  • git diff --check: passed

Tests pin:

  • _hello() defaults to one FileMessage READ
  • protocol.connect() defaults to eager enumeration
  • pyquadcortex.connect() passes True by default
  • the deferred READ list equals the default list with exactly one FileMessage removed
  • "File" remains linked to _SUBSCRIBE_TYPES

Hardware

Exact final head 68f2a35, read-only lifecycle check on 2026-09-08 against a standard Quad Cortex running CorOS 4.1.0:

connect_seconds=3.04
folders=586
listing_seconds=20.07

The connection used profile=QuadCortex41, support=Support.EXPERIMENTAL, and initial_file_listing=False; the explicit list_folders() call then returned the complete listing. This proves deferred connection and on-demand enumeration, not whether the omitted READ changes later file broadcasts.

A maintainer with CorOS 4.0.1 should run the complete hardware suite before merge.

Checklist

  • Returned to draft while addressing review
  • Default behavior pinned at all layers
  • Exact final-head live lifecycle checked
  • Documentation is profile-specific and avoids unmeasured claims
  • Changelog entry added
  • No unrelated changes bundled in

Upstream sync — 2026-09-12

Merged current upstream main at 640a8e9 (including merged PRs #63–#65) into this branch. Current head: 4da0012

Offline suite excluding the three #42-owned Windows portability checks: 2595 passed, 3 skipped; full run: 2722 passed, 3 skipped, 3 known portability failures; mypy: clean across 46 source files; git diff --check: clean.

The three excluded Windows checks are unrelated path-separator and CRLF-fixture assertions fixed within #42; they remain isolated there rather than being copied into this PR.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Updated onto #52 in e879dc9. The initial_file_listing option is now threaded through the profile-resolving connect path while preserving the single shared handshake-patience budget. Focused session/device/client/profile validation: 409 passed; mypy clean; git diff --check clean.

@tony-xmelon
tony-xmelon marked this pull request as draft September 8, 2026 08:41
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Synced with upstream �2331cc at head d545599 and moved back to draft pending the required full hardware suite on this exact commit. Focused validation: 445 passed; mypy clean.

@tony-xmelon
tony-xmelon marked this pull request as ready for review September 8, 2026 09:38

@jonathanstokes jonathanstokes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want this change in. The problem is real. On my unit the folder listing is several hundred messages. They keep arriving for about fifteen seconds after connect() has returned. An app that never asks for a folder tree pays that cost for nothing.

Three things here are good.

You skip one send inside the loop. You did not build a second copy of _SUBSCRIBE_TYPES. So no module state leaks between two connections in one process. I checked this. After a deferred connect the tuple is the same object. A second default connect in the same process still sends the File READ.

The default path sends the same bytes as main. I checked this too. I serialised every message _hello sends on both branches. 26 messages, same bytes.

You also merged the profile work in yourself and fixed the argument tests. That saved me a conflict.

Now the part that is mine. The multi-profile design did not exist when you wrote this. It landed on 2026-09-03 as ADR-0020. The rules for contributors are in contributing.md, under "Working with hardware" and "Adding a device profile". You could not follow a rule that was not written. What I ask for below comes from that design. It is not a mistake on your side.

Here is the design in short. QuadCortex is not only the base class. It is the Quad Cortex on CorOS 4.0.1. That is my unit. QuadCortex41 is a separate class, and it has measured nothing yet. So I must not write anything into QuadCortex that I have not measured on 4.0.1. A 4.1.0 observation goes next to the 4.0.1 record, with the date and the CorOS version. It never replaces it.

Most of my comments are on the lines themselves. Two points below are not, because your PR does not touch those files.

critical: docs/protocol.md needs a matching edit. Line 514 says every READ in step 4 is a subscription. Line 556 calls it "the full 22-type subscription set". Your change makes 21 possible, and the record does not say so. Please add a sentence to section 4.2. Please also put your 4.1.0 measurement next to the 4.0.1 record, with the date. That file is the record. A docstring is not.

nit: changelog.md has an "Unreleased" section. This adds an argument to two public functions. Please add a short entry. That file answers one question: "I upgraded, what is different for me?"

What I will do. I will run pytest tests/hardware --hardware on my 4.0.1 unit before I merge. I will write what I see next to your 4.1.0 record. Your description says you have no 4.0.1 unit. That is the right thing to do, so this run is mine. I will also try to answer the push gate question on the unit. That needs a measurement, and neither of us should guess it.

If this is more work than you want, say so. I can split the change up myself and credit you. Or you can close this, and I will pick up the idea later. Both are fine.

Stokes

Comment thread pyquadcortex/protocol/client.py Resolved
Comment thread pyquadcortex/protocol/client.py Resolved
Comment thread pyquadcortex/protocol/client.py Resolved
Comment thread pyquadcortex/protocol/session.py Resolved
Comment thread tests/test_client.py Resolved
Comment thread docs/api.md Resolved
Comment thread README.md Resolved
@tony-xmelon
tony-xmelon marked this pull request as draft September 8, 2026 18:53
@tony-xmelon
tony-xmelon marked this pull request as ready for review September 8, 2026 18:57
@jonathanstokes

Copy link
Copy Markdown
Contributor

I ran the hardware suite on my 4.0.1 unit, on your final commit 68f2a35.

============== operations on QuadCortex (CorOS 4.0.1, MAINTAINER) ==============
tests: 95 passed, 0 failed, 0 errored, 4 skipped; operations measured: 16 of 16 claimed
passed, not VERIFIED:                   [] (0)   <- candidates to add
VERIFIED and claimed by a test, failed: [] (0)   <- regressions by name

================== 95 passed, 4 skipped in 188.57s (0:03:08) ===================

The offline suite is 2700 passed, 3 skipped and mypy is clean on the same commit.

I also measured the open question. Your docstring and the new passage in docs/protocol.md both call it unmeasured. I have the answer now.

Omitting the File READ does not stop later file change announcements.

I connected twice and did the same thing on the unit each time: save a preset into an empty slot, then change scene. A listener recorded every message in both runs, with no type filter.

connect burst after the on-unit save
default 399 FileMessage 1
initial_file_listing=False 0 1

The messages that follow the save are the same in both runs, in the same order: File, then UndoRedo, then RecallPreset, Scene, RecentsFavorites, SetlistPosition, and the Grid pushes. So the flag drops the 399 message enumeration and nothing else.

Two notes on that.

Your first version of the docstring said this, and you were right. The caution I asked for was still the correct thing at the time: you had no 4.0.1 unit, and this needed one. There is nothing here for you to change.

The correction that follows is mine, not yours. docs/protocol.md line 514 says every READ in step 4 acts as a subscription. That is not true for File, and I can now show it. I will write that up with these numbers, beside the existing record.

One caveat about my own measurement: one trial per arm. It is a positive result in both arms, and 399 against 0 is not ambiguous, so I am confident in it. I will say the same in the record.

Stokes

jonathanstokes
jonathanstokes previously approved these changes Sep 9, 2026

@jonathanstokes jonathanstokes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good to merge. You closed every point, and two of them better than I asked: the deferral test now compares the whole eager list against the deferred one, and the default has its own assertion at both layers.

I checked that the coverage really is there. I flipped each default to false in turn and broke the guard string, and every one of those now fails the suite. Before your fix commit, two of them passed.

Nothing further from you. The docs/protocol.md wording about the push gate is mine to replace, not yours, and I have the measurement now.

Stokes

…ial-file-listing

# Conflicts:
#	changelog.md
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Synced with upstream main at 640a8e9 (including #63–#65). New head: 4da0012. Offline suite excluding the three #42-owned Windows portability checks: 2595 passed, 3 skipped; full run: 2722 passed, 3 skipped, 3 known portability failures; mypy: clean across 46 source files; git diff --check: clean. The three shared Windows-only baseline checks remain isolated in #42.

…l-file-listing

Catch-up merge so the branch is up to date with main, which the ruleset requires
before merging. One conflict, in changelog.md: this branch's "Initial file
enumeration can be deferred" entry against main's "Where a knob's numbers start
comes from the device now" entry. Both kept, this branch's first.

No change to the contribution itself. Verified on the merge result: offline
suite 2796 passed 4 skipped, mypy clean, and the hardware suite on the
maintainer's CorOS 4.0.1 unit 110 passed 3 skipped, 17 of 17 claimed operations
measured with no regressions by name.

@jonathanstokes jonathanstokes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving on e70fd80.

I did the catch-up merge with main myself, because the only conflict was in changelog.md, between your entry and mine. Both are kept, yours first. Nothing else conflicted, and your contribution is unchanged: the diff against main is the same work I read the first time.

Verified on the merge result, which is the commit above:

  • offline suite 2700-plus: 2796 passed, 4 skipped
  • mypy clean
  • hardware suite on my CorOS 4.0.1 unit, firmware d14e:
============== operations on QuadCortex (CorOS 4.0.1, MAINTAINER) ==============
tests: 110 passed, 0 failed, 0 errored, 3 skipped; operations measured: 17 of 17 claimed
failed or skipped:                      [] (0)
passed, not VERIFIED:                   [] (0)   <- candidates to add
VERIFIED and claimed by a test, failed: [] (0)   <- regressions by name

================== 110 passed, 3 skipped in 181.12s (0:03:01) ==================

I also checked the coverage still bites, by breaking the code on purpose and running the suite each time. Each default set to false in turn fails, and dropping File from _SUBSCRIBE_TYPES fails 23 tests. Before your fix commit, two of those passed quietly.

Thanks for the sync merges along the way. The only thing left on this branch was mine to do.

Stokes

@jonathanstokes
jonathanstokes merged commit b670e4b into stokes-audio:main Sep 14, 2026
5 checks passed
@tony-xmelon
tony-xmelon deleted the codex/optional-initial-file-listing branch September 22, 2026 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants