Skip to content

Add physical screen control and capture - #50

Open
tony-xmelon wants to merge 19 commits into
stokes-audio:mainfrom
tony-xmelon:codex/remote-screen-control
Open

tony-xmelon wants to merge 19 commits into
stokes-audio:mainfrom
tony-xmelon:codex/remote-screen-control

Conversation

@tony-xmelon

@tony-xmelon tony-xmelon commented Sep 3, 2026 •

Copy link
Copy Markdown
Contributor

Summary

  • recover and register CorOS 4.1 RemoteControl messages
  • keep capture_screen, graphics_tree, tap_screen, and swipe_screen on the shared operation surface as structured CorOS 4.0.1 refusals
  • implement all four operations on QuadCortex41 and list them in that profile's VERIFIED set
  • validate complete full-display PNG replies and warn about rejected RemoteControl updates
  • prime the remote surface and wait the remainder of the measured 300 ms settle before sending the timed, atomic touch pair
  • take display dimensions from the device profile and make touch release interruption-safe

This follows the maintainer's ADR-0020 guidance from reviews on #47 and #49. The maintainer-owned CorOS 4.0.1 measurement remains intentionally out of this branch.

Testing

Final head: cae5929679a2edbb49fc01417b55c26c6933ad99

  • focused review surface: 694 passed, 1 skipped in 20.40s
  • full Windows run: 2723 passed, 3 skipped, 3 failed in 48.98s; the three failures are the existing Windows path-separator/CRLF fixture checks
  • python -m mypy: passed all 46 source files
  • git diff --check: passed

Offline tests pin the exact protobuf bytes, asynchronous full-frame screenshot matching, rejected-update diagnostics, capture-to-tap settle remainder, timeout context, atomic touch completion after interruption, accepted coordinate boundaries, invalid coordinates, strict-profile behavior, and verification inventory.

Hardware

Historical complete 4.1 hardware run from #42 (head acf3ffe, 2026-09-03, standard Quad Cortex, CorOS 4.1.0):

$env:PYQUADCORTEX_UNDO_SCRATCH_SLOT='255'; .venv\Scripts\python.exe -m pytest --hardware -q
2742 passed, 8 skipped in 225.72s (0:03:45)

That run predates the per-profile operation reporter added on 2026-09-05, so there is no operation-summary block in its retained output to paste without inventing one.

Remote-screen capture was subsequently run on exact head 1762dda on 2026-09-08:

tests: 1 passed, 0 failed, 0 errored, 0 skipped; operations measured: 1 of 1 claimed
passed:                                 ['capture_screen'] (1)
failed or skipped:                      [] (0)
passed, not VERIFIED:                   [] (0)
VERIFIED and claimed by a test, failed: [] (0)
1 passed in 23.66s

The complete 800 x 480 PNG, IHDR, and terminal IEND were validated. tap_screen(184, 147) was manually verified by tony-xmelon on 2026-09-04: it opened the intended top-row Grid block and a following capture showed its editor. This is recorded explicitly because an automated tap changes the owner's screen and the protocol has no reliable restore operation.

The PR remains draft until a complete final-head hardware run can be performed without colliding with the active device-testing work.

Checklist

  • Maintainer review comments addressed
  • Profile ownership and refusals tested
  • Full offline and typing validation run
  • Complete hardware suite run on final head

Upstream sync — 2026-09-12

Merged current upstream main at 640a8e9 (including merged PRs #63–#65) into this branch. Current head: 4dc6c03

Offline suite excluding the three #42-owned Windows portability checks: 2621 passed, 3 skipped; full run: 2748 passed, 3 skipped, 3 known portability failures; mypy: clean across 46 source files; git diff --check: clean.

The three excluded Windows checks are unrelated path-separator and CRLF-fixture assertions fixed within #42; they remain isolated there rather than being copied into this PR.

Local parity sync — 2026-09-14

Local Rust/Windows/Android protocol work exposed the remaining schema-backed RemoteControl surface, so commit 923eaef adds it here without changing the firmware boundary:

  • swipe_screen(x, y, to_x, to_y, timeout=) performs a fresh framebuffer preflight and then atomically sends the exact touch-down / DRAG-with-both-endpoints / touch-up sequence, 20 ms apart; integer bounds and distinct endpoints match the shared host contract
  • graphics_tree(timeout=) installs its waiter before sending RemoteControl{READ, graphics_tree:{}} and accepts only a nonblank, NUL-free, at-most-1-MiB UTF-8 UPDATE
  • both remain structured refusals on the unmeasured CorOS 4.0.1 base and live only on QuadCortex41
  • offline tests pin the exact protobuf bytes, response filtering, size/content bounds, fresh priming, atomic sequencing, timing, invalid coordinates, and timeout context

Validation in an isolated environment: 2765 passed, 3 skipped, 3 deselected; the deselections are the same three documented Windows path/CRLF baseline checks owned by #42. python -m mypy: clean across 46 source files. Wheel and sdist build plus twine check: passed. git diff --check: passed.

The PR remains draft. The read-only graphics_tree hardware test is present; a complete hardware run on exact head 923eaef and any screen-changing swipe confirmation should be recorded before requesting review.

Upstream sync - 2026-09-19

Merged upstream main at cfe39cc6592148d4196fa3028c2776f6618c41b6 (including maintainer PRs #70-#75) and resolved overlaps in favor of the current upstream API and evidence model.

Current head: cae5929679a2edbb49fc01417b55c26c6933ad99

Focused validation: 45 remote-control tests passed; focused coverage and writing-policy checks passed.

GitHub CI on this exact head is green: build + twine check, mypy, and tests on Python 3.11, 3.12, and 3.13 all pass.

No new maintainer comments were posted during this sync. This PR remains draft under its stated hardware/evidence policy; the displayed CHANGES_REQUESTED decision is the earlier review and has not yet been refreshed.

Current scope and status — 2026-09-27

This section supersedes the earlier summary and validation text above. Current head: f8387d9d0c544272186666ded3e2d6c86f08dd5a.

The public surface is narrowed to capture_screen() and tap_screen(); swipe_screen() and graphics_tree() were removed in response to review and need separate review/evidence before returning. The hardware test now skips profiles that do not identify as CorOS 4.1.0. The docs restore the message-type 72 RemoteControl row, preserve the bytes separately from enum-label interpretation, use “physical screen” consistently, and describe only the shipped two-operation scope. ADR-0024 records the atomic gesture transport decision; the 2026-09-27 STEERING entry records the scope change.

Validation at this content: full offline suite 2,996 passed, 7 skipped; focused client/registry/marker/writing suite 458 passed, 3 skipped; mypy clean (46 source files); diff check clean. The direct-HID capture and manual tap observations are recorded above. The complete hardware suite has not been run on this exact head. QC Remote currently reports the device disconnected, so no reconnect was attempted. PR remains draft pending exact-head hardware validation.

Current synced status — 2026-09-27 (head 5c4d80d)

This supersedes the earlier head and validation above. Merged upstream main (d8d40f3) in 5c4d80d, preserving both the screen-control record and upstream catalog-payload record.

  • Full offline suite: 3,002 passed, 7 skipped; exact-head CI run 36289727069 is green (build/twine, mypy, Python 3.11–3.13).
  • Exact-head CorOS 4.1.0 capture_screen hardware test: 1 passed.
  • Complete exact-head hardware suite: 3,116 passed, 1 failed, 16 skipped in 316.16s. The sole failure is the unrelated option-position coverage floor: the clean Fender Deluxe 37 preset exposes 59 positions versus a 100-position minimum; no mismatch was found. The pending Add profile-owned CorOS 4.1 catalog and observations #44 change classifies insufficient coverage as a skip, but is not yet on main, so this is not a clean full-suite pass.
  • QC Remote is restored and synchronized on the original clean preset at 1A; dirty: false, master volume unchanged at 97.

The PR remains draft until the full-suite precondition is resolved and review is refreshed.

Exact-head hardware recheck — 2026-09-27

This supersedes the earlier complete-suite failure recorded above. On exact head 5c4d80d15ec11be7a7f34399916653ff79cc6505, the complete CorOS 4.1.0 hardware suite passed using a saved high-coverage preset:

116 passed, 0 failed, 0 errored, 8 skipped in 290.94s
operations measured: 22 of 22 claimed; 22 passed; no operation-level failures or skips

capture_screen passed. The earlier sole failure was the 100-position coverage floor on Fender Deluxe 37 (59 positions), not a behavioral mismatch; this rerun used Shine On Crazy Synth at 12E and met the coverage precondition. The shared broker was reconnected and QC restored to clean Fender Deluxe 37 at 1A, synchronized, master volume unchanged at 97.

Empty-slot behavior is separate from the stored-preset screenshot request: capture_screen() captures the live display independent of slot occupancy. PR #49's QuadCortex41.preset_screenshot() checks whether the requested slot is already loaded and routes that case directly to capture_screen(), so an empty current slot does not wait for a nonexistent stored image. This was verified against an empty loaded slot (7E): live 800 x 480 PNG returned, test passed in 12.49s, loaded address unchanged. A request for another slot still asks for that slot's stored screenshot.

The PR remains draft pending maintainer re-review; the earlier review's requested scope/evidence/document follow-ups still need to be formally cleared.

Exact final-head validation — 2026-09-27 (1001b61)

This supersedes the earlier hardware-status entries above. After correcting the remaining stale 71 message-type count in the open-questions section, validation was rerun on exact head 1001b61516ad1548b911ac2428593c15f942a745:

  • Offline: 3,002 passed, 7 skipped in 43.24s.
  • Complete direct-HID CorOS 4.1.0 suite, with Shine On Crazy Synth at 12E loaded to satisfy the option-position coverage floor: 116 passed, 0 failed, 0 errored, 8 skipped in 290.72s.
  • All 22 claimed hardware operations passed; no claimed operation failed or skipped. capture_screen passed.
  • Exact-head GitHub CI run 36292984335: build/twine, mypy, and Python 3.11/3.12/3.13 all pass.
  • QC Remote was reconnected and the device restored to clean Fender Deluxe 37 at 1A, synchronized, master volume 97.

All 18 inline review threads are resolved. The requested operation scope is capture and tap only; swipe and graphics-tree remain out of this PR. The 4.0.1 measurement remains with the maintainer, as agreed in the review. The PR is ready for the requested maintainer re-review.

Exact-head full-suite rerun — 2026-09-28

Reran pytest tests/hardware --hardware -q on unchanged head 1001b61516ad1548b911ac2428593c15f942a745 against CorOS 4.1.0 / QuadCortex41, using the saved high-coverage preset at 12E required by the option-position test: 116 passed, 0 failed, 0 errored, 8 skipped in 290.55s. All 22/22 claimed hardware operations passed; no operation-level failures or skips. The original active preset was restored cleanly; the shared broker is ready/synchronized and master volume remained unchanged at 100.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Updated onto #52 in 43c9c1b. capture_screen is now represented by its read-only live test in the new operation inventory; tap_screen remains explicitly excused there because it changes the owner's current screen and has no reliable protocol restore, while its exact sequence and validation stay pinned offline. Focused validation: 409 passed; hardware suite collection 100/100; mypy clean; git diff --check clean.

@tony-xmelon
tony-xmelon marked this pull request as draft September 8, 2026 08:41
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Synced with upstream �2331cc at head 644d4d4 and moved back to draft pending the required full hardware suite on this exact commit. Focused validation: 415 passed; 100 hardware tests collected; mypy clean.

@tony-xmelon
tony-xmelon marked this pull request as ready for review September 8, 2026 09:43
@tony-xmelon
tony-xmelon marked this pull request as draft September 8, 2026 19:10
@tony-xmelon
tony-xmelon marked this pull request as ready for review September 8, 2026 19:13

@Stokes-Quorum Stokes-Quorum left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Posted from the wrong account by mistake. The same review, from the repo owner, is below.

@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@stokes-audio stokes-audio deleted a comment from Stokes-Quorum Sep 9, 2026
@tony-xmelon
tony-xmelon marked this pull request as draft September 9, 2026 22:04
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Thank you for the detailed review. I have addressed the requested changes and accepted every nit in 7c5f1db:

  • record the last successful capture time and wait only the remainder of the 300 ms settle, including explicit capture-then-tap coverage
  • validate full-profile PNG dimensions and terminal IEND, reject region replies, and warn with the received shape when a candidate is rejected
  • add tap_screen(..., timeout=) and contextualise priming timeouts
  • record the dated manual tap verification explicitly instead of treating offline agreement as hardware evidence
  • ensure an interrupted inter-message sleep writes the remaining touch message without another sleep
  • move display dimensions onto Hardware, register RemoteControl explicitly, and exercise the strict default plus both coordinate boundaries
  • separate observed wire bytes from the enum interpretation, bound the TAP/MOVE and settle claims to the retained evidence, correct dates/profile ownership, distinguish preset Screenshot from the live framebuffer, and repair the architecture/ADR/coverage wording

I also removed the unrelated catalog-fingerprint bullet and updated the PR body with the exact retained #42 full-run command and pytest last line. That run predates the operation reporter, so I called out why no operation block exists rather than reconstructing one.

Validation on final head: focused 694 passed, 1 skipped; full offline 2723 passed, 3 skipped plus the three existing Windows-only path/CRLF fixture failures; mypy passed all 46 source files; git diff --check passed.

The shared 4.1 unit is currently occupied by the active device-testing task, so I converted this PR back to draft until the complete hardware suite can be run on the final head. I have left the 4.0.1 measurement entirely to your follow-up, as requested.

…-control

# Conflicts:
#	tests/test_client.py
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Synced with upstream main at 640a8e9 (including #63–#65). New head: 4dc6c03. Offline suite excluding the three #42-owned Windows portability checks: 2621 passed, 3 skipped; full run: 2748 passed, 3 skipped, 3 known portability failures; mypy: clean across 46 source files; git diff --check: clean. The three shared Windows-only baseline checks remain isolated in #42.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Local parity sync is now complete for this head (923eaef54cd627854744377c87d69e37f17575c2). The local contract and gateway fallback now cover the full four-method RemoteControl surface in this PR: capture_screen, graphics_tree, swipe_screen, and tap_screen.

Integrated locally at ca6fe846ebe9692a643954db12ff3ed8b41c2281 through the guarded full software-parity gate. Exact results on the final candidate:

  • Cortex protocol: 139 passed, 1 allowed private-evidence skip, 0 failed
  • Python gateway/MCP/audit: 1,416 passed, plus 1,504 subtests
  • Frontend: 559 passed; Android TypeScript: 63 passed
  • App parity: 16/16 capabilities; MCP parity: 107 actions / 110 gateway methods
  • pyquadcortex contract: 114 upstream + 10 pending = 124 methods
  • Rust/Python extracted extensions: 4/4
  • Rust tests, formatting, clippy, Windows native-shell tests/typecheck, cloud dry-run, dependency audits, and legal inventory: passed

The PR remains draft because this exact head has not yet had a fresh hardware run. Earlier retail CorOS 4.1 evidence is retained in the PR, but I am not treating it as exact-head validation.

…-control

# Conflicts:
#	CLAUDE.md
#	changelog.md
#	docs/ADR.md
#	docs/STEERING.md
#	docs/architecture.md
#	docs/manual-coverage.md
#	docs/protocol.md
#	pyquadcortex/protocol/client.py
#	pyquadcortex/protocol/profiles.py
#	tests/test_profiles.py
#	tests/test_session.py
@jonathanstokes

Copy link
Copy Markdown
Contributor

#42 has merged, and it brought pyquadcortex/protocol/profiles.py into main in a new shape. This branch no longer merges cleanly. The conflicts are in:

  • changelog.md
  • docs/manual-coverage.md
  • pyquadcortex/protocol/profiles.py
  • tests/test_profiles.py
  • tests/test_session.py

Could you merge current main in? A review before that reads code that cannot land.

…-control

# Conflicts:
#	changelog.md
#	docs/manual-coverage.md
#	pyquadcortex/protocol/profiles.py
#	tests/test_profiles.py
#	tests/test_session.py
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Merged current upstream main and resolved the profile and coverage overlaps. The 4.1 profile now retains all four RemoteControl operations plus create_local_backup and inherits main's set_device_name; the manual audit is reconciled at 106 features / 69 yes / 8 partly / 18 no / 11 n/a. Current head: �70ae2d.

Exact-head validation: 3013 passed, 7 skipped; mypy clean; build and twine check passed; diff check clean. This remains draft pending the exact-head hardware policy already recorded.

@jonathanstokes jonathanstokes left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for keeping this current with main. Two things before the detail.

First: swipe_screen and graphics_tree arrived in 923eaef5 on 2026-09-14 and no
maintainer has reviewed them. Roughly 285 lines of new operation surface. I am reading
them as first review, not as a response to anything I wrote.

Second, and the reason most of my 2026-09-09 comments are back: the upstream merge
ce1c8695 on 2026-09-19 deleted your answers to them. At 7c5f1dbc this branch carried
ADR-0021 "A timed device gesture is one atomic transport sequence", a STEERING change-log
entry and its ADR index row, the CLAUDE.md rule about send_sequence, four corrections to
architecture.md, and protocol.md section 7.0 with the probe record and the two dated
coverage rows. None of it is at b70ae2d0. The threads all read resolved. I do not think
you dropped them on purpose - upstream landed its own ADR-0021 under a different title, and
resolving that heading in upstream's favour took the rest with it. Restoring that content is
most of the work below.

Blocking

1. Both hardware tests in tests/hardware/test_remote_screen.py fail on a CorOS 4.0.1
unit rather than skipping.
The guard is if "capture_screen" not in profile.VERIFIED: pytest.skip(...). On the 4.0.1 profile VERIFIED is support.EVERYTHING, and
_Everything.__contains__ returns True for every name, so the skip never fires. The call
then raises ControlNotDrivable, which pytest reports as a failure. Checked offline against
the real classes rather than on a unit: "capture_screen" in QuadCortex.VERIFIED is True,
and QuadCortex(fake).capture_screen() raises. Skip on
profile.VERIFIED is support.EVERYTHING as well, or key the skip off
profile.__name__/MEASURED_ON. This is the one that stops the required hardware run on my
unit from being clean.

2. swipe_screen and graphics_tree are listed in QuadCortex41.VERIFIED with evidence
dated before they existed.
profiles.py says they "were subsequently exercised during
physical 4.1.0 screen sweeps on 2026-09-08", and the UNMARKED_OPERATIONS entry for
swipe_screen says the same. Both methods first appear in 923eaef5, 2026-09-14. The
2026-09-08 run pasted in the description reports passed: ['capture_screen'] (1). I take the
underlying observation to be real - from the Rust/Windows/Android work, not from this branch -
but VERIFIED is what the ADR-0020 guard reads, and listing them there is what lets a default
Support.VERIFIED connection drive an unmeasured swipe across the owner's screen with no
warning. I know __init_subclass__ forces the listing once you override
(client.py:388-393), so the choice is to measure them on a 4.1 unit on this head, or to
land them in a follow-up and keep this PR to capture_screen and tap_screen. I would
prefer the second - it also gets the reviewed half of this branch moving.

3. The evidence that makes the touch order safe is no longer in the tree. Every gesture
here ends on a message named PRESS, and the proto this PR adds declares PRESS = 0; RELEASE = 1. At 7c5f1dbc, protocol.md 7.0 carried the finding that settles it: "Sending
the labelled PRESS/RELEASE order left the touch held and eventually entered Grid drag mode."
That sentence is gone. What is left is docstrings asserting "touch-down ... then touch-up"
with no bytes and no measurement behind them. The next reader sees type=RELEASE opening a
tap, reads it as an obvious inversion, swaps the pair, and gets the measured failure - a held
touch that enters Grid drag mode, on an instrument with no protocol restore. Please restore
7.0, keeping the bytes and the interpretation separate the way that version did. It also
matters for swipe_screen, whose third message is not covered by the tap capture at all: the
swipe lifts because the tap lifts, which is an inference, not a measurement.

4. Three documents are contradicted by this PR's own code, and the record for
send_sequence is missing.
docs/protocol.md:313 says the schema declares 71 types with
NumberOfMessageTypes = 71 as the sentinel; the proto here says 73 and 73.
docs/architecture.md:163 and :166 name four transport methods and say the client depends
on four; there are five now. The send vs request vs await_broadcast table at :285 has no
send_sequence row, and the string send_sequence appears nowhere under docs/ or in
CLAUDE.md. All four were correct at 7c5f1dbc. Separately, CLAUDE.md asks for STEERING,
CLAUDE.md and ADR.md in the same pull request for changes under a section-4 owned path, and
this touches pyquadcortex/ and protocol/. Restoring your ADR-0021 under the next free
number is the piece I most want back - it is the only record of why timing lives in the
transport.

Worth fixing, not blocking

  • docs/protocol.md's Operation coverage table gains no rows, and four operations land here.
    capture_screen and tap_screen have honest rows to write (on-unit, 2026-09-08 and
    2026-09-04, named). The other two have none yet, which is the point of blocker 2.
  • docs/protocol.md:313 also lost the | 72 | RemoteControl | type-table row and the
    ModelPreset = 71 "recovered, not yet observed" note. ModelPreset is not mentioned in the
    description either, and it is the message type that decides RemoteControl is 72.
  • The Screenshot = 25 / RemoteControl = 72 disambiguation from 7.0 is worth restoring on
    its own. Two screenshot paths in one schema is exactly the thing the next reader trips on.
  • tests/hardware/test_remote_screen.py:30 asserts "zenUI" in tree. No run has seen a
    graphics tree, so that string is a guess in an assertion.
  • changelog.md is missing a blank line before ### Rename the unit, drive undo/redo.
  • docs/manual-coverage.md marks the row yes and rolls it into the headline "69 yes", for a
    feature my profile refuses. My earlier note asked for this not to overstate; partly with
    "on QuadCortex41" was closer.
  • Still six words for one thing: physical screen, physical display, remote screen,
    remote-control surface, framebuffer, mouse. One word for the screen, and keep "mouse" to the
    field name.

Not blocking, for the record

  • send_sequence's interruption handling only wraps time.sleep(interval), and does nothing
    when interval is 0, while the docstring promises atomicity unconditionally. As I said in
    September, _write_report swallows except Exception, so the sleep is the realistic window
    and this is mostly fine - but KeyboardInterrupt inside self._dev.write is a
    BaseException and escapes uncaught. Either narrow the docstring or use try/finally.
  • tap_screen(0, 0) serialises its touch-up as mouse{} - every field at a proto3 default,
    so nothing on the wire. The test pins those bytes; nothing says the unit accepts them.
  • The bindings are correct: I regenerated from the committed proto with
    scripts/compile_protos.sh's toolchain and all four files come back byte-identical. Gencode
    stayed 7.35.1, so no protobuf pin or grpcio-tools floor move was owed.
  • Offline suite and mypy are clean on b70ae2d0 itself: 3013 passed, 7 skipped; mypy clean
    across 46 files. The description's numbers name cae59296, two merges back.
  • The branch is 25 commits behind main. Strict status checks make BEHIND block the merge, so
    it needs a sync - but that is the last step, not now.

@tony-xmelon

tony-xmelon commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Updated the branch to e417162 in response to the review. The public PR surface is now only capture_screen and tap_screen; swipe_screen and graphics_tree implementations, tests, and profile claims were removed for a follow-up. Restored the 73-type and Screenshot-versus-RemoteControl protocol record, send_sequence architecture and rule, operation coverage, and ADR-0024. The 4.0.1 hardware-test guard now keys off measured profile firmware.

Offline validation on this head: pytest -q: 2996 passed, 7 skipped; mypy clean across 46 source files; git diff --check clean.

I also exercised the device through the shared Rust broker, not direct pyquadcortex: full 800x480 screen capture succeeded; a tap at the visible plus control (100, 147) opened the Plugins chooser; a second tap restored the Grid. The authoritative preset read stayed dirty=false, with zero blocks before and after. Device diagnostics did not return a firmware identity. The broker owns HID, so I could not run pytest tests/hardware --hardware through Python on this same connection. I have left this PR draft and have not synced main. Please run the required full Python hardware suite on this head, or advise whether broker evidence plus the earlier exact-profile capture/tap evidence is sufficient.

@tony-xmelon tony-xmelon mentioned this pull request Sep 26, 2026
5 tasks done
@tony-xmelon

tony-xmelon commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor Author

Direct HID evidence on exact head e417162, CorOS 4.1.0 (CONTRIBUTED): tests/hardware/test_remote_screen.py::test_capture_screen_returns_the_observed_complete_framebuffer passed (1 passed, 12.31s), returning the complete 800 x 480 frame. I also exercised tap_screen(100, 147) against the empty Grid's + slot. After allowing 1.5s for the UI to settle, direct capture showed the Plugins chooser; a second tap at the same coordinate restored the Grid, and the restored PNG matched the initial framebuffer byte-for-byte (SHA-256 081b3b256d6730198a4a610ae043615df0097eef5f349eda1d0b33e7db1dd087). No block was added and the loaded preset remained unchanged.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Correction and full-suite result on PR #50 exact head e417162, CorOS 4.1.0 / CONTRIBUTED. The preceding short tap comment was truncated by shell quoting; this is the complete record.

The complete command was pytest --hardware -q tests/hardware. Result: 105 passed, 11 failed, 8 skipped in 249.00s. Operation summary: 22 of 22 claimed operations measured; 16 passed; 6 failed or skipped; no VERIFIED operation regressed by name. The remote-screen capture test passed. The direct tap test also passed with 1.5s UI-settle delays: tap at (100,147) opened the Plugins chooser, second tap restored Grid, and restored capture matched the original framebuffer byte-for-byte (SHA-256 081b3b256d6730198a4a610ae043615df0097eef5f349eda1d0b33e7db1dd087).

Failures:

  • generated snapshot checks for models, params, and options: QuadCortex41 has NoSnapshot entries;
  • test_nothing_the_burst_delivered_is_read_again_on_first_access: preset_dirty was read once instead of zero times;
  • test_the_burst_warms_identity_from_connects_own_version_read: current handshake-burst shape differs from its recorded expectation;
  • hidden-option and option-position/count audits: the live 4.1 catalog has 633 models / 398 with resources, while these tests pin a 533-model / 331-resource catalog (live option counts also differ);
  • preset-surface block placement and the parameter/bypass echo controls require an occupied block, but the loaded clean slot is empty.

Cleanup verified directly afterwards: the original My Presets position 52 is loaded, dirty=false, no preset named pyquadcortex scratch remains, and 182 User slots remain free. The tests made no persistent changes. I have not synced main: this run exposed test-data/profile baselines that need maintainer guidance before I can truthfully call the full-suite gate clean.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

@jonathanstokes Thanks for the detailed review. I narrowed this branch to capture_screen and tap_screen as you recommended; swipe_screen and graphics_tree are out of this PR. The CorOS 4.0.1 hardware test now skips by measured profile, not VERIFIED membership. The protocol docs now include message type 72 and the measured protobuf payload bytes separately from the inferred enum labels; the stale four-operation changelog text and coverage wording are corrected. ADR-0024 / the STEERING entry preserve the atomic-gesture decision after upstream's ADR numbering.

Validation on f8387d9: 2,996 passed, 7 skipped; mypy clean; diff check clean. I have not run the complete hardware suite on this exact head: QC Remote currently reports disconnected, and I did not reconnect or disturb the device. The PR remains draft until that exact-head gate can be run.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Exact-head hardware update for f8387d9 (direct USB, CorOS 4.1.0 / CONTRIBUTED; shared broker restored afterward):

3110 passed, 1 failed, 16 skipped in 317.85s
operations measured: 22 of 22 claimed
passed: 20; failed/skipped: read_current_preset, set_param
VERIFIED and claimed by a test, failed: []

The PR's capture_screen hardware test passed in this run. The single failure is the unrelated option-position coverage floor: the currently loaded, clean Fender Deluxe 37 preset exposed 59 positions, below test_every_option_position_lands_where_the_catalog_says's 100-position floor. No parameter mismatch was reported. PR #44 has the pending maintainer-requested correction to skip that insufficient-coverage precondition; it is not yet on main, so this run still cannot be called a clean full-suite pass. I have left #50 draft and have not synced its base yet.

After the run I reconnected QC Remote and verified the device is ready, synchronized, and back on Fender Deluxe 37 at 1A with dirty: false.

…-control

# Conflicts:
#	docs/ADR.md
#	docs/STEERING.md
@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Synced current upstream main (d8d40f3) into this branch at 5c4d80d, resolving the ADR and STEERING overlaps while retaining both the reviewed screen-control record and the upstream catalog-payload record.

Validation on pushed head 5c4d80d:

  • full offline suite: 3,002 passed, 7 skipped
  • exact-head remote-screen capture hardware test: 1 passed on CorOS 4.1.0
  • complete exact-head hardware suite: 3,116 passed, 1 failed, 16 skipped in 316.16s. capture_screen passed. The single failure is the unrelated fixed-list coverage floor: clean preset Fender Deluxe 37 provides 59 positions versus the test's 100-position minimum; it reported no parameter mismatch. The pending Add profile-owned CorOS 4.1 catalog and observations #44 change will classify insufficient coverage as a skip; it is not on main yet, so I am not calling this a clean full-suite pass.
  • after broker restoration, QC is ready/synchronized on the original preset at 1A, dirty: false, master volume unchanged at 97

The PR remains draft until the full-suite precondition is resolved and review is refreshed.

@tony-xmelon

tony-xmelon commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor Author

Exact-head CorOS 4.1.0 hardware validation is now complete on 5c4d80d: 116 passed, 0 failed, 0 errored, 8 skipped in 290.94s; all 22 claimed operations passed, with no operation-level failures or skips. The full-display capture operation passed. The QC was restored afterward to clean Fender Deluxe 37 at 1A; broker ready/synchronized, dirty false, master volume 97.

On the empty-slot point: capture_screen() captures the live display and does not depend on any preset slot being saved. For preset_screenshot() in PR #49, the QuadCortex41 override checks whether the requested address is already loaded; if it is, it calls capture_screen() directly, so an empty currently loaded slot does not wait for a stored-preset PNG that does not exist. I verified that exact case on the device: empty slot 7E returned the live 800x480 screen in 12.49s (1 passed), without changing the loaded address. Requests for a different slot still use the stored preset screenshot operation; they are not silently substituted with the current screen.

@tony-xmelon
tony-xmelon marked this pull request as ready for review September 27, 2026 04:12
@tony-xmelon

tony-xmelon commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review. I’ve addressed the remaining protocol-count drift and pushed exact head 1001b61516ad1548b911ac2428593c15f942a745. All 18 inline threads are resolved, the requested full CorOS 4.1 hardware suite and offline suite pass on this head, and all five CI checks are green. I’ve marked the PR ready; could you please take another look when convenient? The CorOS 4.0.1 measurement remains yours, as agreed.

For clarity on screenshots: capture_screen() reads the live display regardless of preset-slot occupancy. In #49, the loaded-slot preset_screenshot() path delegates to that live capture, so an empty currently loaded slot returns its screen without waiting for a stored preset image. The exact empty-slot hardware test passed on 7E and left the loaded address unchanged.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

I reran the required full hardware suite on exact head 1001b61516ad1548b911ac2428593c15f942a745 (CorOS 4.1.0 / QuadCortex41, direct HID). capture_screen passed, but the complete suite is not clean on the currently loaded preset:

114 passed, 1 failed, 0 errored, 9 skipped in 256.61s

The single failure is test_every_option_position_lands_where_the_catalog_says: the loaded preset exposes only 2 fixed-list positions, below the test's 100-position evidence floor. No parameter mismatch was reported; this is an insufficient-coverage precondition. The test in current #44 head 546b444 now skips that condition explicitly, but that fix is not yet on this PR head or upstream main. The suite teardown restored the original loaded preset; after reconnecting the shared broker the unit is ready, synchronized, back at the same preset/address with dirty: false, and master volume unchanged. This is an exact-head run, not a full-suite pass.

@tony-xmelon

Copy link
Copy Markdown
Contributor Author

Follow-up: the sparse-preset run above was a coverage-precondition failure, not a protocol mismatch. I reran the complete suite on the same exact head with the saved high-coverage slot required by this test. Result: CorOS 4.1.0 / QuadCortex41, 116 passed, 0 failed, 0 errored, 8 skipped in 290.55s; all 22/22 claimed operations passed, including capture_screen. The original active preset was restored cleanly and the shared broker is ready/synchronized. I have added this exact-head result to the PR description. The earlier precondition result is retained for transparency; the successful run is the applicable final-head evidence. Could you please re-review when convenient?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants