Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 3 additions & 2 deletions docs/STEERING.md
Original file line number Diff line number Diff line change
Expand Up @@ -268,8 +268,9 @@ one that lives in the template and the contributor guide is not.
**What changed:** `connect()` reads the unit's `Version` before the handshake,
resolves `(device_type, zenos_git_hash)` in a registry of profile classes, and
refuses an unknown pair with `UnsupportedDevice`. `QuadCortex` declares itself
as the 4.0.1 profile; `QuadCortex41` connects and verifies nothing until a 4.1
unit's suite run fills its `VERIFIED` set; `QuadCortexMini` is recognised and
as the 4.0.1 profile; `QuadCortex41` connects and verifies only operations with
recorded 4.1 evidence; a unit's suite run fills its `VERIFIED` set;
`QuadCortexMini` is recognised and
refused. An operation a profile has not verified refuses under the default
`Support.VERIFIED` and runs with one warning under `Support.EXPERIMENTAL`.
Generated constants live in `pyquadcortex/protocol/catalogs/coros_4_0_1/`;
Expand Down
1 change: 1 addition & 0 deletions docs/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -76,6 +76,7 @@ already read and need no connection; calling them as methods raises
| **Neural Captures** | `captures()`, `list_irs()` to browse the library, `set_capture(cell, entry)` to place one. Creating a capture is the unit's own wizard - disconnect first, since a connected client suppresses it |
| **Discovery** | `list_folders()` - every folder the device knows, including the factory Captures Library and plugin artist presets; `recents()`, `favorites()`, `add_favorite()`, `remove_favorite()` |
| **Manage presets** | `save_current_preset(setlist, slot, name)`, `delete_preset(setlist, name)`, `move_preset(setlist, name, to_slot)` |
| **Local backups** | `create_local_backup()` returns the device's structurally validated portable backup document; serialize it to JSON wherever you keep backups |

**Rows and columns are zero-based, and the unit displays rows 1 to 4.** `row=0` is
the top row on screen and `row=2` is the one labelled 3. This matters more than it
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture.md
Original file line number Diff line number Diff line change
Expand Up @@ -639,7 +639,7 @@ next, roughly in order of how well the ground is prepared:
- **Types not in the registry at all.** The schema declares 71 message types.
Whole feature areas are untouched: `Tuner` / `ShowTuner`, `Looper`,
`MIDISettings`, `NeuralCapture` / `NeuralCapture2`, `Screenshot`,
`Diagnostics`, `LocalBackup` / `CloudBackup`, `Confirmation`,
`Diagnostics`, `CloudBackup`, `Confirmation`,
`GigViewButton`, `SuspendConnection`, `GenericError`, the `*Forward` transport
wrappers, and the production/test-farm messages. Nothing about these has been
observed on the wire by this project, so treat the schema as a starting
Expand Down
2 changes: 1 addition & 1 deletion docs/domain-model.md
Original file line number Diff line number Diff line change
Expand Up @@ -1388,7 +1388,7 @@ the n/a rows below where they intersect the API at all.
| CC-only: device name display/edit | **omitted** | no | see Device name row |
| CC-only: per-scene tempo claim | **omitted** | n/a | contradicts the unit; on-unit presentation wins |
| CC-only: preset / plugin-preset / IR import from computer | **omitted** | no | candidate `File` with payloads; the import flow is unsolved (and IR import probing is hazardous - see CLAUDE.md) |
| CC-only: local backups | **omitted** | no | `LocalBackup` unexplored |
| CC-only: local backups | **omitted** | yes | protocol layer: `create_local_backup()`; no model wrapper yet |
| CC-only: CorOS update via USB | **omitted** - permanently | no | `Updater` |
| CC-only: keyboard shortcuts, window sizing | - | n/a | app UI |
| CC-only: undo/redo shortcuts | **omitted** | no | see Undo/redo row |
Expand Down
8 changes: 4 additions & 4 deletions docs/manual-coverage.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,10 +20,10 @@ or a field in `BinaryPreset`. A named candidate is a lead, not a claim that it w

## Summary

Of 105 features audited: **65 yes**, **8 partly**, **21 no**, **11 n/a**.
Of 105 features audited: **65 yes**, **9 partly**, **20 no**, **11 n/a**.

Of the 93 features a host could plausibly drive - everything above except the 11 marked
n/a - **65 are fully covered** and 8 more are partly covered, which here means the state
Of the 94 features a host could plausibly drive - everything above except the 11 marked
n/a - **65 are fully covered** and 9 more are partly covered, which here means the state
is readable and at least one field of it is confirmed writable, with the neighbours the
same shape but not individually exercised. Only 20 remain untouched.

Expand Down Expand Up @@ -191,7 +191,7 @@ on 25, 9 and 56 as `led_brightness` was 28, 13 and 59).
| CorOS updates | no | `Updater` is decoded and subscribed; never driven. Risky to explore |
| Brightness, power sensitivity, storage | yes | screen, LED and dimmed-LED brightness all confirmed (quantized: 30 reads back 31), plus the three dimming toggles; disk space is reported. `power_option` and `reset_wifi_networks` are refused by `update_settings()` as commands |
| Cloud sign-in and cloud backups | no | `CloudLogin`, `CloudBackup`, `BackupsForward` |
| Local backups | no | `LocalBackup` |
| Local backups | partly | `create_local_backup()` collects and validates the native document; restoring one is not implemented. Confirmed on CorOS 4.1.0: 12 ordered chunks, one marked final |

## 11, 12 Desktop app and reference

Expand Down
1 change: 1 addition & 0 deletions docs/protocol.md
Original file line number Diff line number Diff line change
Expand Up @@ -2799,6 +2799,7 @@ screen; **captured only** = seen on the wire, with no independent read-back.
|---|---|---|---|
| connect handshake | `ResetCommsBuffers` + `Version` UPDATE + `ModelRepo` READ + `Connection` + subscribe READs | read-back | the connect gate; state pushes flow only after it |
| version read | `Version{action: READ}` | read-back | serial and firmware returned. TWO messages come back: the `UPDATE` with fifteen fields, then the device's own `Version{READ}` 0.5-0.8 ms later. `version()` accepts only a `Version` carrying `device_serial_number` or `app_fw_version`, so the second one is never returned as the answer; five back-to-back calls after that change all returned the full reply (2026-09-03) - see "A `Version` READ is answered twice" |
| `create_local_backup` | `LocalBackup{CREATE}` then one or more `LocalBackup{UPDATE, backup_json}` pushes, the last carrying `is_last_chunk` | captured only | Measured 2026-09-09 on Quad Cortex, CorOS 4.0.1 / d14e: a 130,178-character document arrived as ONE push, final marker set, nothing after it; no request id on the reply; the 10-field wrapper and its 64-hex `payload_hash` validated, and the Base64 payload decoded. 9.1 s on a cold connection, about 2.9 s after, and three back-to-back calls on one connection all succeeded. `can_apply_backup` never appeared, so the refusal path is unverified here. Contributed measurement 2026-09-08 on CorOS 4.1.0: 12 chunks (11 x 150,000 characters plus final 144,890), one final marker. Taken together, 150,000 is a MAXIMUM per push and not a fixed chunk size: a document under it arrives whole |
| `recall_preset` / `read_preset` | `SetlistPosition{UPDATE, folder_key, position, is_factory, request_id}` then a `RecallPreset` push | read-back | the push echoes the recall's `request_id` |
| `read_current_preset` / `read_current_preset_push` | `RecallPreset{READ, request_id}` | read-back | the live grid, no side effects. The push variant hands back the whole reply, which carries `reason` beside the preset |
| `loaded_position` | `SetlistPosition{READ, request_id}` | read-back | which slot is loaded; 3 ms measured. A READ names no slot - an UPDATE that did would recall it |
Expand Down
4 changes: 3 additions & 1 deletion pyquadcortex/protocol/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,8 @@
input_level_db, lane_level_db,
tempo_bpm, bpm_to_tempo,
USER_SETLIST_ROOT, Block,
BlockRefused, ControlNotDrivable, Folder,
BlockRefused, ControlNotDrivable,
MalformedLocalBackup, Folder,
MidiOut, QuadCortex, Split,
StompAssignment, ExpressionAssignment,
expression_assignments,
Expand Down Expand Up @@ -128,6 +129,7 @@
"Tempo",
"BlockRefused",
"ControlNotDrivable",
"MalformedLocalBackup",
"UNITY_LEVEL",
"DeviceLostError",
"input_level_db",
Expand Down
140 changes: 139 additions & 1 deletion pyquadcortex/protocol/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -28,8 +28,11 @@
``docs/protocol.md`` for the per-operation coverage table.
"""

import base64
import functools
import json
import logging
import re
import time
import types
import typing
Expand All @@ -51,7 +54,8 @@
from pyquadcortex.protocol.proto import Preset_pb2 as preset

from pyquadcortex.protocol.errors import (BlockRefused, # noqa: F401
ControlNotDrivable)
ControlNotDrivable,
MalformedLocalBackup)
from pyquadcortex.protocol.support import (EVERYTHING, Evidence, Hardware,
Support, measured_firmware,
unverified_text)
Expand Down Expand Up @@ -117,6 +121,10 @@ def _mode_param(message, index: int):
#: :meth:`QuadCortex.set_scene_label` sends this when given ``None``.
SCENE_UNLABELLED = " "

# Observed backups are around 1.7 MB. 32 MiB is a deliberately chosen safety
# ceiling, not a measured device limit, so an unrelated stream is not accepted.
_MAX_LOCAL_BACKUP_BYTES = 32 * 1024 * 1024


# -- typed values for the SETTINGS writes -------------------------------------
#
Expand Down Expand Up @@ -640,6 +648,136 @@ def version(self, timeout: float = 10.0):
or m.HasField("app_fw_version")),
)

def create_local_backup(self, timeout: float = 60.0) -> dict[str, typing.Any]:
Comment thread
tony-xmelon marked this conversation as resolved.
Resolved
"""Create and return the device's portable local-backup document.

The device answers one ``LocalBackup`` CREATE with an uncorrelated
stream of JSON fragments; the final UPDATE carries ``is_last_chunk``.
The fragments are joined in arrival order and the portable wrapper is
structurally validated before it is returned as a dictionary. Its
opaque Base64 payload is deliberately not interpreted, and the native
integrity identifier is checked for shape rather than recomputed.

This does not restore a backup or write a file. The caller can serialize
the returned dictionary wherever it keeps backups.

How many pushes arrive depends on how big the backup is, and 150,000
characters is the MOST one push carries rather than the size it always
is. Measured 2026-09-09 on CorOS 4.0.1 / d14e: a 130,178-character
document arrived in ONE push with the final marker set. Contributed
measurement 2026-09-08 on CorOS 4.1.0: 1,794,890 characters arrived as
12 pushes, eleven of 150,000 and a final 144,890. So join whatever
arrives and stop at the marker, which is what this does.

The replies carry no ``request_id``, even when the CREATE does, so only
one backup should be in flight on a connection at a time. Calling it
again afterwards is fine: three back-to-back calls on one connection
all returned on 4.0.1, the first taking 9.1 s and the rest about 2.9 s.

A contributed 4.1.0 capture showed ``can_apply_backup=false`` refusing
CREATE. That field never appeared in four runs on 4.0.1, so what it
means is inferred from the schema here, not measured. It sits beside
``applied_backup``, so it may belong to restore instead.
"""
def is_chunk(message):
return (
message.action == pa.MessageAction.UPDATE
and (message.HasField("backup_json")
or message.HasField("is_last_chunk")
or message.HasField("can_apply_backup"))
)

def is_final(message):
return message.HasField("is_last_chunk") and message.is_last_chunk

def is_refusal(message):
return (
message.HasField("can_apply_backup")
and not message.can_apply_backup
)

chunks = self._t.collect(
pa.LocalBackupMessage,
lambda: self._t.send(
pa.LocalBackupMessage(action=pa.MessageAction.CREATE)
),
seconds=timeout,
match=is_chunk,
until=lambda message: is_final(message) or is_refusal(message),
)
if any(is_refusal(message) for message in chunks):
raise ControlNotDrivable(
"create_local_backup",
"the CorOS 4.1.0 unit replied can_apply_backup=false to CREATE.",
"Create the backup in Cortex Control and retain the stream for comparison."
)
finals = [message for message in chunks if is_final(message)]
if not chunks or not finals:
raise TimeoutError(
f"the Quad Cortex did not finish a local backup within "
f"{timeout:g} seconds"
)
if len(finals) != 1:
raise MalformedLocalBackup(
f"the Quad Cortex marked {len(finals)} local-backup chunks final"
)
if chunks[-1] is not finals[0]:
raise MalformedLocalBackup(
"the Quad Cortex sent local-backup messages after the final chunk"
)

backup_json = "".join(
message.backup_json
for message in chunks
if message.HasField("backup_json")
)
size = len(backup_json.encode("utf-8"))
if not backup_json or size > _MAX_LOCAL_BACKUP_BYTES:
raise MalformedLocalBackup(
f"the Quad Cortex returned an empty or oversized local backup "
f"({size} bytes)"
)
try:
document = json.loads(backup_json)
except json.JSONDecodeError as error:
raise MalformedLocalBackup(
f"the Quad Cortex returned malformed local-backup JSON: "
f"{error.msg}"
) from error
if (
not isinstance(document, dict)
or document.get("type") != "backup"
or document.get("creator") != "quad"
or not isinstance(document.get("name"), str)
or not document["name"]
):
raise MalformedLocalBackup(
"the Quad Cortex returned an unsupported local-backup document"
)

payload = document.get("payload")
payload_hash = document.get("payload_hash")
if (
not isinstance(payload, str)
or not isinstance(payload_hash, str)
or re.fullmatch(r"[0-9a-fA-F]{64}", payload_hash) is None
Comment thread
tony-xmelon marked this conversation as resolved.
Resolved
):
raise MalformedLocalBackup(
"the Quad Cortex backup is missing its native payload or "
"integrity identifier"
)
try:
decoded = base64.b64decode(payload, validate=True)
except (ValueError, TypeError) as error:
raise MalformedLocalBackup(
"the Quad Cortex backup payload is not valid Base64"
) from error
if not decoded:
raise MalformedLocalBackup(
"the Quad Cortex backup payload decodes to no data"
)
return document

def find_preset(self, name: str, setlist: str = Setlist.USER,
timeout: float = 25.0):
"""Look a preset up by the name shown on the unit.
Expand Down
8 changes: 8 additions & 0 deletions pyquadcortex/protocol/errors.py
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,14 @@ class BlockRefused(RuntimeError):
"""


class MalformedLocalBackup(RuntimeError):
"""A completed local-backup stream is not a usable native document.

This distinguishes framing or payload corruption from a timeout and from
an explicit refusal by the device.
"""


class ControlNotDrivable(ValueError):
"""A request the library will not answer, because answering means guessing.

Expand Down
13 changes: 6 additions & 7 deletions pyquadcortex/protocol/profiles.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,16 +17,15 @@


class QuadCortex41(QuadCortex):
"""Quad Cortex on CorOS 4.1 - connects, and verifies nothing yet.
"""Quad Cortex on CorOS 4.1, with contributed per-operation evidence.

PR #42's description (2026-09-03) reports ``pytest --hardware``: 2742
passed, 8 skipped, on a Quad Cortex running CorOS 4.1.0 / app firmware
d14e, by tony-xmelon. That is a contributor's report and the maintainer has
not reproduced it, which is what ``Evidence.CONTRIBUTED`` says here. The
connection is therefore known to work with this handshake and announce
string. Which operations behave as on 4.0.1 is not known per name, so every
inherited operation refuses under ``Support.VERIFIED`` and runs with a
warning under ``Support.EXPERIMENTAL``. The snapshot is deliberately absent:
not reproduced it, which is what ``Evidence.CONTRIBUTED`` says here.
``create_local_backup`` has its own dated 4.1.0 capture and is VERIFIED;
other inherited operations refuse under ``Support.VERIFIED`` and run with
a warning under ``Support.EXPERIMENTAL``. The snapshot is deliberately absent:
binding the 4.0.1 constants would hand a 4.1 user names their unit does not
use.

Expand All @@ -43,7 +42,7 @@ class QuadCortex41(QuadCortex):

MEASURED_ON = ("4.1.0",)
EVIDENCE = Evidence.CONTRIBUTED
VERIFIED = frozenset()
VERIFIED = frozenset({"create_local_backup"})
models = NoSnapshot("coros_4_1_0")
params = NoSnapshot("coros_4_1_0")
options = NoSnapshot("coros_4_1_0")
Expand Down
19 changes: 17 additions & 2 deletions pyquadcortex/protocol/transport.py
Original file line number Diff line number Diff line change
Expand Up @@ -336,14 +336,20 @@ def request(self, message, timeout=5.0):
self._check_lost() # woken by _confirm_lost, not by a reply
return slot[0]

def collect(self, expected_class, trigger, seconds, match=None):
"""Fire ``trigger()`` and gather EVERY matching message for ``seconds``.
def collect(self, expected_class, trigger, seconds, match=None, until=None):
"""Fire ``trigger()`` and gather matching messages for ``seconds``.

The counterpart of :meth:`await_broadcast` for the case where one request
provokes many pushes rather than one: a single ``File`` READ makes the
device enumerate every folder it knows about, several hundred of them on
the observed unit, arriving over ten to twenty seconds.

``until``, if supplied, is a predicate that ends collection as soon as
it accepts any newly arrived matching message. This serves finite chunk streams
whose final message marks itself; without it the collector keeps the
original fixed-window behavior and gathers every matching message until
``seconds`` expires.

Returns the messages in arrival order. Unlike a waiter, a collector does
not consume messages - they still reach any waiter or other collector.

Expand All @@ -360,7 +366,16 @@ def collect(self, expected_class, trigger, seconds, match=None):
try:
trigger()
deadline = time.monotonic() + seconds
checked = 0
while time.monotonic() < deadline:
if until is not None:
newest = got[checked:]
# Advance only by the snapshot we just inspected. The RX
# thread may append between the slice and this update; a
# fresh len(got) here would skip that arrival forever.
checked += len(newest)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: This is right, and nothing holds it in place.

I put the old line back and ran the whole suite. All 2741 tests still passed. So if someone later tidies this up and reaches for len(got) again, nothing tells them they broke it.

I am not asking you to fix that here. It is genuinely hard to test: the gap you are protecting against is between two lines that run one after the other, so catching it needs a way to make a message arrive in the middle, and there is no hook for that today. I had to use a fake list that appends to itself when it is read.

Your comment above is a fair substitute, and it says the right thing. I wanted it on the record that the comment is what is guarding this, rather than a test.

if any(until(message) for message in newest):
break
if self._device_lost is not None:
break # nothing more is coming; return what arrived
time.sleep(0.1)
Expand Down
21 changes: 21 additions & 0 deletions tests/hardware/test_local_backup.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
"""Read-only hardware coverage for the native local-backup stream."""
import base64

import pytest


@pytest.mark.verifies("create_local_backup")
def test_local_backup_is_a_complete_portable_document(qc, profile, record_property):
if "create_local_backup" not in profile.VERIFIED:
pytest.skip(f"create_local_backup is not VERIFIED on {profile.__name__}")
document = qc.create_local_backup()

assert document["type"] == "backup"
assert document["creator"] == "quad"
assert isinstance(document["name"], str) and document["name"]
Comment thread
tony-xmelon marked this conversation as resolved.
Resolved
assert len(document["payload_hash"]) == 64
assert all(c in "0123456789abcdefABCDEF" for c in document["payload_hash"])
payload = base64.b64decode(document["payload"], validate=True)
assert payload

record_property("local_backup_payload_bytes", len(payload))
Loading
Loading