Skip to content

Commit bee283f

Browse files
committed
Add operations layer + USB Phase 2 chain (rounds 22-47)
Lands the full operations / admin layer plus the USB passthrough chain that grew across rounds 22-47. Operations layer (rounds 22-29): - Folder sync (additive mirror) + coturn TURN config bundle - REST API hardening: bearer auth + per-IP rate limit + audit hook + Prometheus /metrics + browser dashboard at /dashboard - Multi-host admin console (parallel poll + broadcast) persisted to ~/.je_auto_control/admin_hosts.json (mode 0600) - Tamper-evident audit log: SHA-256 hash chain + verify_chain() - WebRTC packet inspector: rolling stats window + summary statistics - USB device enumeration (read-only, cross-platform) - System diagnostics CLI + REST + GUI tab - Web admin dashboard (vanilla JS, sessionStorage token) Test infrastructure + CI (rounds 30-31): - Convert per-round smoke scripts into pytest under test/unit_test/headless/ - Add .github/workflows/quality.yml: ruff + bandit + pytest matrix (Windows 3.10/3.11/3.12) - Pin lint/test deps in dev_requirements.txt - Bandit B105 false-positive fix: exclude language_wrapper dicts via pyproject.toml [tool.bandit] Latent bug fixes (rounds 33, 38): - file_sync._loop no longer marks failed sends as already-synced (next-tick retry promise now honored) - FolderSyncEngine.start() + _RestApiRegistry.start() race fixes - Extract SessionQualityCache to replace two raw dicts shared between asyncio bridge and Qt threads in webrtc_panel OpenAPI + config bundle (rounds 35-36): - /openapi.json (auth-gated) generated from live route table - /docs Swagger UI shell (sessionStorage bearer) - Drift test catches new routes added without metadata - ConfigBundleExporter / ConfigBundleImporter: 8-file allowlist, atomic write with .bak.<ts> backups, version-validated import USB hotplug + Phase 2 chain (rounds 34, 37, 39-47): - UsbHotplugWatcher: bounded ring buffer + sequence-numbered events - Wire-level protocol (10 opcodes, 16 KiB cap, CREDIT-based flow control) over a WebRTC usb DataChannel - Host-side UsbPassthroughSession with full CTRL/BULK/INT dispatch - LibusbBackend with full transfers (Linux end-to-end) - Viewer-side UsbPassthroughClient (blocking open / control_transfer / bulk_transfer / interrupt_transfer / close, outbound credit waits) - Persistent UsbAcl (default deny, mode 0600) + host-side prompt QDialog with cross-thread bridge - WinusbBackend (full ctypes wiring, hardware-unverified) - IokitBackend skeleton (NotImplementedError pending pyobjc work) - Audit-log integration for every ACL decision - Feature flag (default off): enable_usb_passthrough(True) or JE_AUTOCONTROL_USB_PASSTHROUGH=1 - Test-stability fixes for two pre-existing flakes (test_destructive_confirmation_blocks elicitation typo + test_remote_desktop_websocket._wait_until budget bumps) Verified: ruff clean, bandit zero issues, headless suite 605 passed / 7 skipped (cross-platform + aiortc-gated) / 0 failed across three consecutive runs.
1 parent 141d1fe commit bee283f

124 files changed

Lines changed: 23791 additions & 167 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/quality.yml

Lines changed: 79 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,79 @@
1+
name: AutoControl Code Quality
2+
3+
# Static analysis (ruff, bandit) plus the headless pytest suite added in
4+
# rounds 22-30. Decoupled from the existing dev/stable workflows, which
5+
# run legacy standalone test scripts and exist for hardware integration
6+
# coverage on Windows runners.
7+
8+
on:
9+
push:
10+
branches: [ "dev", "main", "stable" ]
11+
pull_request:
12+
branches: [ "dev", "main", "stable" ]
13+
workflow_dispatch:
14+
15+
permissions:
16+
contents: read
17+
18+
jobs:
19+
lint:
20+
runs-on: ubuntu-latest
21+
steps:
22+
- uses: actions/checkout@v4
23+
24+
- name: Set up Python
25+
uses: actions/setup-python@v5
26+
with:
27+
python-version: "3.12"
28+
cache: "pip"
29+
30+
- name: Install ruff
31+
run: |
32+
python -m pip install --upgrade pip
33+
pip install ruff
34+
35+
- name: Run ruff
36+
run: ruff check je_auto_control/
37+
38+
security:
39+
runs-on: ubuntu-latest
40+
steps:
41+
- uses: actions/checkout@v4
42+
43+
- name: Set up Python
44+
uses: actions/setup-python@v5
45+
with:
46+
python-version: "3.12"
47+
cache: "pip"
48+
49+
- name: Install bandit
50+
run: |
51+
python -m pip install --upgrade pip
52+
pip install bandit
53+
54+
- name: Run bandit (recursive, skip tests + i18n dicts)
55+
run: bandit -r je_auto_control/ -c pyproject.toml
56+
57+
pytest-headless:
58+
runs-on: windows-2022
59+
strategy:
60+
fail-fast: false
61+
matrix:
62+
python-version: [ "3.10", "3.11", "3.12" ]
63+
steps:
64+
- uses: actions/checkout@v4
65+
66+
- name: Set up Python ${{ matrix.python-version }}
67+
uses: actions/setup-python@v5
68+
with:
69+
python-version: ${{ matrix.python-version }}
70+
cache: "pip"
71+
72+
- name: Install dependencies
73+
run: |
74+
python -m pip install --upgrade pip wheel
75+
pip install -r dev_requirements.txt
76+
pip install -e .
77+
78+
- name: Run headless pytest suite
79+
run: pytest test/unit_test/headless/ -v --tb=short --timeout=120

dev_requirements.txt

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,9 @@ PySide6==6.11.0
88
qt-material==2.17
99
mss==10.2.0
1010
defusedxml==0.7.1
11+
12+
# Quality tooling — used by .github/workflows/quality.yml and locally.
13+
ruff==0.15.9
14+
bandit==1.9.4
15+
pytest==9.0.2
16+
pytest-timeout==2.4.0
Lines changed: 251 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,251 @@
1+
============================================================
2+
USB Passthrough — Operator Guide
3+
============================================================
4+
5+
Step-by-step recipe for getting a USB device on a host machine to
6+
respond to traffic from a remote viewer. Assumes Phase 2a.1 (current
7+
shipping state) — host-side end-to-end works on Linux libusb; Windows
8+
WinUSB is hardware-unverified; macOS IOKit is not yet implemented.
9+
10+
If you're a security reviewer instead of an operator, you want
11+
:doc:`usb_passthrough_security_review`. If you're a developer wanting
12+
the protocol details, :doc:`usb_passthrough_design`.
13+
14+
.. contents::
15+
:local:
16+
:depth: 2
17+
18+
19+
Prerequisites
20+
=============
21+
22+
On the **host** (the machine with the physical USB device):
23+
24+
- Python 3.10+ with AutoControl installed.
25+
- The optional ``webrtc`` extra: ``pip install je_auto_control[webrtc]``.
26+
- ``pyusb`` installed if you want the libusb backend:
27+
``pip install pyusb``.
28+
- The USB device the viewer will use, plugged in.
29+
- Per-OS setup (see *Driver setup* below).
30+
31+
On the **viewer** (the remote machine that will use the device):
32+
33+
- Python 3.10+ with AutoControl installed.
34+
- Network reach to the host's REST API port (default 9939) **and** to
35+
the WebRTC signalling / TURN endpoints if the viewer is behind NAT.
36+
- The host's bearer token (operator hands it over out-of-band).
37+
38+
39+
Driver setup (per OS)
40+
=====================
41+
42+
Linux (libusb)
43+
--------------
44+
45+
The libusb backend is the most-tested path today. Steps:
46+
47+
1. Install ``libusb-1.0`` development files (e.g. ``apt install
48+
libusb-1.0-0``).
49+
2. Add a ``udev`` rule so the AutoControl host process can claim the
50+
device without root. Example for a YubiKey 5
51+
(vendor ``1050``, product ``0407``)::
52+
53+
# /etc/udev/rules.d/99-autocontrol-usb.rules
54+
SUBSYSTEM=="usb", ATTRS{idVendor}=="1050",
55+
ATTRS{idProduct}=="0407", MODE="0660",
56+
GROUP="plugdev"
57+
58+
Then ``sudo udevadm control --reload && sudo udevadm trigger``.
59+
3. Make sure your AutoControl user is in ``plugdev``.
60+
4. If the device is a HID, AutoControl's libusb wrapper detaches
61+
``usbhid`` on ``open`` and re-attaches on ``close``. Don't be
62+
alarmed if your local keyboard input briefly hiccups during a
63+
claim of a HID device.
64+
65+
Windows (WinUSB) — *hardware-unverified*
66+
----------------------------------------
67+
68+
The ctypes wiring exists but has not been validated against real
69+
hardware. Treat as alpha. Steps:
70+
71+
1. Use `Zadig <https://zadig.akeo.ie/>`_ or libwdi to associate the
72+
target device with the WinUSB driver. Do not do this for devices
73+
the host OS already manages (printers, hubs, keyboards).
74+
2. After binding, the device should appear in
75+
``WinusbBackend().list()``.
76+
3. Hardware testing is required before relying on transfers. See
77+
the security review checklist for the expected test matrix.
78+
79+
macOS (IOKit) — *not yet implemented*
80+
-------------------------------------
81+
82+
The skeleton exists; ``IokitBackend()`` constructs but ``list`` /
83+
``open`` raise ``NotImplementedError``. Track Phase 2c.
84+
85+
86+
Enabling the feature
87+
====================
88+
89+
USB passthrough is **off by default**. Two ways to opt in:
90+
91+
- Environment variable, picked up at process start::
92+
93+
export JE_AUTOCONTROL_USB_PASSTHROUGH=1
94+
python -m je_auto_control.cli start-rest
95+
96+
- Programmatic, in your bootstrap script (overrides env)::
97+
98+
from je_auto_control.utils.usb.passthrough import enable_usb_passthrough
99+
enable_usb_passthrough(True)
100+
101+
Confirm with :func:`is_usb_passthrough_enabled`::
102+
103+
from je_auto_control.utils.usb.passthrough import is_usb_passthrough_enabled
104+
assert is_usb_passthrough_enabled()
105+
106+
107+
ACL setup
108+
=========
109+
110+
The ACL defaults to ``"deny"`` so a viewer cannot claim a device the
111+
operator hasn't approved. Add per-device rules:
112+
113+
1. From the GUI — the *USB* tab on the host shows the prompt dialog
114+
on first OPEN of an unknown device. Tick *Remember this decision*
115+
to persist a permanent allow rule.
116+
2. From Python::
117+
118+
from je_auto_control.utils.usb.passthrough import (
119+
AclRule, UsbAcl,
120+
)
121+
acl = UsbAcl()
122+
acl.add_rule(AclRule(
123+
vendor_id="1050", product_id="0407",
124+
serial=None, # match any serial
125+
label="YubiKey 5",
126+
allow=True,
127+
prompt_on_open=False, # silent allow once approved
128+
))
129+
130+
3. By editing ``~/.je_auto_control/usb_acl.json`` directly. The file
131+
is permission-checked (mode ``0600`` on POSIX). Bad JSON or an
132+
unknown ``version`` falls back to default-deny.
133+
134+
Decision precedence:
135+
136+
- First matching rule wins. ``prompt_on_open=True`` means re-ask the
137+
operator each time, even if the rule is ``allow=True``.
138+
- If no rule matches, the file's ``default`` (``"deny"`` out of the
139+
box) applies.
140+
141+
142+
Starting the host
143+
=================
144+
145+
The host needs the REST API running (so the viewer can enumerate)
146+
and a WebRTC peer connection to the viewer (so transfers can flow).
147+
148+
REST::
149+
150+
from je_auto_control.utils.rest_api import start_rest_api_server
151+
server = start_rest_api_server(host="0.0.0.0", port=9939)
152+
print("Bearer:", server.token)
153+
154+
WebRTC: use the existing remote desktop pipeline (see
155+
:doc:`operations_layer_doc`) to bring up a session. The viewer's
156+
``UsbPassthroughClient`` then plugs into the negotiated DataChannel.
157+
158+
159+
Viewer-side: claim and transfer
160+
===============================
161+
162+
Enumerate
163+
---------
164+
165+
From Python::
166+
167+
import urllib.request, json
168+
req = urllib.request.Request(
169+
"http://host:9939/usb/devices",
170+
headers={"Authorization": f"Bearer {token}"},
171+
)
172+
with urllib.request.urlopen(req) as r:
173+
body = json.loads(r.read())
174+
for d in body["devices"]:
175+
print(d["vendor_id"], d["product_id"], d.get("product"))
176+
177+
Or via the *USB Browser* GUI tab on the viewer side: paste the host's
178+
REST URL + token, click *Fetch devices*.
179+
180+
Open + transfer
181+
---------------
182+
183+
::
184+
185+
from je_auto_control.utils.usb.passthrough import (
186+
UsbPassthroughClient, encode_frame, decode_frame,
187+
)
188+
189+
# `data_channel` is your WebRTC RTCDataChannel for the "usb" channel.
190+
def send(frame):
191+
data_channel.send(encode_frame(frame))
192+
193+
client = UsbPassthroughClient(send_frame=send)
194+
# Wire the channel's on-message callback:
195+
data_channel.on("message")(lambda raw: client.feed_frame(decode_frame(raw)))
196+
197+
handle = client.open(vendor_id="1050", product_id="0407")
198+
response = handle.control_transfer(
199+
bm_request_type=0xC0, b_request=6, w_value=0x0100, length=18,
200+
)
201+
print("device descriptor:", response.hex())
202+
handle.close()
203+
client.shutdown()
204+
205+
Errors:
206+
207+
- ``UsbClientTimeout`` — the host took longer than ``reply_timeout_s``
208+
(default 10s) to respond. Check the network / host process.
209+
- ``UsbClientError`` — the host replied with ``{ok: false, error: ...}``.
210+
The most common case is *denied by ACL policy* — go check the
211+
prompt dialog or the ACL rule on the host.
212+
- ``UsbClientClosed`` — the client or its handle was already shut down.
213+
214+
215+
Troubleshooting matrix
216+
======================
217+
218+
========================================== =====================================================
219+
Symptom Likely cause / fix
220+
========================================== =====================================================
221+
``open`` returns ``denied by ACL policy`` No allow rule + ``default = deny``. Add a rule or
222+
enable a prompt callback.
223+
``open`` returns ``no device matches`` Device not enumerated. Check ``UsbHotplugWatcher``
224+
output or run ``list_usb_devices()`` directly.
225+
On Windows, confirm Zadig binding.
226+
``credit exhausted`` on transfer Viewer sent more frames than the host's
227+
``initial_credits`` window allows. Either lower
228+
request rate or raise ``initial_credits`` on
229+
the session.
230+
Transfer ``UsbClientTimeout`` Host process is busy or the WebRTC channel is
231+
broken. Inspect the *Packet Inspector* tab for
232+
RTT / packet loss.
233+
After OPEN, host's keyboard stops working Linux: a HID device was claimed and
234+
``usbhid`` was detached. The driver re-attaches
235+
on CLOSE; if not, ``udevadm trigger`` to recover.
236+
Audit chain shows ``broken_at_id`` Someone edited ``audit.db`` directly. Restore
237+
from a backup; investigate.
238+
========================================== =====================================================
239+
240+
241+
What is *not* shipped yet
242+
=========================
243+
244+
- WebRTC viewer GUI does not auto-wire the ``usb`` DataChannel — the
245+
*USB Browser* tab's *Open* button shows a "not yet wired" message.
246+
You can drive the protocol from Python today.
247+
- Windows WinUSB transfer methods are written but not validated
248+
against real hardware. Do not use in production.
249+
- macOS IOKit backend is unimplemented (Phase 2c).
250+
- Phase 2e external security review has not been signed; the feature
251+
flag must remain explicit opt-in.

0 commit comments

Comments
 (0)