You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Three opt-in backends for games / apps that ignore the default
SendInput (Win) or XTest (Linux) paths because they read raw
input via GetRawInputData / evdev, plus a virtual-gamepad
facade for games that only accept controller input.
Interception (Windows)
- New sub-package ``je_auto_control/windows/interception/``
with ctypes bindings to ``interception.dll`` and
drop-in-compatible ``keyboard.py`` / ``mouse.py`` modules
matching the existing SendInput surface.
- Wired into ``wrapper/_platform_windows.py`` via a new
``_select_input_backend`` helper triggered by
``JE_AUTOCONTROL_WIN32_BACKEND=interception``. Falls back to
SendInput with a warning when the driver isn't installed, so
deployments can roll the driver out lazily.
- Mouse-button tuples are remapped to Interception flag bits
when the backend is active so the wrapper's
``mouse_keys_table`` dispatches correctly without changing
callers.
uinput (Linux)
- New sub-package ``je_auto_control/linux_with_x11/uinput/``.
``_device.py`` is a small ctypes + ioctl wrapper around
``/dev/uinput`` (no third-party deps).
- ``keyboard.py`` and ``mouse.py`` mirror
``x11_linux_keyboard_control`` / ``x11_linux_mouse_control``;
``set_position`` synthesises the absolute move as a relative
delta off the current cursor reported by Xlib so callers
keep the same contract.
- Wrapper selector via ``JE_AUTOCONTROL_LINUX_BACKEND=uinput``
with the same XTest fallback semantics on permission failure.
ViGEm virtual gamepad (Windows)
- New module ``je_auto_control/utils/gamepad/`` providing
``VirtualGamepad`` (string-keyed buttons / dpad / sticks /
triggers, context manager) backed by the optional
``vgamepad`` package.
- ``default_virtual_gamepad`` / ``is_virtual_gamepad_available``
re-exported from the top-level ``je_auto_control`` facade so
scripts can ``from je_auto_control import VirtualGamepad``.
- Executor commands ``AC_gamepad_press`` / ``_release`` /
``_click`` / ``_dpad`` / ``_left_stick`` / ``_right_stick`` /
``_left_trigger`` / ``_right_trigger`` / ``_reset`` route
through the singleton.
- MCP tools ``ac_gamepad_*`` registered via a new
``gamepad_tools()`` factory; all destructive, all stripped
under ``--readonly``.
Tests
- ``test_input_backends.py`` covers (a) the optional sub-
packages import on every platform without raising, (b)
``is_available()`` probes return ``False`` rather than
raising when the driver / kernel device is missing, (c) the
wrapper's env-var selectors fall back cleanly, (d) MCP
registers the seven new gamepad tools with the right
schema + annotations.
- 600 / 600 headless pytest pass; ruff clean on
``je_auto_control/``.
Docs / READMEs
- ``new_features_doc.rst`` (Eng + Zh) gains a "Driver-level
input backends" section with installer steps, env-var setup
for each backend, and an explicit anti-cheat caveat.
- ``README.md`` and the two CN/TW READMEs gain a feature
bullet describing all three backends and their fallback
behaviour.
Copy file name to clipboardExpand all lines: README.md
+1Lines changed: 1 addition & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -64,6 +64,7 @@
64
64
-**LLM Action Planner** — translate a plain-language description into a validated `AC_*` action list using Claude
65
65
-**Runtime Variables & Control Flow** — `${var}` substitution at execution time, plus `AC_set_var` / `AC_inc_var` / `AC_if_var` / `AC_for_each` / `AC_loop` / `AC_retry` for data-driven scripts
66
66
- **Remote Desktop** — stream this machine's screen and accept remote input over a token-authenticated TCP protocol, *or* connect to another machine and view + control it (host + viewer GUIs included). Optional TLS (HTTPS-grade encryption), WebSocket transport (ws:// + wss:// for browser / firewall-friendly clients), persistent 9-digit Host ID, host→viewer audio streaming, bidirectional clipboard sync (text + image), and chunked file transfer (drag-drop + progress bar; arbitrary destination path; no size cap). Plus folder sync (additive mirror — local deletions never propagate) and a self-hosted coturn TURN config bundle generator (turnserver.conf + systemd unit + docker-compose + README). **AnyDesk-style popout**: when the viewer authenticates, the live remote desktop opens in its own resizable top-level window so the control panel stays uncluttered. The Remote Desktop tabs are wrapped in `QScrollArea` so the panel stays usable on small windows and stretches edge-to-edge on 4K displays. Driveable headlessly via `je_auto_control` and over MCP through the new `ac_remote_*` tools
67
+
-**Driver-level input backends (opt-in)** — for games / apps that ignore SendInput (Win) or XTest (Linux): **Interception driver backend** for Windows (HID-layer keyboard / mouse injection via Oblita's WHQL-signed driver, opt-in via `JE_AUTOCONTROL_WIN32_BACKEND=interception`), **uinput backend** for Linux (kernel `/dev/uinput` synthetic HID device, opt-in via `JE_AUTOCONTROL_LINUX_BACKEND=uinput`), and **ViGEm virtual gamepad** for Windows games that read controllers (virtual Xbox 360 pad with friendly button / dpad / stick / trigger API, exposed as `AC_gamepad_*` executor commands and `ac_gamepad_*` MCP tools). All three fall back gracefully when the driver isn't installed, so existing deployments keep working unchanged
67
68
-**Clipboard** — read/write system clipboard text on Windows, macOS, and Linux
68
69
-**Screenshot & Screen Recording** — capture full screen or regions as images, record screen to video (AVI/MP4)
69
70
-**Action Recording & Playback** — record mouse/keyboard events and replay them
0 commit comments