Merge pull request #172 from Integration-Automation/dev

Add clipboard, hotkeys, triggers, cron, REST, CLI, plugins, i18n

Author: JE-Chen

CLAUDE.md

@@ -74,6 +74,27 @@ python -m pytest test/integrated_test/
 python -m build
 ```
 
+## Feature Delivery Rules
+
+### Every feature must ship both a headless API and a GUI surface
+
+No feature is complete unless it can be driven entirely without the GUI **and** has a corresponding GUI affordance. Concretely:
+
+- **Headless core in `utils/` or `wrapper/`**: all business logic lives in a module with zero `PySide6` imports. Users must be able to `import je_auto_control` and call the feature without ever instantiating a Qt class.
+- **Re-export from the package facade**: add the public functions / classes to `je_auto_control/__init__.py` and its `__all__` so `import je_auto_control as ac; ac.<feature>(...)` works out of the box.
+- **Executor command coverage**: wire an `AC_*` command into `utils/executor/action_executor.py` so the feature is usable from JSON action files, the socket server, the scheduler, and the visual script builder — all without Python glue.
+- **GUI tab or control is a thin wrapper**: the Qt widget must only translate user input into calls on the headless core. It must not contain business logic that would be unreachable headlessly.
+- **The top-level package stays Qt-free**: `import je_auto_control` MUST NOT import `PySide6`. The GUI entry point is loaded lazily inside `start_autocontrol_gui()`. Verify with:
+
+  ```python
+  import sys, je_auto_control  # noqa
+  assert not any("PySide6" in m for m in sys.modules)
+  ```
+
+- **Tests cover the headless path**: at least one unit test in `test/unit_test/` must exercise the feature through its non-GUI API with no Qt imports.
+
+Features that are inherently interactive (e.g. region picking with the mouse, template cropping) still count as GUI-only — but they must accept programmatic equivalents (e.g. `screenshot(screen_region=[...])` with explicit coordinates) so scripts can replay the same effect headlessly.
+
 ## Coding Standards
 
 ### Security First

docs/source/Eng/doc/cli/cli_doc.rst

@@ -2,47 +2,90 @@
 Command-Line Interface
 =======================
 
-AutoControl can be used directly from the command line to execute automation scripts.
+Two CLI entry points are provided:
 
-Execute a Single Action File
-=============================
+- ``python -m je_auto_control`` — legacy flag-style runner for one-off
+  execute / create-project operations. Also launches the GUI when called
+  with no arguments.
+- ``python -m je_auto_control.cli`` — subcommand-based runner for running
+  scripts, listing scheduler jobs, and starting the socket / REST servers.
+
+Subcommand CLI (``python -m je_auto_control.cli``)
+==================================================
+
+Run a script
+------------
 
 .. code-block:: bash
 
-   python -m je_auto_control --execute_file "path/to/actions.json"
+   python -m je_auto_control.cli run script.json
+   python -m je_auto_control.cli run script.json --var count=10 --var name=alice
+   python -m je_auto_control.cli run script.json --dry-run
+
+``--var name=value`` is parsed as JSON when the value parses, otherwise
+it is treated as a plain string. ``--dry-run`` records every action
+through the executor without invoking any side effects.
+
+List scheduler jobs
+-------------------
+
+.. code-block:: bash
+
+   python -m je_auto_control.cli list-jobs
+
+Start the TCP socket server
+---------------------------
+
+.. code-block:: bash
+
+   python -m je_auto_control.cli start-server --host 127.0.0.1 --port 9938
 
-   # Short form
+Start the REST API server
+-------------------------
+
+.. code-block:: bash
+
+   python -m je_auto_control.cli start-rest --host 127.0.0.1 --port 9939
+
+Endpoints: ``GET /health``, ``GET /jobs``, ``POST /execute`` with
+``{"actions": [...]}``.
+
+Legacy flag-style CLI (``python -m je_auto_control``)
+=====================================================
+
+Execute a single action file
+----------------------------
+
+.. code-block:: bash
+
+   python -m je_auto_control --execute_file "path/to/actions.json"
    python -m je_auto_control -e "path/to/actions.json"
 
-Execute All Files in a Directory
-================================
+Execute all files in a directory
+--------------------------------
 
 .. code-block:: bash
 
    python -m je_auto_control --execute_dir "path/to/action_files/"
-
-   # Short form
    python -m je_auto_control -d "path/to/action_files/"
 
-Execute a JSON String Directly
-==============================
+Execute a JSON string directly
+------------------------------
 
 .. code-block:: bash
 
    python -m je_auto_control --execute_str '[["AC_screenshot", {"file_path": "test.png"}]]'
 
-Create a Project Template
-=========================
+Create a project template
+-------------------------
 
 .. code-block:: bash
 
    python -m je_auto_control --create_project "path/to/my_project"
-
-   # Short form
    python -m je_auto_control -c "path/to/my_project"
 
 Launch the GUI
-==============
+--------------
 
 .. code-block:: bash
 

docs/source/Eng/doc/new_features/new_features_doc.rst

@@ -0,0 +1,211 @@
+=====================
+New Features (2026-04)
+=====================
+
+This page documents the April 2026 additions to AutoControl. Every new
+feature ships with a headless Python API **and** a GUI affordance, and is
+wired into the executor so it works from JSON scripts, the socket server,
+the REST API, and the CLI without any Python glue.
+
+.. contents::
+   :local:
+   :depth: 2
+
+
+Clipboard
+=========
+
+Headless::
+
+   import je_auto_control as ac
+   ac.set_clipboard("hello")
+   text = ac.get_clipboard()
+
+Action-JSON commands::
+
+   [["AC_clipboard_set", {"text": "hello"}]]
+   [["AC_clipboard_get", {}]]
+
+Backends: Windows (Win32 via ``ctypes``), macOS (``pbcopy``/``pbpaste``),
+Linux (``xclip`` or ``xsel``). A ``RuntimeError`` is raised if no backend
+is available.
+
+
+Dry-run / step-debug executor
+=============================
+
+Run an action list through the executor without invoking any side effects
+— useful for validating JSON scripts::
+
+   from je_auto_control.utils.executor.action_executor import executor
+   record = executor.execute_action(actions, dry_run=True)
+
+``step_callback`` lets you observe each action before it runs::
+
+   executor.execute_action(actions, step_callback=lambda a: print(a))
+
+From the CLI::
+
+   python -m je_auto_control.cli run script.json --dry-run
+
+
+Global hotkey daemon (Windows)
+==============================
+
+Bind OS-level hotkeys to action-JSON scripts::
+
+   from je_auto_control import default_hotkey_daemon
+   default_hotkey_daemon.bind("ctrl+alt+1", "scripts/greet.json")
+   default_hotkey_daemon.start()
+
+Supported modifiers: ``ctrl``, ``alt``, ``shift``, ``win`` / ``super`` /
+``meta``. Keys: letters, digits, ``f1`` … ``f12``, arrows, ``space``,
+``enter``, ``tab``, ``escape``, ``home``, ``end``, ``insert``, ``delete``,
+``pageup``, ``pagedown``.
+
+macOS and Linux currently raise ``NotImplementedError`` on
+``start()`` — the Strategy-pattern interface is in place so backends can
+be added later.
+
+GUI: **Hotkeys** tab (bind/unbind, start/stop daemon, live fired count).
+
+
+Event triggers
+==============
+
+Poll-based triggers fire an action script when a screen/state change is
+detected::
+
+   from je_auto_control import default_trigger_engine, ImageAppearsTrigger
+   default_trigger_engine.add(ImageAppearsTrigger(
+       trigger_id="", script_path="scripts/click_ok.json",
+       image_path="templates/ok_button.png", threshold=0.85,
+       repeat=True,
+   ))
+   default_trigger_engine.start()
+
+Available trigger types:
+
+- ``ImageAppearsTrigger`` — template match on the current screen
+- ``WindowAppearsTrigger`` — title substring match
+- ``PixelColorTrigger`` — pixel color within tolerance
+- ``FilePathTrigger`` — mtime change on a path
+
+GUI: **Triggers** tab (add/remove/start/stop, live fired count).
+
+
+Cron scheduling
+===============
+
+Five-field cron (``minute hour day-of-month month day-of-week``) with
+``*``, comma-lists, ``*/step``, and ``start-stop`` ranges::
+
+   from je_auto_control import default_scheduler
+   job = default_scheduler.add_cron_job(
+       script_path="scripts/daily.json",
+       cron_expression="0 9 * * 1-5",   # 09:00 on weekdays
+   )
+   default_scheduler.start()
+
+Interval and cron jobs coexist in the same scheduler; ``job.is_cron``
+tells them apart. GUI: **Scheduler** tab has cron/interval radio.
+
+
+Plugin loader
+=============
+
+A plugin file is any ``.py`` defining top-level callables whose names
+start with ``AC_``. Each one becomes a new executor command::
+
+   # my_plugins/greeting.py
+   def AC_greet(args=None):
+       return f"hello, {args['name']}"
+
+::
+
+   from je_auto_control import (
+       load_plugin_directory, register_plugin_commands,
+   )
+   commands = load_plugin_directory("my_plugins/")
+   register_plugin_commands(commands)
+
+   # Now usable from JSON:
+   # [["AC_greet", {"name": "world"}]]
+
+GUI: **Plugins** tab (browse directory, one-click register).
+
+.. warning::
+   Plugin files execute arbitrary Python. Only load from directories
+   under your own control.
+
+
+REST API server
+===============
+
+A stdlib-only HTTP server that exposes the executor and scheduler::
+
+   from je_auto_control import start_rest_api_server
+   server = start_rest_api_server(host="127.0.0.1", port=9939)
+
+Endpoints:
+
+- ``GET /health`` — liveness probe
+- ``GET /jobs`` — scheduler job list
+- ``POST /execute`` with body ``{"actions": [...]}`` — run actions
+
+GUI: **Socket Server** tab now has a separate REST section with its own
+host/port and a ``0.0.0.0`` opt-in.
+
+.. note::
+   Defaults to ``127.0.0.1`` per CLAUDE.md policy. Bind to ``0.0.0.0``
+   only when you have authenticated the network boundary.
+
+
+CLI runner
+==========
+
+A thin subcommand-based CLI over the headless APIs::
+
+   python -m je_auto_control.cli run script.json
+   python -m je_auto_control.cli run script.json --var name=alice --dry-run
+   python -m je_auto_control.cli list-jobs
+   python -m je_auto_control.cli start-server --port 9938
+   python -m je_auto_control.cli start-rest --port 9939
+
+``--var name=value`` is parsed as JSON when possible (so ``count=10``
+becomes an int), otherwise treated as a string.
+
+
+Multi-language GUI (i18n)
+=========================
+
+Live language switching via the **Language** menu. Built-in packs:
+
+- English
+- Traditional Chinese (繁體中文)
+- Simplified Chinese (简体中文)
+- Japanese (日本語)
+
+Register additional languages at runtime::
+
+   from je_auto_control.gui.language_wrapper.multi_language_wrapper import (
+       language_wrapper,
+   )
+   language_wrapper.register_language("French", {"menu_file": "Fichier", ...})
+
+Missing keys fall through to the English default, so a feature ships
+with usable labels even before its translations land.
+
+
+Closable tabs + menu bar
+========================
+
+The main window is now a ``QMainWindow`` with:
+
+- **File** → Open Script, Exit
+- **View → Tabs** → checkable entries for every tab (restore closed tabs)
+- **Tools** → Start hotkey daemon / scheduler / trigger engine
+- **Language** → select a registered language pack
+- **Help** → About
+
+Close any tab with its ✕ button; re-open it via *View → Tabs*.

docs/source/Eng/eng_index.rst

@@ -22,3 +22,4 @@ Comprehensive guides for all AutoControl features.
    doc/critical_exit/critical_exit_doc
    doc/cli/cli_doc
    doc/create_project/create_project_doc
+   doc/new_features/new_features_doc