|
| 1 | +# CLAUDE.md — AutoControl |
| 2 | + |
| 3 | +## Project Overview |
| 4 | + |
| 5 | +AutoControl (`je_auto_control`) is a cross-platform Python GUI automation framework supporting Windows (Win32 API), macOS (pyobjc/Quartz), and Linux (X11). It provides mouse/keyboard control, image recognition, screen capture, action scripting, and report generation through a unified API. |
| 6 | + |
| 7 | +- **Package name**: `je_auto_control` |
| 8 | +- **Python**: >= 3.10 |
| 9 | +- **License**: MIT |
| 10 | +- **Author**: JE-Chen |
| 11 | + |
| 12 | +## Architecture & Design Patterns |
| 13 | + |
| 14 | +### Strategy Pattern — Platform Abstraction |
| 15 | + |
| 16 | +`wrapper/platform_wrapper.py` auto-detects the OS and loads the correct backend. All wrapper modules (`auto_control_mouse.py`, `auto_control_keyboard.py`, etc.) delegate to the platform-specific implementation. New platform support is added by implementing the backend interface — no wrapper changes needed. |
| 17 | + |
| 18 | +### Facade Pattern — Unified API Surface |
| 19 | + |
| 20 | +`je_auto_control/__init__.py` re-exports all public functions from wrapper and utility modules, providing a single entry point. Users import only `je_auto_control` and access all features. |
| 21 | + |
| 22 | +### Command Pattern — JSON Action Executor |
| 23 | + |
| 24 | +`utils/executor/action_executor.py` maps string command names (e.g., `AC_click_mouse`) to callable functions. JSON action files define sequences of commands with parameters, enabling recording, serialization, and replay of automation flows. |
| 25 | + |
| 26 | +### Observer Pattern — Callback Executor |
| 27 | + |
| 28 | +`utils/callback/callback_function_executor.py` allows registering callback functions that fire after automation actions complete, supporting event-driven chaining. |
| 29 | + |
| 30 | +### Template Method — Report Generation |
| 31 | + |
| 32 | +`utils/generate_report/` provides HTML, JSON, and XML report generators sharing a common structure: collect test records, format output, write file. Each format implements its own rendering. |
| 33 | + |
| 34 | +## Directory Structure |
| 35 | + |
| 36 | +``` |
| 37 | +je_auto_control/ |
| 38 | +├── wrapper/ # Platform-agnostic API (Strategy consumers) |
| 39 | +├── windows/ # Win32 backend (ctypes) |
| 40 | +├── osx/ # macOS backend (pyobjc/Quartz) |
| 41 | +├── linux_with_x11/ # Linux X11 backend (python-Xlib) |
| 42 | +├── gui/ # PySide6 GUI application |
| 43 | +└── utils/ |
| 44 | + ├── executor/ # JSON action executor (Command pattern) |
| 45 | + ├── callback/ # Callback executor (Observer pattern) |
| 46 | + ├── cv2_utils/ # OpenCV: screenshot, template matching, video |
| 47 | + ├── socket_server/ # TCP server for remote automation |
| 48 | + ├── shell_process/ # Shell command manager |
| 49 | + ├── generate_report/ # HTML/JSON/XML report generators |
| 50 | + ├── test_record/ # Test action recording |
| 51 | + ├── json/ # JSON action file I/O |
| 52 | + ├── project/ # Project scaffolding |
| 53 | + ├── package_manager/ # Dynamic package loading |
| 54 | + ├── logging/ # Logging |
| 55 | + └── exception/ # Custom exceptions |
| 56 | +``` |
| 57 | + |
| 58 | +## Development Commands |
| 59 | + |
| 60 | +```bash |
| 61 | +# Install dependencies |
| 62 | +pip install -r dev_requirements.txt |
| 63 | + |
| 64 | +# Install with GUI support |
| 65 | +pip install -e .[gui] |
| 66 | + |
| 67 | +# Run unit tests |
| 68 | +python -m pytest test/unit_test/ |
| 69 | + |
| 70 | +# Run integration tests |
| 71 | +python -m pytest test/integrated_test/ |
| 72 | + |
| 73 | +# Build package |
| 74 | +python -m build |
| 75 | +``` |
| 76 | + |
| 77 | +## Coding Standards |
| 78 | + |
| 79 | +### Security First |
| 80 | + |
| 81 | +- **Input validation**: Validate all external inputs (user input, file content, network data, JSON action commands) at system boundaries. Sanitize file paths to prevent path traversal. Never trust data from TCP socket clients without validation. |
| 82 | +- **Injection prevention**: When executing shell commands (`shell_process`), never construct command strings from unsanitized input. Use parameterized approaches or allowlists. |
| 83 | +- **Deserialization safety**: JSON action files and socket server payloads must be validated against expected schemas before execution. Reject unknown command names. |
| 84 | +- **No secrets in code**: Never commit credentials, API keys, tokens, or `.env` files. Keep secrets out of logs and reports. |
| 85 | +- **Principle of least privilege**: Socket server should bind to localhost by default. Document security implications of exposing to network. |
| 86 | +- **Dependency awareness**: Pin dependency versions. Review transitive dependencies for known vulnerabilities. |
| 87 | + |
| 88 | +### Performance Best Practices |
| 89 | + |
| 90 | +- **Lazy imports**: Platform-specific backends are loaded only for the current OS — do not import all backends unconditionally. |
| 91 | +- **Avoid redundant screenshots**: Image recognition operations should reuse screen captures when performing multiple searches on the same frame. |
| 92 | +- **Buffer management**: Screen recording and video capture must properly release resources (file handles, codec buffers) in `finally` blocks or context managers. |
| 93 | +- **Thread safety**: Socket server and recording threads must use proper synchronization. Avoid shared mutable state without locks. |
| 94 | +- **Minimize allocations in hot paths**: Mouse/keyboard event dispatch should avoid unnecessary object creation per event. |
| 95 | + |
| 96 | +### Software Engineering Principles |
| 97 | + |
| 98 | +- **SOLID**: Each module has a single responsibility. Platform backends are open for extension (new OS) without modifying wrappers. Depend on abstractions (wrapper API), not concrete implementations (Win32/X11/Quartz). |
| 99 | +- **DRY**: Common logic belongs in `wrapper/` or `utils/`, not duplicated across platform backends. |
| 100 | +- **YAGNI**: Do not add speculative features. Implement what is needed now. |
| 101 | +- **Fail fast**: Raise clear, specific exceptions (`AutoControlMouseException`, `AutoControlKeyboardException`, etc.) at the point of failure. Do not silently swallow errors. |
| 102 | +- **Immutable data where possible**: Action lists and configuration should be treated as read-only once loaded. |
| 103 | + |
| 104 | +### Code Style |
| 105 | + |
| 106 | +- Follow PEP 8. |
| 107 | +- Use type hints for all public function signatures. |
| 108 | +- Keep functions focused and short — one function, one task. |
| 109 | +- Prefer composition over inheritance for extending functionality. |
| 110 | +- Remove dead code immediately — no commented-out blocks, no unused imports, no unreachable branches. |
| 111 | + |
| 112 | +## Commit Conventions |
| 113 | + |
| 114 | +- Write concise commit messages focused on **why**, not what. |
| 115 | +- **Do not mention any AI tools, assistants, or models in commit messages** — no "Co-Authored-By" AI attributions, no references to AI-generated code. |
| 116 | +- Use imperative mood: "Add feature", "Fix bug", "Remove unused code". |
| 117 | +- Examples: |
| 118 | + - `Add image threshold parameter validation` |
| 119 | + - `Fix mouse scroll direction on macOS` |
| 120 | + - `Remove deprecated screen capture fallback` |
| 121 | + |
| 122 | +## Testing |
| 123 | + |
| 124 | +- **Unit tests**: `test/unit_test/` — test individual functions in isolation. |
| 125 | +- **Integration tests**: `test/integrated_test/` — test cross-module workflows. |
| 126 | +- **Manual tests**: `test/manual_test/` — require human verification (GUI, visual). |
| 127 | +- **GUI tests**: `test/gui_test/` — PySide6 interface tests. |
| 128 | +- All tests must pass before merging. Ensure cross-platform compatibility. |
| 129 | + |
| 130 | +## Key Conventions |
| 131 | + |
| 132 | +- All public API functions are exported from `je_auto_control/__init__.py` and listed in `__all__`. |
| 133 | +- JSON action command names use `AC_` prefix (e.g., `AC_click_mouse`). |
| 134 | +- Platform backends follow naming: `{platform}_{function}.py` (e.g., `win32_ctype_mouse_control.py`). |
| 135 | +- Virtual key mappings are in `core/utils/*_vk.py` per platform. |
0 commit comments