diff --git a/.GCC/branches/plan_accessibility_phase_1.md b/.GCC/branches/plan_accessibility_phase_1.md new file mode 100644 index 0000000..4de9f5c --- /dev/null +++ b/.GCC/branches/plan_accessibility_phase_1.md @@ -0,0 +1,221 @@ +# Execution Plan: Médiation d'Accessibilité Programmatique via AT-SPI / D-Bus (Issue #130) + +## 📋 Target Invariant & Pre-requisites +- **Target Invariant**: CI 100% PASS (`./ci.sh` : compileall, workflows, ruff check, ruff format, mypy, pytest 100%). Intégrité et robustesse du connecteur AT-SPI en Rust (`linux/crates/atspi_mediator`) et de la couche Python `linux.layers.accessibility`. Zéro régression sur les 65 tests initiaux. +- **Pre-requisites**: Branche `feat/accessibility-mediation-phase-1`. Compilation locale du binaire Rust `linux/crates/atspi_mediator/target/release/gui-agent-atspi` et connectivité D-Bus AT-SPI (`org.a11y.Bus`). + +## 🛠️ Step-by-Step Sequence + +### Step 1: Création & Compilation du Crate Rust Natif `linux/crates/atspi_mediator` +- [x] **Action**: Implémentation de `linux/crates/atspi_mediator/` avec `Cargo.toml`, `src/lib.rs` et `src/main.rs`. Implémentation des primitives AT-SPI2 / D-Bus (`snapshot_tree`, `perform_action`, `set_element_value`, `list_accessible_apps`, `connect`). Support CLI unifié (`state`, `action`, `value`, `apps`, `doctor`) et serveur MCP stdio JSON-RPC (`mcp`) pour les requêtes distantes. Résolution automatique des index numériques vers `object_ref` et repli sur les actions génériques (`activate`, `click`, `press`, `default`). +- [x] **Verify**: `cargo build --release --manifest-path linux/crates/atspi_mediator/Cargo.toml` +- **Verification Proof**: +```text + Compiling atspi-mediator v0.1.0 (/home/omni/Code/gui_agent/linux/crates/atspi_mediator) + Finished `release` profile [optimized] target(s) in 1m 06s +-rwxrwxr-x 2 omni omni 3,0M 12 sept. 02:28 /home/omni/Code/gui_agent/linux/crates/atspi_mediator/target/release/gui-agent-atspi +``` + +### Step 2: Implémentation & Durcissement de la couche Python `linux/layers/accessibility.py` +- [x] **Action**: Connecteur AT-SPI dans `linux/layers/accessibility.py`. Détection prioritaire du binaire Rust natif compilé localement (`gui-agent-atspi` en release ou debug), extraction d'arbres sémantiques en JSON, cache local de correspondance `_last_node_cache` (`element_index -> object_ref`), priorisation de `element_identifier` pour éviter les échecs de nœuds non trouvés, déclenchement d'actions et mutation de valeurs. +- [x] **Verify**: `./venv/bin/python -c "from linux.layers.accessibility import get_app_state, perform_action; s = get_app_state(app_name='antigravity'); print('State:', s['status'], s['count']); print('Action on 0:', perform_action('0', '0'))"` +- **Verification Proof**: +```text +State: success 2 +Action on 0: True +``` + +### Step 3: Tests Unitaires & Intégration de la Couche Accessibilité +- [x] **Action**: Ajout de 18 tests unitaires dans `linux/tests/test_accessibility.py` et 9 tests unitaires Rust dans `linux/crates/atspi_mediator/src/lib.rs` couvrant : détection du binaire, extraction nominale et filtrée, gestion des timeouts et erreurs subprocess, mocks d'état et de handlers, protocole MCP JSON-RPC, résolution par cache de nœuds (`_last_node_cache`), gestion stricte de `snapshot_id`, sélection déterministe d'action (`select_action_index`), rejet fail-closed des index périmés, rejet des noms dupliqués et des actions non liées, et intégration façade `mcp_core`. +- [x] **Verify**: `./venv/bin/pytest linux/tests/test_accessibility.py -v && cargo test --manifest-path linux/crates/atspi_mediator/Cargo.toml` +- **Verification Proof**: +```text +============================= test session starts ============================== +platform linux -- Python 3.13.5, pytest-9.1.1, pluggy-1.6.0 -- /home/omni/Code/gui_agent/venv/bin/python3 +cachedir: .pytest_cache +rootdir: /home/omni/Code/gui_agent +configfile: pyproject.toml +plugins: anyio-4.13.0 +collecting ... collected 18 items + +tests/test_accessibility.py::test_find_atspi_mediator_binary PASSED [ 5%] +tests/test_accessibility.py::test_get_app_state_nominal_or_error PASSED [ 11%] +tests/test_accessibility.py::test_get_app_state_with_screenshot PASSED [ 16%] +tests/test_accessibility.py::test_get_app_state_mock PASSED [ 22%] +tests/test_accessibility.py::test_get_app_state_binary_missing PASSED [ 27%] +tests/test_accessibility.py::test_get_app_state_subprocess_error PASSED [ 33%] +tests/test_accessibility.py::test_get_app_state_subprocess_timeout PASSED [ 38%] +tests/test_accessibility.py::test_perform_action_mock PASSED [ 44%] +tests/test_accessibility.py::test_perform_action_binary_missing PASSED [ 50%] +tests/test_accessibility.py::test_perform_action_mcp_mock_protocol PASSED [ 55%] +tests/test_accessibility.py::test_set_value_mock PASSED [ 61%] +tests/test_accessibility.py::test_set_value_binary_missing PASSED [ 66%] +tests/test_accessibility.py::test_set_value_mcp_mock_protocol PASSED [ 72%] +tests/test_accessibility.py::test_perform_action_and_set_value_with_node_cache PASSED [ 77%] +tests/test_accessibility.py::test_mcp_core_sdk_facade_integration PASSED [ 83%] +tests/test_accessibility.py::test_get_app_state_invalid_tree_payload PASSED [ 88%] +tests/test_accessibility.py::test_perform_action_and_set_value_with_snapshot_id_mismatch PASSED [ 94%] +tests/test_accessibility.py::test_perform_action_and_set_value_with_missing_cache_index PASSED [100%] + +============================== 18 passed in 7.75s ============================== + +running 9 tests +test tests::test_select_action_index_duplicate_exact_names_rejected ... ok +test tests::test_select_action_index_ambiguous_rejected ... ok +test tests::test_select_action_index_empty_actions ... ok +test tests::test_select_action_index_exact_match ... ok +test tests::test_select_action_index_numeric_index ... ok +test tests::test_select_action_index_single_action ... ok +test tests::test_select_action_index_single_unrelated_action_rejected_for_click ... ok +test tests::test_split_object_ref_id_invalid ... ok +test tests::test_split_object_ref_id_valid ... ok + +test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.01s +``` + +### Step 4: Validation Complète de la Suite CI +- [x] **Action**: Exécution du runner CI local `./ci.sh` pour valider l'ensemble des étapes de qualité (compileall, workflows, linter Ruff, formateur Ruff, typage Mypy strict, suite Pytest 83 tests). +- [x] **Verify**: `./ci.sh` +- **Verification Proof**: +```text +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +📊 RÉSUMÉ D'EXÉCUTION CI (CI Summary) +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +| Étape de Validation | Statut | Durée | +|--------------------------------------------|------------|------------| +| Compilation Bytecode Python (compileall) | PASS | 216ms | +| Validation Workflows GitHub Actions | PASS | 69ms | +| Linter de Code (Ruff Check) | PASS | 68ms | +| Formatage de Code (Ruff Format) | PASS | 81ms | +| Typage Statique Strict (Mypy) | PASS | 529ms | +| Suite de Tests Pytest | PASS | 34169ms | +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +🎉 Toutes les étapes CI sont validées avec succès ! +============================= 83 passed in 33.29s ============================== +``` + +### Step 5: Durcissement exhaustif suite aux retours de revues (Greptile & CodeRabbit) +- [x] **Action**: Correction complète de l'ensemble des constats : + 1. Résolution du bus AT-SPI dédié (`_get_atspi_bus_address` interrogeant `AT_SPI_BUS_ADDRESS`, `org.a11y.Bus.GetAddress` et `/run/user//at-spi/bus_0`). + 2. Durcissement du serveur stdio Rust MCP : interruption de boucle sur EOF/erreur stdin (`break`), liaison stricte des index numériques aux snapshots (`snapshot_token`). + 3. Sécurisation XDG et répertoires d'exécution (`_validate_xdg_env_path` absolu, `_is_secure_runtime_directory` vérifiant UID et mode `0700`). + 4. Isolation du build Cargo dans `linux/install.sh` avec `TARGET_DIR` dédié et validation d'artefact. + 5. Correction de l'interpolation de chemin hatchling dans `hatch_build.py`. + 6. Décalage des coordonnées d'écran virtuel sous KDE spectacle multi-écrans dans `linux/server.py`. + 7. Correction des chemins de test CI dans `.github/workflows/ci.yml` (`linux/tests/`). + 8. Ajout des tests de non-régression (110 tests au total). +- [x] **Verify**: `./ci.sh && cargo check` +- **Verification Proof**: +```text +============================= 110 passed in 33.40s ============================= +✔ Validé (35302ms) +``` + +### Step 6: Résolution finale des 3 constats bloquants Greptile (P1) +- [x] **Action**: Traitement chirurgical et exhaustif : + 1. Transmission directe de `element_identifier` sans `element_index` depuis Python et priorité aux identifiants non-numériques dans Rust `resolve_mcp_target` (suppression de l'erreur "Aucun snapshot actif" sur les processus MCP fraîchement démarrés). + 2. Parsing universel de `_get_atspi_bus_address` acceptant les sorties `busctl` et `dbus-send` avec ou sans guillemets (`s unix:path=...` ou `s "unix:path=..."`). + 3. Confinement strict de `--purge-data` dans `linux/uninstall.sh` au cache approuvé de gui-agent, ignorant les variables d'environnement arbitraires externes. + 4. Nouveaux tests de validation comportementale : 111 tests unitaires et d'intégration validés. +- [x] **Verify**: `./ci.sh && cargo test --all-features && cargo fmt --check` +- **Verification Proof**: +```text +============================= 111 passed in 38.12s ============================= +✔ Validé (40012ms) + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +📊 RÉSUMÉ D'EXÉCUTION CI (CI Summary) +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +| Étape de Validation | Statut | Durée | +|--------------------------------------------|------------|------------| +| Compilation Bytecode Python (compileall) | PASS | 234ms | +| Validation Workflows GitHub Actions | PASS | 93ms | +| Linter de Code (Ruff Check) | PASS | 22ms | +| Formatage de Code (Ruff Format) | PASS | 21ms | +| Typage Statique Strict (Mypy) | PASS | 554ms | +| Suite de Tests Pytest | PASS | 40012ms | +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +🎉 Toutes les étapes CI sont validées avec succès ! +``` + +### Step 7: Prise en charge sécurisée des répertoires de captures personnalisés lors de la désinstallation (Greptile 5/5) +- [x] **Action**: Mise à jour de `linux/uninstall.sh` pour consulter et purger `GUI_AGENT_SCREENSHOTS_DIR` en plus du cache par défaut, avec frontière stricte d'approbation (`$APPROVED_CACHE`, `$APPROVED_DATA`) pour la suppression récursive, protection contre les répertoires système/racines/utilisateurs critiques (`/`, `$HOME`, `/tmp`, `/var`, etc.), vérification stricte de propriété UID (`-O`), et pour tout répertoire personnalisé/externe, purge ciblée exclusive des motifs exacts de captures et vidéos générés (`screenshot_[0-9a-zA-Z_-]*.@(png|jpg|jpeg|webp)`, `raw_screenshot_[0-9a-zA-Z_-]*`, `web_screenshot_[0-9a-zA-Z_-]*`, `video_[0-9a-zA-Z_-]*.mp4`, `recording_[0-9a-zA-Z_-]*.mp4`, `_mcp_screen_tmp_*.png`) préservant 100% des fichiers tiers (`capture-vacation.txt`, `video-notes.txt`, repos de dev contenant `gui-agent`), et tests unitaires exhaustifs dans `linux/tests/test_package.py` (112 tests PASS). +- [x] **Verify**: `./ci.sh && cargo test --all-features` +- **Verification Proof**: +```text +============================= 112 passed in 38.04s ============================= +✔ Validé (39766ms) + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +📊 RÉSUMÉ D'EXÉCUTION CI (CI Summary) +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +| Étape de Validation | Statut | Durée | +|--------------------------------------------|------------|------------| +| Compilation Bytecode Python (compileall) | PASS | 282ms | +| Validation Workflows GitHub Actions | PASS | 86ms | +| Linter de Code (Ruff Check) | PASS | 23ms | +| Formatage de Code (Ruff Format) | PASS | 20ms | +| Typage Statique Strict (Mypy) | PASS | 631ms | +| Suite de Tests Pytest | PASS | 39766ms | +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +### Step 8: Restriction stricte des motifs de purge personnalisés aux formats authentiques de l'application (Greptile 5/5 & CodeRabbit) +- [x] **Action**: Mise à jour de `linux/uninstall.sh` pour remplacer les commandes `find -name` avec globs génériques par une boucle de validation stricte par expressions régulières directes (`[[ "$fname" =~ ... ]]`). Les fichiers ciblés dans les emplacements personnalisés sont strictement validés : + 1. Captures : `^(screenshot|raw_screenshot|web_screenshot)_[0-9]+(_[0-9]+)?(\ \([0-9]+\))?\.(png|jpg|jpeg|webp)$` + 2. Enregistrements : `^recording_([0-9]+(_[0-9]+)?|[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}|[0-9a-fA-F]{32})(\ \([0-9]+\))?\.mp4$` + 3. Fichiers MCP temporaires : `^_mcp_screen_tmp_[0-9a-zA-Z_]+\.png$` + Préservation garantie de 100% des fichiers médias tiers, y compris ceux ayant des préfixes numériques comme `recording_1_interview.mp4`, `screenshot_1_final.png`, `video_projet.mp4` ou les UUID tronqués. Enrichissement des assertions dans `linux/tests/test_package.py`. +- [x] **Verify**: `./ci.sh && cargo test --all-features` +- **Verification Proof**: +```text +============================= 112 passed in 37.56s ============================= +✔ Validé (39509ms) + +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +📊 RÉSUMÉ D'EXÉCUTION CI (CI Summary) +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +| Étape de Validation | Statut | Durée | +|--------------------------------------------|------------|------------| +| Compilation Bytecode Python (compileall) | PASS | 326ms | +| Validation Workflows GitHub Actions | PASS | 79ms | +| Linter de Code (Ruff Check) | PASS | 37ms | +| Formatage de Code (Ruff Format) | PASS | 20ms | +| Typage Statique Strict (Mypy) | PASS | 765ms | +| Suite de Tests Pytest | PASS | 39509ms | +━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ +🎉 Toutes les étapes CI sont validées avec succès ! +``` + +### Step 9: Validation stricte et rejet des index numériques hors plage u32 dans le médiateur Rust +- [x] **Action**: Mise à jour de `resolve_mcp_target` dans `linux/crates/atspi_mediator/src/main.rs` pour valider `target_index` via `u32::try_from(i)` et rejeter immédiatement les valeurs supérieures à `u32::MAX` (`> 4294967295`) au lieu d'autoriser un débordement silencieux vers 0. Préservation de la priorité absolue des identifiants non-numériques explicites (`element_identifier`), et ajout de 4 tests unitaires dédiés dans `src/main.rs`. +- [x] **Verify**: `cargo test --manifest-path linux/crates/atspi_mediator/Cargo.toml` +- **Verification Proof**: +```text +running 9 tests +test tests::test_select_action_index_ambiguous_rejected ... ok +test tests::test_select_action_index_duplicate_exact_names_rejected ... ok +test tests::test_select_action_index_empty_actions ... ok +test tests::test_select_action_index_exact_match ... ok +test tests::test_select_action_index_numeric_index ... ok +test tests::test_select_action_index_single_action ... ok +test tests::test_select_action_index_single_unrelated_action_rejected_for_click ... ok +test tests::test_split_object_ref_id_invalid ... ok +test tests::test_split_object_ref_id_valid ... ok + +test result: ok. 9 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s + +running 4 tests +test tests::test_resolve_mcp_target_element_index_overflow_rejected ... ok +test tests::test_resolve_mcp_target_explicit_identifier_priority ... ok +test tests::test_resolve_mcp_target_element_index_valid ... ok +test tests::test_resolve_mcp_target_numeric_identifier_fallback ... ok + +test result: ok. 4 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s +``` + +## ⚠️ Mitigations & Edge Cases +- **Risk**: Saturation CPU / mémoire lors de la compilation de `atspi` et `zbus` sur machine modeste. +- **Mitigation**: Compilation séquentielle contrôlée avec build profile release optimisé (`codegen-units = 1`, `lto = "fat"`, `strip = "symbols"`), exclusion de `target/` de git via `.gitignore` pour préserver la légèreté du dépôt (binaire release 3,0M). +- **Risk**: Absence de bus AT-SPI dans les runners CI ou environnements conteneurisés sans interface graphique. +- **Mitigation**: Détection automatique de la présence du bus AT-SPI (`AT_SPI_BUS_ADDRESS`, `org.a11y.Bus` ou socket `/run/user//at-spi/bus_0`). En cas d'indisponibilité du bus ou de l'exécutable Rust, retour gracieux d'un état d'erreur structuré sans crash ni exception non gérée. +- **Risk**: Perte de correspondance entre index numérique et identifiant AT-SPI lors d'appels `perform_action` ou `set_value` consécutifs. +- **Mitigation**: Implémentation du cache mémoire `_last_node_cache` associant chaque `index` à son `object_ref`, injectant automatiquement `element_identifier` auprès du moteur Rust. diff --git a/.GCC/branches/test.md b/.GCC/branches/test.md index 8ddb402..16a356b 100644 --- a/.GCC/branches/test.md +++ b/.GCC/branches/test.md @@ -109,4 +109,36 @@ La règle opérationnelle est désormais : `./ci.sh` + tests ciblés + Greptile ## 🟢 Conclusion & Qualification La campagne d'exécution atteste d'une qualification à **100% PASS** des 21 outils MCP GUI ainsi que du packaging standard Python (`gui-agent`), de la construction des artefacts de distribution, de l'isolation via `uv tool install`, du script d'installation Linux `install.sh`, du script PowerShell Windows `install.ps1`, du skill d'installation Windows `skills/gui-agent-windows-install/SKILL.md`, ainsi que du correctif de sécurité et de conformité du chemin de sortie pour `gui_take_screenshot` (PR #7). +--- + +## 🦾 Phase 1 : Médiation d'Accessibilité Programmatique via AT-SPI / D-Bus (Issue #130) (2026-09-12) + +| Cible / Scénario | Commande de Test | Résultat Attendu | Résultat Constaté | Statut | +|---|---|---|---|---| +| **Détection du moteur Rust** | `pytest tests/test_accessibility.py -k test_find_atspi_mediator_binary` | Résolution prioritaire variable d'env puis binaire local/upstream | Priorité respectée, détection `/bin/sh` et binaire Rust standard | **PASS** | +| **Extraction d'arbre AT-SPI** | `pytest tests/test_accessibility.py -k 'test_get_app_state_nominal_or_error or test_get_app_state_with_screenshot'` | Format JSON textuel pur avec count et tree | Structure validée (`status in ('success', 'error')`, `layer: 'accessibility'`) | **PASS** | +| **Gestion des erreurs et timeouts** | `pytest tests/test_accessibility.py -k 'test_get_app_state_subprocess_error or test_get_app_state_subprocess_timeout or test_get_app_state_binary_missing'` | Pas de crash, dict d'erreur clair et borné | Retours d'erreur structurés avec capture propre | **PASS** | +| **Mocks d'état applicatif** | `pytest tests/test_accessibility.py -k test_get_app_state_mock` | Découplage complet sans bus graphique requis | Arbre synthétique injecté et inspecté fidèlement | **PASS** | +| **Déclenchement d'actions RAM** | `pytest tests/test_accessibility.py -k 'test_perform_action_mock or test_perform_action_mcp_mock_protocol or test_perform_action_binary_missing'` | Format MCP element_index/element_identifier | Protocole JSON-RPC validé, exécution d'actions sans curseur | **PASS** | +| **Écriture directe en mémoire** | `pytest tests/test_accessibility.py -k 'test_set_value_mock or test_set_value_mcp_mock_protocol or test_set_value_binary_missing'` | Mutation Value/EditableText directe | Valeurs assignées sans perte ni layout clavier | **PASS** | +| **Résolution par cache de nœuds** | `pytest tests/test_accessibility.py -k test_perform_action_and_set_value_with_node_cache` | Injection de element_identifier à partir du cache local | Résolution d'index numérique vérifiée et validée | **PASS** | +| **Intégration façade SDK** | `pytest tests/test_accessibility.py -k test_mcp_core_sdk_facade_integration` | Exposition cohérente dans `mcp_core` | `mcp_core.get_app_state`, `perform_action`, `set_value` opérationnels | **PASS** | +| **Régression Scaffolding** | `pytest tests/test_package.py -k test_modular_architecture_scaffolding` | Non-régression sur le découpage modulaire | Validation 100% de la structure core/layers/utils | **PASS** | +| **Validation stricte de l'arbre** | `pytest tests/test_accessibility.py -k test_get_app_state_invalid_tree_payload` | Rejet explicite des payloads non conformes | Erreur structurée retournée, purge déterministe du cache de nœuds | **PASS** | +| **Identité de snapshot & non-obsolescence** | `pytest tests/test_accessibility.py -k test_perform_action_and_set_value_with_snapshot_id_mismatch` | Rejet immédiat (return False) si snapshot_id mismatch | Évite toute action accidentelle sur un index périmé | **PASS** | +| **Rejet fail-closed d'index absent du cache** | `pytest tests/test_accessibility.py -k test_perform_action_and_set_value_with_missing_cache_index` | Rejet immédiat sans subprocess si index inconnu | Aucune action non coordonnée sur arbre sauvage | **PASS** | +| **Validation CodeRabbit locale** | `coderabbit review --agent -t uncommitted` | 0 finding bloquant / avertissement sur l'ensemble des fichiers modifiés | 0 finding, review completed avec succès | **PASS** | +| **Durcissement index bounds & hydratation D-Bus** | `cargo test --manifest-path crates/atspi_mediator/Cargo.toml` | Rejet immédiat sur index numérique hors limites + mutex d'hydratation réessayable | 8/8 tests passés, 0 warning clippy | **PASS** | +| **Rejet obligatoire d'index sans snapshot_id** | `pytest tests/test_accessibility.py -k test_numeric_index_without_snapshot_id_rejected` | Rejet fail-closed systématique sans Popen si snapshot_id manquant sur index numérique | Test unitaire validé | **PASS** | +| **Livraison native et fallbacks install.sh** | `./install.sh --dry-run` | Détection cibles workspace/crate target et fallback git cargo install | 0 erreur, code 0 | **PASS** | +| **Validation Globale CI** | `./ci.sh` | 100% des étapes CI vertes (compileall, workflows, ruff check/format, mypy, pytest) | 84/84 tests passés en 32.31s, 0 avertissement, 0 erreur | **PASS** | +| **Validation CodeRabbit 3 remarques mineures** | `coderabbit review --agent --uncommitted` | 0 finding bloquant / avertissement sur `windows/install.ps1`, `linux/core/repl.py`, `linux/tests/test_package.py` | `review_completed`, `findings: 0` | **PASS** | +| **Gardiens repl incrémental UTF-8 & install.ps1** | `pytest linux/tests/test_package.py -k "test_repl_safe_read_incremental_utf8 or test_repl_execute_script_multibyte_utf8 or test_windows_install_ps1_guards"` | Validation fragments multi-octets UTF-8, script repl émettant des emojis/accents et gardes PowerShell | 3/3 tests unitaires passés | **PASS** | +| **Validation Globale CI (Phase 1 finale)** | `./ci.sh` | 100% des étapes CI vertes (compileall, verify_workflows, ruff check, ruff format, mypy, pytest) | 105/105 tests passés en 36.20s, 0 avertissement, 0 erreur | **PASS** | +| **Compilation Rust native Workspace** | `cargo check` | Vérification du crate natif `atspi_mediator` | Compilé en 0.09s (code 0) | **PASS** | +| **Revue Greptile finale sur commit `45ca723`** | `greptile review --agent --branch main` | Vérification exhaustive de la branche contre main | `Confidence: 5/5`, 0 finding, `The PR appears safe to merge` | **PASS** | +| **Certification Greptile finale sur commit `f8c261c`** | `greptile review --agent --branch main --instructions "..."` | Certification des 3 ajustements finaux (Review ID `9369eadb-09f9-439d-94e5-ca3a78d6c94a`) | `Confidence: 5/5`, 0 comment, `The reviewed adjustments appear safe to merge` | **PASS** | +| **Certification CodeRabbit finale sur commit `f8c261c`** | `coderabbit review --agent --base-commit 6d6686c` | 0 finding sur l'ensemble des fichiers modifiés | `review_completed`, `findings: 0` | **PASS** | +| **Validation Globale CI (Phase 1 durcie)** | `./ci.sh` | 100% des étapes CI vertes (compileall, verify_workflows, ruff check, ruff format, mypy, pytest) | 112/112 tests passés en 54.98s, 0 avertissement, 0 erreur | **PASS** | +| **Rejet index u32 hors plage & tests MCP** | `cargo test --manifest-path linux/crates/atspi_mediator/Cargo.toml` | Rejet immédiat sur index numérique > u32::MAX + priorité identifiant explicite | 13/13 tests passés (9 lib, 4 main), 0 avertissement | **PASS** | diff --git a/.GCC/main.md b/.GCC/main.md index 301b0cc..bd60a33 100644 --- a/.GCC/main.md +++ b/.GCC/main.md @@ -12,6 +12,7 @@ - [2026-08-27] Synchronisation Concurrente et Nettoyage Déterministe de l'Enregistrement Vidéo (PR #57, Confidence Score 5/5 Greptile) - [2026-08-27] Bornage Déterministe et Deadline Globale pour le Listing X11 (PR #55, Confidence Score 5/5 Greptile) - [2026-09-12] Modular Architecture Scaffolding, Execution Primitives Hardening & Dependencies Modernization (PR #136, Confidence Score 5/5 Greptile, 0 findings CodeRabbit, 65/65 tests) +- [2026-09-13] Médiation d'Accessibilité Programmatique AT-SPI & Durcissement Purge Uninstall (PR #137, Confidence Score 5/5 Greptile, 12/12 fils CodeRabbit résolus, 112/112 tests) ## 🎯 Objective High-performance FastMCP server engineered with a decoupled modular architecture (core, layers, utils) for direct, low-latency Computer Use on Linux (X11/XWayland) and Windows desktop environments (<50 MB RAM, 21 tools, zero-leak process lifecycle). @@ -43,6 +44,8 @@ High-performance FastMCP server engineered with a decoupled modular architecture - **Règle d'échec sans boucle** : si une revue locale ou GitHub trouve un bug, ne pas relancer immédiatement l'agent. Lire 100 % du verdict, regrouper tous les constats, corriger localement, puis refaire toute la validation locale. Une nouvelle revue GitHub nécessite un changement réel, une validation locale complète et une autorisation explicite tenant compte du coût/quota. - **Portée Greptile à ne jamais confondre** : `greptile review` local analyse le commit/working tree selon le prompt fourni ; Greptile GitHub analyse la PR complète après push. Un score local 5/5 ne prédit donc pas le score GitHub. - Après chaque revue GitHub, lire le message principal de la PR, le résumé Greptile complet, le bilan T-Rex et 100 % des commentaires/threads ; ne jamais conclure à partir du seul score ou d'un check-run. + - **Temporisation d'attente Greptile** : Les revues automatiques Greptile GitHub prennent environ **8 minutes**. Ne jamais régler de timer d'attente court (ex: 30 secondes ou 5 minutes) qui entraîne des boucles de vérification prématurées ou inutiles ; configurer impérativement un délai d'attente de **8 minutes (480 secondes)** avant de relever le verdict Greptile. + - **Identité de publication de la PR** : La Pull Request doit être créée et finalisée avec le compte GitHub **personal agent**. - CodeRabbit distant est déclenché explicitement par un commentaire `@coderabbitai review` uniquement après la validation locale et avec autorisation explicite ; ne pas le relancer si une revue est déjà en cours, rate-limitée ou non nécessaire. - Les check-runs seuls ne valent pas validation ; le critère final reste Greptile **5/5**, zéro échec de sécurité, zéro commentaire actionnable et zéro bloquant Optibot/CodeRabbit. La priorité est de réduire le nombre total de reviews, pas de boucler jusqu'à un score parfait. - **Analyse des vulnérabilités de rollback capture identifiées par Greptile (PR #7)** : @@ -50,6 +53,18 @@ High-performance FastMCP server engineered with a decoupled modular architecture - *P1 - Course TOCTOU lors de la suppression par chemin* : La séquence `os.stat()` puis `os.unlink(filename, dir_fd)` permet à un attaquant de remplacer l'entrée entre les deux appels et d'entraîner la suppression de son fichier tiers. Solution : bannir la suppression destructive basée sur le nom dans un répertoire concurrent ; retenir le descripteur ouvert de la réservation à l'écriture, ou s'abstenir de tout `unlink` non lié de manière exclusive. ## 🧠 Decisions Made +- [2026-09-12] Bundle Unique Natif par Système d'Exploitation (Rust + REPL PyO3) + - **Context**: Besoin de livrer un artefact directement exécutable par OS (`gui-agent` sous Linux, `gui-agent.exe` sous Windows, `gui-agent` sous macOS), compilable directement sur la machine hôte via Cargo sans surcoût d'environnement virtuel Python ni fragmentation multi-processus. + - **Discarded Options**: Bundle Python auto-extractible via PyInstaller/Nuitka (>150 Mo, latence au démarrage, décompression) ; distribution multi-binaires fragmentée (FastMCP Python appelant des sous-processus séparés). + - **Rationale**: Un binaire autonome Rust garantit une latence quasi-nulle (<5 ms), une consommation mémoire minime (<15 Mo), une compilation locale unifiée (`cargo build --release`), une étanchéité par plateforme via des crates dédiés (`linux/crates/`, `windows/crates/`), et une intégration native du REPL CodeAct (`execute_script`) via PyO3. +- [2026-09-12] Durcissement de la Résolution d'Index, Livraison Native et Sélection Déterministe d'Actions (Phase 1 #130) + - **Context**: Retours de revue Greptile et CodeRabbit : repli arbitraire sur l'index 1 dans `select_action_index`, maintien de l'exécution sur index périmés sans snapshot actif ou sur cache absent, et omission de la compilation/installation de `gui-agent-atspi` dans le workflow `install.sh`. + - **Discarded Options**: Tolérer un repli permissif sur une action arbitraire ; ignorer l'invalidation de cache et relancer un snapshot non filtré en tâche de fond ; exiger l'installation manuelle d'outils tiers. + - **Rationale**: Rejet fail-closed strict (return False) de tout index numérique non résolu dans `_last_node_cache` ou sur snapshot mismatch ; sélection d'action déterministe (exacte, numérique, sous-chaîne unique, synonymes sémantiques ou rejet avec listing des actions disponibles) ; build et installation automatique de `gui-agent-atspi` dans `install.sh` ; création du workspace `Cargo.toml` racine activant la détection CI native. +- [2026-09-12] Médiation d'Accessibilité Programmatique via Moteur Natif Rust AT-SPI / D-Bus (Phase 1 #130) + - **Context**: L'accès à l'interface graphique Linux par perception visuelle seule (captures d'écran et OCR) souffre de cécité sémantique (menus contextuels éphémères, scaling HiDPI, dialogues modaux bloquants). L'accès direct à AT-SPI2 via D-Bus (`org.a11y.Bus`) est requis avec une latence d'extraction minimale (< 50 ms). + - **Discarded Options**: Bibliothèque Python `pyatspi` (obsolète, fuites mémoire et dépendances C non isolées) ; binding C pur / ctypes ; réécriture complète du serveur MCP en Rust (rupture de compatibilité avec l'écosystème FastMCP Python existant). + - **Rationale**: Moteur autonome bivalent écrit en Rust (`linux/crates/atspi_mediator` produisant le binaire release autonome `gui-agent-atspi` de 3,0M épuré) exploitant `atspi` et `zbus`, avec support CLI et mode serveur stdio JSON-RPC MCP (`initialize`, `tools/call`), interfacé depuis Python via `linux/layers/accessibility.py` avec mise en cache synchronisée par verrou (`_cache_lock`) des index vers `object_ref`, communication par flux process borné avec timeouts stricts et mocks complets pour CI headless. - [2026-09-11] Architecture Modulaire Découplée (core, layers, utils #129) - **Context**: Monolithe historique couplant REPL, gestion PTY, drivers bas niveau et helpers. - **Discarded Options**: Monolithe persistant ; micro-paquets distribués séparément. @@ -112,6 +127,7 @@ High-performance FastMCP server engineered with a decoupled modular architecture - **Rationale**: Geler la structure jusqu'à la revue utilisateur afin de ne pas invalider les chemins de son audit, et reporter les corrections futures dans l'audit. ## 🌿 Active Branches / Plans +- `feat/accessibility-mediation-phase-1` : Médiation d'accessibilité programmatique via AT-SPI / D-Bus (Issue #130) [plan_accessibility_phase_1.md](.GCC/branches/plan_accessibility_phase_1.md) - Pull Request [#137](https://github.com/leandre755/gui_agent/pull/137) soumise par `personnal-agent` - `main` : Production release with decoupled modular architecture (core, layers, utils), bilingual landing pages, 65/65 Zero-Slop test harness, hardened screenshot rollback lifecycle, bounded X11 timeouts and thread-safe video recording. ## 📈 Current Status @@ -119,18 +135,25 @@ High-performance FastMCP server engineered with a decoupled modular architecture - Suppression définitive des 8 issues obsolètes (#3, #4, #5, #9, #12, #28, #30, #48). - Création des 7 issues d'architecture v1.0 (#129 à #135) couvrant l'arborescence, les phases 1-4 et la recherche d'équivalents Windows/macOS. - Fusion de la PR #136 (`refactor/modular-architecture-issue-129`, Closes #129) avec Confidence Score 5/5 sur Greptile et 0 findings CodeRabbit (65/65 tests validés). + - Implémentation et durcissement complets de la Phase 1 (#130) : Médiation d'accessibilité programmatique via AT-SPI / D-Bus (moteur natif Rust `linux/crates/atspi_mediator` produisant `gui-agent-atspi`, couche Python `linux/layers/accessibility.py`, 112/112 tests CI validés au dernier `./ci.sh`). - Fermeture de la PR obsolète #112 (traitement de la sécurité subprocess #44 transféré à l'Issue #132). - Fusion des PRs précédentes (#7, #8, #16, #35, #50, #57, #55). - Fermeture des issues résolues (#42, #56, #69, #106, #70, #68, #63, #59, #46, #45, #13, #107, #43). - Nettoyage et suppression de l'ensemble des branches résiduelles distantes et locales. - - Validation CI 65/65 tests, quality gate PASS sur `main`. -- 🔄 In progress: Aucun (arbre propre sur `main`). + - Restructuration étanche par système d'exploitation (`linux/`, `windows/`, `macos/`), migration de `tests/` et `examples/` dans `linux/`, zéro code/test/cache à la racine. + - Durcissement exhaustif de `.gitignore` et purge des caches résiduels (1,5 Go de target crate et __pycache__). + - Alignement du workspace Cargo racine (`Cargo.toml`) sur `linux/crates/atspi_mediator` validé par `cargo check`. + - Décision d'architecture actée : Bundle Unique Natif par OS en Rust (avec REPL PyO3 embarqué) directement exécutable et compilable sur l'hôte. + - Validation CI 112/112 tests PASS, Mypy strict (36 fichiers), Bandit, Semgrep et quality gate pre-commit PASS sur la branche `feat/accessibility-mediation-phase-1`. + - Application intégrale et exhaustive des retours de revue Greptile et CodeRabbit : transmission directe d'identifiant résolu et priorité dans le médiateur MCP Rust, parsing universel de l'adresse de bus AT-SPI (formats bruts/cités busctl et dbus-send), prise en charge sécurisée des répertoires de captures personnalisés (`GUI_AGENT_SCREENSHOTS_DIR`) avec protection stricte des racines système/utilisateurs, vérification de propriété UID, restriction chirurgicale aux motifs applicatifs authentiques (timestamps numériques et UUID stricts), et préservation à 100% des fichiers médias tiers plausibles (`video_projet.mp4`, `recording_interview.mp4`, `screenshot_final.png`). + - Validation et certification officielle de la Pull Request [#137](https://github.com/leandre755/gui_agent/pull/137) : **Confidence Score: 5/5 sur Greptile**, **0 commentaire ajouté**, verdict *Safe to merge; there are no outstanding blocking issues*, et **12/12 fils CodeRabbit résolus** sur GitHub. + - Validation CI complète : 112/112 tests PASS, Mypy strict (36 fichiers), Bandit, Semgrep, Rust clippy/test/fmt et quality gate pre-commit PASS. +- 🔄 In progress: Approbation et fusion de la Pull Request #137 par le mainteneur. - ⏳ Pending: - - 1. **Phase 1 (#130)** : Médiation d'accessibilité programmatique via AT-SPI / D-Bus (`layers/accessibility.py`). - 2. **Phase 2 (#131)** : Moteur d'exécution local CodeAct et SDK unifié `mcp_core` (`core/repl.py`). - 3. **Phase 3 (#132)** : Émulation d'entrées noyau (`uinput/evdev`), perception visuelle (`RapidOCR`) et gestion de fenêtrage (`process_run` sécurisé). - 4. **Phase 4 (#133)** : Déclaration FastMCP des 13 outils chirurgicaux, suppression des redondances et mise à jour CI. - 5. **Recherche OS tiers (#134, #135)** : Adaptation Windows (UI Automation) et macOS (NSAccessibility). ## 👉 Next Session Direction -Initier la Phase 1 sur une nouvelle branche dédiée : Implémentation de l'Issue #130 (Médiation d'accessibilité programmatique via AT-SPI / D-Bus). +Finaliser la Pull Request pour la Phase 1 (Issue #130 : Médiation d'accessibilité programmatique via AT-SPI / D-Bus) puis initier la Phase 2 (Issue #131). diff --git a/.GCC/resume.md b/.GCC/resume.md index 59da379..ad54224 100644 --- a/.GCC/resume.md +++ b/.GCC/resume.md @@ -1,50 +1,64 @@ # Session Handoff ## 🎯 Functional Outcome & Task Reality -- **Requested Task**: Fin de session demandée par l'utilisateur : supprimer la branche locale `refactor/modular-architecture-issue-129`, basculer sur `main` et effectuer un `git pull` depuis GitHub. +- **Requested Task**: Résolution intégrale de tous les constats de revue Greptile (passage de 1/5 à 5/5) et CodeRabbit (résolution de 100% des fils, confirmation officielle de levée des blocages) sur la PR #137 (`feat/accessibility-mediation-phase-1`). - **Functional Status**: SUCCESS - **Behavioral Proof**: - - PR #136 (`refactor/modular-architecture-issue-129`) fusionnée avec succès sur GitHub (`mergedAt: 2026-09-12T01:47:04Z`, Closes #129). - - PR #112 obsolète fermée (`state: CLOSED`). - - Branche locale `refactor/modular-architecture-issue-129` supprimée sans résidu (`git branch -d`). - - Branche `main` synchronisée avec l'amont (`origin/main`) via avance rapide (`f78c5b4..aced1bd`). - - Validation exhaustive de fin de session exécutée sur `main` via `./ci.sh` : 100% PASS (65/65 tests Pytest, Ruff check 0 erreur, Ruff format 0 modif, Mypy strict 17 fichiers 0 erreur). + - Exécution complète de `./ci.sh` : 112/112 tests unitaires et d'intégration validés sans aucune erreur (`112 passed in 54.98s` en local et `112 passed in 80.93s` en pré-push). + - Validation Rust complète : `cargo fmt --check`, `cargo clippy --all-targets --all-features -- -D warnings`, `cargo test` validés avec 13/13 tests PASS (9 tests dans `lib.rs`, 4 tests dans `main.rs`). + - Binaire autonome `gui-agent-atspi` recompilé en release et synchronisé dans `linux/bin/` et `~/.local/bin/`. + - Linter Ruff : `ruff check .` validé avec 0 erreur. + - Formateur Ruff : `ruff format --check .` validé avec 0 anomalie. + - Typage Mypy : `mypy -p linux` validé avec 0 erreur sur 36 fichiers sources. + - Validation des workflows GitHub Actions : `verify_workflows.py` validé. + - Confirmation textuelle officielle CodeRabbit : *"Oui. Les corrections demandées ont été vérifiées dans le commit 2e931e8... Je ne vois plus de blocage lié à ma demande de changements."* + - 100% des fils de discussion de revue résolus sur GitHub (25/25 threads résolus, 0 thread non résolu). ## ⚡ Technical Diffs / Atomic Modifications -- **Branch**: `main` (synchronisée sur commit `aced1bd`) -- **Package `gui_agent/`**: - - `core/` : Moteur d'exécution local CodeAct (`repl.py`), SDK unifié (`mcp_core.py`), gestionnaire PTY interactif avec bornage et nettoyage de processus orphelins (`pty_session.py`). - - `layers/` : Médiation d'accessibilité programmatique (`accessibility.py`), perception visuelle & OCR (`visual_perception.py`), émulation d'entrées bas-niveau (`input_emulation.py`), gestion de fenêtrage et processus (`window_management.py`). - - `utils/` : Géométrie d'écran durcie (`coordinates.py`), cinématique humaine (`human_mimic.py`), gestion vidéo synchronisée (`video.py`). -- **Dépendances & scripts**: - - `pyproject.toml` : Ajout de `rapidocr-onnxruntime>=1.3.0`, `evdev>=1.7.0`, retrait de `playwright` et `pytesseract`. - - `install.sh` : Paquets système Linux (`python3-dbus`, `at-spi2-core`, `libatspi-dev`, `python3-tk`). -- **Tests**: - - `tests/test_package.py` : Tests d'échafaudage modulaire et régressions PTY / affichage. +- **File**: `linux/crates/atspi_mediator/src/main.rs` + - **Scope**: Médiateur Rust AT-SPI et serveur stdio MCP. + - **Exact Technical Change**: Sécurisation de `resolve_mcp_target` via `u32::try_from(i)` rejetant immédiatement les entiers `element_index` hors limites (`> u32::MAX`) évitant tout débordement silencieux vers le nœud 0, tout en préservant la priorité absolue aux identifiants non-numériques explicites (`element_identifier`). Ajout de 4 tests unitaires dédiés dans `src/main.rs`. +- **File**: `.GCC/main.md` + - **Scope**: Registre macro du projet. + - **Exact Technical Change**: Harmonisation des mentions de tests historiques (lignes 138 et 147) vers 112/112 tests CI validés. +- **File**: `.GCC/resume.md` + - **Scope**: Journal de transition technique. + - **Exact Technical Change**: Remplacement des anciennes instructions de re-tag/push par des directives de vérification en lecture seule et d'attente d'approbation. +- **File**: `.GCC/branches/plan_accessibility_phase_1.md` + - **Scope**: Plan tactique Phase 1. + - **Exact Technical Change**: Ajout du Step 9 documentant la validation de `element_index` et l'exécution des 13 tests unitaires Rust. +- **File**: `.GCC/branches/test.md` + - **Scope**: Journal de test persistent. + - **Exact Technical Change**: Enregistrement des exécutions `./ci.sh` (112/112 PASS) et `cargo test` (13/13 PASS). ## 🛠️ Static Codebase Health -- **Verification Command Run**: `./ci.sh` sur `main` +- **Verification Command Run**: `./ci.sh && cargo test --manifest-path linux/crates/atspi_mediator/Cargo.toml` - **Linter/Compiler Status**: ```text +============================= 112 passed in 38.37s ============================= +✔ Validé (40648ms) + ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 📊 RÉSUMÉ D'EXÉCUTION CI (CI Summary) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ | Étape de Validation | Statut | Durée | |--------------------------------------------|------------|------------| -| Compilation Bytecode Python (compileall) | PASS | 162ms | -| Validation Workflows GitHub Actions | PASS | 63ms | -| Linter de Code (Ruff Check) | PASS | 6854ms | -| Formatage de Code (Ruff Format) | PASS | 74ms | -| Typage Statique Strict (Mypy) | PASS | 446ms | -| Suite de Tests Pytest | PASS | 20385ms | +| Compilation Bytecode Python (compileall) | PASS | 452ms | +| Validation Workflows GitHub Actions | PASS | 78ms | +| Linter de Code (Ruff Check) | PASS | 112ms | +| Formatage de Code (Ruff Format) | PASS | 23ms | +| Typage Statique Strict (Mypy) | PASS | 2102ms | +| Suite de Tests Pytest | PASS | 40648ms | ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ -🎉 Toutes les étapes CI sont validées avec succès ! (65 passed in 19.56s) +🎉 Toutes les étapes CI sont validées avec succès ! ``` ## 🚧 Unfinished Work & Technical Failures -- Aucun échec. Arbre de travail sur `main` propre et aligné sur `origin/main`. +- PR #137 entièrement soumise et passée en revue avec score Greptile 5/5 validé. +- Tous les constats de revue résolus et validés localement (112/112 tests CI et 13/13 tests Rust). +- En attente de consultation en lecture seule de l'état de la PR et de l'approbation formelle du mainteneur. ## 👉 Handover Directives for the Next Agent -1. **Target File**: `gui_agent/layers/accessibility.py` -2. **Immediate Action**: Créer la branche dédiée `feat/accessibility-mediation-phase-1` pour traiter l'Issue #130 (`[FEAT] Phase 1 : Médiation d'accessibilité programmatique via AT-SPI / D-Bus`). -3. **Verification Command**: `./ci.sh` +1. **Target File**: `linux/crates/atspi_mediator/src/main.rs` +2. **Immediate Action**: Effectuer une consultation en lecture seule de l'état de la PR #137 et attendre l'approbation du mainteneur avant toute action distante. +3. **Verification Command**: `gh pr view 137 --json state,reviewDecision,statusCheckRollup` diff --git a/.githooks/pre-commit b/.githooks/pre-commit index 127c3e1..b30de04 100755 --- a/.githooks/pre-commit +++ b/.githooks/pre-commit @@ -81,12 +81,12 @@ $UV_CMD run --with ruff ruff format --check . || { # 5. Typage Statique Strict (Mypy) echo "[Pre-Commit] 5/8 - Typage Statique Global Strict (Mypy)..." if [ -f "./venv/bin/mypy" ]; then - ./venv/bin/mypy gui_agent mcp_gui_server.py --ignore-missing-imports || { + ./venv/bin/mypy -p linux --ignore-missing-imports || { echo "[Erreur Pre-Commit] Échec de la vérification de types Mypy." exit 1 } else - $UV_CMD run --with mypy mypy gui_agent mcp_gui_server.py --ignore-missing-imports || { + $UV_CMD run --with mypy mypy -p linux --ignore-missing-imports || { echo "[Erreur Pre-Commit] Échec de la vérification de types Mypy." exit 1 } @@ -101,7 +101,7 @@ $UV_CMD run --with ruff ruff check --select B,SIM,C90,UP,RUF --quiet . || { # 7. Audit Sécurité Code AST (Bandit) echo "[Pre-Commit] 7/8 - Audit Sécurité Code AST (Bandit)..." -$UV_CMD run --with bandit bandit -c pyproject.toml -q -r gui_agent mcp_gui_server.py || { +$UV_CMD run --with bandit bandit -c pyproject.toml -q -r linux || { echo "[Erreur Pre-Commit] Bandit a identifié un risque de sécurité dans le code !" exit 1 } diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 3266ab6..826881e 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -80,9 +80,9 @@ jobs: python -m compileall -q . ruff check . if command -v xvfb-run >/dev/null 2>&1; then - xvfb-run -a pytest -v tests/ + xvfb-run -a pytest -v linux/tests/ else - pytest -v tests/ + pytest -v linux/tests/ fi - name: Set up Java diff --git a/.gitignore b/.gitignore index 44d868a..8e7f8f3 100644 --- a/.gitignore +++ b/.gitignore @@ -1,22 +1,59 @@ -# Environnement virtuel Python +# Environnements virtuels Python venv/ .venv/ +env/ +ENV/ -# Caches Python +# Caches Python, compilateurs et linters __pycache__/ *.pyc *.pyo *.pyd +.mypy_cache/ +.pytest_cache/ +.ruff_cache/ +.coverage +htmlcov/ -# Captures d'écran locales de test +# Artefacts de build et packaging Python +build/ +dist/ +*.egg-info/ +*.egg + +# Caches d'exécution et médias locaux +.cache/ screenshots/ +videos/ raw_screenshot_*.png screenshot_*.png +*.mp4 +*.avi + +# Build Rust et binaires exécutables natifs multi-plateforme +target/ +**/target/ +bin/ +**/bin/ +*.exe +*.dll +*.so +*.dylib -# Fichiers système ou d'IDE +# Fichiers système et IDE .DS_Store +.AppleDouble +.LSOverride +._* +Thumbs.db +ehthumbs.db +Desktop.ini +$RECYCLE.BIN/ .vscode/ .idea/ +*.swp +*.swo +*~ # Identifiants et secrets sensibles git_credential*.json @@ -31,4 +68,3 @@ assets/ *.excalidraw mermaid_diagrams.txt todo.md - diff --git a/Cargo.lock b/Cargo.lock new file mode 100644 index 0000000..4bdfce2 --- /dev/null +++ b/Cargo.lock @@ -0,0 +1,1058 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "async-broadcast" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "435a87a52755b8f27fcf321ac4f04b2802e337c8c4872923137471ec39c37532" +dependencies = [ + "event-listener", + "event-listener-strategy", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-channel" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2" +dependencies = [ + "concurrent-queue", + "event-listener-strategy", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-executor" +version = "1.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c96bf972d85afc50bf5ab8fe2d54d1586b4e0b46c97c50a0c9e71e2f7bcd812a" +dependencies = [ + "async-task", + "concurrent-queue", + "fastrand", + "futures-lite", + "pin-project-lite", + "slab", +] + +[[package]] +name = "async-io" +version = "2.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "456b8a8feb6f42d237746d4b3e9a178494627745c3c56c6ea55d92ba50d026fc" +dependencies = [ + "autocfg", + "cfg-if", + "concurrent-queue", + "futures-io", + "futures-lite", + "parking", + "polling", + "rustix", + "slab", + "windows-sys", +] + +[[package]] +name = "async-lock" +version = "3.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "290f7f2596bd5b78a9fec8088ccd89180d7f9f55b94b0576823bbbdc72ee8311" +dependencies = [ + "event-listener", + "event-listener-strategy", + "pin-project-lite", +] + +[[package]] +name = "async-process" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc50921ec0055cdd8a16de48773bfeec5c972598674347252c0399676be7da75" +dependencies = [ + "async-channel", + "async-io", + "async-lock", + "async-signal", + "async-task", + "blocking", + "cfg-if", + "event-listener", + "futures-lite", + "rustix", +] + +[[package]] +name = "async-recursion" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b43422f69d8ff38f95f1b2bb76517c91589a924d1559a0e935d7c8ce0274c11" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "async-signal" +version = "0.2.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52b5aaafa020cf5053a01f2a60e8ff5dccf550f0f77ec54a4e47285ac2bab485" +dependencies = [ + "async-io", + "async-lock", + "atomic-waker", + "cfg-if", + "futures-core", + "futures-io", + "rustix", + "signal-hook-registry", + "slab", + "windows-sys", +] + +[[package]] +name = "async-task" +version = "4.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de" + +[[package]] +name = "async-trait" +version = "0.1.92" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "atomic-waker" +version = "1.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" + +[[package]] +name = "atspi" +version = "0.29.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c77886257be21c9cd89a4ae7e64860c6f0eefca799bb79127913052bd0eefb3d" +dependencies = [ + "atspi-common", + "atspi-proxies", + "zbus", +] + +[[package]] +name = "atspi-common" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "20c5617155740c98003016429ad13fe43ce7a77b007479350a9f8bf95a29f63d" +dependencies = [ + "enumflags2", + "serde", + "static_assertions", + "zbus", + "zbus-lockstep", + "zbus-lockstep-macros", + "zbus_names", + "zvariant", +] + +[[package]] +name = "atspi-connection" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43b28a59b7c709ce0f1196405e6eacee35424dad07b4b005c0f9e417d8c2116f" +dependencies = [ + "atspi-common", + "atspi-proxies", + "futures-lite", + "zbus", +] + +[[package]] +name = "atspi-mediator" +version = "0.1.0" +dependencies = [ + "anyhow", + "atspi", + "atspi-connection", + "futures-util", + "serde", + "serde_json", + "tokio", + "zbus", +] + +[[package]] +name = "atspi-proxies" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2230e48787ed3eb4088996eab66a32ca20c0b67bbd4fd6cdfe79f04f1f04c9fc" +dependencies = [ + "atspi-common", + "serde", + "zbus", +] + +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + +[[package]] +name = "bitflags" +version = "2.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06" + +[[package]] +name = "blocking" +version = "1.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a70e4329df6cb94385eed412ec92375c3cdd8a6e502493d1229b6414e4036dfa" +dependencies = [ + "async-channel", + "async-task", + "futures-io", + "futures-lite", + "piper", +] + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "bytes" +version = "1.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "concurrent-queue" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ca0197aee26d1ae37445ee532fefce43251d24cc7c166799f4d46817f1d3973" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6" + +[[package]] +name = "endi" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "66b7e2430c6dff6a955451e2cfc438f09cea1965a9d6f87f7e3b90decc014099" + +[[package]] +name = "enumflags2" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1027f7680c853e056ebcec683615fb6fbbc07dbaa13b4d5d9442b146ded4ecef" +dependencies = [ + "enumflags2_derive", + "serde", +] + +[[package]] +name = "enumflags2_derive" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67c78a4d8fdf9953a5c9d458f9efe940fd97a0cab0941c075a813ac594733827" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "event-listener" +version = "5.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a23add41df1562121a9393cb065eab5146a1242410f23a644851e90cfd669d2" +dependencies = [ + "parking", + "pin-project-lite", +] + +[[package]] +name = "event-listener-strategy" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93" +dependencies = [ + "event-listener", + "pin-project-lite", +] + +[[package]] +name = "fastrand" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223" + +[[package]] +name = "futures-core" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e" + +[[package]] +name = "futures-io" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed" + +[[package]] +name = "futures-lite" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" +dependencies = [ + "fastrand", + "futures-core", + "futures-io", + "parking", + "pin-project-lite", +] + +[[package]] +name = "futures-macro" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "futures-task" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd" + +[[package]] +name = "futures-util" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc" +dependencies = [ + "futures-core", + "futures-macro", + "futures-task", + "pin-project-lite", + "slab", +] + +[[package]] +name = "getrandom" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" +dependencies = [ + "cfg-if", + "libc", + "r-efi", +] + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" + +[[package]] +name = "hermit-abi" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e17592d60ebacc7d5e169f4663c5f84f9161cc90328abcfe8456f41e4dfcb284" + +[[package]] +name = "hex" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" + +[[package]] +name = "indexmap" +version = "2.14.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855" +dependencies = [ + "equivalent", + "hashbrown", +] + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "js-sys" +version = "0.3.105" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce57d20d1ea864ce2ac172ab472d409214f4fd359f0b2a2775abdf522e2af99e" +dependencies = [ + "cfg-if", + "futures-util", + "wasm-bindgen", +] + +[[package]] +name = "libc" +version = "0.2.189" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "memoffset" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a" +dependencies = [ + "autocfg", +] + +[[package]] +name = "mio" +version = "1.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b18443e9c262bfe8fa82f51666e2642c53393f7e5c27b3e1aeab922cff5b9d8" +dependencies = [ + "libc", + "wasi", + "windows-sys", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "ordered-stream" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9aa2b01e1d916879f73a53d01d1d6cee68adbb31d6d9177a8cfce093cced1d50" +dependencies = [ + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "parking" +version = "2.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" + +[[package]] +name = "pin-project-lite" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" + +[[package]] +name = "piper" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1" +dependencies = [ + "atomic-waker", + "fastrand", + "futures-io", +] + +[[package]] +name = "polling" +version = "3.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d0e4f59085d47d8241c88ead0f274e8a0cb551f3625263c05eb8dd897c34218" +dependencies = [ + "cfg-if", + "concurrent-queue", + "hermit-abi", + "pin-project-lite", + "rustix", + "windows-sys", +] + +[[package]] +name = "proc-macro-crate" +version = "3.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e67ba7e9b2b56446f1d419b1d807906278ffa1a658a8a5d8a39dcb1f5a78614f" +dependencies = [ + "toml_edit", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "rustversion" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" + +[[package]] +name = "serde" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "serde_json" +version = "1.0.151" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "serde_repr" +version = "0.1.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8d3b1629de253c70a0508c3899572da79ca359fdab27c7920ff00406df418906" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "signal-hook-registry" +version = "1.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b" +dependencies = [ + "errno", + "libc", +] + +[[package]] +name = "slab" +version = "0.4.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" + +[[package]] +name = "socket2" +version = "0.6.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "static_assertions" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f" + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "3.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12df2e0110f65b775f769bb17ef989067a1d931b2eb822bd4346631eeada89f9" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "tempfile" +version = "3.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" +dependencies = [ + "fastrand", + "getrandom", + "once_cell", + "rustix", + "windows-sys", +] + +[[package]] +name = "tokio" +version = "1.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "202caea871b69668250d242070849eb495be178ed697a3e98aebce5bc81a0bed" +dependencies = [ + "bytes", + "libc", + "mio", + "pin-project-lite", + "signal-hook-registry", + "socket2", + "tokio-macros", + "tracing", + "windows-sys", +] + +[[package]] +name = "tokio-macros" +version = "2.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "toml_datetime" +version = "1.1.1+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7" +dependencies = [ + "serde_core", +] + +[[package]] +name = "toml_edit" +version = "0.25.15+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1340ea94a5856333492c9064b02c778b191dd2c853778d9609debdcdfea3a614" +dependencies = [ + "indexmap", + "toml_datetime", + "toml_parser", + "winnow", +] + +[[package]] +name = "toml_parser" +version = "1.1.3+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d38ac1cf9b95face32296c0a3ede1fdc270627c9d9c02a7274dd6d960dc4d56" +dependencies = [ + "winnow", +] + +[[package]] +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "pin-project-lite", + "tracing-attributes", + "tracing-core", +] + +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "uds_windows" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2f6fb2847f6742cd76af783a2a2c49e9375d0a111c7bef6f71cd9e738c72d6e" +dependencies = [ + "memoffset", + "tempfile", + "windows-sys", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "uuid" +version = "1.26.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ef6dac1e96601b4fb3acccccff2139741fcb757cb9a36089bf5be91cfb285ce" +dependencies = [ + "js-sys", + "serde_core", + "wasm-bindgen", +] + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "wasm-bindgen" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aecb87a33d3b0c5e3b7aa46336eaf486cffafbd281b195e4c8b80d50df2351bf" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a690d511e3c1a8b3a55e33511e3c2c00c78415cd23650f32b808627f5696b9ed" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "411e4887f0071ef2d2164a9d5fdf2d20efbef78fccd3a78b0c10a1dc5295e48a" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 3.0.5", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "81941cd78d0c92026c33e5e01312845a4cb1e9af3407f9134b100dd03144103e" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "winnow" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "23b97319f7b8343df12cc98938e5c3eb436064524c8d2b4e30a1d3a36eecdf81" +dependencies = [ + "memchr", +] + +[[package]] +name = "zbus" +version = "5.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5db4be7c075cb421e4b7ee645541604239bd243ba7c357511f4ff3a74b555907" +dependencies = [ + "async-broadcast", + "async-executor", + "async-io", + "async-lock", + "async-process", + "async-recursion", + "async-task", + "async-trait", + "blocking", + "enumflags2", + "event-listener", + "futures-core", + "futures-lite", + "hex", + "libc", + "ordered-stream", + "rustix", + "serde", + "serde_repr", + "tokio", + "tracing", + "uds_windows", + "uuid", + "windows-sys", + "winnow", + "zbus_macros", + "zbus_names", + "zvariant", +] + +[[package]] +name = "zbus-lockstep" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6998de05217a084b7578728a9443d04ea4cd80f2a0839b8d78770b76ccd45863" +dependencies = [ + "zbus_xml", + "zvariant", +] + +[[package]] +name = "zbus-lockstep-macros" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "10da05367f3a7b7553c8cdf8fa91aee6b64afebe32b51c95177957efc47ca3a0" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "zbus-lockstep", + "zbus_xml", + "zvariant", +] + +[[package]] +name = "zbus_macros" +version = "5.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2990635d09ade6df1868f72f8cac69a876a90981e8bd3c40b1be413f8dc88f40" +dependencies = [ + "proc-macro-crate", + "proc-macro2", + "quote", + "syn 3.0.5", + "zbus_names", + "zvariant", + "zvariant_utils", +] + +[[package]] +name = "zbus_names" +version = "4.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d8bf88b4a3ff53e883001e0e0115b297a9d53c31b9c1edd2bfdd853e3428624e" +dependencies = [ + "serde", + "winnow", + "zvariant", +] + +[[package]] +name = "zbus_xml" +version = "5.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1586c021a01ca0a9216dcd874e546382e156a5cbab5fab6cb5f10087e22682a" +dependencies = [ + "serde", + "winnow", + "zbus_names", + "zvariant", +] + +[[package]] +name = "zcheapstr" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1afec51604565183aeb5c54c20aeab286120d4e4460f7f76e3e8bb8c0d99473" +dependencies = [ + "serde", +] + +[[package]] +name = "zmij" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" + +[[package]] +name = "zvariant" +version = "5.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c1d34c27cc6cdd1f458427519dd6b8612f7b7e3f7b9a0b2355d041dda9869147" +dependencies = [ + "endi", + "enumflags2", + "serde", + "winnow", + "zcheapstr", + "zvariant_derive", + "zvariant_utils", +] + +[[package]] +name = "zvariant_derive" +version = "5.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "864155e69b4352db0c7f374917bf45d1e0c8d17659c8b3dbf9795f3673f8c497" +dependencies = [ + "proc-macro-crate", + "proc-macro2", + "quote", + "syn 3.0.5", + "zvariant_utils", +] + +[[package]] +name = "zvariant_utils" +version = "4.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bad0294361a320b694a328460dc73add56c306150f5cb6bfafc44446120008a3" +dependencies = [ + "proc-macro2", + "quote", + "serde", + "syn 3.0.5", + "winnow", +] diff --git a/Cargo.toml b/Cargo.toml new file mode 100644 index 0000000..42e8b46 --- /dev/null +++ b/Cargo.toml @@ -0,0 +1,12 @@ +[workspace] +members = [ + "linux/crates/atspi_mediator", +] +resolver = "2" + +[profile.release] +codegen-units = 1 +opt-level = 3 +lto = "fat" +strip = "symbols" + diff --git a/INSTALL.md b/INSTALL.md index 5ec31ad..88eafe5 100644 --- a/INSTALL.md +++ b/INSTALL.md @@ -11,13 +11,15 @@ Ce guide détaille l'installation, la configuration et le dépannage du serveur Ouvrez une invite de commande **PowerShell** (en utilisateur standard ou administrateur) et lancez : ```powershell -powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/leandre755/gui_agent/main/install.ps1 | iex" +# Téléchargement et exécution vérifiée du script d'installation : +Invoke-WebRequest -Uri "https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/windows/install.ps1" -OutFile "install.ps1" +powershell -ExecutionPolicy Bypass -File .\install.ps1 ``` Ou en local depuis le dépôt cloné : ```powershell -.\install.ps1 -Local +.\windows\install.ps1 -Local ``` #### Ce que prend en charge automatiquement le script `install.ps1` : @@ -113,8 +115,9 @@ Ajoutez ou fusionnez dans le fichier `$env:USERPROFILE\.gemini\config\mcp_config Pour désinstaller complètement le serveur et nettoyer les configurations MCP : ```powershell -# Désinstallation automatique -powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/leandre755/gui_agent/main/uninstall.ps1 | iex" +# Téléchargement et exécution vérifiée du désinstallateur : +Invoke-WebRequest -Uri "https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/windows/uninstall.ps1" -OutFile "uninstall.ps1" +powershell -ExecutionPolicy Bypass -File .\uninstall.ps1 # Ou avec purge complète des captures d'écran : .\uninstall.ps1 -PurgeData -Yes @@ -122,14 +125,24 @@ powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.c --- -## 🐧 Installation sous Linux & macOS +## 🐧 Installation sous Linux -### Installation en une seule ligne : +*(Note : le support dédié macOS fait l'objet de la Phase 5 ultérieure).* + +### Installation d'une release versionnée sous Linux : ```bash -curl -fsSL https://raw.githubusercontent.com/leandre755/gui_agent/main/install.sh | bash +curl -fsSLO https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/linux/install.sh +chmod +x install.sh && ./install.sh ``` +Le script d'installation configure automatiquement : +1. **Les paquets système Linux** : X11, capture d'écran, OCR, AT-SPI2 Core (`at-spi2-core`, `python3-dbus`, `cargo`). +2. **Le gestionnaire Astral `uv`** : Installation autonome dans `~/.local/bin/uv`. +3. **Le serveur FastMCP `gui-agent`** : Déploiement isolé via `uv tool install`. +4. **Le médiateur natif AT-SPI `gui-agent-atspi`** : Compilation Rust ultra-rapide et installation dans `~/.local/bin/gui-agent-atspi`. +5. **L'intégration MCP** : Enregistrement direct pour Claude Code CLI et Antigravity CLI (`~/.gemini/config/mcp_config.json`). + ### Désinstallation sous Linux : ```bash -./uninstall.sh --purge-data --yes +./linux/uninstall.sh --purge-data --yes ``` diff --git a/README.fr.md b/README.fr.md index 1ff755a..2870ab7 100644 --- a/README.fr.md +++ b/README.fr.md @@ -82,6 +82,15 @@ Le serveur expose 21 outils FastMCP monolithiques couvrant l'intégralité du cy 4. **Répartiteur d'Entrées et de Fenêtres OS Natif** : Les frappes, raccourcis, clics de souris et opérations de glisser sont acheminés via des pilotes natifs à faible latence (`xdotool` et `python-xlib` sous Linux, API Win32 sous Windows). Des micro-délais humanisés émulent une interaction utilisateur naturelle. Les commandes de gestion de fenêtres (`wmctrl` / `xprop`) inspectent et manipulent l'état des fenêtres sans verrouiller le gestionnaire de fenêtres. 5. **Vision Locale, OCR & Automatisation Playwright** : La correspondance de motifs (`cv2.matchTemplate`) permet une détection robuste des icônes malgré les variations de thèmes. La détection de texte combine Tesseract OCR avec le repli ONNX RapidOCR. L'automatisation web s'appuie sur Playwright pour inspecter les arbres ARIA et manipuler directement les nœuds DOM sans ambiguïté visuelle. +### Architecture Multi-Plateforme à la Racine + +Le projet structure les implémentations par système d'exploitation dans des répertoires dédiés à la racine : +- **`linux/`** : Implémentation principale sous Linux regroupant le moteur `gui_agent`, le médiateur natif d'accessibilité AT-SPI2 / D-Bus (`linux/crates/atspi_mediator`), les scripts d'installation dédiés (`linux/install.sh`, `linux/uninstall.sh`) et la résolution dynamique des chemins conforme au standard XDG Base Directory (`XDG_CACHE_HOME`, `XDG_DATA_HOME`, `XDG_CONFIG_HOME`). +- **`windows/`** : Répertoire réservé pour l'implémentation Windows via UI Automation et l'API Win32 (Phase 5 #134). +- **`macos/`** : Répertoire réservé pour l'implémentation macOS via NSAccessibility et Quartz Event Taps (Phase 5 #135). + +Tous les répertoires de cache (captures d'écran avec grille cartésienne et enregistrements vidéo MP4 gérés par `gui_start_video_recording` et `gui_stop_video_recording`) sont résolus dynamiquement sans aucun chemin utilisateur en dur. + --- ## Package Installation @@ -95,21 +104,22 @@ Exécutez le script d'installation automatisé pour vérifier les dépendances, ```bash # Installateur curl en une ligne -curl -fsSL https://raw.githubusercontent.com/leandre755/gui_agent/main/install.sh | bash +curl -fsSL https://raw.githubusercontent.com/leandre755/gui_agent/main/linux/install.sh | bash # Ou localement depuis un dépôt cloné -./install.sh +./linux/install.sh ``` #### Microsoft Windows (PowerShell) Lancez PowerShell (utilisateur standard ou administrateur) et exécutez : ```powershell -# Installateur PowerShell en une ligne -powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/leandre755/gui_agent/main/install.ps1 | iex" +# Téléchargement et exécution vérifiée d'une release versionnée +Invoke-WebRequest -Uri "https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/windows/install.ps1" -OutFile "install.ps1" +powershell -ExecutionPolicy Bypass -File .\install.ps1 # Ou localement depuis un dépôt cloné -.\install.ps1 -Local +.\windows\install.ps1 -Local ``` ### 2. Déploiement Isolé via uv tool @@ -392,24 +402,26 @@ Arrête proprement l'enregistrement FFmpeg en cours et valide le conteneur du fi Pour purger proprement `gui-agent`, supprimer les environnements isolés et retirer les configurations MCP enregistrées : -### 1. Linux & macOS (Bash) +### 1. Linux (Bash) ```bash -# Désinstallateur distant automatisé -curl -fsSL https://raw.githubusercontent.com/leandre755/gui_agent/main/uninstall.sh | bash +# Téléchargement et exécution vérifiée d'une release versionnée +curl -fsSLO https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/linux/uninstall.sh +chmod +x uninstall.sh && ./uninstall.sh --purge-data --yes -# Désinstallation locale avec purge complète des données et captures -./uninstall.sh --purge-data --yes +# Ou désinstallation locale avec purge complète des données et captures +./linux/uninstall.sh --purge-data --yes ``` ### 2. Microsoft Windows (PowerShell) ```powershell -# Désinstallateur distant automatisé -powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/leandre755/gui_agent/main/uninstall.ps1 | iex" +# Téléchargement et exécution vérifiée d'une release versionnée +Invoke-WebRequest -Uri "https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/windows/uninstall.ps1" -OutFile "uninstall.ps1" +powershell -ExecutionPolicy Bypass -File .\uninstall.ps1 -PurgeData -Yes -# Désinstallation locale avec purge complète des données et captures -.\uninstall.ps1 -PurgeData -Yes +# Ou désinstallation locale avec purge complète des données et captures +.\windows\uninstall.ps1 -PurgeData -Yes ``` #### Éléments nettoyés par le désinstallateur : diff --git a/README.md b/README.md index 3a9e18b..494e55c 100644 --- a/README.md +++ b/README.md @@ -83,6 +83,15 @@ The server exposes 21 monolithic FastMCP tools covering the complete lifecycle o 4. **Native OS Input & Window Dispatcher**: Keystrokes, hotkeys, mouse clicks, and drag operations are routed through low-latency native drivers (`xdotool` and `python-xlib` under Linux, Win32 API under Windows). Humanized delays and micro-jitter emulate natural user interaction. Window management commands (`wmctrl` / `xprop`) inspect and manipulate window states without window manager locks. 5. **Local Vision, OCR & Playwright Automation**: Template matching (`cv2.matchTemplate`) enables robust icon detection even under theme variations. Text discovery combines Tesseract OCR with RapidOCR ONNX fallback. Web automation leverages Playwright to inspect ARIA trees and manipulate DOM nodes directly without visual ambiguity. +### Multi-Platform Root Architecture + +The codebase organizes platform implementations into dedicated root directories: +- **`linux/`** : Primary Linux implementation containing the `gui_agent` core engine, native AT-SPI2 / D-Bus accessibility mediator (`linux/crates/atspi_mediator`), dedicated installation scripts (`linux/install.sh`, `linux/uninstall.sh`), and dynamic XDG Base Directory path resolution (`XDG_CACHE_HOME`, `XDG_DATA_HOME`, `XDG_CONFIG_HOME`). +- **`windows/`** : Reserved directory for Windows UI Automation & Win32 API implementation (Phase 5 #134). +- **`macos/`** : Reserved directory for macOS NSAccessibility & Quartz Event Taps implementation (Phase 5 #135). + +All cache paths (including dynamic Cartesian screenshot buffers and low-overhead MP4 video recordings via `gui_start_video_recording` and `gui_stop_video_recording`) are resolved dynamically without hardcoded user directories. + --- ## Package Installation @@ -95,22 +104,24 @@ The server exposes 21 monolithic FastMCP tools covering the complete lifecycle o Run the automated installer to check dependencies, install Astral `uv`, configure the isolated environment, and register the MCP server: ```bash -# Single-line curl installer -curl -fsSL https://raw.githubusercontent.com/leandre755/gui_agent/main/install.sh | bash +# Download and verify a versioned release installer +curl -fsSLO https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/linux/install.sh +chmod +x install.sh && ./install.sh # Or locally from a cloned repository -./install.sh +./linux/install.sh ``` #### Microsoft Windows (PowerShell) Launch PowerShell (standard user or administrator) and execute: ```powershell -# Single-line PowerShell installer -powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/leandre755/gui_agent/main/install.ps1 | iex" +# Download and execute a verified release installer +Invoke-WebRequest -Uri "https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/windows/install.ps1" -OutFile "install.ps1" +powershell -ExecutionPolicy Bypass -File .\install.ps1 # Or locally from a cloned repository -.\install.ps1 -Local +.\windows\install.ps1 -Local ``` ### 2. Isolated Deployment via uv tool @@ -393,24 +404,26 @@ Cleanly terminates the ongoing FFmpeg recording and validates the generated MP4 To cleanly purge `gui-agent`, delete isolated environments, and remove registered MCP configurations: -### 1. Linux & macOS (Bash) +### 1. Linux (Bash) ```bash -# Automated remote uninstaller -curl -fsSL https://raw.githubusercontent.com/leandre755/gui_agent/main/uninstall.sh | bash +# Download and verify a versioned release uninstaller +curl -fsSLO https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/linux/uninstall.sh +chmod +x uninstall.sh && ./uninstall.sh --purge-data --yes -# Local uninstall with full data and screenshot purge -./uninstall.sh --purge-data --yes +# Or local uninstall with full data and screenshot purge +./linux/uninstall.sh --purge-data --yes ``` ### 2. Microsoft Windows (PowerShell) ```powershell -# Automated remote uninstaller -powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/leandre755/gui_agent/main/uninstall.ps1 | iex" +# Download and execute a verified release uninstaller +Invoke-WebRequest -Uri "https://raw.githubusercontent.com/leandre755/gui_agent/v0.1.0/windows/uninstall.ps1" -OutFile "uninstall.ps1" +powershell -ExecutionPolicy Bypass -File .\uninstall.ps1 -PurgeData -Yes -# Local uninstall with full data and screenshot purge -.\uninstall.ps1 -PurgeData -Yes +# Or local uninstall with full data and screenshot purge +.\windows\uninstall.ps1 -PurgeData -Yes ``` #### What the uninstaller cleans: diff --git a/ci.sh b/ci.sh index d2d54e6..3c51511 100755 --- a/ci.sh +++ b/ci.sh @@ -178,11 +178,11 @@ if [ "$QUICK_MODE" -eq 0 ]; then run_step "Formatage de Code (Ruff Format)" "$RUFF_CMD format --check ." # 5. Typage Statique - run_step "Typage Statique Strict (Mypy)" "$MYPY_CMD gui_agent mcp_gui_server.py" + run_step "Typage Statique Strict (Mypy)" "$MYPY_CMD -p linux" fi # 6. Tests Pytest -run_step "Suite de Tests Pytest" "$TEST_RUNNER -v tests/" +run_step "Suite de Tests Pytest" "$TEST_RUNNER -v linux/tests/" # Affichage du tableau récapitulatif echo -e "${CYAN}━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━${NC}" diff --git a/examples/capture_to_artifacts.py b/examples/capture_to_artifacts.py deleted file mode 100644 index aa0571d..0000000 --- a/examples/capture_to_artifacts.py +++ /dev/null @@ -1,28 +0,0 @@ -#!/usr/bin/env python3 -import os -import sys -import shutil - -sys.path.append("/home/omni/Code/gui_agent") -import mcp_gui_server - - -def main(): - print("Prise de la capture d'écran...") - res = mcp_gui_server.gui_take_screenshot(apply_grid=True, grid_interval=100) - if res.get("status") != "success": - print(f"Erreur : {res.get('message')}") - sys.exit(1) - - scr_path = res.get("screenshot_path") - dest_dir = "/home/omni/.gemini/antigravity/brain/e48ec26a-978a-4f3e-ad66-71e604f5934e" - dest_path = os.path.join(dest_dir, "screenshot_xboard.png") - - os.makedirs(dest_dir, exist_ok=True) - shutil.copy(scr_path, dest_path) - print(f"Capture enregistrée et copiée vers : {dest_path}") - print(f"Dimensions : {res.get('width')}x{res.get('height')}") - - -if __name__ == "__main__": - main() diff --git a/gui_agent/core/__init__.py b/gui_agent/core/__init__.py deleted file mode 100644 index 494af17..0000000 --- a/gui_agent/core/__init__.py +++ /dev/null @@ -1,7 +0,0 @@ -"""Moteur d'exécution local CodeAct et SDK unifié mcp_core.""" - -from gui_agent.core.mcp_core import mcp_core -from gui_agent.core.pty_session import PTYSession -from gui_agent.core.repl import execute_script - -__all__ = ["PTYSession", "execute_script", "mcp_core"] diff --git a/gui_agent/layers/__init__.py b/gui_agent/layers/__init__.py deleted file mode 100644 index d708c88..0000000 --- a/gui_agent/layers/__init__.py +++ /dev/null @@ -1,21 +0,0 @@ -"""Couches d'interaction et de médiation système pour gui_agent.""" - -from gui_agent.layers.accessibility import get_app_state, perform_action, set_value -from gui_agent.layers.input_emulation import key_tap, mouse_click_at, mouse_drag_smooth, mouse_scroll, screen_capture -from gui_agent.layers.visual_perception import find_text -from gui_agent.layers.window_management import activate_window, process_list, process_run - -__all__ = [ - "activate_window", - "find_text", - "get_app_state", - "key_tap", - "mouse_click_at", - "mouse_drag_smooth", - "mouse_scroll", - "perform_action", - "process_list", - "process_run", - "screen_capture", - "set_value", -] diff --git a/gui_agent/layers/accessibility.py b/gui_agent/layers/accessibility.py deleted file mode 100644 index 8e5d7b5..0000000 --- a/gui_agent/layers/accessibility.py +++ /dev/null @@ -1,23 +0,0 @@ -"""Couche d'Accessibilité Programmatique (AT-SPI / D-Bus).""" - -from __future__ import annotations - -import logging -from typing import Any - -logger = logging.getLogger("gui_agent.layers.accessibility") - - -def get_app_state(include_screenshot: bool = False) -> dict[str, Any]: - """Extrait l'arbre d'accessibilité applicatif en JSON texte pur via AT-SPI.""" - return {"status": "not_implemented", "layer": "accessibility", "include_screenshot": include_screenshot} - - -def perform_action(element_id: str, action: str) -> bool: - """Déclenche l'action du composant accessible par le bus.""" - return False - - -def set_value(element_id: str, text: str) -> bool: - """Écrit directement la valeur textuelle dans la mémoire du composant.""" - return False diff --git a/gui_agent/utils/__init__.py b/gui_agent/utils/__init__.py deleted file mode 100644 index 79a928f..0000000 --- a/gui_agent/utils/__init__.py +++ /dev/null @@ -1,14 +0,0 @@ -"""Utilitaires et drivers partagés pour gui_agent.""" - -from gui_agent.utils.coordinates import get_monitor_geometry, normalize_coordinates -from gui_agent.utils.human_mimic import generate_smooth_path, sleep_human, type_char_human -from gui_agent.utils.video import validate_video_recording_params - -__all__ = [ - "generate_smooth_path", - "get_monitor_geometry", - "normalize_coordinates", - "sleep_human", - "type_char_human", - "validate_video_recording_params", -] diff --git a/hatch_build.py b/hatch_build.py new file mode 100644 index 0000000..63e5f79 --- /dev/null +++ b/hatch_build.py @@ -0,0 +1,146 @@ +"""Hatchling custom build hook pour gui-agent. + +Compile automatiquement le médiateur Rust natif gui-agent-atspi lors de la +construction de la wheel (pip install ., uv tool install git+..., python -m build) +afin d'embarquer le binaire natif dans le paquet distribuable. +""" + +from __future__ import annotations + +import contextlib +import logging +import os +import platform +import shutil +import stat +import subprocess +import sys +import tempfile +from typing import Any + +from hatchling.builders.hooks.plugin.interface import BuildHookInterface + +logger = logging.getLogger(__name__) + + +def _detect_linux_platform_tag() -> str: + """Détermine le tag de compatibilité manylinux ou musllinux audité de la plateforme hôte.""" + try: + import packaging.tags as p_tags + + for t in p_tags.sys_tags(): + if t.platform.startswith(("manylinux", "musllinux")): + return t.platform + except Exception as exc: + logger.debug("Échec de détection packaging.tags: %s", exc) + + libc_name, libc_version = platform.libc_ver() + arch = platform.machine().lower() + if libc_name == "glibc" and libc_version: + parts = libc_version.split(".") + if len(parts) >= 2: + return f"manylinux_{parts[0]}_{parts[1]}_{arch}" + elif "musl" in sys.version.lower() or os.path.exists("/lib/ld-musl-x86_64.so.1"): + return f"musllinux_1_2_{arch}" + + raise RuntimeError( + f"Impossible de déterminer un tag de compatibilité manylinux ou musllinux audité pour l'hôte " + f"({libc_name} {libc_version} {arch})." + ) + + +class CustomBuildHook(BuildHookInterface): + """Hook de construction personnalisée pour compiler le médiateur AT-SPI Rust.""" + + def initialize(self, version: str, build_data: dict[str, Any]) -> None: + """Initialise la compilation Rust et copie le binaire dans linux/bin/.""" + project_root = self.root + + if version == "editable": + self.build_config.target_config["dev-mode-dirs"] = ["linux"] + shim_content = f'''"""Editable installation shim for gui_agent pointing to linux/.""" +import os + +_linux_dir = os.path.abspath({os.path.join(project_root, "linux")!r}) +__path__ = [_linux_dir] +__file__ = os.path.join(_linux_dir, "__init__.py") + +if os.path.isfile(__file__): + with open(__file__, encoding="utf-8") as _f: + exec(compile(_f.read(), __file__, "exec")) +''' + with tempfile.NamedTemporaryFile("w", encoding="utf-8", suffix=".py", delete=False) as shim: + shim.write(shim_content) + shim.flush() + self._shim_file = shim.name + + if "force_include_editable" not in build_data: + build_data["force_include_editable"] = {} + build_data["force_include_editable"][self._shim_file] = "gui_agent/__init__.py" + cargo_bin = shutil.which("cargo") + cargo_manifest = os.path.join(project_root, "Cargo.toml") + + dest_dir = os.path.join(project_root, "linux", "bin") + os.makedirs(dest_dir, exist_ok=True) + dest_bin = os.path.join(dest_dir, "gui-agent-atspi") + + if sys.platform.startswith("linux") and cargo_bin and os.path.isfile(cargo_manifest): + # Nettoyer l'artefact de destination préalable uniquement avant recompilation + if os.path.exists(dest_bin): + with contextlib.suppress(OSError): + os.remove(dest_bin) + + target_dir = os.environ.get("CARGO_TARGET_DIR") or os.path.join(project_root, "target") + try: + subprocess.run( + [ + cargo_bin, + "build", + "--release", + "--manifest-path", + cargo_manifest, + "--bin", + "gui-agent-atspi", + "--target-dir", + target_dir, + ], + cwd=project_root, + capture_output=True, + check=True, + ) + except Exception as exc: + raise RuntimeError(f"Échec de la compilation préalable du médiateur Rust via Cargo: {exc}") from exc + + built_bin = os.path.join(target_dir, "release", "gui-agent-atspi") + if not os.path.isfile(built_bin): + raise RuntimeError(f"L'artefact compilé attendu est introuvable après cargo build: {built_bin}") + + shutil.copy2(built_bin, dest_bin) + mode = os.stat(dest_bin).st_mode + os.chmod(dest_bin, mode | stat.S_IXUSR) + + # Le binaire natif ELF est embarqué : marquer la wheel avec un tag de compatibilité audité (manylinux / musllinux) + build_data["pure_python"] = False + plat = _detect_linux_platform_tag() + build_data["tag"] = f"py3-none-{plat}" + else: + # Aucun binaire natif embarqué : garantir une wheel pure Python cohérente + if os.path.exists(dest_bin): + with contextlib.suppress(OSError): + os.remove(dest_bin) + + def finalize(self, version: str, build_data: dict[str, Any], artifact_path: str) -> None: + """Nettoie les artefacts temporaires après la construction.""" + shim = getattr(self, "_shim_file", None) + if shim and os.path.exists(shim): + with contextlib.suppress(OSError): + os.remove(shim) + self._shim_file = None + + def clean(self, versions: list[str]) -> None: + """Nettoie les fichiers temporaires du hook.""" + shim = getattr(self, "_shim_file", None) + if shim and os.path.exists(shim): + with contextlib.suppress(OSError): + os.remove(shim) + self._shim_file = None diff --git a/linux/README.md b/linux/README.md new file mode 100644 index 0000000..62a62fb --- /dev/null +++ b/linux/README.md @@ -0,0 +1,25 @@ +# Plateforme Linux — FastMCP GUI Agent + +Ce répertoire regroupe les composants natifs et l'outillage d'intégration spécifiques à **Linux** (X11 / XWayland / AT-SPI2). + +--- + +## 1. Architecture des Modules Linux + +- **`paths.py`** : Résolution dynamique des chemins système conforme aux spécifications **XDG Base Directory** : + - Données : `$XDG_DATA_HOME/gui-agent` (défaut : `~/.local/share/gui-agent`) + - Configuration : `$XDG_CONFIG_HOME/gui-agent` (défaut : `~/.config/gui-agent`) + - Cache : `$XDG_CACHE_HOME/gui-agent` (défaut : `~/.cache/gui-agent`) + - Captures d'écran : `$XDG_CACHE_HOME/gui-agent/screenshots` (ou `GUI_AGENT_SCREENSHOTS_DIR`) + - Captures vidéo : `$XDG_CACHE_HOME/gui-agent/videos` (ou `GUI_AGENT_VIDEOS_DIR`) + - Découverte dynamique de binaires natifs (`gui-agent-atspi`, `computer-use-linux`). +- **`accessibility.py`** : Médiation sémantique d'accessibilité programmatique via **AT-SPI2 / D-Bus** (`get_app_state`, `perform_action`, `set_value`). +- **`input.py`** : Émulation d'entrées clavier et souris (X11 / `uinput` / `evdev`). +- **`window.py`** : Ciblage et manipulation des fenêtres (EWMH, `wmctrl`, `xdotool`). + +--- + +## 2. Scripts d'Installation et d'Automatisation + +- **`install.sh`** : Détection des prérequis système (`xdotool`, `wmctrl`, `ffmpeg`, `libatspi`), configuration de l'environnement virtuel Python et compilation/installation du binaire natif Rust `gui-agent-atspi`. +- **`uninstall.sh`** : Nettoyage propre et déterministe des binaires autonomes et caches spécifiques. diff --git a/gui_agent/__init__.py b/linux/__init__.py similarity index 98% rename from gui_agent/__init__.py rename to linux/__init__.py index 71dfcc2..4d0989e 100644 --- a/gui_agent/__init__.py +++ b/linux/__init__.py @@ -1,6 +1,6 @@ """GUI Agent - FastMCP Computer Use Monolithic Server for Linux Desktops.""" -from gui_agent.server import ( +from .server import ( FastMCP, SCREENSHOTS_DIR, capture_screen_pil, diff --git a/gui_agent/__main__.py b/linux/__main__.py similarity index 75% rename from gui_agent/__main__.py rename to linux/__main__.py index 4d391f0..33fb6bf 100644 --- a/gui_agent/__main__.py +++ b/linux/__main__.py @@ -1,6 +1,6 @@ """Point d'entrée exécutable pour `python -m gui_agent`.""" -from gui_agent.server import main +from .server import main if __name__ == "__main__": main() diff --git a/linux/accessibility.py b/linux/accessibility.py new file mode 100644 index 0000000..37ab299 --- /dev/null +++ b/linux/accessibility.py @@ -0,0 +1,23 @@ +"""Médiation sémantique d'accessibilité programmatique via AT-SPI2 / D-Bus sous Linux.""" + +from __future__ import annotations + +from .layers.accessibility import ( + find_atspi_mediator_binary, + get_app_state, + perform_action, + set_mock_action_handler, + set_mock_state, + set_mock_value_handler, + set_value, +) + +__all__ = [ + "find_atspi_mediator_binary", + "get_app_state", + "perform_action", + "set_mock_action_handler", + "set_mock_state", + "set_mock_value_handler", + "set_value", +] diff --git a/linux/core/__init__.py b/linux/core/__init__.py new file mode 100644 index 0000000..5fec5b4 --- /dev/null +++ b/linux/core/__init__.py @@ -0,0 +1,8 @@ +"""Moteur d'exécution local CodeAct et SDK unifié mcp_core.""" + +from . import pty_session, repl +from .mcp_core import MCPCoreSDK, mcp_core +from .pty_session import PTYSession +from .repl import execute_script + +__all__ = ["MCPCoreSDK", "PTYSession", "execute_script", "mcp_core", "pty_session", "repl"] diff --git a/gui_agent/core/mcp_core.py b/linux/core/mcp_core.py similarity index 97% rename from gui_agent/core/mcp_core.py rename to linux/core/mcp_core.py index 7e83106..a216480 100644 --- a/gui_agent/core/mcp_core.py +++ b/linux/core/mcp_core.py @@ -2,7 +2,7 @@ from __future__ import annotations -from gui_agent import layers +from .. import layers class MCPCoreSDK: diff --git a/gui_agent/core/pty_session.py b/linux/core/pty_session.py similarity index 79% rename from gui_agent/core/pty_session.py rename to linux/core/pty_session.py index 94f0e9a..1bb5407 100644 --- a/gui_agent/core/pty_session.py +++ b/linux/core/pty_session.py @@ -40,11 +40,20 @@ def execute( total_chars = 0 start = time.monotonic() proc: subprocess.Popen[Any] | None = None + returncode: int | None = None try: os.set_blocking(master_fd, False) - proc = subprocess.Popen( - cmd, stdin=slave_fd, stdout=slave_fd, stderr=slave_fd, close_fds=True, preexec_fn=os.setsid - ) + try: + proc = subprocess.Popen( + cmd, + stdin=slave_fd, + stdout=slave_fd, + stderr=slave_fd, + close_fds=True, + start_new_session=True, + ) + except OSError as exc: + return -1, f"Échec de démarrage du processus PTY : {exc}" with contextlib.suppress(OSError): os.close(slave_fd) slave_fd = -1 @@ -88,15 +97,24 @@ def execute( if total_chars > limit: _kill_pty(proc) return -1, "Taille de sortie PTY maximale dépassée." - returncode = proc.wait() + returncode = proc.poll() finally: for fd in (slave_fd, master_fd): with contextlib.suppress(OSError): if fd >= 0: os.close(fd) if proc is not None: - _kill_pty(proc) - return returncode, "".join(chunks) + if proc.poll() is None: + _kill_pty(proc) + else: + with contextlib.suppress(Exception): + os.killpg(proc.pid, signal.SIGKILL) + with contextlib.suppress(Exception): + if returncode is None: + returncode = proc.wait(timeout=1.0) + else: + proc.wait(timeout=0.5) + return (returncode if returncode is not None else 0), "".join(chunks) def _read_pty(fd: int) -> str: @@ -108,4 +126,6 @@ def _read_pty(fd: int) -> str: def _kill_pty(proc: subprocess.Popen[Any]) -> None: with contextlib.suppress(Exception): os.killpg(proc.pid, signal.SIGKILL) + if proc.poll() is None: + proc.kill() proc.wait(timeout=0.5) diff --git a/gui_agent/core/repl.py b/linux/core/repl.py similarity index 55% rename from gui_agent/core/repl.py rename to linux/core/repl.py index 62340d3..e3738e9 100644 --- a/gui_agent/core/repl.py +++ b/linux/core/repl.py @@ -2,6 +2,7 @@ from __future__ import annotations +import codecs import contextlib import logging import os @@ -14,10 +15,26 @@ logger = logging.getLogger("gui_agent.core.repl") MAX_OUTPUT_CHARS = 1_000_000 + +_PACKAGE_ROOT = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +_PROJECT_ROOT = os.path.dirname(_PACKAGE_ROOT) + RUNNER_HARNESS = ( - "import sys, os\nsys.path.insert(0, os.getcwd())\n" - "import gui_agent.core.mcp_core as _sdk\nsys.modules['mcp_core'] = sys.modules['gui_agent.core.mcp_core']\n" - "code = sys.stdin.read()\nexec(compile(code, '', 'exec'), {'__name__': '__main__', 'mcp_core': _sdk.mcp_core})\n" + "import sys, os\n" + f"sys.path.insert(0, {_PACKAGE_ROOT!r})\n" + f"sys.path.insert(0, {_PROJECT_ROOT!r})\n" + "sys.path.insert(0, os.getcwd())\n" + "try:\n" + " import linux.core.mcp_core as _sdk\n" + " import linux as _linux\n" + " sys.modules['gui_agent'] = _linux\n" + " sys.modules['gui_agent.core'] = sys.modules.get('linux.core')\n" + " sys.modules['gui_agent.core.mcp_core'] = _sdk\n" + "except Exception:\n" + " import gui_agent.core.mcp_core as _sdk\n" + "sys.modules['mcp_core'] = _sdk\n" + "code = sys.stdin.read()\n" + "exec(compile(code, '', 'exec'), {'__name__': '__main__', 'mcp_core': _sdk.mcp_core})\n" ) @@ -45,6 +62,10 @@ def execute_script(code: str, timeout: float = 30.0, max_output_chars: int = MAX for fd in (in_fd, out_fd, err_fd): os.set_blocking(fd, False) inp, off, in_closed, out_len, err_len, readers = code.encode("utf-8"), 0, False, 0, 0, [out_fd, err_fd] + decoders: dict[int, codecs.IncrementalDecoder] = { + out_fd: codecs.getincrementaldecoder("utf-8")(errors="replace"), + err_fd: codecs.getincrementaldecoder("utf-8")(errors="replace"), + } while True: el = time.monotonic() - start @@ -64,26 +85,18 @@ def execute_script(code: str, timeout: float = 30.0, max_output_chars: int = MAX r, w, _ = select.select(readers, wl, [], min(max(0.01, timeout - el), 0.05)) if w and not in_closed: off, in_closed = _write_input(proc, in_fd, inp, off) - for fd in r: - chunk = _safe_read(fd) - if not chunk: - if fd in readers: - readers.remove(fd) - continue - (out_ch if fd == out_fd else err_ch).append(chunk) - out_len += len(chunk) if fd == out_fd else 0 - err_len += len(chunk) if fd != out_fd else 0 - if out_len > max_output_chars or err_len > max_output_chars: - _kill_proc(proc) - status, err = "error", f"Taille de sortie maximale dépassée ({max_output_chars} caractères)." - break + out_len, err_len, exceeded = _handle_readable_fds( + r, readers, out_fd, out_ch, err_ch, decoders, out_len, err_len, max_output_chars + ) + if exceeded: + _kill_proc(proc) + status, err = "error", f"Taille de sortie maximale dépassée ({max_output_chars} caractères)." + break if err: break if not err: - for fd in readers: - while c := _safe_read(fd): - (out_ch if fd == out_fd else err_ch).append(c) + _drain_readers(readers, out_fd, out_ch, err_ch, decoders) if proc.returncode is None: _close_proc(proc) if proc.returncode != 0 and status == "success": @@ -104,6 +117,50 @@ def execute_script(code: str, timeout: float = 30.0, max_output_chars: int = MAX return res +def _handle_readable_fds( + r: list[int], + readers: list[int], + out_fd: int, + out_ch: list[str], + err_ch: list[str], + decoders: dict[int, codecs.IncrementalDecoder], + out_len: int, + err_len: int, + max_output_chars: int, +) -> tuple[int, int, bool]: + for fd in r: + chunk = _safe_read(fd, decoders.get(fd)) + if chunk is None: + continue + if not chunk: + if fd in readers: + readers.remove(fd) + continue + (out_ch if fd == out_fd else err_ch).append(chunk) + if fd == out_fd: + out_len += len(chunk) + else: + err_len += len(chunk) + if out_len > max_output_chars or err_len > max_output_chars: + return out_len, err_len, True + return out_len, err_len, False + + +def _drain_readers( + readers: list[int], + out_fd: int, + out_ch: list[str], + err_ch: list[str], + decoders: dict[int, codecs.IncrementalDecoder], +) -> None: + for fd in readers: + while c := _safe_read(fd, decoders.get(fd)): + (out_ch if fd == out_fd else err_ch).append(c) + for fd, dec in decoders.items(): + if rest := dec.decode(b"", final=True): + (out_ch if fd == out_fd else err_ch).append(rest) + + def _clamp_output( out_ch: list[str], err_ch: list[str], max_chars: int, status: str, err: str | None ) -> tuple[str, str, str, str | None]: @@ -124,10 +181,34 @@ def _close_proc(proc: subprocess.Popen[str]) -> None: s.close() -def _safe_read(fd: int) -> str: - with contextlib.suppress(OSError): - return os.read(fd, 4096).decode("utf-8", errors="replace") - return "" +def _safe_read(fd: int, decoder: codecs.IncrementalDecoder | None = None) -> str | None: + try: + data = os.read(fd, 4096) + except (BlockingIOError, InterruptedError): + return None + except OSError: + return decoder.decode(b"", final=True) if decoder else "" + + if not data: + return decoder.decode(b"", final=True) if decoder else "" + + if decoder is None: + return data.decode("utf-8", errors="replace") + + text = decoder.decode(data) + for _ in range(4): + if text: + break + try: + more = os.read(fd, 4096) + if not more: + return decoder.decode(b"", final=True) + text = decoder.decode(more) + except (BlockingIOError, InterruptedError): + return None + except OSError: + return decoder.decode(b"", final=True) + return text if text else None def _kill_proc(proc: subprocess.Popen[str]) -> None: @@ -143,6 +224,8 @@ def _write_input(proc: subprocess.Popen[str], in_fd: int, inp: bytes, off: int) if proc.stdin: proc.stdin.close() return off, True + except (BlockingIOError, InterruptedError): + return off, False except OSError: if proc.stdin: proc.stdin.close() diff --git a/linux/crates/atspi_mediator/Cargo.lock b/linux/crates/atspi_mediator/Cargo.lock new file mode 100644 index 0000000..4bdfce2 --- /dev/null +++ b/linux/crates/atspi_mediator/Cargo.lock @@ -0,0 +1,1058 @@ +# This file is automatically @generated by Cargo. +# It is not intended for manual editing. +version = 4 + +[[package]] +name = "anyhow" +version = "1.0.104" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "330a5ed07fa54e4702c9d6c4174f74427fc0ef6e214bbd677ae50a5099946470" + +[[package]] +name = "async-broadcast" +version = "0.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "435a87a52755b8f27fcf321ac4f04b2802e337c8c4872923137471ec39c37532" +dependencies = [ + "event-listener", + "event-listener-strategy", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-channel" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "924ed96dd52d1b75e9c1a3e6275715fd320f5f9439fb5a4a11fa51f4221158d2" +dependencies = [ + "concurrent-queue", + "event-listener-strategy", + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "async-executor" +version = "1.14.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c96bf972d85afc50bf5ab8fe2d54d1586b4e0b46c97c50a0c9e71e2f7bcd812a" +dependencies = [ + "async-task", + "concurrent-queue", + "fastrand", + "futures-lite", + "pin-project-lite", + "slab", +] + +[[package]] +name = "async-io" +version = "2.6.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "456b8a8feb6f42d237746d4b3e9a178494627745c3c56c6ea55d92ba50d026fc" +dependencies = [ + "autocfg", + "cfg-if", + "concurrent-queue", + "futures-io", + "futures-lite", + "parking", + "polling", + "rustix", + "slab", + "windows-sys", +] + +[[package]] +name = "async-lock" +version = "3.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "290f7f2596bd5b78a9fec8088ccd89180d7f9f55b94b0576823bbbdc72ee8311" +dependencies = [ + "event-listener", + "event-listener-strategy", + "pin-project-lite", +] + +[[package]] +name = "async-process" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc50921ec0055cdd8a16de48773bfeec5c972598674347252c0399676be7da75" +dependencies = [ + "async-channel", + "async-io", + "async-lock", + "async-signal", + "async-task", + "blocking", + "cfg-if", + "event-listener", + "futures-lite", + "rustix", +] + +[[package]] +name = "async-recursion" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3b43422f69d8ff38f95f1b2bb76517c91589a924d1559a0e935d7c8ce0274c11" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "async-signal" +version = "0.2.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "52b5aaafa020cf5053a01f2a60e8ff5dccf550f0f77ec54a4e47285ac2bab485" +dependencies = [ + "async-io", + "async-lock", + "atomic-waker", + "cfg-if", + "futures-core", + "futures-io", + "rustix", + "signal-hook-registry", + "slab", + "windows-sys", +] + +[[package]] +name = "async-task" +version = "4.7.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8b75356056920673b02621b35afd0f7dda9306d03c79a30f5c56c44cf256e3de" + +[[package]] +name = "async-trait" +version = "0.1.92" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "82f6aeea286b8eb4dd3431a1be1b59d290ace00f5bfd8e2a159bc2a05e2c1667" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "atomic-waker" +version = "1.1.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1505bd5d3d116872e7271a6d4e16d81d0c8570876c8de68093a09ac269d8aac0" + +[[package]] +name = "atspi" +version = "0.29.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c77886257be21c9cd89a4ae7e64860c6f0eefca799bb79127913052bd0eefb3d" +dependencies = [ + "atspi-common", + "atspi-proxies", + "zbus", +] + +[[package]] +name = "atspi-common" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "20c5617155740c98003016429ad13fe43ce7a77b007479350a9f8bf95a29f63d" +dependencies = [ + "enumflags2", + "serde", + "static_assertions", + "zbus", + "zbus-lockstep", + "zbus-lockstep-macros", + "zbus_names", + "zvariant", +] + +[[package]] +name = "atspi-connection" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "43b28a59b7c709ce0f1196405e6eacee35424dad07b4b005c0f9e417d8c2116f" +dependencies = [ + "atspi-common", + "atspi-proxies", + "futures-lite", + "zbus", +] + +[[package]] +name = "atspi-mediator" +version = "0.1.0" +dependencies = [ + "anyhow", + "atspi", + "atspi-connection", + "futures-util", + "serde", + "serde_json", + "tokio", + "zbus", +] + +[[package]] +name = "atspi-proxies" +version = "0.13.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2230e48787ed3eb4088996eab66a32ca20c0b67bbd4fd6cdfe79f04f1f04c9fc" +dependencies = [ + "atspi-common", + "serde", + "zbus", +] + +[[package]] +name = "autocfg" +version = "1.5.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53" + +[[package]] +name = "bitflags" +version = "2.13.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06" + +[[package]] +name = "blocking" +version = "1.7.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a70e4329df6cb94385eed412ec92375c3cdd8a6e502493d1229b6414e4036dfa" +dependencies = [ + "async-channel", + "async-task", + "futures-io", + "futures-lite", + "piper", +] + +[[package]] +name = "bumpalo" +version = "3.20.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649" + +[[package]] +name = "bytes" +version = "1.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "fc652a48c352aef3ea3aed32080501cf3ef6ed5da78602a020c991775b0aff04" + +[[package]] +name = "cfg-if" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801" + +[[package]] +name = "concurrent-queue" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4ca0197aee26d1ae37445ee532fefce43251d24cc7c166799f4d46817f1d3973" +dependencies = [ + "crossbeam-utils", +] + +[[package]] +name = "crossbeam-utils" +version = "0.8.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a31eee39dddec8330830986fcd7625edb5a24ec90ea038215273bbc3adb08ac6" + +[[package]] +name = "endi" +version = "1.1.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "66b7e2430c6dff6a955451e2cfc438f09cea1965a9d6f87f7e3b90decc014099" + +[[package]] +name = "enumflags2" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1027f7680c853e056ebcec683615fb6fbbc07dbaa13b4d5d9442b146ded4ecef" +dependencies = [ + "enumflags2_derive", + "serde", +] + +[[package]] +name = "enumflags2_derive" +version = "0.7.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67c78a4d8fdf9953a5c9d458f9efe940fd97a0cab0941c075a813ac594733827" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "equivalent" +version = "1.0.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f" + +[[package]] +name = "errno" +version = "0.3.14" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "event-listener" +version = "5.4.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5a23add41df1562121a9393cb065eab5146a1242410f23a644851e90cfd669d2" +dependencies = [ + "parking", + "pin-project-lite", +] + +[[package]] +name = "event-listener-strategy" +version = "0.5.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8be9f3dfaaffdae2972880079a491a1a8bb7cbed0b8dd7a347f668b4150a3b93" +dependencies = [ + "event-listener", + "pin-project-lite", +] + +[[package]] +name = "fastrand" +version = "2.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223" + +[[package]] +name = "futures-core" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "92d699e522242e69e3003b94ecc1f960f3a5e015aa7c5d7486e65ad01dd94f5e" + +[[package]] +name = "futures-io" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "53c0fa8157de1303bfffdaa1cc2a673bfffb60102f76b0ef4441659124373fed" + +[[package]] +name = "futures-lite" +version = "2.6.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f78e10609fe0e0b3f4157ffab1876319b5b0db102a2c60dc4626306dc46b44ad" +dependencies = [ + "fastrand", + "futures-core", + "futures-io", + "parking", + "pin-project-lite", +] + +[[package]] +name = "futures-macro" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9fb9654ba8355388abeb8dcb4fc62f511300867002afc858860463bdd9fe0c44" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "futures-task" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cd417de3d1d015fc3bfd2b1ea46dfc7bab72ef86f1cc7cc9c78e728b34a6d1fd" + +[[package]] +name = "futures-util" +version = "0.3.34" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0d50a92467f8ba5dd6e3ee5d4bd04d73ab2e4e1c44474a0674821dfce14b79bc" +dependencies = [ + "futures-core", + "futures-macro", + "futures-task", + "pin-project-lite", + "slab", +] + +[[package]] +name = "getrandom" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099" +dependencies = [ + "cfg-if", + "libc", + "r-efi", +] + +[[package]] +name = "hashbrown" +version = "0.17.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a" + +[[package]] +name = "hermit-abi" +version = "0.5.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e17592d60ebacc7d5e169f4663c5f84f9161cc90328abcfe8456f41e4dfcb284" + +[[package]] +name = "hex" +version = "0.4.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7f24254aa9a54b5c858eaee2f5bccdb46aaf0e486a595ed5fd8f86ba55232a70" + +[[package]] +name = "indexmap" +version = "2.14.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cc4e190f5d26ca7051642629da2c52fc03bde85a03197c99408dcd291734c855" +dependencies = [ + "equivalent", + "hashbrown", +] + +[[package]] +name = "itoa" +version = "1.0.18" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682" + +[[package]] +name = "js-sys" +version = "0.3.105" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ce57d20d1ea864ce2ac172ab472d409214f4fd359f0b2a2775abdf522e2af99e" +dependencies = [ + "cfg-if", + "futures-util", + "wasm-bindgen", +] + +[[package]] +name = "libc" +version = "0.2.189" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2" + +[[package]] +name = "linux-raw-sys" +version = "0.12.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53" + +[[package]] +name = "memchr" +version = "2.8.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98" + +[[package]] +name = "memoffset" +version = "0.9.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a" +dependencies = [ + "autocfg", +] + +[[package]] +name = "mio" +version = "1.2.3" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4b18443e9c262bfe8fa82f51666e2642c53393f7e5c27b3e1aeab922cff5b9d8" +dependencies = [ + "libc", + "wasi", + "windows-sys", +] + +[[package]] +name = "once_cell" +version = "1.21.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50" + +[[package]] +name = "ordered-stream" +version = "0.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "9aa2b01e1d916879f73a53d01d1d6cee68adbb31d6d9177a8cfce093cced1d50" +dependencies = [ + "futures-core", + "pin-project-lite", +] + +[[package]] +name = "parking" +version = "2.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f38d5652c16fde515bb1ecef450ab0f6a219d619a7274976324d5e377f7dceba" + +[[package]] +name = "pin-project-lite" +version = "0.2.17" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd" + +[[package]] +name = "piper" +version = "0.2.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c835479a4443ded371d6c535cbfd8d31ad92c5d23ae9770a61bc155e4992a3c1" +dependencies = [ + "atomic-waker", + "fastrand", + "futures-io", +] + +[[package]] +name = "polling" +version = "3.11.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5d0e4f59085d47d8241c88ead0f274e8a0cb551f3625263c05eb8dd897c34218" +dependencies = [ + "cfg-if", + "concurrent-queue", + "hermit-abi", + "pin-project-lite", + "rustix", + "windows-sys", +] + +[[package]] +name = "proc-macro-crate" +version = "3.5.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e67ba7e9b2b56446f1d419b1d807906278ffa1a658a8a5d8a39dcb1f5a78614f" +dependencies = [ + "toml_edit", +] + +[[package]] +name = "proc-macro2" +version = "1.0.107" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "quote" +version = "1.0.47" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001" +dependencies = [ + "proc-macro2", +] + +[[package]] +name = "r-efi" +version = "6.0.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf" + +[[package]] +name = "rustix" +version = "1.1.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190" +dependencies = [ + "bitflags", + "errno", + "libc", + "linux-raw-sys", + "windows-sys", +] + +[[package]] +name = "rustversion" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f" + +[[package]] +name = "serde" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba" +dependencies = [ + "serde_core", + "serde_derive", +] + +[[package]] +name = "serde_core" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48" +dependencies = [ + "serde_derive", +] + +[[package]] +name = "serde_derive" +version = "1.0.229" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "serde_json" +version = "1.0.151" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14" +dependencies = [ + "itoa", + "memchr", + "serde", + "serde_core", + "zmij", +] + +[[package]] +name = "serde_repr" +version = "0.1.21" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "8d3b1629de253c70a0508c3899572da79ca359fdab27c7920ff00406df418906" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "signal-hook-registry" +version = "1.4.8" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c4db69cba1110affc0e9f7bcd48bbf87b3f4fc7c61fc9155afd4c469eb3d6c1b" +dependencies = [ + "errno", + "libc", +] + +[[package]] +name = "slab" +version = "0.4.12" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "0c790de23124f9ab44544d7ac05d60440adc586479ce501c1d6d7da3cd8c9cf5" + +[[package]] +name = "socket2" +version = "0.6.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c3d1e2c7f27f8d4cb10542a02c49005dbd6e93095799d6f3be745fae9f8fedd4" +dependencies = [ + "libc", + "windows-sys", +] + +[[package]] +name = "static_assertions" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a2eb9349b6444b326872e140eb1cf5e7c522154d69e7a0ffb0fb81c06b37543f" + +[[package]] +name = "syn" +version = "2.0.119" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "syn" +version = "3.0.5" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "12df2e0110f65b775f769bb17ef989067a1d931b2eb822bd4346631eeada89f9" +dependencies = [ + "proc-macro2", + "quote", + "unicode-ident", +] + +[[package]] +name = "tempfile" +version = "3.27.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd" +dependencies = [ + "fastrand", + "getrandom", + "once_cell", + "rustix", + "windows-sys", +] + +[[package]] +name = "tokio" +version = "1.53.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "202caea871b69668250d242070849eb495be178ed697a3e98aebce5bc81a0bed" +dependencies = [ + "bytes", + "libc", + "mio", + "pin-project-lite", + "signal-hook-registry", + "socket2", + "tokio-macros", + "tracing", + "windows-sys", +] + +[[package]] +name = "tokio-macros" +version = "2.7.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "78773a2a397f451582ce068015985c33193cf6dea8b74d2a639fe457b2f07b0e" +dependencies = [ + "proc-macro2", + "quote", + "syn 3.0.5", +] + +[[package]] +name = "toml_datetime" +version = "1.1.1+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "3165f65f62e28e0115a00b2ebdd37eb6f3b641855f9d636d3cd4103767159ad7" +dependencies = [ + "serde_core", +] + +[[package]] +name = "toml_edit" +version = "0.25.15+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1340ea94a5856333492c9064b02c778b191dd2c853778d9609debdcdfea3a614" +dependencies = [ + "indexmap", + "toml_datetime", + "toml_parser", + "winnow", +] + +[[package]] +name = "toml_parser" +version = "1.1.3+spec-1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "1d38ac1cf9b95face32296c0a3ede1fdc270627c9d9c02a7274dd6d960dc4d56" +dependencies = [ + "winnow", +] + +[[package]] +name = "tracing" +version = "0.1.44" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100" +dependencies = [ + "pin-project-lite", + "tracing-attributes", + "tracing-core", +] + +[[package]] +name = "tracing-attributes" +version = "0.1.31" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", +] + +[[package]] +name = "tracing-core" +version = "0.1.36" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a" +dependencies = [ + "once_cell", +] + +[[package]] +name = "uds_windows" +version = "1.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f2f6fb2847f6742cd76af783a2a2c49e9375d0a111c7bef6f71cd9e738c72d6e" +dependencies = [ + "memoffset", + "tempfile", + "windows-sys", +] + +[[package]] +name = "unicode-ident" +version = "1.0.24" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75" + +[[package]] +name = "uuid" +version = "1.26.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2ef6dac1e96601b4fb3acccccff2139741fcb757cb9a36089bf5be91cfb285ce" +dependencies = [ + "js-sys", + "serde_core", + "wasm-bindgen", +] + +[[package]] +name = "wasi" +version = "0.11.1+wasi-snapshot-preview1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ccf3ec651a847eb01de73ccad15eb7d99f80485de043efb2f370cd654f4ea44b" + +[[package]] +name = "wasm-bindgen" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "aecb87a33d3b0c5e3b7aa46336eaf486cffafbd281b195e4c8b80d50df2351bf" +dependencies = [ + "cfg-if", + "once_cell", + "rustversion", + "wasm-bindgen-macro", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-macro" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "a690d511e3c1a8b3a55e33511e3c2c00c78415cd23650f32b808627f5696b9ed" +dependencies = [ + "quote", + "wasm-bindgen-macro-support", +] + +[[package]] +name = "wasm-bindgen-macro-support" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "411e4887f0071ef2d2164a9d5fdf2d20efbef78fccd3a78b0c10a1dc5295e48a" +dependencies = [ + "bumpalo", + "proc-macro2", + "quote", + "syn 3.0.5", + "wasm-bindgen-shared", +] + +[[package]] +name = "wasm-bindgen-shared" +version = "0.2.128" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "81941cd78d0c92026c33e5e01312845a4cb1e9af3407f9134b100dd03144103e" +dependencies = [ + "unicode-ident", +] + +[[package]] +name = "windows-link" +version = "0.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5" + +[[package]] +name = "windows-sys" +version = "0.61.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc" +dependencies = [ + "windows-link", +] + +[[package]] +name = "winnow" +version = "1.0.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "23b97319f7b8343df12cc98938e5c3eb436064524c8d2b4e30a1d3a36eecdf81" +dependencies = [ + "memchr", +] + +[[package]] +name = "zbus" +version = "5.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "5db4be7c075cb421e4b7ee645541604239bd243ba7c357511f4ff3a74b555907" +dependencies = [ + "async-broadcast", + "async-executor", + "async-io", + "async-lock", + "async-process", + "async-recursion", + "async-task", + "async-trait", + "blocking", + "enumflags2", + "event-listener", + "futures-core", + "futures-lite", + "hex", + "libc", + "ordered-stream", + "rustix", + "serde", + "serde_repr", + "tokio", + "tracing", + "uds_windows", + "uuid", + "windows-sys", + "winnow", + "zbus_macros", + "zbus_names", + "zvariant", +] + +[[package]] +name = "zbus-lockstep" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "6998de05217a084b7578728a9443d04ea4cd80f2a0839b8d78770b76ccd45863" +dependencies = [ + "zbus_xml", + "zvariant", +] + +[[package]] +name = "zbus-lockstep-macros" +version = "0.5.2" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "10da05367f3a7b7553c8cdf8fa91aee6b64afebe32b51c95177957efc47ca3a0" +dependencies = [ + "proc-macro2", + "quote", + "syn 2.0.119", + "zbus-lockstep", + "zbus_xml", + "zvariant", +] + +[[package]] +name = "zbus_macros" +version = "5.19.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "2990635d09ade6df1868f72f8cac69a876a90981e8bd3c40b1be413f8dc88f40" +dependencies = [ + "proc-macro-crate", + "proc-macro2", + "quote", + "syn 3.0.5", + "zbus_names", + "zvariant", + "zvariant_utils", +] + +[[package]] +name = "zbus_names" +version = "4.3.4" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d8bf88b4a3ff53e883001e0e0115b297a9d53c31b9c1edd2bfdd853e3428624e" +dependencies = [ + "serde", + "winnow", + "zvariant", +] + +[[package]] +name = "zbus_xml" +version = "5.2.1" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1586c021a01ca0a9216dcd874e546382e156a5cbab5fab6cb5f10087e22682a" +dependencies = [ + "serde", + "winnow", + "zbus_names", + "zvariant", +] + +[[package]] +name = "zcheapstr" +version = "1.1.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "d1afec51604565183aeb5c54c20aeab286120d4e4460f7f76e3e8bb8c0d99473" +dependencies = [ + "serde", +] + +[[package]] +name = "zmij" +version = "1.0.23" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b" + +[[package]] +name = "zvariant" +version = "5.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "c1d34c27cc6cdd1f458427519dd6b8612f7b7e3f7b9a0b2355d041dda9869147" +dependencies = [ + "endi", + "enumflags2", + "serde", + "winnow", + "zcheapstr", + "zvariant_derive", + "zvariant_utils", +] + +[[package]] +name = "zvariant_derive" +version = "5.15.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "864155e69b4352db0c7f374917bf45d1e0c8d17659c8b3dbf9795f3673f8c497" +dependencies = [ + "proc-macro-crate", + "proc-macro2", + "quote", + "syn 3.0.5", + "zvariant_utils", +] + +[[package]] +name = "zvariant_utils" +version = "4.2.0" +source = "registry+https://github.com/rust-lang/crates.io-index" +checksum = "bad0294361a320b694a328460dc73add56c306150f5cb6bfafc44446120008a3" +dependencies = [ + "proc-macro2", + "quote", + "serde", + "syn 3.0.5", + "winnow", +] diff --git a/linux/crates/atspi_mediator/Cargo.toml b/linux/crates/atspi_mediator/Cargo.toml new file mode 100644 index 0000000..e67cd49 --- /dev/null +++ b/linux/crates/atspi_mediator/Cargo.toml @@ -0,0 +1,26 @@ +[package] +name = "atspi-mediator" +version = "0.1.0" +edition = "2021" +authors = ["Leandre "] +description = "High-performance AT-SPI2 / D-Bus accessibility mediation engine for Linux desktop" +license = "MIT" + +[[bin]] +name = "gui-agent-atspi" +path = "src/main.rs" + +[lib] +name = "atspi_mediator" +path = "src/lib.rs" + +[dependencies] +anyhow = "1.0" +atspi = { version = "0.29.0", default-features = false, features = ["proxies", "wrappers", "tokio"] } +atspi-connection = { version = "0.13.0", default-features = false, features = ["wrappers"] } +futures-util = "0.3" +serde = { version = "1.0", features = ["derive"] } +serde_json = "1.0" +tokio = { version = "1.40", features = ["io-std", "io-util", "macros", "rt-multi-thread", "sync", "time"] } +zbus = "5.14.0" + diff --git a/linux/crates/atspi_mediator/src/lib.rs b/linux/crates/atspi_mediator/src/lib.rs new file mode 100644 index 0000000..4738d05 --- /dev/null +++ b/linux/crates/atspi_mediator/src/lib.rs @@ -0,0 +1,1028 @@ +//! AT-SPI2 / D-Bus accessibility mediation engine for gui_agent. + +use anyhow::{anyhow, Context, Result}; +use atspi::{ + proxy::{ + accessible::{AccessibleProxy, ObjectRefExt}, + proxy_ext::ProxyExt, + }, + CoordType, ObjectRef, ObjectRefOwned, StateSet, +}; +use atspi_connection::AccessibilityConnection; +use futures_util::{stream, StreamExt}; +use serde::{Deserialize, Serialize}; +use std::{collections::VecDeque, future::Future, time::Duration}; +use tokio::time::timeout; +use zbus::{ + fdo::DBusProxy, + names::{BusName, UniqueName}, + zvariant::ObjectPath, +}; + +const MAX_TEXT_READBACK_CHARS: i32 = 4096; +const MAX_TEXT_SELECTIONS: i32 = 8; +const DEFAULT_SNAPSHOT_MAX_NODES: usize = 1_000; +const HARD_SNAPSHOT_MAX_NODES: usize = 2_000; +const DEFAULT_SNAPSHOT_MAX_DEPTH: u32 = 32; +const HARD_SNAPSHOT_MAX_DEPTH: u32 = 64; +const CHILD_READ_CONCURRENCY: usize = 16; +const SNAPSHOT_TIMEOUT: Duration = Duration::from_secs(10); +const MAX_DISCOVERY_ROOTS: usize = 256; +const ROOT_MATCH_CHILD_LIMIT: usize = 8; +const MAX_DISCOVERY_CHILD_READS: usize = MAX_DISCOVERY_ROOTS * ROOT_MATCH_CHILD_LIMIT; + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AccessibleAppSummary { + pub object_ref: String, + pub name: Option, + pub pid: Option, + pub role: String, + pub child_count: i32, + pub bounds: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AccessibilityNode { + pub index: u32, + pub parent_index: Option, + pub depth: u32, + pub object_ref: String, + pub role: String, + pub name: Option, + pub description: Option, + pub child_count: i32, + pub bounds: Option, + pub states: Vec, + pub actions: Vec, + pub value: Option, + pub text: Option, + pub supports_editable_text: bool, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct Bounds { + pub x: i32, + pub y: i32, + pub width: i32, + pub height: i32, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AccessibilityAction { + pub index: i32, + pub name: String, + pub description: String, + pub keybinding: String, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AccessibilityValue { + pub current: f64, + pub minimum: f64, + pub maximum: f64, + pub minimum_increment: f64, + pub text: Option, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AccessibilityText { + pub character_count: i32, + pub caret_offset: Option, + pub content: Option, + pub truncated: bool, + pub selections: Vec, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct AccessibilityTextSelection { + pub start_offset: i32, + pub end_offset: i32, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub struct ActionInvocation { + pub action_index: i32, + pub action_name: Option, + pub ok: bool, +} + +#[derive(Debug, Clone, Serialize, Deserialize)] +pub enum ValueSetInvocation { + Numeric { value: f64 }, + EditableText, +} + +static HYDRATE_BUS_ENV_MUTEX: std::sync::Mutex<()> = std::sync::Mutex::new(()); + +pub fn hydrate_session_bus_env() { + if std::env::var_os("DBUS_SESSION_BUS_ADDRESS").is_some() { + return; + } + if let Ok(_guard) = HYDRATE_BUS_ENV_MUTEX.lock() { + if std::env::var_os("DBUS_SESSION_BUS_ADDRESS").is_none() { + if let Ok(runtime) = std::env::var("XDG_RUNTIME_DIR") { + let bus = std::path::Path::new(&runtime).join("bus"); + if bus.exists() { + std::env::set_var( + "DBUS_SESSION_BUS_ADDRESS", + format!("unix:path={}", bus.display()), + ); + } + } + } + } +} + +pub fn snapshot_limits( + requested_max_nodes: Option, + requested_max_depth: Option, +) -> (usize, u32) { + ( + requested_max_nodes + .unwrap_or(DEFAULT_SNAPSHOT_MAX_NODES) + .clamp(1, HARD_SNAPSHOT_MAX_NODES), + requested_max_depth + .unwrap_or(DEFAULT_SNAPSHOT_MAX_DEPTH) + .min(HARD_SNAPSHOT_MAX_DEPTH), + ) +} + +fn snapshot_child_read_budgets(max_nodes: usize) -> (usize, usize, usize) { + (MAX_DISCOVERY_ROOTS, MAX_DISCOVERY_CHILD_READS, max_nodes) +} + +struct BoundedTraversal { + queue: VecDeque, + attempted: usize, + max_items: usize, +} + +impl BoundedTraversal { + fn new(max_items: usize) -> Self { + Self { + queue: VecDeque::new(), + attempted: 0, + max_items, + } + } + + fn enqueue(&mut self, items: impl IntoIterator) { + self.queue + .extend(items.into_iter().take(self.remaining_capacity())); + } + + fn pop(&mut self) -> Option { + if self.attempted >= self.max_items { + return None; + } + let item = self.queue.pop_front()?; + self.attempted += 1; + Some(item) + } + + fn remaining_capacity(&self) -> usize { + self.max_items + .saturating_sub(self.attempted.saturating_add(self.queue.len())) + } +} + +fn bounded_child_count(reported: i32, limit: usize) -> usize { + usize::try_from(reported).unwrap_or_default().min(limit) +} + +struct IndexedReadBatch { + items: Vec, + attempted: usize, +} + +impl IndexedReadBatch { + fn all_failed(&self) -> bool { + self.attempted > 0 && self.items.is_empty() + } +} + +async fn fetch_indexed_up_to( + reported: i32, + limit: usize, + remaining_attempts: &mut usize, + fetch: F, +) -> IndexedReadBatch +where + F: Fn(i32) -> Fut, + Fut: Future>, +{ + let attempt_count = bounded_child_count(reported, limit).min(*remaining_attempts); + *remaining_attempts = (*remaining_attempts).saturating_sub(attempt_count); + let end_index = i32::try_from(attempt_count).unwrap_or(i32::MAX); + + let items = stream::iter(0..end_index) + .map(fetch) + .buffered(CHILD_READ_CONCURRENCY) + .filter_map(|result| async move { result.ok() }) + .collect() + .await; + + IndexedReadBatch { + items, + attempted: attempt_count, + } +} + +async fn children_up_to( + proxy: &AccessibleProxy<'_>, + limit: usize, + remaining_attempts: &mut usize, +) -> std::result::Result, atspi::AtspiError> { + if limit == 0 || *remaining_attempts == 0 { + return Ok(IndexedReadBatch { + items: Vec::new(), + attempted: 0, + }); + } + + let child_count = proxy.child_count().await?; + Ok( + fetch_indexed_up_to(child_count, limit, remaining_attempts, |index| { + proxy.get_child_at_index(index) + }) + .await, + ) +} + +pub async fn connect() -> Result { + hydrate_session_bus_env(); + AccessibilityConnection::new() + .await + .context("Échec de connexion au bus d'accessibilité AT-SPI (org.a11y.Bus)") +} + +async fn open_accessible<'r>( + conn: &AccessibilityConnection, + object_ref: &'r ObjectRefOwned, +) -> Result, atspi::AtspiError> { + object_ref.as_accessible_proxy(conn.connection()).await +} + +async fn registry_children( + conn: &AccessibilityConnection, + limit: usize, + remaining_child_reads: &mut usize, +) -> Result> { + let root = conn + .root_accessible_on_registry() + .await + .context("Impossible d'ouvrir la racine AT-SPI du registre")?; + let batch = children_up_to(&root, limit, remaining_child_reads) + .await + .context("Impossible de lire les enfants du registre AT-SPI")?; + if batch.all_failed() { + return Err(anyhow!( + "Le registre AT-SPI a signalé des enfants, mais chaque lecture a échoué" + )); + } + Ok(batch.items) +} + +async fn object_ref_pid(dbus: Option<&DBusProxy<'_>>, object_ref: &ObjectRefOwned) -> Option { + let dbus = dbus?; + let bus_name = BusName::try_from(object_ref.name_as_str()?.to_string()).ok()?; + dbus.get_connection_unix_process_id(bus_name).await.ok() +} + +async fn select_roots( + conn: &AccessibilityConnection, + roots: Vec, + app_name_or_bundle_identifier: Option<&str>, + target_pid: Option, + remaining_child_reads: &mut usize, +) -> Vec { + let needle = app_name_or_bundle_identifier + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(|value| value.to_ascii_lowercase()); + let dbus = DBusProxy::new(conn.connection()).await.ok(); + let remaining = roots; + + if let Some(target_pid) = target_pid { + let mut pid_and_filter_matches = Vec::new(); + let mut pid_matches = Vec::new(); + + for object_ref in remaining { + if object_ref_pid(dbus.as_ref(), &object_ref).await == Some(target_pid) { + if let Some(needle) = needle.as_deref() { + if root_matches(conn, &object_ref, needle, remaining_child_reads).await { + pid_and_filter_matches.push(object_ref); + } else { + pid_matches.push(object_ref); + } + } else { + pid_matches.push(object_ref); + } + } + } + + if !pid_and_filter_matches.is_empty() { + return pid_and_filter_matches; + } + return pid_matches; + } + + if let Some(needle) = needle.as_deref() { + let mut filter_matches = Vec::new(); + for object_ref in remaining { + if root_matches(conn, &object_ref, needle, remaining_child_reads).await { + filter_matches.push(object_ref); + } + } + return filter_matches; + } + + remaining +} + +async fn root_matches( + conn: &AccessibilityConnection, + object_ref: &ObjectRefOwned, + needle: &str, + remaining_child_reads: &mut usize, +) -> bool { + let Ok(proxy) = open_accessible(conn, object_ref).await else { + return object_ref_id(object_ref) + .to_ascii_lowercase() + .contains(needle); + }; + + if proxy_matches(&proxy, object_ref, needle).await { + return true; + } + + for child_ref in children_up_to(&proxy, ROOT_MATCH_CHILD_LIMIT, remaining_child_reads) + .await + .map(|batch| batch.items) + .unwrap_or_default() + { + let Ok(child_proxy) = open_accessible(conn, &child_ref).await else { + continue; + }; + if proxy_matches(&child_proxy, &child_ref, needle).await { + return true; + } + } + + false +} + +async fn proxy_matches( + proxy: &AccessibleProxy<'_>, + object_ref: &ObjectRefOwned, + needle: &str, +) -> bool { + let name = proxy.name().await.unwrap_or_default(); + let role = proxy.get_role_name().await.unwrap_or_default(); + format!("{} {} {}", object_ref_id(object_ref), name, role) + .to_ascii_lowercase() + .contains(needle) +} + +pub async fn list_accessible_apps(limit: usize) -> Result> { + let conn = connect().await?; + let mut remaining_child_reads = limit; + let roots = registry_children(&conn, limit, &mut remaining_child_reads).await?; + let dbus = DBusProxy::new(conn.connection()).await.ok(); + let mut apps = Vec::new(); + + for object_ref in roots.into_iter().take(limit) { + if let Ok(proxy) = open_accessible(&conn, &object_ref).await { + apps.push(read_app_summary(&proxy, &object_ref, dbus.as_ref()).await); + } + } + + Ok(apps) +} + +pub async fn snapshot_tree( + app_name_or_bundle_identifier: Option<&str>, + target_pid: Option, + max_nodes: usize, + max_depth: u32, +) -> Result> { + let (max_nodes, max_depth) = snapshot_limits(Some(max_nodes), Some(max_depth)); + timeout( + SNAPSHOT_TIMEOUT, + snapshot_tree_inner( + app_name_or_bundle_identifier, + target_pid, + max_nodes, + max_depth, + ), + ) + .await + .context("AT-SPI snapshot a dépassé le délai imparti de 10 secondes")? +} + +async fn snapshot_tree_inner( + app_name_or_bundle_identifier: Option<&str>, + target_pid: Option, + max_nodes: usize, + max_depth: u32, +) -> Result> { + let conn = connect().await?; + let (mut remaining_registry_reads, mut remaining_filter_reads, mut remaining_traversal_reads) = + snapshot_child_read_budgets(max_nodes); + let roots = + registry_children(&conn, MAX_DISCOVERY_ROOTS, &mut remaining_registry_reads).await?; + let selected_roots = select_roots( + &conn, + roots, + app_name_or_bundle_identifier, + target_pid, + &mut remaining_filter_reads, + ) + .await; + let mut nodes = Vec::new(); + let mut traversal = BoundedTraversal::new(max_nodes); + + traversal.enqueue( + selected_roots + .into_iter() + .map(|object_ref| (object_ref, 0_u32, None)), + ); + + while let Some((object_ref, depth, parent_index)) = traversal.pop() { + let Ok(proxy) = open_accessible(&conn, &object_ref).await else { + continue; + }; + let index = nodes.len() as u32; + let remaining = traversal.remaining_capacity(); + let child_refs = if depth < max_depth && remaining > 0 { + children_up_to(&proxy, remaining, &mut remaining_traversal_reads) + .await + .map(|batch| batch.items) + .unwrap_or_default() + } else { + Vec::new() + }; + + nodes.push(read_node(&proxy, &object_ref, index, parent_index, depth).await); + + traversal.enqueue( + child_refs + .into_iter() + .map(|child| (child, depth + 1, Some(index))), + ); + } + + Ok(nodes) +} + +pub async fn perform_action( + object_ref_id: &str, + requested_action: Option<&str>, +) -> Result { + let conn = connect().await?; + let object_ref = object_ref_from_id(object_ref_id)?; + let proxy = open_accessible(&conn, &object_ref) + .await + .with_context(|| format!("Impossible d'ouvrir l'objet AT-SPI {object_ref_id}"))?; + let action = proxy + .proxies() + .await? + .action() + .await + .context("Le composant n'implémente pas l'interface AT-SPI Action")?; + let actions = action.get_actions().await.unwrap_or_default(); + let action_index = select_action_index(&actions, requested_action)?; + let action_name = actions + .get(action_index as usize) + .map(|act| act.name.clone()); + let ok = action + .do_action(action_index) + .await + .with_context(|| format!("Échec d'exécution de l'action AT-SPI index {action_index}"))?; + + Ok(ActionInvocation { + action_index, + action_name, + ok, + }) +} + +pub async fn set_element_value(object_ref_id: &str, value: &str) -> Result { + let conn = connect().await?; + let object_ref = object_ref_from_id(object_ref_id)?; + let proxy = open_accessible(&conn, &object_ref) + .await + .with_context(|| format!("Impossible d'ouvrir l'objet AT-SPI {object_ref_id}"))?; + let proxies = proxy.proxies().await?; + + if let Ok(numeric_value) = value.parse::() { + if let Ok(value_proxy) = proxies.value().await { + value_proxy + .set_current_value(numeric_value) + .await + .with_context(|| { + format!("Échec de modification de la valeur numérique à {numeric_value}") + })?; + return Ok(ValueSetInvocation::Numeric { + value: numeric_value, + }); + } + } + + if let Ok(editable_text) = proxies.editable_text().await { + let ok = editable_text + .set_text_contents(value) + .await + .context("Échec d'écriture directe dans l'interface AT-SPI EditableText")?; + if ok { + return Ok(ValueSetInvocation::EditableText); + } + return Err(anyhow!( + "L'interface AT-SPI EditableText a rejeté le contenu" + )); + } + + if value.parse::().is_err() && proxies.value().await.is_ok() { + return Err(anyhow!( + "Le composant expose l'interface Value mais la valeur fournie n'est pas numérique" + )); + } + + Err(anyhow!( + "Le composant n'implémente ni l'interface Value ni EditableText" + )) +} + +async fn read_app_summary( + proxy: &AccessibleProxy<'_>, + object_ref: &ObjectRefOwned, + dbus: Option<&DBusProxy<'_>>, +) -> AccessibleAppSummary { + let name = optional_string(proxy.name().await.ok()); + let role = role_name(proxy).await; + let child_count = proxy.child_count().await.unwrap_or(0); + let pid = object_ref_pid(dbus, object_ref).await; + AccessibleAppSummary { + object_ref: object_ref_id(object_ref), + name, + pid, + role, + child_count, + bounds: bounds(proxy).await, + } +} + +async fn read_node( + proxy: &AccessibleProxy<'_>, + object_ref: &ObjectRefOwned, + index: u32, + parent_index: Option, + depth: u32, +) -> AccessibilityNode { + let proxies = proxy.proxies().await.ok(); + AccessibilityNode { + index, + parent_index, + depth, + object_ref: object_ref_id(object_ref), + role: role_name(proxy).await, + name: optional_string(proxy.name().await.ok()), + description: optional_string(proxy.description().await.ok()), + child_count: proxy.child_count().await.unwrap_or_default(), + bounds: bounds_from_proxies(proxies.as_ref(), proxy).await, + states: states_from_proxy(proxy).await, + actions: actions_from_proxies(proxies.as_ref()).await, + value: value_from_proxies(proxies.as_ref()).await, + text: text_from_proxies(proxies.as_ref()).await, + supports_editable_text: supports_editable_text(proxies.as_ref()).await, + } +} + +async fn role_name(proxy: &AccessibleProxy<'_>) -> String { + if let Ok(role) = proxy.get_role_name().await { + if !role.trim().is_empty() { + return role; + } + } + proxy + .get_role() + .await + .map(|role| format!("{role:?}")) + .unwrap_or_else(|_| "unknown".to_string()) +} + +async fn bounds(proxy: &AccessibleProxy<'_>) -> Option { + bounds_from_proxies(proxy.proxies().await.ok().as_ref(), proxy).await +} + +async fn bounds_from_proxies( + proxies: Option<&atspi::proxy::proxy_ext::Proxies<'_>>, + proxy: &AccessibleProxy<'_>, +) -> Option { + let owned_proxies; + let proxies = if let Some(proxies) = proxies { + proxies + } else { + owned_proxies = proxy.proxies().await.ok()?; + &owned_proxies + }; + let component = proxies.component().await.ok()?; + let (x, y, width, height) = component.get_extents(CoordType::Screen).await.ok()?; + normalize_bounds(Bounds { + x, + y, + width, + height, + }) +} + +fn normalize_bounds(bounds: Bounds) -> Option { + if bounds.width <= 0 || bounds.height <= 0 { + return None; + } + if bounds.x <= i32::MIN / 2 || bounds.y <= i32::MIN / 2 { + return None; + } + Some(bounds) +} + +async fn actions_from_proxies( + proxies: Option<&atspi::proxy::proxy_ext::Proxies<'_>>, +) -> Vec { + let Some(proxies) = proxies else { + return Vec::new(); + }; + let Ok(action_proxy) = proxies.action().await else { + return Vec::new(); + }; + + action_proxy + .get_actions() + .await + .unwrap_or_default() + .into_iter() + .enumerate() + .map(|(index, action)| AccessibilityAction { + index: index as i32, + name: action.name, + description: action.description, + keybinding: action.keybinding, + }) + .collect() +} + +async fn states_from_proxy(proxy: &AccessibleProxy<'_>) -> Vec { + proxy + .get_state() + .await + .map(state_labels) + .unwrap_or_default() +} + +async fn value_from_proxies( + proxies: Option<&atspi::proxy::proxy_ext::Proxies<'_>>, +) -> Option { + let value = proxies?.value().await.ok()?; + Some(AccessibilityValue { + current: value.current_value().await.ok()?, + minimum: value.minimum_value().await.ok()?, + maximum: value.maximum_value().await.ok()?, + minimum_increment: value.minimum_increment().await.ok()?, + text: optional_string(value.text().await.ok()), + }) +} + +async fn text_from_proxies( + proxies: Option<&atspi::proxy::proxy_ext::Proxies<'_>>, +) -> Option { + let text = proxies?.text().await.ok()?; + let character_count = text.character_count().await.ok()?.max(0); + let caret_offset = text.caret_offset().await.ok(); + let capped_count = character_count.min(MAX_TEXT_READBACK_CHARS); + let content = if capped_count > 0 { + optional_string(text.get_text(0, capped_count).await.ok()) + } else { + None + }; + let selection_count = text + .get_nselections() + .await + .unwrap_or_default() + .clamp(0, MAX_TEXT_SELECTIONS); + let mut selections = Vec::new(); + for index in 0..selection_count { + if let Ok((start_offset, end_offset)) = text.get_selection(index).await { + selections.push(AccessibilityTextSelection { + start_offset, + end_offset, + }); + } + } + + Some(AccessibilityText { + character_count, + caret_offset, + content, + truncated: character_count > MAX_TEXT_READBACK_CHARS, + selections, + }) +} + +async fn supports_editable_text(proxies: Option<&atspi::proxy::proxy_ext::Proxies<'_>>) -> bool { + let Some(proxies) = proxies else { + return false; + }; + proxies.editable_text().await.is_ok() +} + +fn state_labels(state_set: StateSet) -> Vec { + state_set.iter().map(|state| state.to_string()).collect() +} + +fn select_action_index(actions: &[atspi::Action], requested_action: Option<&str>) -> Result { + if actions.is_empty() { + return Err(anyhow!("Le composant n'expose aucune action AT-SPI")); + } + + if let Some(requested) = requested_action + .map(str::trim) + .filter(|value| !value.is_empty()) + { + let requested_lower = requested.to_ascii_lowercase(); + + // 1. Correspondance exacte sur le nom ou la description (insensible à la casse) + let exact_matches: Vec = actions + .iter() + .enumerate() + .filter(|(_, action)| { + action.name.to_ascii_lowercase() == requested_lower + || action.description.to_ascii_lowercase() == requested_lower + }) + .map(|(idx, _)| idx) + .collect(); + + if exact_matches.len() == 1 { + return Ok(exact_matches[0] as i32); + } else if exact_matches.len() > 1 { + let available = exact_matches + .iter() + .map(|&idx| format!("#{idx}: '{}'", actions[idx].name)) + .collect::>() + .join(", "); + return Err(anyhow!( + "Plusieurs actions correspondent exactement à '{requested}' ([{available}]) ; veuillez spécifier l'index numérique" + )); + } + + // 2. Correspondance exacte sur un index numérique (ex: "0", "1") + if let Ok(index) = requested.parse::() { + if index < actions.len() { + return Ok(index as i32); + } else { + return Err(anyhow!( + "Index d'action numérique {index} hors limites (actions disponibles: 0..{})", + actions.len() - 1 + )); + } + } + + // 3. Correspondance par sous-chaîne univoque sur le nom ou la description + let substring_matches: Vec = actions + .iter() + .enumerate() + .filter(|(_, action)| { + let name = action.name.to_ascii_lowercase(); + let desc = action.description.to_ascii_lowercase(); + name.contains(&requested_lower) || desc.contains(&requested_lower) + }) + .map(|(idx, _)| idx) + .collect(); + + if substring_matches.len() == 1 { + return Ok(substring_matches[0] as i32); + } + + // 4. Synonymes sémantiques pour les requêtes génériques ("activate", "click", "press", "default", "primary") + let is_generic_action = matches!( + requested_lower.as_str(), + "activate" | "click" | "press" | "default" | "primary" + ); + if is_generic_action { + let primary_matches: Vec = actions + .iter() + .enumerate() + .filter(|(_, action)| { + let name = action.name.to_ascii_lowercase(); + matches!( + name.as_str(), + "activate" | "click" | "press" | "primary" | "default" | "toggle" | "open" + ) + }) + .map(|(idx, _)| idx) + .collect(); + + if primary_matches.len() == 1 { + return Ok(primary_matches[0] as i32); + } + } + + let available = actions + .iter() + .enumerate() + .map(|(idx, a)| { + if a.description.trim().is_empty() { + format!("#{idx}: '{}'", a.name) + } else { + format!("#{idx}: '{}' ({})", a.name, a.description) + } + }) + .collect::>() + .join(", "); + + return Err(anyhow!( + "L'action demandée '{requested}' est introuvable ou ambiguë ; actions disponibles: [{available}]" + )); + } + + // Aucune action spécifiée (None ou chaîne vide) : sélection univoque de l'action unique ou primaire + if actions.len() == 1 { + return Ok(0); + } + + let default_matches: Vec = actions + .iter() + .enumerate() + .filter(|(_, action)| { + let name = action.name.to_ascii_lowercase(); + matches!( + name.as_str(), + "activate" | "click" | "press" | "primary" | "default" + ) + }) + .map(|(idx, _)| idx) + .collect(); + + if default_matches.len() == 1 { + return Ok(default_matches[0] as i32); + } + + let available = actions + .iter() + .enumerate() + .map(|(idx, a)| { + if a.description.trim().is_empty() { + format!("#{idx}: '{}'", a.name) + } else { + format!("#{idx}: '{}' ({})", a.name, a.description) + } + }) + .collect::>() + .join(", "); + + Err(anyhow!( + "Plusieurs actions sont disponibles ([{available}]) mais aucune action par défaut univoque n'a pu être déterminée ; veuillez spécifier l'action" + )) +} + +fn optional_string(value: Option) -> Option { + value + .map(|value| value.trim().to_string()) + .filter(|value| !value.is_empty()) +} + +pub fn split_object_ref_id(object_ref_id: &str) -> Result<(&str, &str)> { + let Some(path_start) = object_ref_id.find('/') else { + return Err(anyhow!( + "invalid AT-SPI object ref '{object_ref_id}'; expected ':bus/path'" + )); + }; + let (name, path) = object_ref_id.split_at(path_start); + if name.is_empty() || path.is_empty() { + return Err(anyhow!( + "invalid AT-SPI object ref '{object_ref_id}'; expected ':bus/path'" + )); + } + Ok((name, path)) +} + +pub fn object_ref_from_id(object_ref_id: &str) -> Result { + let (name, path) = split_object_ref_id(object_ref_id)?; + let name = UniqueName::try_from(name.to_string()) + .with_context(|| format!("invalid AT-SPI bus name in object ref {object_ref_id}"))?; + let path = ObjectPath::try_from(path.to_string()) + .with_context(|| format!("invalid AT-SPI object path in object ref {object_ref_id}"))?; + Ok(ObjectRef::new_owned(name, path)) +} + +pub fn object_ref_id(object_ref: &ObjectRefOwned) -> String { + format!( + "{}{}", + object_ref.name_as_str().unwrap_or(""), + object_ref.path_as_str() + ) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn make_action(name: &str, description: &str) -> atspi::Action { + atspi::Action { + name: name.to_string(), + description: description.to_string(), + keybinding: String::new(), + } + } + + #[test] + fn test_split_object_ref_id_valid() { + let (name, path) = split_object_ref_id(":1.42/org/a11y/atspi/accessible/root").unwrap(); + assert_eq!(name, ":1.42"); + assert_eq!(path, "/org/a11y/atspi/accessible/root"); + } + + #[test] + fn test_split_object_ref_id_invalid() { + assert!(split_object_ref_id("invalid").is_err()); + assert!(split_object_ref_id("/only/path").is_err()); + assert!(split_object_ref_id(":busonly").is_err()); + } + + #[test] + fn test_select_action_index_empty_actions() { + let actions = vec![]; + let err = select_action_index(&actions, Some("click")).unwrap_err(); + assert!(err.to_string().contains("expose aucune action")); + } + + #[test] + fn test_select_action_index_single_action() { + let actions = vec![make_action("press", "Presses the button")]; + assert_eq!(select_action_index(&actions, None).unwrap(), 0); + assert_eq!(select_action_index(&actions, Some("press")).unwrap(), 0); + assert_eq!(select_action_index(&actions, Some("click")).unwrap(), 0); + assert_eq!(select_action_index(&actions, Some("0")).unwrap(), 0); + } + + #[test] + fn test_select_action_index_single_unrelated_action_rejected_for_click() { + let actions = vec![make_action("delete", "Delete item")]; + assert_eq!(select_action_index(&actions, None).unwrap(), 0); + assert_eq!(select_action_index(&actions, Some("delete")).unwrap(), 0); + assert_eq!(select_action_index(&actions, Some("0")).unwrap(), 0); + assert!(select_action_index(&actions, Some("click")).is_err()); + assert!(select_action_index(&actions, Some("press")).is_err()); + } + + #[test] + fn test_select_action_index_exact_match() { + let actions = vec![ + make_action("show-menu", "Show context menu"), + make_action("activate", "Activate the primary item"), + make_action("delete", "Delete item"), + ]; + assert_eq!(select_action_index(&actions, Some("delete")).unwrap(), 2); + assert_eq!(select_action_index(&actions, Some("activate")).unwrap(), 1); + assert_eq!(select_action_index(&actions, Some("show-menu")).unwrap(), 0); + } + + #[test] + fn test_select_action_index_numeric_index() { + let actions = vec![ + make_action("action-a", "First"), + make_action("action-99", "Second"), + ]; + assert_eq!(select_action_index(&actions, Some("1")).unwrap(), 1); + assert_eq!(select_action_index(&actions, Some("0")).unwrap(), 0); + // "99" est un index numérique hors limites (taille 2) : il doit échouer immédiatement + // sans tomber dans la recherche de sous-chaîne pour "action-99". + let err = select_action_index(&actions, Some("99")).unwrap_err(); + assert!(err.to_string().contains("hors limites")); + // En revanche, nommer explicitement "action-99" doit correspondre exactement. + assert_eq!(select_action_index(&actions, Some("action-99")).unwrap(), 1); + } + + #[test] + fn test_select_action_index_ambiguous_rejected() { + let actions = vec![ + make_action("custom_first", "First operation"), + make_action("custom_second", "Second operation"), + ]; + // En l'absence d'action "activate/click" standard et avec 2 actions personnalisées, + // une demande générique "click" ou None DOIT échouer et lister les choix disponibles. + assert!(select_action_index(&actions, Some("click")).is_err()); + assert!(select_action_index(&actions, None).is_err()); + } + + #[test] + fn test_select_action_index_duplicate_exact_names_rejected() { + let actions = vec![ + make_action("duplicate_action", "First variant"), + make_action("duplicate_action", "Second variant"), + ]; + let err = select_action_index(&actions, Some("duplicate_action")).unwrap_err(); + assert!(err + .to_string() + .contains("Plusieurs actions correspondent exactement")); + } +} diff --git a/linux/crates/atspi_mediator/src/main.rs b/linux/crates/atspi_mediator/src/main.rs new file mode 100644 index 0000000..4e7a502 --- /dev/null +++ b/linux/crates/atspi_mediator/src/main.rs @@ -0,0 +1,696 @@ +//! Point d'entrée CLI et serveur MCP pour le médiateur AT-SPI gui_agent. + +use anyhow::{anyhow, Result}; +use atspi_mediator::{ + connect, hydrate_session_bus_env, list_accessible_apps, perform_action, set_element_value, + snapshot_tree, AccessibilityNode, +}; +use serde_json::Value; +use std::env; +use std::sync::Arc; +use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader}; +use tokio::sync::Mutex; + +async fn resolve_ref( + object_ref_or_index: &str, + cached: Option<&[AccessibilityNode]>, +) -> Result { + if let Ok(idx) = object_ref_or_index.parse::() { + if let Some(nodes) = cached { + return nodes + .iter() + .find(|n| n.index == idx) + .map(|n| n.object_ref.clone()) + .ok_or_else(|| { + anyhow!("Aucun nœud d'accessibilité trouvé dans le cache pour l'index {idx}") + }); + } + Err(anyhow!( + "L'index numérique '{idx}' nécessite un cache de snapshot actif. Veuillez fournir une référence AT-SPI explicite (ex: ':1.42/path')." + )) + } else { + Ok(object_ref_or_index.to_string()) + } +} + +#[tokio::main] +async fn main() -> Result<()> { + hydrate_session_bus_env(); + let args: Vec = env::args().collect(); + let command = args.get(1).map(String::as_str).unwrap_or("help"); + + match command { + "state" => { + let app_name = args.get(2).map(String::as_str); + match snapshot_tree(app_name, None, 1000, 32).await { + Ok(nodes) => { + let out = serde_json::json!({ + "status": "success", + "count": nodes.len(), + "tree": nodes, + }); + println!("{}", serde_json::to_string(&out)?); + } + Err(err) => { + let out = serde_json::json!({ + "status": "error", + "error": format!("{err:#}"), + }); + eprintln!("{}", serde_json::to_string(&out)?); + std::process::exit(1); + } + } + } + "action" => { + let Some(target) = args.get(2) else { + eprintln!("Usage: gui-agent-atspi action [action_name]"); + std::process::exit(1); + }; + let action_name = args.get(3).map(String::as_str); + let object_ref = match resolve_ref(target, None).await { + Ok(r) => r, + Err(err) => { + let out = serde_json::json!({ + "status": "error", + "error": format!("{err:#}"), + }); + eprintln!("{}", serde_json::to_string(&out)?); + std::process::exit(1); + } + }; + match perform_action(&object_ref, action_name).await { + Ok(inv) => { + let out = serde_json::json!({ + "status": "success", + "action_index": inv.action_index, + "action_name": inv.action_name, + "ok": inv.ok, + }); + println!("{}", serde_json::to_string(&out)?); + } + Err(err) => { + let out = serde_json::json!({ + "status": "error", + "error": format!("{err:#}"), + }); + eprintln!("{}", serde_json::to_string(&out)?); + std::process::exit(1); + } + } + } + "value" => { + let Some(target) = args.get(2) else { + eprintln!("Usage: gui-agent-atspi value "); + std::process::exit(1); + }; + let Some(val) = args.get(3) else { + eprintln!("Usage: gui-agent-atspi value "); + std::process::exit(1); + }; + let object_ref = match resolve_ref(target, None).await { + Ok(r) => r, + Err(err) => { + let out = serde_json::json!({ + "status": "error", + "error": format!("{err:#}"), + }); + eprintln!("{}", serde_json::to_string(&out)?); + std::process::exit(1); + } + }; + match set_element_value(&object_ref, val).await { + Ok(inv) => { + let out = serde_json::json!({ + "status": "success", + "result": format!("{inv:?}"), + "ok": true, + }); + println!("{}", serde_json::to_string(&out)?); + } + Err(err) => { + let out = serde_json::json!({ + "status": "error", + "error": format!("{err:#}"), + "ok": false, + }); + eprintln!("{}", serde_json::to_string(&out)?); + std::process::exit(1); + } + } + } + "apps" => match list_accessible_apps(50).await { + Ok(apps) => { + let out = serde_json::json!({ + "status": "success", + "apps": apps, + }); + println!("{}", serde_json::to_string(&out)?); + } + Err(err) => { + let out = serde_json::json!({ + "status": "error", + "error": format!("{err:#}"), + }); + eprintln!("{}", serde_json::to_string(&out)?); + std::process::exit(1); + } + }, + "doctor" => match connect().await { + Ok(_) => { + let out = serde_json::json!({ + "status": "success", + "at_spi_available": true, + }); + println!("{}", serde_json::to_string(&out)?); + } + Err(err) => { + let out = serde_json::json!({ + "status": "error", + "at_spi_available": false, + "error": format!("{err:#}"), + }); + println!("{}", serde_json::to_string(&out)?); + std::process::exit(1); + } + }, + "mcp" => { + run_mcp_server().await?; + } + _ => { + eprintln!("gui-agent-atspi: AT-SPI2 / D-Bus accessibility bridge CLI & MCP server"); + eprintln!("Commandes disponibles : state, action, value, apps, doctor, mcp"); + std::process::exit(1); + } + } + + Ok(()) +} + +#[derive(Clone)] +struct SnapshotState { + token: String, + nodes: Vec, +} + +fn resolve_mcp_target( + arguments: &Value, + cache: &Option, +) -> Result { + let target_ident = arguments.get("element_identifier").and_then(Value::as_str); + let target_index = arguments.get("element_index").and_then(Value::as_u64); + let snapshot_token = arguments.get("snapshot_token").and_then(Value::as_str); + + // 1. Si un identifiant explicite non-numérique est fourni (ex: ":1.42/path"), il a priorité absolue + if let Some(ident) = target_ident { + let trimmed = ident.trim(); + if !trimmed.is_empty() && trimmed.parse::().is_err() { + return Ok(trimmed.to_string()); + } + } + + let numeric_idx = match target_index { + Some(i) => Some(u32::try_from(i).map_err(|_| ( + serde_json::json!({ + "status": "error", + "error": "element_index hors plage (doit tenir dans un entier 32 bits non signé).", + "ok": false + }), + true, + ))?), + None => target_ident.and_then(|id| id.trim().parse::().ok()), + }; + + if let Some(idx) = numeric_idx { + let Some(snap) = cache else { + return Err(( + serde_json::json!({ + "status": "error", + "error": "Aucun snapshot actif. Veuillez d'abord appeler get_app_state.", + "ok": false + }), + true, + )); + }; + let Some(token) = snapshot_token else { + return Err(( + serde_json::json!({ + "status": "error", + "error": "L'utilisation d'un index numérique (element_index) requiert le paramètre snapshot_token pour éviter d'agir sur un contrôle obsolète.", + "ok": false + }), + true, + )); + }; + if token != snap.token { + return Err(( + serde_json::json!({ + "status": "error", + "error": format!("snapshot_token périmé ('{token}' != '{}'). Veuillez rafraîchir get_app_state.", snap.token), + "ok": false + }), + true, + )); + } + let node = snap.nodes.iter().find(|n| n.index == idx); + let Some(node) = node else { + return Err(( + serde_json::json!({ + "status": "error", + "error": format!("Index {idx} introuvable dans le snapshot actuel ({})", snap.token), + "ok": false + }), + true, + )); + }; + Ok(node.object_ref.clone()) + } else if let Some(ident) = target_ident { + if ident.is_empty() { + Err(( + serde_json::json!({ + "status": "error", + "error": "L'identifiant d'élément (element_identifier) ne peut pas être vide.", + "ok": false + }), + true, + )) + } else { + Ok(ident.to_string()) + } + } else { + Err(( + serde_json::json!({ + "status": "error", + "error": "Paramètres element_index ou element_identifier requis.", + "ok": false + }), + true, + )) + } +} + +async fn run_mcp_server() -> Result<()> { + let stdin = tokio::io::stdin(); + let mut stdout = tokio::io::stdout(); + let mut reader = BufReader::new(stdin).lines(); + let cached_snapshot: Arc>> = Arc::new(Mutex::new(None)); + let snapshot_seq = Arc::new(std::sync::atomic::AtomicU64::new(1)); + + loop { + let line = match reader.next_line().await { + Ok(Some(line)) => line, + Ok(None) => break, + Err(err) => { + eprintln!("gui-agent-atspi: erreur de lecture stdin : {err}"); + break; + } + }; + let trimmed = line.trim(); + if trimmed.is_empty() { + continue; + } + + let parsed: Value = match serde_json::from_str(trimmed) { + Ok(v) => v, + Err(_) => { + let err_resp = serde_json::json!({ + "jsonrpc": "2.0", + "id": Value::Null, + "error": { + "code": -32700, + "message": "Parse error" + } + }); + stdout + .write_all(format!("{}\n", serde_json::to_string(&err_resp)?).as_bytes()) + .await?; + stdout.flush().await?; + continue; + } + }; + + if !parsed.is_object() { + let err_resp = serde_json::json!({ + "jsonrpc": "2.0", + "id": Value::Null, + "error": { + "code": -32600, + "message": "Invalid Request" + } + }); + stdout + .write_all(format!("{}\n", serde_json::to_string(&err_resp)?).as_bytes()) + .await?; + stdout.flush().await?; + continue; + } + + let method = parsed.get("method").and_then(Value::as_str).unwrap_or(""); + let id = parsed.get("id").cloned(); + + // Ignorer les notifications sans id (ex: notifications/initialized) + if id.is_none() || id == Some(Value::Null) { + continue; + } + + match method { + "initialize" => { + let resp = serde_json::json!({ + "jsonrpc": "2.0", + "id": id, + "result": { + "protocolVersion": "2024-11-05", + "capabilities": { + "tools": {} + }, + "serverInfo": { + "name": "gui-agent-atspi", + "version": "0.1.0" + } + } + }); + stdout + .write_all(format!("{}\n", serde_json::to_string(&resp)?).as_bytes()) + .await?; + stdout.flush().await?; + } + "tools/list" => { + let resp = serde_json::json!({ + "jsonrpc": "2.0", + "id": id, + "result": { + "tools": [ + { + "name": "get_app_state", + "description": "Extracts the accessibility tree for a given desktop application or desktop root", + "inputSchema": { + "type": "object", + "properties": { + "app_name": { + "type": "string", + "description": "Optional application name or window title to filter by" + } + } + } + }, + { + "name": "perform_action", + "description": "Performs an action on an accessible element identified by element_identifier or element_index", + "inputSchema": { + "type": "object", + "properties": { + "element_identifier": { + "type": "string", + "description": "D-Bus object path of the accessible element" + }, + "element_index": { + "type": "integer", + "description": "Numeric index from the latest get_app_state snapshot" + }, + "action": { + "type": "string", + "description": "Action name to perform, defaults to activate" + }, + "snapshot_token": { + "type": "string", + "description": "Snapshot token returned by get_app_state (required when targeting by element_index)" + } + } + } + }, + { + "name": "set_value", + "description": "Sets the text content or numeric value of an accessible element", + "inputSchema": { + "type": "object", + "properties": { + "element_identifier": { + "type": "string", + "description": "D-Bus object path of the accessible element" + }, + "element_index": { + "type": "integer", + "description": "Numeric index from the latest get_app_state snapshot" + }, + "snapshot_token": { + "type": "string", + "description": "Snapshot token returned by get_app_state (required when targeting by element_index)" + }, + "value": { + "type": "string", + "description": "Value to assign to the accessible element" + } + }, + "required": ["value"] + } + } + ] + } + }); + stdout + .write_all(format!("{}\n", serde_json::to_string(&resp)?).as_bytes()) + .await?; + stdout.flush().await?; + } + "tools/call" => { + let params = parsed.get("params").cloned().unwrap_or(Value::Null); + let tool_name = params.get("name").and_then(Value::as_str).unwrap_or(""); + let arguments = params.get("arguments").cloned().unwrap_or(Value::Null); + + let (result_data, is_error) = match tool_name { + "get_app_state" => { + let app_name = arguments.get("app_name").and_then(Value::as_str); + match snapshot_tree(app_name, None, 1000, 32).await { + Ok(nodes) => { + let token = format!( + "snap-{}", + snapshot_seq.fetch_add(1, std::sync::atomic::Ordering::SeqCst) + ); + let mut cache = cached_snapshot.lock().await; + *cache = Some(SnapshotState { + token: token.clone(), + nodes: nodes.clone(), + }); + ( + serde_json::json!({ + "status": "success", + "count": nodes.len(), + "tree": nodes, + "snapshot_token": token, + "ok": true + }), + false, + ) + } + Err(err) => ( + serde_json::json!({ + "status": "error", + "error": format!("{err:#}"), + "ok": false + }), + true, + ), + } + } + "perform_action" => { + let action_name = arguments.get("action").and_then(Value::as_str); + let cache = cached_snapshot.lock().await; + let target_res = resolve_mcp_target(&arguments, &cache); + drop(cache); + + match target_res { + Ok(resolved) => match perform_action(&resolved, action_name).await { + Ok(inv) => ( + serde_json::json!({ + "status": "success", + "ok": inv.ok, + "action_index": inv.action_index, + "action_name": inv.action_name + }), + !inv.ok, + ), + Err(err) => ( + serde_json::json!({ + "status": "error", + "error": format!("{err:#}"), + "ok": false + }), + true, + ), + }, + Err(err_tuple) => err_tuple, + } + } + "set_value" => { + let val = arguments.get("value").and_then(Value::as_str).unwrap_or(""); + let cache = cached_snapshot.lock().await; + let target_res = resolve_mcp_target(&arguments, &cache); + drop(cache); + + match target_res { + Ok(resolved) => match set_element_value(&resolved, val).await { + Ok(inv) => ( + serde_json::json!({ + "status": "success", + "ok": true, + "result": format!("{inv:?}") + }), + false, + ), + Err(err) => ( + serde_json::json!({ + "status": "error", + "error": format!("{err:#}"), + "ok": false + }), + true, + ), + }, + Err(err_tuple) => err_tuple, + } + } + _ => ( + serde_json::json!({ + "status": "error", + "error": format!("Outil inconnu : {tool_name}"), + "ok": false + }), + true, + ), + }; + + let resp = serde_json::json!({ + "jsonrpc": "2.0", + "id": id, + "result": { + "isError": is_error, + "structuredContent": result_data, + "content": [ + { + "type": "text", + "text": serde_json::to_string(&result_data).unwrap_or_default() + } + ] + } + }); + stdout + .write_all(format!("{}\n", serde_json::to_string(&resp)?).as_bytes()) + .await?; + stdout.flush().await?; + } + _ => { + if let Some(id_val) = id { + let resp = serde_json::json!({ + "jsonrpc": "2.0", + "id": id_val, + "error": { + "code": -32601, + "message": format!("Method not found: {method}") + } + }); + stdout + .write_all(format!("{}\n", serde_json::to_string(&resp)?).as_bytes()) + .await?; + stdout.flush().await?; + } + } + } + } + + Ok(()) +} + +#[cfg(test)] +mod tests { + use super::*; + use serde_json::json; + + #[test] + fn test_resolve_mcp_target_explicit_identifier_priority() { + let args = json!({ + "element_identifier": ":1.42/org/a11y/root", + "element_index": 4294967296u64, + }); + let res = resolve_mcp_target(&args, &None); + assert_eq!(res.unwrap(), ":1.42/org/a11y/root"); + } + + #[test] + fn test_resolve_mcp_target_element_index_overflow_rejected() { + let args = json!({ + "element_index": 4294967296u64, + "snapshot_token": "token_123" + }); + let res = resolve_mcp_target(&args, &None); + assert!(res.is_err()); + let (err_val, is_err) = res.unwrap_err(); + assert!(is_err); + assert_eq!( + err_val["error"], + "element_index hors plage (doit tenir dans un entier 32 bits non signé)." + ); + } + + #[test] + fn test_resolve_mcp_target_element_index_valid() { + let node: AccessibilityNode = serde_json::from_value(json!({ + "index": 42, + "parent_index": null, + "depth": 0, + "object_ref": ":1.42/button_42", + "role": "push button", + "name": "Test Button", + "description": null, + "child_count": 0, + "bounds": null, + "states": ["visible"], + "actions": [], + "value": null, + "text": null, + "supports_editable_text": false + })) + .unwrap(); + + let cache = Some(SnapshotState { + token: "tok42".to_string(), + nodes: vec![node], + }); + let args = json!({ + "element_index": 42, + "snapshot_token": "tok42" + }); + let res = resolve_mcp_target(&args, &cache); + assert_eq!(res.unwrap(), ":1.42/button_42"); + } + + #[test] + fn test_resolve_mcp_target_numeric_identifier_fallback() { + let node: AccessibilityNode = serde_json::from_value(json!({ + "index": 5, + "parent_index": null, + "depth": 0, + "object_ref": ":1.42/item_5", + "role": "list item", + "name": "Item", + "description": null, + "child_count": 0, + "bounds": null, + "states": [], + "actions": [], + "value": null, + "text": null, + "supports_editable_text": false + })) + .unwrap(); + + let cache = Some(SnapshotState { + token: "tok5".to_string(), + nodes: vec![node], + }); + let args = json!({ + "element_identifier": "5", + "snapshot_token": "tok5" + }); + let res = resolve_mcp_target(&args, &cache); + assert_eq!(res.unwrap(), ":1.42/item_5"); + } +} diff --git a/linux/examples/capture_to_artifacts.py b/linux/examples/capture_to_artifacts.py new file mode 100644 index 0000000..37b9506 --- /dev/null +++ b/linux/examples/capture_to_artifacts.py @@ -0,0 +1,48 @@ +#!/usr/bin/env python3 +import os +import sys +import shutil + +_ROOT_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) +sys.path.insert(0, _ROOT_DIR) + +try: + import mcp_gui_server +except ImportError as err: + print(f"Erreur d'importation : {err}", file=sys.stderr) + sys.exit(1) + + +def main(): + print("Prise de la capture d'écran...") + res = mcp_gui_server.gui_take_screenshot(apply_grid=True, grid_interval=100) + if res.get("status") != "success": + print(f"Erreur : {res.get('message')}") + sys.exit(1) + + scr_path = res.get("screenshot_path") + if not scr_path or not os.path.isfile(scr_path): + print("Erreur : Fichier de capture d'écran introuvable dans le retour.") + sys.exit(1) + + try: + try: + from paths import get_screenshots_dir + except ImportError: + from linux.paths import get_screenshots_dir + + fallback_artifacts = os.path.join(os.path.dirname(get_screenshots_dir()), "artifacts") + except Exception: + fallback_artifacts = os.path.join(os.getcwd(), "artifacts") + + dest_dir = sys.argv[1] if len(sys.argv) > 1 else os.environ.get("GUI_AGENT_ARTIFACTS_DIR", fallback_artifacts) + dest_path = os.path.join(dest_dir, "screenshot_xboard.png") + + os.makedirs(dest_dir, exist_ok=True) + shutil.copy(scr_path, dest_path) + print(f"Capture enregistrée et copiée vers : {dest_path}") + print(f"Dimensions : {res.get('width')}x{res.get('height')}") + + +if __name__ == "__main__": + main() diff --git a/examples/test_e2e_validation.py b/linux/examples/test_e2e_validation.py similarity index 98% rename from examples/test_e2e_validation.py rename to linux/examples/test_e2e_validation.py index c2a4200..dd70c59 100644 --- a/examples/test_e2e_validation.py +++ b/linux/examples/test_e2e_validation.py @@ -7,9 +7,12 @@ - Brique 4 : Pont Hybride Web (Playwright) & Enregistrement Vidéo FFmpeg """ +import asyncio import os +import sys import time -import asyncio + +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) # Import direct des fonctions exposées du serveur MCP GUI from mcp_gui_server import ( diff --git a/examples/test_evolutions.py b/linux/examples/test_evolutions.py similarity index 96% rename from examples/test_evolutions.py rename to linux/examples/test_evolutions.py index c370d95..d08adba 100644 --- a/examples/test_evolutions.py +++ b/linux/examples/test_evolutions.py @@ -4,7 +4,7 @@ import tempfile from PIL import Image -sys.path.append(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) import mcp_gui_server @@ -65,6 +65,7 @@ def run_evolution_tests(): match_x = res_match.get("x") match_y = res_match.get("y") confidence = res_match.get("confidence") + assert confidence is not None, "Confiance de matching absente" print(f"Modèle trouvé aux coordonnées : ({match_x}, {match_y}) avec confiance {confidence:.4f}") assert match_x is not None and match_x > 0 and match_y is not None and match_y > 0, ( diff --git a/examples/test_gui_agent.py b/linux/examples/test_gui_agent.py similarity index 75% rename from examples/test_gui_agent.py rename to linux/examples/test_gui_agent.py index b8193e7..7e02881 100755 --- a/examples/test_gui_agent.py +++ b/linux/examples/test_gui_agent.py @@ -4,7 +4,7 @@ import time # Permettre l'importation de mcp_gui_server -sys.path.append(os.path.dirname(os.path.abspath(__file__))) +sys.path.insert(0, os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) try: import mcp_gui_server @@ -28,16 +28,22 @@ def run_integration_test(): width = screen_info.get("width") height = screen_info.get("height") - assert width > 0 and height > 0, "Assertion échouée : Résolution invalide !" + if not isinstance(width, int) or not isinstance(height, int) or width <= 0 or height <= 0: + print(f"Échec : Résolution invalide ({width}x{height}) !", file=sys.stderr) + sys.exit(1) # 2. Prendre une capture d'écran initiale avec grille de coordonnées print("\n[Étape 2] Prise de capture d'écran initiale (avec grille)...") screenshot_res = mcp_gui_server.gui_take_screenshot(apply_grid=True, grid_interval=150) print(f"Résultat : {screenshot_res}") - assert screenshot_res.get("status") == "success", "Assertion échouée : La capture a échoué !" + if screenshot_res.get("status") != "success": + print("Échec : La capture a échoué !", file=sys.stderr) + sys.exit(1) screenshot_path = screenshot_res.get("screenshot_path") - assert os.path.exists(screenshot_path), f"Assertion échouée : Fichier image absent ({screenshot_path}) !" + if not screenshot_path or not os.path.exists(screenshot_path): + print(f"Échec : Fichier image absent ({screenshot_path}) !", file=sys.stderr) + sys.exit(1) print(f"Capture enregistrée à : {screenshot_path}") # 3. Simulation sécurisée de mouvements de souris au centre de l'écran @@ -51,7 +57,9 @@ def run_integration_test(): for ox, oy in move_offsets: res = mcp_gui_server.gui_mouse_move(ox, oy, duration=0.15) print(f" Déplacement vers ({ox}, {oy}) -> {res.get('status')}") - assert res.get("status") == "success", "Assertion échouée : Le mouvement a échoué !" + if res.get("status") != "success": + print("Échec : Le mouvement a échoué !", file=sys.stderr) + sys.exit(1) time.sleep(0.1) # 4. Simulation de raccourci clavier sécurisé @@ -76,7 +84,13 @@ def run_integration_test(): print("\n=== TEST DE L'AGENT GUI TERMINÉ AVEC SUCCÈS ===") print("Vous pouvez inspecter les captures d'écran générées dans le dossier :") - print(" /home/omni/Code/gui_agent/screenshots/") + try: + from linux.paths import get_screenshots_dir + + screenshots_dir = get_screenshots_dir() + except Exception: + screenshots_dir = os.path.dirname(screenshot_path) if screenshot_path else "screenshots/" + print(f" {screenshots_dir}") if __name__ == "__main__": diff --git a/linux/input.py b/linux/input.py new file mode 100644 index 0000000..712800b --- /dev/null +++ b/linux/input.py @@ -0,0 +1,19 @@ +"""Émulation d'entrées clavier et souris pour Linux (X11 / uinput / evdev).""" + +from __future__ import annotations + +from .layers.input_emulation import ( + key_tap, + mouse_click_at, + mouse_drag_smooth, + mouse_scroll, + screen_capture, +) + +__all__ = [ + "key_tap", + "mouse_click_at", + "mouse_drag_smooth", + "mouse_scroll", + "screen_capture", +] diff --git a/install.sh b/linux/install.sh similarity index 55% rename from install.sh rename to linux/install.sh index 0d7cc0c..1b6441a 100755 --- a/install.sh +++ b/linux/install.sh @@ -50,7 +50,7 @@ print_banner() { show_help() { print_banner - echo "Usage: ./install.sh [OPTIONS]" + echo "Usage: ./linux/install.sh [OPTIONS]" echo "" echo "Options:" echo " -y, --yes, --no-prompt Non-interactive mode (accept all defaults)" @@ -131,17 +131,18 @@ fi python3 -c "import dbus" >/dev/null 2>&1 || MISSING_SYS_DEPS+=("python3-dbus") python3 -c "import tkinter" >/dev/null 2>&1 || MISSING_SYS_DEPS+=("python3-tk") pkg-config --exists atspi-2 2>/dev/null || [ -d "/usr/include/at-spi-2.0" ] || command -v at-spi-bus-launcher >/dev/null 2>&1 || MISSING_SYS_DEPS+=("at-spi2-core") +command -v cargo >/dev/null 2>&1 || MISSING_SYS_DEPS+=("cargo") if [[ ${#MISSING_SYS_DEPS[@]} -gt 0 ]]; then log_warn "Dépendances système manquantes détectées : ${MISSING_SYS_DEPS[*]}" # Détection du gestionnaire de paquets INSTALL_CMD="" if command -v apt-get >/dev/null 2>&1; then - INSTALL_CMD="sudo apt-get update && sudo apt-get install -y xdotool wmctrl spectacle ffmpeg xclip tesseract-ocr python3-dbus at-spi2-core python3-tk" + INSTALL_CMD="sudo apt-get update && sudo apt-get install -y xdotool wmctrl spectacle ffmpeg xclip tesseract-ocr python3-dbus at-spi2-core python3-tk cargo" elif command -v dnf >/dev/null 2>&1; then - INSTALL_CMD="sudo dnf install -y xdotool wmctrl spectacle ffmpeg xclip tesseract python3-dbus at-spi2-core python3-tkinter" + INSTALL_CMD="sudo dnf install -y xdotool wmctrl spectacle ffmpeg xclip tesseract python3-dbus at-spi2-core python3-tkinter cargo" elif command -v pacman >/dev/null 2>&1; then - INSTALL_CMD="sudo pacman -S --needed xdotool wmctrl spectacle ffmpeg xclip tesseract python-dbus at-spi2-core tk" + INSTALL_CMD="sudo pacman -S --needed --noconfirm xdotool wmctrl spectacle ffmpeg xclip tesseract python-dbus at-spi2-core tk rust" fi if [[ -n "$INSTALL_CMD" ]]; then @@ -150,9 +151,12 @@ if [[ ${#MISSING_SYS_DEPS[@]} -gt 0 ]]; then elif [[ "$NON_INTERACTIVE" == "true" ]]; then log_info "Installation automatique des dépendances système..." eval "$INSTALL_CMD" || log_warn "Échec d'installation avec sudo, poursuite de l'installation..." + elif [[ ! -t 0 ]]; then + log_warn "Entrée standard non interactive sans flag -y/--yes. Poursuite sans installer les paquets système." else echo -e "${YELLOW}Voulez-vous installer automatiquement les paquets système manquants ? [O/n]${NC} " - read -r response + response="" + read -r response || response="n" if [[ "$response" =~ ^([oO][uU][iI]|[oO]|"")$ ]]; then eval "$INSTALL_CMD" || log_warn "Échec lors de l'installation automatique. Poursuite..." else @@ -198,11 +202,25 @@ log_info "4/5 - Installation isolée du package '$PACKAGE_NAME' via 'uv tool ins if [[ "$DRY_RUN" == "true" ]]; then log_info "[Dry-Run] uv tool install --force $PACKAGE_NAME" + log_info "[Dry-Run] cargo build --release --manifest-path crates/atspi_mediator/Cargo.toml (si cargo est présent)" else - SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" - if [[ "$LOCAL_INSTALL" == "true" ]] || [[ -f "${SCRIPT_DIR}/pyproject.toml" && -d "${SCRIPT_DIR}/gui_agent" ]]; then - log_info "Installation locale depuis : $SCRIPT_DIR" - "$UV_PATH" tool install "$SCRIPT_DIR" --force + SCRIPT_DIR="" + if [[ -n "${BASH_SOURCE[0]:-}" && -f "${BASH_SOURCE[0]}" ]]; then + SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" + fi + + PROJECT_DIR="" + if [[ -n "$SCRIPT_DIR" ]]; then + if [[ -f "${SCRIPT_DIR}/pyproject.toml" ]]; then + PROJECT_DIR="$SCRIPT_DIR" + elif [[ -f "$(dirname "$SCRIPT_DIR")/pyproject.toml" ]]; then + PROJECT_DIR="$(dirname "$SCRIPT_DIR")" + fi + fi + + if [[ "$LOCAL_INSTALL" == "true" && -n "$PROJECT_DIR" ]] || [[ -n "$PROJECT_DIR" && -f "${PROJECT_DIR}/pyproject.toml" ]]; then + log_info "Installation locale depuis : $PROJECT_DIR" + "$UV_PATH" tool install "$PROJECT_DIR" --force else # Tentative depuis PyPI avec fallback sur Git repo log_info "Installation depuis le registre de packages..." @@ -212,9 +230,87 @@ else fi fi + # Compilation et installation du médiateur AT-SPI Rust si Cargo est présent + if command -v cargo >/dev/null 2>&1; then + log_info "Compilation et installation du médiateur natif AT-SPI Rust (gui-agent-atspi)..." + mkdir -p "${HOME}/.local/bin" + if [[ -n "$PROJECT_DIR" && -d "$PROJECT_DIR" ]]; then + mkdir -p "${PROJECT_DIR}/linux/bin" 2>/dev/null || true + fi + + WORKSPACE_CARGO_PATH="${PROJECT_DIR:+$PROJECT_DIR/Cargo.toml}" + CARGO_TOML_PATH="${PROJECT_DIR:+$PROJECT_DIR/linux/crates/atspi_mediator/Cargo.toml}" + if [[ -n "$PROJECT_DIR" && ! -f "$CARGO_TOML_PATH" ]]; then + CARGO_TOML_PATH="${PROJECT_DIR}/crates/atspi_mediator/Cargo.toml" + fi + + TARGET_DIR="${PROJECT_DIR:+$PROJECT_DIR/target}" + if [[ -z "$TARGET_DIR" ]]; then + TARGET_DIR="$(mktemp -d /tmp/gui-agent-cargo-target-XXXXXX)" + fi + mkdir -p "$TARGET_DIR" + + if [[ -n "$WORKSPACE_CARGO_PATH" && -f "$WORKSPACE_CARGO_PATH" ]]; then + if CARGO_TARGET_DIR="$TARGET_DIR" cargo build --release --manifest-path "$WORKSPACE_CARGO_PATH" --bin gui-agent-atspi; then + if [[ -f "${TARGET_DIR}/release/gui-agent-atspi" ]]; then + cp "${TARGET_DIR}/release/gui-agent-atspi" "${HOME}/.local/bin/gui-agent-atspi" + if [[ -n "$PROJECT_DIR" && -d "$PROJECT_DIR" ]]; then + cp "${TARGET_DIR}/release/gui-agent-atspi" "${PROJECT_DIR}/linux/bin/gui-agent-atspi" 2>/dev/null || true + chmod +x "${PROJECT_DIR}/linux/bin/gui-agent-atspi" 2>/dev/null || true + fi + chmod +x "${HOME}/.local/bin/gui-agent-atspi" 2>/dev/null || true + log_success "Médiateur AT-SPI natif compilé et installé avec succès dans ~/.local/bin/gui-agent-atspi" + else + log_warn "Binaire compilé introuvable dans ${TARGET_DIR}/release. Le fallback Python/D-Bus sera utilisé." + fi + else + log_warn "Échec de la compilation Cargo du médiateur natif AT-SPI. Le fallback Python/D-Bus sera utilisé." + fi + elif [[ -n "$CARGO_TOML_PATH" && -f "$CARGO_TOML_PATH" ]]; then + if CARGO_TARGET_DIR="$TARGET_DIR" cargo build --release --manifest-path "$CARGO_TOML_PATH"; then + if [[ -f "${TARGET_DIR}/release/gui-agent-atspi" ]]; then + cp "${TARGET_DIR}/release/gui-agent-atspi" "${HOME}/.local/bin/gui-agent-atspi" + if [[ -n "$PROJECT_DIR" && -d "$PROJECT_DIR" ]]; then + cp "${TARGET_DIR}/release/gui-agent-atspi" "${PROJECT_DIR}/linux/bin/gui-agent-atspi" 2>/dev/null || true + chmod +x "${PROJECT_DIR}/linux/bin/gui-agent-atspi" 2>/dev/null || true + fi + chmod +x "${HOME}/.local/bin/gui-agent-atspi" 2>/dev/null || true + log_success "Médiateur AT-SPI natif installé avec succès dans ~/.local/bin/gui-agent-atspi" + else + log_warn "Binaire compilé introuvable après build. Le fallback Python/D-Bus sera utilisé." + fi + else + log_warn "Échec de la compilation Cargo du médiateur natif AT-SPI. Le fallback Python/D-Bus sera utilisé." + fi + else + # Cas d'une installation distante (curl | bash) sans clone local du dépôt + log_info "Dépôt local non détecté, installation du médiateur natif depuis Git (${GIT_REPO_URL})..." + if cargo install --git "${GIT_REPO_URL}" atspi-mediator --root "${HOME}/.local" --force; then + chmod +x "${HOME}/.local/bin/gui-agent-atspi" 2>/dev/null || true + log_success "Médiateur AT-SPI natif installé depuis Git dans ~/.local/bin/gui-agent-atspi" + else + log_warn "Échec de l'installation de gui-agent-atspi depuis Git. L'accessibilité AT-SPI utilisera le fallback Python/D-Bus." + fi + fi + else + log_warn "Cargo (Rust) non détecté : le binaire d'accessibilité 'gui-agent-atspi' ne sera pas compilé." + log_warn "Pour bénéficier de l'accessibilité AT-SPI native : curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh" + fi + # Vérification des commandes installées + INSTALLED_BINS=() if command -v gui-agent >/dev/null 2>&1 || [[ -x "${HOME}/.local/bin/gui-agent" ]]; then - log_success "Exécutables 'gui-agent' et 'mcp-gui-server' installés avec succès dans ~/.local/bin !" + INSTALLED_BINS+=("gui-agent") + fi + if command -v mcp-gui-server >/dev/null 2>&1 || [[ -x "${HOME}/.local/bin/mcp-gui-server" ]]; then + INSTALLED_BINS+=("mcp-gui-server") + fi + if command -v gui-agent-atspi >/dev/null 2>&1 || [[ -x "${HOME}/.local/bin/gui-agent-atspi" ]]; then + INSTALLED_BINS+=("gui-agent-atspi") + fi + + if [[ ${#INSTALLED_BINS[@]} -gt 0 ]]; then + log_success "Exécutables installés avec succès dans ~/.local/bin : ${INSTALLED_BINS[*]}" else log_warn "Les exécutables ont été installés mais ~/.local/bin n'est pas encore dans votre PATH." log_warn "Ajoutez 'export PATH=\"\$HOME/.local/bin:\$PATH\"' dans votre ~/.bashrc ou ~/.zshrc." @@ -231,9 +327,10 @@ else if command -v claude >/dev/null 2>&1; then log_info "Client Claude Code CLI détecté." CONFIGURE_CLAUDE=true - if [[ "$NON_INTERACTIVE" != "true" ]]; then + if [[ "$NON_INTERACTIVE" != "true" && -t 0 ]]; then echo -e "${YELLOW}Voulez-vous enregistrer automatiquement gui-agent dans Claude Code CLI ? [O/n]${NC} " - read -r response + response="" + read -r response || response="n" if [[ ! "$response" =~ ^([oO][uU][iI]|[oO]|"")$ ]]; then CONFIGURE_CLAUDE=false fi @@ -241,8 +338,11 @@ else if [[ "$CONFIGURE_CLAUDE" == "true" ]]; then log_info "Enregistrement dans Claude Code : claude mcp add gui-agent -- gui-agent" - claude mcp add gui-agent -- gui-agent 2>/dev/null || log_warn "Claude MCP add a retourné un statut non nul (le serveur est peut-être déjà configuré)." - log_success "Serveur MCP configuré pour Claude Code." + if claude mcp add gui-agent -- gui-agent 2>/dev/null; then + log_success "Serveur MCP configuré pour Claude Code." + else + log_warn "Claude MCP add a retourné un statut non nul (le serveur est peut-être déjà configuré)." + fi fi fi @@ -254,9 +354,11 @@ else mkdir -p "$ANTIGRAVITY_CONFIG_DIR" # Injection / fusion sécurisée dans mcp_config.json via python - "$UV_PATH" run python - < object_ref synchronisé par verrou +_snapshots: dict[str, dict[str, Any]] = {} +_last_node_cache: dict[str, str] = {} +_last_snapshot_id: str | None = None +_cache_lock = threading.Lock() + + +def _clear_accessibility_cache() -> None: + """Réinitialise l'ensemble des caches de snapshots et de nœuds.""" + global _last_snapshot_id + with _cache_lock: + _snapshots.clear() + _last_node_cache.clear() + _last_snapshot_id = None + + +def set_mock_state(state: dict[str, Any] | None) -> None: + """Configure un état mocké d'accessibilité pour les tests.""" + global _mock_state + _mock_state = state + + +def set_mock_action_handler(handler: Any | None) -> None: + """Configure un gestionnaire mocké pour perform_action.""" + global _mock_action_handler + _mock_action_handler = handler + + +def set_mock_value_handler(handler: Any | None) -> None: + """Configure un gestionnaire mocké pour set_value.""" + global _mock_value_handler + _mock_value_handler = handler + + +def _is_elf_binary(path: str) -> bool: + """Vérifie si le fichier cible est un exécutable binaire natif ELF compatible avec l'hôte.""" + try: + with open(path, "rb") as f: + header = f.read(20) + if len(header) < 20 or header[:4] != b"\x7fELF": + return False + + ei_data = header[5] # 1 = little-endian, 2 = big-endian + endian = "<" if ei_data == 1 else ">" + e_machine = struct.unpack(endian + "H", header[18:20])[0] + + host = platform.machine().lower() + expected_machines: dict[str, set[int]] = { + "x86_64": {0x3E}, + "amd64": {0x3E}, + "aarch64": {0xB7}, + "arm64": {0xB7}, + "i386": {0x03}, + "i686": {0x03}, + "x86": {0x03}, + "arm": {0x28}, + "armv7l": {0x28}, + "riscv64": {0xF3}, + } + allowed = expected_machines.get(host) + return allowed is None or e_machine in allowed + except (OSError, struct.error): + return False + + +def find_atspi_mediator_binary(exclude_scripts: bool = False) -> str | None: + """Détecte l'exécutable du moteur AT-SPI disponible.""" + # 1. Variable d'environnement prioritaire + custom_bin = os.environ.get("GUI_AGENT_ATSPI_BIN") + if ( + custom_bin + and os.path.isfile(custom_bin) + and os.access(custom_bin, os.X_OK) + and (not exclude_scripts or _is_elf_binary(custom_bin)) + ): + return custom_bin + + # 2. Binaire local compilé dans workspace root ou crates/atspi_mediator (release puis debug) + project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + for profile in ("release", "debug"): + workspace_bin = os.path.join(project_root, "target", profile, "gui-agent-atspi") + if ( + os.path.isfile(workspace_bin) + and os.access(workspace_bin, os.X_OK) + and (not exclude_scripts or _is_elf_binary(workspace_bin)) + ): + return workspace_bin + for candidate_crate in ( + os.path.join(project_root, "linux", "crates", "atspi_mediator", "target", profile, "gui-agent-atspi"), + os.path.join(project_root, "crates", "atspi_mediator", "target", profile, "gui-agent-atspi"), + ): + if ( + os.path.isfile(candidate_crate) + and os.access(candidate_crate, os.X_OK) + and (not exclude_scripts or _is_elf_binary(candidate_crate)) + ): + return candidate_crate + + # 2b. Binaire natif packagé directement dans le package Python (gui_agent/bin/gui-agent-atspi) + package_bin = os.path.join(os.path.dirname(os.path.dirname(os.path.abspath(__file__))), "bin", "gui-agent-atspi") + if ( + os.path.isfile(package_bin) + and os.access(package_bin, os.X_OK) + and (not exclude_scripts or _is_elf_binary(package_bin)) + ): + return package_bin + + # 3. Exécutable système standard dans ~/.local/bin ou préfixe d'environnement Python + standard_paths = [ + os.path.expanduser("~/.local/bin/gui-agent-atspi"), + os.path.join(sys.prefix, "bin", "gui-agent-atspi"), + ] + for candidate in standard_paths: + if ( + os.path.isfile(candidate) + and os.access(candidate, os.X_OK) + and (not exclude_scripts or _is_elf_binary(candidate)) + ): + return candidate + + # 4. Exécutable système 'gui-agent-atspi' dans le PATH + which_agent = shutil.which("gui-agent-atspi") + if which_agent and (not exclude_scripts or _is_elf_binary(which_agent)): + return which_agent + + # 5. Exécutable système computer-use-linux (Rust upstream) + which_bin = shutil.which("computer-use-linux") + if which_bin and (not exclude_scripts or _is_elf_binary(which_bin)): + return which_bin + + # 6. Chemins NVM éventuels dans le répertoire utilisateur + user_home = os.path.expanduser("~") + nvm_dir = os.path.join(user_home, ".nvm", "versions", "node") + if os.path.isdir(nvm_dir): + try: + for node_ver in sorted(os.listdir(nvm_dir), reverse=True): + candidate = os.path.join(nvm_dir, node_ver, "bin", "computer-use-linux") + if ( + os.path.isfile(candidate) + and os.access(candidate, os.X_OK) + and (not exclude_scripts or _is_elf_binary(candidate)) + ): + return candidate + except OSError as nvm_err: + logger.debug("Erreur lors du scan du répertoire NVM: %s", nvm_err) + + return None + + +def _get_atspi_bus_address() -> str | None: + """Résout l'adresse du bus d'accessibilité dédié AT-SPI / D-Bus. + + Recherche en priorité AT_SPI_BUS_ADDRESS, puis interroge org.a11y.Bus sur le + bus de session utilisateur, et se replie sur le socket /run/user/$UID/at-spi/bus_0 + ou le bus de session si aucun bus dédié n'est joignable. + """ + # 1. Variable d'environnement explicite AT_SPI_BUS_ADDRESS + env_atspi = os.environ.get("AT_SPI_BUS_ADDRESS") + if env_atspi and env_atspi.strip(): + return env_atspi.strip() + + # 2. Interrogation d'org.a11y.Bus.GetAddress sur le bus de session via busctl + busctl_bin = shutil.which("busctl") + if busctl_bin: + try: + res = subprocess.run( + [busctl_bin, "--user", "call", "org.a11y.Bus", "/org/a11y/bus", "org.a11y.Bus", "GetAddress"], + capture_output=True, + text=True, + timeout=2.0, + check=False, + ) + if res.returncode == 0 and res.stdout.strip(): + addr = res.stdout.strip() + if addr.startswith("s "): + addr = addr[2:].strip() + if addr.startswith('"') and addr.endswith('"') and len(addr) >= 2: + addr = addr[1:-1].strip() + if addr: + return addr + except Exception as exc: + logger.debug("Échec de la résolution de l'adresse AT-SPI via busctl: %s", exc) + + # 3. Interrogation via dbus-send si busctl est indisponible ou a échoué + dbus_send_bin = shutil.which("dbus-send") + if dbus_send_bin: + try: + res = subprocess.run( + [ + dbus_send_bin, + "--session", + "--print-reply=literal", + "--dest=org.a11y.Bus", + "/org/a11y/bus", + "org.a11y.Bus.GetAddress", + ], + capture_output=True, + text=True, + timeout=2.0, + check=False, + ) + if res.returncode == 0 and res.stdout.strip(): + addr = res.stdout.strip() + if addr.startswith("string "): + addr = addr[7:].strip() + if addr.startswith('"') and addr.endswith('"') and len(addr) >= 2: + addr = addr[1:-1].strip() + if addr: + return addr + except Exception as exc: + logger.debug("Échec de la résolution de l'adresse AT-SPI via dbus-send: %s", exc) + + # 4. Socket standard /run/user/$UID/at-spi/bus_0 + uid = os.getuid() if hasattr(os, "getuid") else None + if uid is not None: + atspi_socket = f"/run/user/{uid}/at-spi/bus_0" + if os.path.exists(atspi_socket): + return f"unix:path={atspi_socket}" + + # 5. Repli sur le bus de session en dernier recours + env_addr = os.environ.get("DBUS_SESSION_BUS_ADDRESS") + if env_addr and env_addr.strip(): + return env_addr.strip() + if uid is not None: + user_bus = f"/run/user/{uid}/bus" + if os.path.exists(user_bus): + return f"unix:path={user_bus}" + return None + + +def _dbus_get_actions( + busctl_bin: str, addr_args: list[str], dest: str, obj_path: str, timeout: float = 5.0 +) -> list[str]: + """Récupère la liste des noms d'actions exposées par org.a11y.atspi.Action.""" + if not dest or dest.startswith("-") or not obj_path.startswith("/"): + return [] + cmd_count = [ + busctl_bin, + *addr_args, + "call", + dest, + obj_path, + "org.a11y.atspi.Action", + "GetNActions", + ] + try: + res = subprocess.run(cmd_count, capture_output=True, timeout=timeout, text=True, check=False) + if res.returncode != 0: + return [] + count_str = res.stdout.strip().split()[-1] + count = max(0, min(int(count_str), 32)) + except Exception: + return [] + + names: list[str] = [] + for idx in range(count): + cmd_name = [ + busctl_bin, + *addr_args, + "call", + dest, + obj_path, + "org.a11y.atspi.Action", + "GetName", + "i", + str(idx), + ] + try: + res_name = subprocess.run(cmd_name, capture_output=True, timeout=timeout, text=True, check=False) + if res_name.returncode == 0 and 's "' in res_name.stdout: + name = res_name.stdout.split('s "', 1)[1].split('"', 1)[0] + names.append(name) + else: + names.append("") + except Exception: + names.append("") + return names + + +def _dbus_call_action_or_value(tool_name: str, arguments: dict[str, Any], timeout: float = 10.0) -> bool: + """Exécute l'action ou l'écriture de valeur via busctl directement sur le bus AT-SPI.""" + element_id = str(arguments.get("element_identifier") or arguments.get("element_index") or "") + if not element_id or "/" not in element_id: + return False + + parts = element_id.split("/", 1) + dest = parts[0].rstrip(":") + obj_path = "/" + parts[1] + if not dest or dest.startswith("-") or not obj_path.startswith("/"): + return False + + busctl_bin = shutil.which("busctl") + if not busctl_bin: + return False + + bus_addr = _get_atspi_bus_address() + addr_args = ["--address=" + bus_addr] if bus_addr else ["--user"] + + if tool_name == "perform_action": + action_name = str(arguments.get("action", "")).strip() + actions = _dbus_get_actions(busctl_bin, addr_args, dest, obj_path, timeout=timeout) + + action_idx: int | None = None + if action_name.isdigit(): + idx = int(action_name) + if actions and not (0 <= idx < len(actions)): + return False + action_idx = idx + elif not actions: + return False + elif not action_name: + action_idx = 0 + else: + req_lower = action_name.lower() + exact_matches = [i for i, name in enumerate(actions) if name.lower() == req_lower] + if len(exact_matches) == 1: + action_idx = exact_matches[0] + elif len(exact_matches) > 1: + return False + else: + sub_matches = [i for i, name in enumerate(actions) if req_lower in name.lower()] + if len(sub_matches) == 1: + action_idx = sub_matches[0] + elif req_lower in ("activate", "click", "press", "primary", "default"): + primary = [ + i + for i, name in enumerate(actions) + if name.lower() in ("activate", "click", "press", "primary", "default", "toggle", "open") + ] + if len(primary) == 1: + action_idx = primary[0] + + if action_idx is None: + return False + + cmd = [ + busctl_bin, + *addr_args, + "call", + dest, + obj_path, + "org.a11y.atspi.Action", + "DoAction", + "i", + str(action_idx), + ] + try: + res = subprocess.run(cmd, capture_output=True, timeout=timeout, text=True, check=False) + return res.returncode == 0 and "b true" in res.stdout + except Exception as exc: + logger.debug("Échec du fallback busctl perform_action: %s", exc) + return False + + if tool_name == "set_value": + val_str = str(arguments.get("value", "")) + # 1. Tentative SetTextContents sur EditableText + cmd_text = [ + busctl_bin, + *addr_args, + "call", + dest, + obj_path, + "org.a11y.atspi.EditableText", + "SetTextContents", + "s", + val_str, + ] + try: + res = subprocess.run(cmd_text, capture_output=True, timeout=timeout, text=True, check=False) + if res.returncode == 0 and "b true" in res.stdout: + return True + except Exception as exc: + logger.debug("Échec du fallback busctl SetTextContents: %s", exc) + + # 2. Tentative CurrentValue sur Value (si valeur numérique) + try: + num_val = float(val_str) + cmd_val = [ + busctl_bin, + *addr_args, + "set-property", + dest, + obj_path, + "org.a11y.atspi.Value", + "CurrentValue", + "d", + str(num_val), + ] + res = subprocess.run(cmd_val, capture_output=True, timeout=timeout, text=True, check=False) + return res.returncode == 0 + except ValueError: + pass + except Exception as exc: + logger.debug("Échec du fallback busctl set-property Value: %s", exc) + + return False + + +def _dbus_read_children( + busctl_bin: str, addr_args: list[str], dest: str, obj_path: str, timeout: float = 1.5 +) -> list[tuple[str, str]]: + """Lit les enfants d'un nœud Accessible via l'interface D-Bus.""" + if not dest or dest.startswith("-") or not obj_path.startswith("/"): + return [] + cmd = [ + busctl_bin, + *addr_args, + "call", + dest, + obj_path, + "org.a11y.atspi.Accessible", + "GetChildren", + ] + try: + res = subprocess.run(cmd, capture_output=True, timeout=timeout, text=True, check=False) + if res.returncode != 0: + return [] + tokens = res.stdout.strip().split() + items = [tok.strip('"') for tok in tokens if tok.startswith(('":', '"/'))] + children: list[tuple[str, str]] = [] + for i in range(0, len(items) - 1, 2): + c_dest, c_path = items[i], items[i + 1] + if c_dest and not c_dest.startswith("-") and c_path.startswith("/"): + children.append((c_dest, c_path)) + return children + except Exception: + return [] + + +def _dbus_get_app_state( + app_name: str | None = None, + max_nodes: int = 50, + max_depth: int = 4, + timeout: float = 10.0, +) -> list[dict[str, Any]]: + """Extrait l'arbre d'accessibilité applicatif directement via D-Bus / busctl en explorant les contrôles descendants.""" + busctl_bin = shutil.which("busctl") + if not busctl_bin: + return [] + + if timeout <= 0: + return [] + + deadline = time.monotonic() + timeout + + def _remaining(cap: float = 1.0) -> float: + rem = deadline - time.monotonic() + return min(cap, max(0.001, rem)) + + bus_addr = _get_atspi_bus_address() + addr_args = ["--address=" + bus_addr] if bus_addr else ["--user"] + + roots = _dbus_read_children( + busctl_bin, addr_args, "org.a11y.atspi.Registry", "/org/a11y/atspi/accessible/root", timeout=_remaining(timeout) + ) + if not roots: + return [] + + target_needle = app_name.strip().lower() if app_name else "" + queue: collections.deque[tuple[str, str, int]] = collections.deque() + + # Découverte et filtrage des applications racines + for bus_dest, root_path in roots: + if time.monotonic() >= deadline: + break + cmd_name = [ + busctl_bin, + *addr_args, + "get-property", + bus_dest, + root_path, + "org.a11y.atspi.Accessible", + "Name", + ] + try: + name_res = subprocess.run(cmd_name, capture_output=True, timeout=_remaining(1.0), text=True, check=False) + app_root_name = "" + if name_res.returncode == 0 and 's "' in name_res.stdout: + app_root_name = name_res.stdout.split('s "', 1)[1].split('"', 1)[0] + except Exception as exc: + logger.debug("Échec de lecture du nom d'application racine D-Bus: %s", exc) + continue + + if target_needle and target_needle not in app_root_name.lower(): + continue + + queue.append((bus_dest, root_path, 0)) + + nodes: list[dict[str, Any]] = [] + curr_index = 0 + + while queue and len(nodes) < max_nodes: + if time.monotonic() >= deadline: + break + dest, obj_path, depth = queue.popleft() + + # Lecture du nom + name = "" + if time.monotonic() < deadline: + try: + cmd_n = [busctl_bin, *addr_args, "get-property", dest, obj_path, "org.a11y.atspi.Accessible", "Name"] + res_n = subprocess.run(cmd_n, capture_output=True, timeout=_remaining(1.0), text=True, check=False) + if res_n.returncode == 0 and 's "' in res_n.stdout: + name = res_n.stdout.split('s "', 1)[1].split('"', 1)[0] + except Exception as exc: + logger.debug("Échec de lecture du nom Accessible: %s", exc) + + # Lecture du rôle + role = "application" if depth == 0 else "unknown" + if time.monotonic() < deadline: + try: + cmd_r = [busctl_bin, *addr_args, "call", dest, obj_path, "org.a11y.atspi.Accessible", "GetRoleName"] + res_r = subprocess.run(cmd_r, capture_output=True, timeout=_remaining(1.0), text=True, check=False) + if res_r.returncode == 0 and 's "' in res_r.stdout: + role = res_r.stdout.split('s "', 1)[1].split('"', 1)[0] + except Exception as exc: + logger.debug("Échec de lecture du rôle Accessible: %s", exc) + + # Actions disponibles sur le composant (collectées uniquement pour les contrôles interactifs descendants) + actions: list[str] = [] + if depth > 0 and time.monotonic() < deadline: + actions = _dbus_get_actions(busctl_bin, addr_args, dest, obj_path, timeout=_remaining(0.2)) + + # Exploration des enfants descendants si la profondeur maximale n'est pas atteinte + child_count = 0 + if depth < max_depth and (len(nodes) + len(queue)) < max_nodes and time.monotonic() < deadline: + children = _dbus_read_children(busctl_bin, addr_args, dest, obj_path, timeout=_remaining(1.0)) + child_count = len(children) + for child_dest, child_path in children: + if (len(nodes) + len(queue)) >= max_nodes: + break + queue.append((child_dest, child_path, depth + 1)) + + nodes.append( + { + "index": curr_index, + "object_ref": f"{dest}{obj_path}", + "name": name or ("application" if depth == 0 else ""), + "role": role, + "depth": depth, + "children_count": child_count, + "states": ["visible", "showing"], + "actions": actions, + } + ) + curr_index += 1 + + return nodes + + +def _run_python_atspi_cli(args: list[str]) -> None: + """Fallback d'exécution du médiateur AT-SPI en pur Python.""" + if not args or args[0] in ("-h", "--help", "help"): + print( + "gui-agent-atspi: interface de médiation AT-SPI / D-Bus (commandes: doctor, apps, state, action, value, mcp)" + ) + return + + cmd = args[0] + if cmd == "doctor": + bus_addr = os.environ.get("DBUS_SESSION_BUS_ADDRESS", "") + status = "ok" if bus_addr or os.path.exists(f"/run/user/{os.getuid()}/bus") else "no_bus" + print(json.dumps({"status": status, "engine": "python-fallback", "bus": bus_addr})) + elif cmd == "apps": + apps_tree = _dbus_get_app_state(max_depth=0) + app_names = [n["name"] for n in apps_tree if n.get("name")] + print(json.dumps(app_names)) + elif cmd == "state": + app_filt = args[1] if len(args) > 1 else None + state_tree = _dbus_get_app_state(app_name=app_filt) + if not state_tree: + print( + json.dumps( + { + "status": "error", + "layer": "accessibility", + "error": "Moteur natif gui-agent-atspi introuvable et aucun contrôle accessible D-Bus détecté", + "count": 0, + "tree": [], + } + ) + ) + else: + print( + json.dumps( + { + "status": "success", + "layer": "accessibility", + "count": len(state_tree), + "tree": state_tree, + "degraded": True, + } + ) + ) + elif cmd == "action": + target = args[1] if len(args) > 1 else "" + act_name = args[2] if len(args) > 2 else "0" + ok = _dbus_call_action_or_value("perform_action", {"element_identifier": target, "action": act_name}) + print(json.dumps({"ok": ok})) + elif cmd == "value": + target = args[1] if len(args) > 1 else "" + val = args[2] if len(args) > 2 else "" + ok = _dbus_call_action_or_value("set_value", {"element_identifier": target, "value": val}) + print(json.dumps({"ok": ok})) + elif cmd == "mcp": + for line in sys.stdin: + line_str = line.strip() + if not line_str: + continue + try: + req = json.loads(line_str) + except Exception: + sys.stdout.write( + json.dumps({"jsonrpc": "2.0", "id": None, "error": {"code": -32700, "message": "Parse error"}}) + + "\n" + ) + sys.stdout.flush() + continue + + if not isinstance(req, dict): + sys.stdout.write( + json.dumps({"jsonrpc": "2.0", "id": None, "error": {"code": -32600, "message": "Invalid Request"}}) + + "\n" + ) + sys.stdout.flush() + continue + + req_id = req.get("id") + method = req.get("method") + + # Ignorer les notifications sans id (ex: notifications/initialized) + if req_id is None: + continue + + try: + if method == "initialize": + res = { + "jsonrpc": "2.0", + "id": req_id, + "result": { + "protocolVersion": "2024-11-05", + "capabilities": {"tools": {}}, + "serverInfo": {"name": "gui-agent-atspi-py", "version": "0.1.0"}, + }, + } + elif method == "tools/list": + res = { + "jsonrpc": "2.0", + "id": req_id, + "result": { + "tools": [ + { + "name": "get_app_state", + "description": "Extrait l'arbre d'accessibilité d'une application ou du bureau", + "inputSchema": { + "type": "object", + "properties": { + "app_name": {"type": "string", "description": "Nom de l'application cible"}, + "include_screenshot": {"type": "boolean", "default": False}, + }, + }, + }, + { + "name": "perform_action", + "description": "Déclenche une action AT-SPI sur un composant accessible", + "inputSchema": { + "type": "object", + "properties": { + "element_identifier": {"type": "string"}, + "action": {"type": "string", "default": "activate"}, + "snapshot_id": {"type": "string"}, + }, + "required": ["element_identifier"], + }, + }, + { + "name": "set_value", + "description": "Modifie la valeur textuelle d'un composant accessible", + "inputSchema": { + "type": "object", + "properties": { + "element_identifier": {"type": "string"}, + "value": {"type": "string"}, + "snapshot_id": {"type": "string"}, + }, + "required": ["element_identifier", "value"], + }, + }, + ] + }, + } + elif method == "tools/call": + params = req.get("params", {}) + t_name = str(params.get("name", "")) + t_args = params.get("arguments", {}) + if not isinstance(t_args, dict): + t_args = {} + ok = _dbus_call_action_or_value(t_name, t_args) + res = {"jsonrpc": "2.0", "id": req_id, "result": {"structuredContent": {"ok": ok}}} + else: + res = {"jsonrpc": "2.0", "id": req_id, "error": {"code": -32601, "message": "Method not found"}} + sys.stdout.write(json.dumps(res) + "\n") + sys.stdout.flush() + except Exception as loop_exc: + sys.stdout.write( + json.dumps( + { + "jsonrpc": "2.0", + "id": req_id, + "error": {"code": -32603, "message": f"Internal error: {loop_exc}"}, + } + ) + + "\n" + ) + sys.stdout.flush() + continue + else: + print(f"Commande inconnue: {cmd}", file=sys.stderr) + sys.exit(1) + + +def main_cli() -> None: + """CLI unifié gui-agent-atspi déployé nativement avec le paquet Python. + + Délègue directement au binaire compilé Rust natif à haute performance (<50ms) + si présent, compile à la volée si Cargo est disponible, ou exécute le fallback + programmé. + """ + args = sys.argv[1:] + # 1. Recherche prioritaire du binaire Rust natif (ELF) + native_bin = find_atspi_mediator_binary(exclude_scripts=True) + if native_bin and os.path.isfile(native_bin) and os.access(native_bin, os.X_OK): + with contextlib.suppress(OSError): + res = subprocess.run([native_bin, *args], check=False) + sys.exit(res.returncode) + + # 2. Si le binaire compilé n'est pas encore disponible, tentative de compilation locale via Cargo + project_root = os.path.dirname(os.path.dirname(os.path.dirname(os.path.abspath(__file__)))) + cargo_manifest = os.path.join(project_root, "Cargo.toml") + if os.path.isfile(cargo_manifest) and shutil.which("cargo"): + try: + logger.info("Compilation initiale du médiateur Rust gui-agent-atspi via Cargo...") + subprocess.run( + ["cargo", "build", "--release", "--manifest-path", cargo_manifest, "--bin", "gui-agent-atspi"], + cwd=project_root, + capture_output=True, + check=True, + ) + built_bin = os.path.join(project_root, "target", "release", "gui-agent-atspi") + if os.path.isfile(built_bin) and os.access(built_bin, os.X_OK): + built_res = subprocess.run([built_bin, *args], check=False) + sys.exit(built_res.returncode) + except Exception as exc: + logger.debug("Échec de compilation à la volée: %s", exc) + + # 3. Fallback d'exécution Python pour les commandes de base + _run_python_atspi_cli(args) + + +def _call_mcp_action_or_value(tool_name: str, arguments: dict[str, Any], timeout: float = 15.0) -> bool: + """Invoque l'outil MCP perform_action ou set_value via le serveur stdio Rust ou le fallback D-Bus.""" + binary = find_atspi_mediator_binary(exclude_scripts=True) + if not binary: + logger.info("Binaire AT-SPI Rust absent, repli sur l'invocation D-Bus directe pour %s", tool_name) + return _dbus_call_action_or_value(tool_name, arguments, timeout=timeout) + + try: + proc = subprocess.Popen( + [binary, "mcp"], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + except Exception as exc: + logger.error("Impossible de lancer le processus MCP %s: %s, repli sur D-Bus", binary, exc) + return _dbus_call_action_or_value(tool_name, arguments, timeout=timeout) + + init_req = { + "jsonrpc": "2.0", + "id": 1, + "method": "initialize", + "params": { + "protocolVersion": "2024-11-05", + "capabilities": {}, + "clientInfo": {"name": "gui-agent", "version": "0.1.0"}, + }, + } + call_req = { + "jsonrpc": "2.0", + "id": 2, + "method": "tools/call", + "params": {"name": tool_name, "arguments": arguments}, + } + payload = ( + json.dumps(init_req) + + "\n" + + json.dumps({"jsonrpc": "2.0", "method": "notifications/initialized"}) + + "\n" + + json.dumps(call_req) + + "\n" + ) + + try: + stdout_data, _ = proc.communicate(input=payload, timeout=timeout) + for line in stdout_data.strip().splitlines(): + line = line.strip() + if not line: + continue + try: + data = json.loads(line) + except Exception as parse_exc: + logger.debug("Ligne non JSON ignorée dans la sortie MCP: %s (%s)", line, parse_exc) + continue + if data.get("id") == 2: + result = data.get("result", {}) + structured = result.get("structuredContent") + if isinstance(structured, dict) and "ok" in structured: + return bool(structured["ok"]) + + content = result.get("content", []) + if content and isinstance(content[0], dict): + raw_text = content[0].get("text", "") + try: + content_json = json.loads(raw_text) + if isinstance(content_json, dict) and "ok" in content_json: + return bool(content_json["ok"]) + except Exception as json_exc: + logger.debug("Échec du parsing JSON du champ content: %s", json_exc) + + return False + return False + except subprocess.TimeoutExpired: + logger.error("Timeout dépassé (%.1fs) lors de l'appel MCP %s", timeout, tool_name) + try: + proc.kill() + proc.wait(timeout=1.0) + except Exception as kill_exc: + logger.debug("Échec du kill après timeout: %s", kill_exc) + return False + except Exception as exc: + logger.error("Erreur lors de l'appel MCP %s: %s", tool_name, exc) + return False + finally: + if proc.poll() is None: + try: + proc.terminate() + proc.wait(timeout=1.0) + except Exception as term_exc: + logger.debug("Échec de la terminaison du processus MCP: %s", term_exc) + try: + proc.kill() + except Exception as kill_exc: + logger.debug("Échec du kill du processus MCP: %s", kill_exc) + for stream in (proc.stdin, proc.stdout, proc.stderr): + if stream and hasattr(stream, "closed") and not stream.closed: + try: + stream.close() + except Exception as close_exc: + logger.debug("Échec de la fermeture de flux MCP: %s", close_exc) + + +def _format_snapshot_result( + tree: list[dict[str, Any]], + app_name: str | None, + include_screenshot: bool, + snapshot_id: str | None = None, +) -> dict[str, Any]: + """Enregistre l'arbre dans le cache de snapshots et formate le dictionnaire de retour.""" + global _last_snapshot_id + + new_cache: dict[str, str] = {} + for node in tree: + if isinstance(node, dict): + idx = node.get("index") + obj_ref = node.get("object_ref") + if idx is not None and obj_ref: + new_cache[str(idx)] = str(obj_ref) + + sid = snapshot_id or uuid.uuid4().hex[:12] + with _cache_lock: + if len(_snapshots) >= 20: + oldest_id = next(iter(_snapshots)) + del _snapshots[oldest_id] + _snapshots[sid] = { + "nodes": new_cache, + "app_name": app_name, + } + _last_node_cache.clear() + _last_node_cache.update(new_cache) + _last_snapshot_id = sid + + result: dict[str, Any] = { + "status": "success", + "layer": "accessibility", + "snapshot_id": sid, + "count": len(tree), + "tree": tree, + "include_screenshot": include_screenshot, + } + if include_screenshot: + with contextlib.suppress(Exception): + from gui_agent.layers.input_emulation import screen_capture + + cap = screen_capture() + if cap.get("status") == "success": + result["screenshot"] = cap.get("image") + + return result + + +def get_app_state(include_screenshot: bool = False, app_name: str | None = None) -> dict[str, Any]: + """Extrait l'arbre d'accessibilité applicatif en JSON texte pur via AT-SPI. + + Args: + include_screenshot: Si True, capture et inclut également l'image de l'écran. + app_name: Nom optionnel de l'application ou filtre textuel de fenêtre. + + Returns: + dict contenant status, layer, count, tree et include_screenshot. + """ + # 1. Vérification du mock + if _mock_state is not None: + mock_res = dict(_mock_state) + mock_res["layer"] = "accessibility" + mock_res["include_screenshot"] = include_screenshot + tree_mock = mock_res.get("tree", []) + if isinstance(tree_mock, list): + formatted = _format_snapshot_result( + tree_mock, app_name, include_screenshot, snapshot_id=mock_res.get("snapshot_id") + ) + formatted.update(mock_res) + return formatted + return mock_res + + # 2. Résolution de l'exécutable Rust + binary = find_atspi_mediator_binary() + if not binary: + logger.info("Moteur Rust AT-SPI introuvable, tentative de repli D-Bus pour get_app_state") + dbus_tree = _dbus_get_app_state(app_name=app_name) + if dbus_tree: + res = _format_snapshot_result(dbus_tree, app_name, include_screenshot) + res["degraded"] = True + return res + + _clear_accessibility_cache() + return { + "status": "error", + "layer": "accessibility", + "error": "Moteur Rust AT-SPI introuvable (computer-use-linux ou gui-agent-atspi manquant)", + "include_screenshot": include_screenshot, + "tree": [], + "count": 0, + } + + # 3. Commande CLI d'extraction de l'arbre + cmd = [binary, "state"] + if app_name: + cmd.append(app_name) + + try: + proc = subprocess.run( + cmd, + capture_output=True, + text=True, + timeout=10.0, + check=False, + ) + if proc.returncode != 0: + err_msg = proc.stderr.strip() or f"Code sortie {proc.returncode}" + logger.warning("Échec de l'extraction de l'arbre AT-SPI (%s), tentative de repli D-Bus", err_msg) + dbus_tree = _dbus_get_app_state(app_name=app_name) + if dbus_tree: + res = _format_snapshot_result(dbus_tree, app_name, include_screenshot) + res["degraded"] = True + return res + + _clear_accessibility_cache() + return { + "status": "error", + "layer": "accessibility", + "error": err_msg, + "include_screenshot": include_screenshot, + "tree": [], + "count": 0, + } + + tree_data = json.loads(proc.stdout) + # Validation stricte du format de sortie de l'arbre + if isinstance(tree_data, list): + tree = tree_data + elif isinstance(tree_data, dict) and isinstance(tree_data.get("tree"), list): + if tree_data.get("status") == "error": + _clear_accessibility_cache() + return { + "status": "error", + "layer": "accessibility", + "error": str(tree_data.get("error") or "Échec de l'extraction AT-SPI"), + "include_screenshot": include_screenshot, + "tree": [], + "count": 0, + } + tree = tree_data["tree"] + else: + _clear_accessibility_cache() + return { + "status": "error", + "layer": "accessibility", + "error": "Format de sortie AT-SPI non pris en charge (liste ou dict avec 'tree' attendu)", + "include_screenshot": include_screenshot, + "tree": [], + "count": 0, + } + + return _format_snapshot_result(tree, app_name, include_screenshot) + + except subprocess.TimeoutExpired: + _clear_accessibility_cache() + logger.error("Timeout dépassé (10s) lors de la lecture de l'arbre AT-SPI") + return { + "status": "error", + "layer": "accessibility", + "error": "Timeout de lecture AT-SPI dépassé (10s)", + "include_screenshot": include_screenshot, + "tree": [], + "count": 0, + } + except Exception as exc: + _clear_accessibility_cache() + logger.error("Erreur inattendue lors de l'extraction AT-SPI: %s", exc) + return { + "status": "error", + "layer": "accessibility", + "error": str(exc), + "include_screenshot": include_screenshot, + "tree": [], + "count": 0, + } + + +def perform_action( + element_id: str | int, + action: str = "activate", + snapshot_id: str | None = None, + timeout: float = 15.0, +) -> bool: + """Déclenche l'action du composant accessible directement en mémoire par le bus AT-SPI. + + Args: + element_id: Index numérique de l'élément (ex: "42") ou référence AT-SPI (ex: ":1.14/path"). + action: Nom de l'action à exécuter (ex: "activate", "press", "click", "toggle"). + snapshot_id: Identifiant du snapshot pour s'assurer que l'index correspond à la vue actuelle (obligatoire si element_id est numérique). + timeout: Délai maximal en secondes alloué à l'opération (défaut: 15.0s). + + Returns: + bool indiquant si l'action a été déclenchée avec succès. + """ + if _mock_action_handler is not None: + if callable(_mock_action_handler): + return bool(_mock_action_handler(element_id, action)) + return bool(_mock_action_handler) + + element_id = str(element_id) + + with _cache_lock: + if snapshot_id is not None and snapshot_id not in _snapshots: + logger.error( + "Snapshot ID '%s' inconnu ou expiré. Action rejetée.", + snapshot_id, + ) + return False + + if element_id.isdigit(): + if not snapshot_id: + logger.error( + "L'index numérique %s requiert obligatoirement un 'snapshot_id' valide issu de get_app_state() pour garantir la cohérence sémantique et éviter d'agir sur un état applicatif obsolète ou différent. Action rejetée.", + element_id, + ) + return False + + with _cache_lock: + snapshot_entry = _snapshots.get(snapshot_id) + if snapshot_entry is None: + logger.error( + "Snapshot ID '%s' inconnu ou expiré pour l'index %s. Action rejetée.", + snapshot_id, + element_id, + ) + return False + resolved_ref = snapshot_entry["nodes"].get(element_id) + + if not resolved_ref: + logger.error( + "Index d'élément %s introuvable dans le snapshot '%s' (app: %s). Action rejetée.", + element_id, + snapshot_id, + snapshot_entry.get("app_name"), + ) + return False + + args: dict[str, Any] = { + "action": action, + "element_identifier": resolved_ref, + } + else: + args = { + "action": action, + "element_identifier": element_id, + } + + return _call_mcp_action_or_value("perform_action", args, timeout=timeout) + + +def set_value( + element_id: str | int, + text: str, + snapshot_id: str | None = None, + timeout: float = 15.0, +) -> bool: + """Écrit directement la valeur textuelle dans la mémoire du composant via AT-SPI. + + Args: + element_id: Index numérique de l'élément (ex: "42") ou référence AT-SPI (ex: ":1.14/path"). + text: Valeur textuelle ou numérique à assigner au composant. + snapshot_id: Identifiant du snapshot pour garantir que l'index correspond à la vue actuelle (obligatoire si element_id est numérique). + timeout: Délai maximal en secondes alloué à l'opération (défaut: 15.0s). + + Returns: + bool indiquant si l'assignation a été acceptée par le composant. + """ + if _mock_value_handler is not None: + if callable(_mock_value_handler): + return bool(_mock_value_handler(element_id, text)) + return bool(_mock_value_handler) + + element_id = str(element_id) + + with _cache_lock: + if snapshot_id is not None and snapshot_id not in _snapshots: + logger.error( + "Snapshot ID '%s' inconnu ou expiré. Écriture de valeur rejetée.", + snapshot_id, + ) + return False + + if element_id.isdigit(): + if not snapshot_id: + logger.error( + "L'index numérique %s requiert obligatoirement un 'snapshot_id' valide issu de get_app_state() pour garantir la cohérence sémantique et éviter d'agir sur un état applicatif obsolète ou différent. Écriture de valeur rejetée.", + element_id, + ) + return False + + with _cache_lock: + snapshot_entry = _snapshots.get(snapshot_id) + if snapshot_entry is None: + logger.error( + "Snapshot ID '%s' inconnu ou expiré pour l'index %s. Écriture de valeur rejetée.", + snapshot_id, + element_id, + ) + return False + resolved_ref = snapshot_entry["nodes"].get(element_id) + + if not resolved_ref: + logger.error( + "Index d'élément %s introuvable dans le snapshot '%s' (app: %s). Écriture de valeur rejetée.", + element_id, + snapshot_id, + snapshot_entry.get("app_name"), + ) + return False + + args: dict[str, Any] = { + "element_identifier": resolved_ref, + "value": text, + } + else: + args = { + "element_identifier": element_id, + "value": text, + } + + return _call_mcp_action_or_value("set_value", args, timeout=timeout) diff --git a/gui_agent/layers/input_emulation.py b/linux/layers/input_emulation.py similarity index 100% rename from gui_agent/layers/input_emulation.py rename to linux/layers/input_emulation.py diff --git a/gui_agent/layers/visual_perception.py b/linux/layers/visual_perception.py similarity index 100% rename from gui_agent/layers/visual_perception.py rename to linux/layers/visual_perception.py diff --git a/gui_agent/layers/window_management.py b/linux/layers/window_management.py similarity index 100% rename from gui_agent/layers/window_management.py rename to linux/layers/window_management.py diff --git a/mcp_gui_server.py b/linux/mcp_gui_server.py similarity index 69% rename from mcp_gui_server.py rename to linux/mcp_gui_server.py index d8a8b6f..807778b 100755 --- a/mcp_gui_server.py +++ b/linux/mcp_gui_server.py @@ -1,7 +1,18 @@ -""" -Serveur MCP GUI Agent - Module de rétrocompatibilité. -Redirige vers le package standard `gui_agent.server`. -""" +import importlib.util +import os +import sys + +# Si 'gui_agent' n'est pas encore enregistré dans l'environnement (ex: exécution directe depuis source), +# initialiser le package dynamique pour mapper 'gui_agent' vers le répertoire source courant (linux/) +if "gui_agent" not in sys.modules: + _pkg_dir = os.path.dirname(os.path.abspath(__file__)) + _pkg_init = os.path.join(_pkg_dir, "__init__.py") + if os.path.isfile(_pkg_init): + _spec = importlib.util.spec_from_file_location("gui_agent", _pkg_init, submodule_search_locations=[_pkg_dir]) + if _spec and _spec.loader: + _pkg = importlib.util.module_from_spec(_spec) + sys.modules["gui_agent"] = _pkg + _spec.loader.exec_module(_pkg) from gui_agent.server import ( FastMCP, diff --git a/linux/paths.py b/linux/paths.py new file mode 100644 index 0000000..291df9c --- /dev/null +++ b/linux/paths.py @@ -0,0 +1,281 @@ +"""Résolution dynamique des chemins système sous Linux (spécifications XDG Base Directory). + +Garantit l'agnosticisme complet de l'emplacement de gui_agent : aucun chemin utilisateur en dur, +stockage conforme sous XDG_DATA_HOME, XDG_CONFIG_HOME, XDG_CACHE_HOME et découverte dynamique +des binaires exécutables natifs (ex. gui-agent-atspi). +""" + +from __future__ import annotations + +import os +from pathlib import Path +import shutil +import sys +import tempfile + + +def _validate_xdg_env_path(var_name: str) -> Path | None: + """Valide une variable d'environnement XDG selon la spécification XDG Base Directory. + + Une variable XDG doit contenir un chemin absolu. Si elle est vide, non définie + ou relative, elle doit être ignorée et le repli par défaut utilisé. + """ + raw = os.environ.get(var_name) + if not raw or not raw.strip(): + return None + raw_path = Path(raw.strip()).expanduser() + if not raw_path.is_absolute(): + return None + return raw_path.resolve() + + +def _is_secure_runtime_directory(p: Path, uid: int | None) -> bool: + """Vérifie qu'un répertoire d'exécution est un dossier réel, non-symlink, appartenant à l'UID avec mode 0700.""" + try: + if not p.is_dir() or p.is_symlink(): + return False + st = p.stat() + if uid is not None and st.st_uid != uid: + return False + return (st.st_mode & 0o077) == 0 + except OSError: + return False + + +class LinuxPaths: + """Gestionnaire de chemins système pour Linux conforme aux standards XDG.""" + + @property + def platform_name(self) -> str: + return "linux" + + def get_xdg_data_home(self) -> Path: + """XDG_DATA_HOME (défaut : $HOME/.local/share).""" + valid = _validate_xdg_env_path("XDG_DATA_HOME") + if valid is not None: + return valid + return Path(os.environ.get("HOME", "~")).expanduser().resolve() / ".local" / "share" + + def get_xdg_config_home(self) -> Path: + """XDG_CONFIG_HOME (défaut : $HOME/.config).""" + valid = _validate_xdg_env_path("XDG_CONFIG_HOME") + if valid is not None: + return valid + return Path(os.environ.get("HOME", "~")).expanduser().resolve() / ".config" + + def get_xdg_cache_home(self) -> Path: + """XDG_CACHE_HOME (défaut : $HOME/.cache).""" + valid = _validate_xdg_env_path("XDG_CACHE_HOME") + if valid is not None: + return valid + return Path(os.environ.get("HOME", "~")).expanduser().resolve() / ".cache" + + def get_xdg_runtime_dir(self) -> Path: + """XDG_RUNTIME_DIR (défaut : /run/user/$UID ou /tmp/gui-agent-$UID sécurisé).""" + uid = os.getuid() if hasattr(os, "getuid") else None + valid_xdg = _validate_xdg_env_path("XDG_RUNTIME_DIR") + if valid_xdg is not None and _is_secure_runtime_directory(valid_xdg, uid): + return valid_xdg + + if uid is not None: + user_run = Path(f"/run/user/{uid}") + if _is_secure_runtime_directory(user_run, uid): + return user_run + + temp_dir = Path(tempfile.gettempdir()) + fallback_dir = temp_dir / (f"gui-agent-{uid}" if uid is not None else "gui-agent") + try: + if fallback_dir.is_symlink(): + try: + fallback_dir.unlink() + except OSError: + return Path(tempfile.mkdtemp(prefix=f"gui-agent-{uid or 'safe'}-")) + if fallback_dir.exists(): + st = fallback_dir.stat() + if uid is not None and st.st_uid != uid: + return Path(tempfile.mkdtemp(prefix=f"gui-agent-{uid}-")) + if (st.st_mode & 0o077) != 0: + fallback_dir.chmod(0o700) + else: + fallback_dir.mkdir(mode=0o700, parents=True, exist_ok=True) + fallback_dir.chmod(0o700) + except OSError: + return Path(tempfile.mkdtemp(prefix=f"gui-agent-{uid or 'safe'}-")) + return fallback_dir + + def get_data_dir(self) -> Path: + """Répertoire persistant de données pour gui-agent.""" + return self.get_xdg_data_home() / "gui-agent" + + def get_config_dir(self) -> Path: + """Répertoire de configuration pour gui-agent.""" + return self.get_xdg_config_home() / "gui-agent" + + def get_cache_dir(self) -> Path: + """Répertoire de cache général pour gui-agent.""" + return self.get_xdg_cache_home() / "gui-agent" + + def get_screenshots_dir(self) -> Path: + """Répertoire dynamique des captures d'écran.""" + env_val = os.environ.get("GUI_AGENT_SCREENSHOTS_DIR") + if env_val and env_val.strip(): + return Path(env_val).expanduser().resolve() + return self.get_cache_dir() / "screenshots" + + def get_videos_dir(self) -> Path: + """Répertoire dynamique des enregistrements vidéo.""" + env_val = os.environ.get("GUI_AGENT_VIDEOS_DIR") + if env_val and env_val.strip(): + return Path(env_val).expanduser().resolve() + return self.get_cache_dir() / "videos" + + def get_binary_install_dir(self) -> Path: + """Répertoire d'installation standard pour les binaires exécutables.""" + if sys.prefix != sys.base_prefix: + return Path(sys.prefix) / "bin" + return Path(os.environ.get("HOME", "~")).expanduser().resolve() / ".local" / "bin" + + def get_binary_candidates(self, binary_name: str) -> list[Path]: + """Retourne la liste ordonnée des chemins candidats de découverte du binaire.""" + candidates: list[Path] = [] + + # 1. Variables d'environnement prioritaires + clean_name = binary_name.replace("-", "_").replace(".", "_").upper() + short_name = clean_name[len("GUI_AGENT_") :] if clean_name.startswith("GUI_AGENT_") else clean_name + for env_var in ( + f"GUI_AGENT_{short_name}_BIN", + f"GUI_AGENT_{clean_name}_BIN", + "GUI_AGENT_LINUX_BIN", + "GUI_AGENT_BIN", + ): + val = os.environ.get(env_var) + if val and val.strip(): + candidates.append(Path(val).expanduser().resolve()) + + # 2. Cibles de compilation Cargo locale (release puis debug) + base_dir = Path(__file__).resolve().parent.parent + for profile in ("release", "debug"): + candidates.append(base_dir / "target" / profile / binary_name) + candidates.append(base_dir / "crates" / "atspi_mediator" / "target" / profile / binary_name) + candidates.append(base_dir / "linux" / "crates" / "atspi_mediator" / "target" / profile / binary_name) + + # 3. Binaire packagé directement dans le package Python (gui_agent/bin/...) + package_bin_dir = Path(__file__).resolve().parent / "bin" + candidates.append(package_bin_dir / binary_name) + + # 4. Environnement virtuel actif (sys.prefix/bin) + candidates.append(Path(sys.prefix) / "bin" / binary_name) + + # 5. Répertoire utilisateur ~/.local/bin + candidates.append(Path(os.environ.get("HOME", "~")).expanduser().resolve() / ".local" / "bin" / binary_name) + + # 6. Recherche PATH système via shutil.which + which_path = shutil.which(binary_name) + if which_path: + candidates.append(Path(which_path).resolve()) + + # 7. Répertoires système standards + for sys_dir in ("/usr/local/bin", "/usr/bin", "/bin"): + candidates.append(Path(sys_dir) / binary_name) + + # Déduplication avec préservation de l'ordre + seen: set[str] = set() + deduped: list[Path] = [] + for p in candidates: + str_p = str(p) + if str_p not in seen: + seen.add(str_p) + deduped.append(p) + + return deduped + + def find_binary(self, binary_name: str) -> Path | None: + """Trouve le premier binaire exécutable valide parmi les candidats.""" + for candidate in self.get_binary_candidates(binary_name): + try: + if candidate.is_file() and os.access(candidate, os.X_OK): + return candidate + except (OSError, PermissionError): + continue + return None + + def ensure_dirs(self) -> None: + """Crée tous les répertoires nécessaires avec gestion des permissions.""" + for d in ( + self.get_data_dir(), + self.get_config_dir(), + self.get_cache_dir(), + self.get_screenshots_dir(), + self.get_videos_dir(), + ): + d.mkdir(parents=True, exist_ok=True) + + +# Instance globale Linux +_paths = LinuxPaths() + + +def get_paths() -> LinuxPaths: + """Retourne l'instance des chemins Linux.""" + return _paths + + +def get_screenshots_dir() -> str: + """Retourne le chemin absolu du répertoire de captures d'écran.""" + p = _paths.get_screenshots_dir() + p.mkdir(parents=True, exist_ok=True) + return str(p) + + +def get_videos_dir() -> str: + """Retourne le chemin absolu du répertoire des enregistrements vidéo.""" + p = _paths.get_videos_dir() + p.mkdir(parents=True, exist_ok=True) + return str(p) + + +def get_cache_dir() -> str: + """Retourne le chemin absolu du cache.""" + p = _paths.get_cache_dir() + p.mkdir(parents=True, exist_ok=True) + return str(p) + + +def get_data_dir() -> str: + """Retourne le chemin absolu des données.""" + p = _paths.get_data_dir() + p.mkdir(parents=True, exist_ok=True) + return str(p) + + +def get_config_dir() -> str: + """Retourne le chemin absolu de configuration.""" + p = _paths.get_config_dir() + p.mkdir(parents=True, exist_ok=True) + return str(p) + + +def get_binary_install_dir() -> str: + """Retourne le répertoire d'installation des binaires.""" + p = _paths.get_binary_install_dir() + p.mkdir(parents=True, exist_ok=True) + return str(p) + + +def find_platform_binary(binary_name: str) -> str | None: + """Recherche un binaire exécutable natif.""" + found = _paths.find_binary(binary_name) + return str(found) if found else None + + +__all__ = [ + "LinuxPaths", + "find_platform_binary", + "get_binary_install_dir", + "get_cache_dir", + "get_config_dir", + "get_data_dir", + "get_paths", + "get_screenshots_dir", + "get_videos_dir", +] diff --git a/gui_agent/server.py b/linux/server.py similarity index 96% rename from gui_agent/server.py rename to linux/server.py index a1e18bc..2fd51b6 100755 --- a/gui_agent/server.py +++ b/linux/server.py @@ -7,12 +7,15 @@ import shlex import shutil import subprocess +import sys import tempfile import threading import time from typing import Any from PIL import Image, ImageDraw +from .paths import get_screenshots_dir, get_videos_dir + # S'assurer que la variable DISPLAY existe pour éviter KeyError sous Linux headless/CI os.environ.setdefault("DISPLAY", ":0") import pyautogui @@ -20,15 +23,19 @@ # S'assurer que le SDK MCP est accessible (compatibilité mcp 1.x et 2.x+) FastMCP: Any = None try: - from mcp.server.fastmcp import FastMCP as _FastMCP + import mcp.server.fastmcp as _fastmcp_mod - FastMCP = _FastMCP -except ImportError: + FastMCP = getattr(_fastmcp_mod, "FastMCP", None) + if FastMCP is None: + raise AttributeError("FastMCP non trouvé dans mcp.server.fastmcp") +except (ImportError, AttributeError): try: - from mcp.server.mcpserver import MCPServer as _MCPServer + import mcp.server.mcpserver as _mcpserver_mod - FastMCP = _MCPServer - except ImportError as err: + FastMCP = getattr(_mcpserver_mod, "MCPServer", None) + if FastMCP is None: + raise AttributeError("MCPServer non trouvé dans mcp.server.mcpserver") + except (ImportError, AttributeError) as err: raise ImportError( "Erreur : Le SDK 'mcp' (FastMCP ou MCPServer) est requis pour faire fonctionner gui-agent." ) from err @@ -44,15 +51,13 @@ # Création du serveur MCP mcp = FastMCP("GUI Agent Server") -# Répertoire de stockage des captures d'écran (configurable via GUI_AGENT_SCREENSHOTS_DIR) -SCREENSHOTS_DIR = os.path.abspath( - os.environ.get( - "GUI_AGENT_SCREENSHOTS_DIR", - os.path.join(os.path.expanduser("~"), ".local", "share", "gui-agent", "screenshots"), - ) -) +# Répertoires de stockage des artefacts et caches (résolution dynamique selon l'OS) +SCREENSHOTS_DIR = os.path.abspath(get_screenshots_dir()) os.makedirs(SCREENSHOTS_DIR, exist_ok=True) +VIDEOS_DIR = os.path.abspath(get_videos_dir()) +os.makedirs(VIDEOS_DIR, exist_ok=True) + def check_display_env() -> None: """Vérifie que la variable d'environnement DISPLAY est définie pour X11.""" @@ -86,6 +91,13 @@ def capture_screen_pil(monitor_index: int = 1) -> Image.Image: import numpy as np if np.std(np.array(img)) > 0.001: + with contextlib.suppress(Exception): + left, top, width, height = get_monitor_geometry(monitor_index) + if (width, height) != img.size: + v_left, v_top, _, _ = get_monitor_geometry(0) + rel_left = left - v_left + rel_top = top - v_top + img = img.crop((rel_left, rel_top, rel_left + width, rel_top + height)) return img except Exception as e_spec: logger.debug(f"Fallback spectacle ignoré : {e_spec}") @@ -153,10 +165,10 @@ def normalize_coordinates( if normalized: fx = float(x) fy = float(y) - # Auto-détection si l'utilisateur transmet des valeurs float dans [0.0, 1.0] au lieu de [0, 1000] - if 0.0 <= fx <= 1.0: + # Auto-détection si l'utilisateur transmet des valeurs float dans ]0.0, 1.0[ au lieu de [0, 1000] + if 0.0 < fx < 1.0: fx *= 1000.0 - if 0.0 <= fy <= 1.0: + if 0.0 < fy < 1.0: fy *= 1000.0 x_clamped = max(0.0, min(1000.0, fx)) @@ -973,7 +985,7 @@ def gui_window_focus(window_id: int) -> dict[str, Any]: """ Active et met au premier plan une fenêtre X11 par son ID. """ - if not isinstance(window_id, int) or window_id <= 0: + if not isinstance(window_id, int) or isinstance(window_id, bool) or window_id <= 0: return {"status": "error", "message": "window_id doit être un entier positif."} check_display_env() xdotool_bin = shutil.which("xdotool") or "/bin/xdotool" @@ -995,7 +1007,7 @@ def gui_window_resize_move(window_id: int, x: int, y: int, width: int, height: i """ Redimensionne et déplace une fenêtre de manière atomique en une seule exécution xdotool sous Linux. """ - if not isinstance(window_id, int) or window_id < 0 or isinstance(window_id, bool): + if not isinstance(window_id, int) or isinstance(window_id, bool) or window_id <= 0: return {"status": "error", "message": "window_id doit être un entier positif."} if not isinstance(x, int) or not isinstance(y, int) or isinstance(x, bool) or isinstance(y, bool): @@ -1005,7 +1017,7 @@ def gui_window_resize_move(window_id: int, x: int, y: int, width: int, height: i return {"status": "error", "message": "width et height doivent être des entiers."} if width <= 0 or height <= 0: - return {"status": "error", "message": "width et height doivent être strictly positifs."} + return {"status": "error", "message": "width et height doivent être strictement positifs."} check_display_env() xdotool_bin = shutil.which("xdotool") or "/bin/xdotool" @@ -1056,7 +1068,7 @@ def gui_window_close(window_id: int) -> dict[str, Any]: """ Ferme proprement une fenêtre X11 par son ID via xdotool windowclose. """ - if not isinstance(window_id, int) or window_id <= 0: + if not isinstance(window_id, int) or isinstance(window_id, bool) or window_id <= 0: return {"status": "error", "message": "window_id doit être un entier positif."} check_display_env() xdotool_bin = shutil.which("xdotool") or "/bin/xdotool" @@ -1122,7 +1134,11 @@ def gui_find_template(template_path: str, threshold: float = 0.8, monitor_index: if not isinstance(template_path, str) or not os.path.exists(template_path): return {"status": "error", "message": "Fichier template introuvable."} - threshold_val = max(0.01, min(1.0, float(threshold))) + try: + threshold_val = max(0.01, min(1.0, float(threshold))) + except (ValueError, TypeError): + return {"status": "error", "message": "Seuil 'threshold' invalide (nombre attendu)."} + screen_img = None try: import cv2 @@ -1164,7 +1180,8 @@ def gui_mouse_scroll(clicks: int, direction: str = "down") -> dict[str, Any]: btn_map = {"up": "4", "down": "5", "left": "6", "right": "7"} btn = btn_map[dir_clean] for _ in range(clicks): - run_xdotool(["click", btn]) + if not run_xdotool(["click", btn]): + return {"status": "error", "message": f"Échec du défilement vers le '{dir_clean}' via xdotool."} time.sleep(0.02) return {"status": "success", "action": f"Défilement de {clicks} pas vers le '{dir_clean}'"} except Exception as e: @@ -1533,7 +1550,8 @@ def _validate_video_recording_params( if not output_path: timestamp = int(time.time()) - output_path = os.path.join(SCREENSHOTS_DIR, f"recording_{timestamp}.mp4") + target_dir = VIDEOS_DIR if os.path.isdir(VIDEOS_DIR) else SCREENSHOTS_DIR + output_path = os.path.join(target_dir, f"recording_{timestamp}.mp4") normalized_path = os.path.abspath(os.path.expanduser(str(output_path))) if os.path.isdir(normalized_path): @@ -1727,7 +1745,7 @@ def gui_start_video_recording( "-f", "x11grab", "-i", - f"{display_str}.0+{left},{top}", + f"{display_str if '.' in display_str.split(':')[-1] else f'{display_str}.0'}+{left},{top}", "-c:v", "libx264", "-preset", @@ -1839,6 +1857,12 @@ def gui_stop_video_recording() -> dict[str, Any]: def main() -> None: """Point d'entrée principal pour démarrer le serveur MCP GUI Agent.""" + if any(arg in sys.argv[1:] for arg in ("-h", "--help")): + print( + "Usage: gui-agent [OPTIONS]\n\n" + "Serveur FastMCP pour l'automatisation d'interface graphique et l'interaction desktop." + ) + sys.exit(0) logger.info("Démarrage du serveur MCP GUI Agent...") check_display_env() mcp.run() diff --git a/linux/tests/conftest.py b/linux/tests/conftest.py new file mode 100644 index 0000000..00f1cfc --- /dev/null +++ b/linux/tests/conftest.py @@ -0,0 +1,29 @@ +"""Configuration pytest et chargement direct du package gui_agent depuis linux/. + +Permet d'importer `gui_agent` et ses sous-modules directement depuis le répertoire `linux/` +sans nécessiter de dossier physique `gui_agent/` ni de symlink à la racine du projet. +""" + +from __future__ import annotations + +import importlib.machinery +import importlib.util +from pathlib import Path +import sys + +linux_dir = Path(__file__).resolve().parent.parent +project_root = linux_dir.parent + +if str(linux_dir) not in sys.path: + sys.path.insert(0, str(linux_dir)) +if str(project_root) not in sys.path: + sys.path.insert(0, str(project_root)) + +if "gui_agent" not in sys.modules: + loader = importlib.machinery.SourceFileLoader("gui_agent", str(linux_dir / "__init__.py")) + spec = importlib.util.spec_from_loader("gui_agent", loader, origin=str(linux_dir / "__init__.py")) + if spec is not None: + spec.submodule_search_locations = [str(linux_dir)] + mod = importlib.util.module_from_spec(spec) + sys.modules["gui_agent"] = mod + loader.exec_module(mod) diff --git a/linux/tests/test_accessibility.py b/linux/tests/test_accessibility.py new file mode 100644 index 0000000..606416e --- /dev/null +++ b/linux/tests/test_accessibility.py @@ -0,0 +1,932 @@ +"""Tests unitaires et d'intégration pour la couche d'accessibilité (AT-SPI / D-Bus).""" + +from __future__ import annotations + +import json +import os +from pathlib import Path +import shutil +import subprocess +from typing import Any +from unittest.mock import MagicMock + +import pytest + +from gui_agent.core.mcp_core import mcp_core +import gui_agent.layers.accessibility as accessibility +from gui_agent.layers.accessibility import ( + find_atspi_mediator_binary, + get_app_state, + perform_action, + set_mock_action_handler, + set_mock_state, + set_mock_value_handler, + set_value, +) + + +@pytest.fixture(autouse=True) +def cleanup_mocks() -> Any: + """Réinitialise les gestionnaires mocks avant et après chaque test.""" + set_mock_state(None) + set_mock_action_handler(None) + set_mock_value_handler(None) + accessibility._snapshots.clear() + accessibility._last_node_cache.clear() + accessibility._last_snapshot_id = None + yield + set_mock_state(None) + set_mock_action_handler(None) + set_mock_value_handler(None) + accessibility._snapshots.clear() + accessibility._last_node_cache.clear() + accessibility._last_snapshot_id = None + + +def test_find_atspi_mediator_binary(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie la détection de l'exécutable Rust et la priorité des variables d'environnement.""" + # Test avec override d'environnement valide + monkeypatch.setenv("GUI_AGENT_ATSPI_BIN", "/bin/sh") + assert find_atspi_mediator_binary() == "/bin/sh" + + # Test avec override inexistant + monkeypatch.setenv("GUI_AGENT_ATSPI_BIN", "/nonexistent/binary") + detected = find_atspi_mediator_binary() + assert detected != "/nonexistent/binary" + + +def test_get_app_state_nominal_or_error() -> None: + """Vérifie le contrat de retour de get_app_state en conditions réelles.""" + res = get_app_state(include_screenshot=False) + assert isinstance(res, dict) + assert res["layer"] == "accessibility" + assert res["status"] in ("success", "error") + assert "count" in res + assert "tree" in res + assert res["include_screenshot"] is False + + +def test_get_app_state_with_screenshot() -> None: + """Vérifie la transmission du paramètre include_screenshot.""" + res = get_app_state(include_screenshot=True) + assert res["include_screenshot"] is True + assert res["layer"] == "accessibility" + + +def test_get_app_state_mock() -> None: + """Vérifie le comportement de get_app_state sous mock.""" + mock_payload = { + "status": "success", + "count": 2, + "tree": [ + {"index": 0, "role": "application", "name": "test_app"}, + {"index": 1, "role": "push button", "name": "Valider"}, + ], + } + set_mock_state(mock_payload) + res = get_app_state(include_screenshot=False) + assert res["status"] == "success" + assert res["count"] == 2 + assert len(res["tree"]) == 2 + assert res["tree"][1]["name"] == "Valider" + assert res["layer"] == "accessibility" + + +def test_get_app_state_binary_missing(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie la gestion gracieuse en cas d'absence de moteur Rust.""" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: None) + monkeypatch.setattr(accessibility, "_dbus_get_app_state", lambda app_name=None: []) + res = get_app_state() + assert res["status"] == "error" + assert "introuvable" in res["error"] + assert res["count"] == 0 + assert res["tree"] == [] + + +def test_get_app_state_subprocess_error(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie la gestion d'un échec d'exécution du binaire Rust.""" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: "/bin/sh") + + mock_proc = MagicMock() + mock_proc.returncode = 1 + mock_proc.stderr = "Bus D-Bus inaccessible" + monkeypatch.setattr(subprocess, "run", lambda *args, **kwargs: mock_proc) + + res = get_app_state() + assert res["status"] == "error" + assert "Bus D-Bus inaccessible" in res["error"] + + +def test_get_app_state_subprocess_timeout(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie la levée et la capture propre d'un timeout sur le binaire.""" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: "/bin/sh") + + def raise_timeout(*args: Any, **kwargs: Any) -> Any: + raise subprocess.TimeoutExpired(cmd=["test"], timeout=10.0) + + monkeypatch.setattr(subprocess, "run", raise_timeout) + + res = get_app_state() + assert res["status"] == "error" + assert "Timeout" in res["error"] + + +def test_perform_action_mock() -> None: + """Vérifie le fonctionnement de perform_action avec mock.""" + set_mock_action_handler(lambda el, act: el == "10" and act == "activate") + assert perform_action("10", "activate") is True + assert perform_action("10", "press") is False + assert perform_action("99", "activate") is False + + set_mock_action_handler(True) + assert perform_action("anything") is True + + +def test_perform_action_binary_missing(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie le retour False immédiat si aucun binaire Rust n'est présent et fallback D-Bus indisponible.""" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: None) + monkeypatch.setattr(accessibility, "_get_atspi_bus_address", lambda: None) + monkeypatch.setattr(shutil, "which", lambda cmd: None) + assert perform_action("0") is False + assert perform_action(":1.1/org/a11y/atspi/accessible/root", "activate") is False + + +def test_perform_action_mcp_mock_protocol(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie l'interaction JSON-RPC avec le protocole MCP stdio pour perform_action.""" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: "/bin/sh") + + written_lines: list[str] = [] + + class MockStdin: + def write(self, text: str) -> None: + written_lines.append(text) + + def flush(self) -> None: + pass + + class MockStdout: + def __init__(self) -> None: + self.lines = [ + json.dumps({"jsonrpc": "2.0", "id": 1, "result": {}}) + "\n", + json.dumps({"jsonrpc": "2.0", "id": 2, "result": {"structuredContent": {"ok": True}}}) + "\n", + ] + + def readline(self) -> str: + if self.lines: + return self.lines.pop(0) + return "" + + class MockProcess: + def __init__(self) -> None: + self.stdin = MockStdin() + self.stdout = MockStdout() + self.stderr = MagicMock() + + def communicate(self, input: str | None = None, timeout: float | None = None) -> tuple[str, str]: + if input: + for line in input.splitlines(keepends=True): + self.stdin.write(line) + out = "".join(self.stdout.lines) + return out, "" + + def poll(self) -> int | None: + return 0 + + def terminate(self) -> None: + pass + + def wait(self, timeout: float = 1.0) -> None: + pass + + def kill(self) -> None: + pass + + monkeypatch.setattr(subprocess, "Popen", lambda *args, **kwargs: MockProcess()) + + # Appel numérique (avec index résolu dans le cache du snapshot) + accessibility._snapshots["snap_mcp"] = {"nodes": {"42": ":1.42/node/42"}, "app_name": "test_app"} + accessibility._last_snapshot_id = "snap_mcp" + accessibility._last_node_cache["42"] = ":1.42/node/42" + res_num = perform_action("42", "press", snapshot_id="snap_mcp") + assert res_num is True + assert any('"element_identifier": ":1.42/node/42"' in line for line in written_lines) + assert not any('"element_index"' in line for line in written_lines) + + # Appel sélecteur textuel direct (element_identifier) + written_lines.clear() + res_str = perform_action(":1.14/root", "click") + assert res_str is True + assert any('"element_identifier": ":1.14/root"' in line for line in written_lines) + + +def test_set_value_mock() -> None: + """Vérifie le fonctionnement de set_value avec mock.""" + set_mock_value_handler(lambda el, val: el == "15" and val == "valid_text") + assert set_value("15", "valid_text") is True + assert set_value("15", "wrong") is False + assert set_value("99", "valid_text") is False + + set_mock_value_handler(True) + assert set_value("any", "val") is True + + +def test_set_value_binary_missing(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie le retour False immédiat si aucun binaire Rust n'est présent et fallback D-Bus indisponible.""" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: None) + monkeypatch.setattr(accessibility, "_get_atspi_bus_address", lambda: None) + monkeypatch.setattr(shutil, "which", lambda cmd: None) + assert set_value("0", "text") is False + assert set_value(":1.1/org/a11y/atspi/accessible/root", "text") is False + + +def test_set_value_mcp_mock_protocol(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie l'interaction JSON-RPC avec le protocole MCP stdio pour set_value.""" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: "/bin/sh") + + written_lines: list[str] = [] + + class MockStdin: + def write(self, text: str) -> None: + written_lines.append(text) + + def flush(self) -> None: + pass + + class MockStdout: + def __init__(self) -> None: + self.lines = [ + json.dumps({"jsonrpc": "2.0", "id": 1, "result": {}}) + "\n", + json.dumps({"jsonrpc": "2.0", "id": 2, "result": {"structuredContent": {"ok": True}}}) + "\n", + ] + + def readline(self) -> str: + if self.lines: + return self.lines.pop(0) + return "" + + class MockProcess: + def __init__(self) -> None: + self.stdin = MockStdin() + self.stdout = MockStdout() + self.stderr = MagicMock() + + def communicate(self, input: str | None = None, timeout: float | None = None) -> tuple[str, str]: + if input: + for line in input.splitlines(keepends=True): + self.stdin.write(line) + out = "".join(self.stdout.lines) + return out, "" + + def poll(self) -> int | None: + return 0 + + def terminate(self) -> None: + pass + + def wait(self, timeout: float = 1.0) -> None: + pass + + def kill(self) -> None: + pass + + monkeypatch.setattr(subprocess, "Popen", lambda *args, **kwargs: MockProcess()) + + accessibility._snapshots["snap_val"] = {"nodes": {"100": ":1.100/input/100"}, "app_name": "test_app"} + accessibility._last_snapshot_id = "snap_val" + accessibility._last_node_cache["100"] = ":1.100/input/100" + res = set_value("100", "nouveau_texte", snapshot_id="snap_val") + assert res is True + assert any('"value": "nouveau_texte"' in line for line in written_lines) + assert any('"element_identifier": ":1.100/input/100"' in line for line in written_lines) + assert not any('"element_index"' in line for line in written_lines) + + +def test_perform_action_and_set_value_with_node_cache(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie que element_identifier est résolu à partir de _last_node_cache.""" + written_lines: list[str] = [] + + class MockStdin: + def write(self, text: str) -> None: + written_lines.append(text) + + def flush(self) -> None: + pass + + class MockStdout: + def __init__(self) -> None: + self.lines = [ + json.dumps({"jsonrpc": "2.0", "id": 1, "result": {}}) + "\n", + json.dumps({"jsonrpc": "2.0", "id": 2, "result": {"structuredContent": {"ok": True}}}) + "\n", + json.dumps({"jsonrpc": "2.0", "id": 1, "result": {}}) + "\n", + json.dumps({"jsonrpc": "2.0", "id": 2, "result": {"structuredContent": {"ok": True}}}) + "\n", + ] + + def readline(self) -> str: + if self.lines: + return self.lines.pop(0) + return "" + + class MockProcess: + def __init__(self) -> None: + self.stdin = MockStdin() + self.stdout = MockStdout() + self.stderr = MagicMock() + + def communicate(self, input: str | None = None, timeout: float | None = None) -> tuple[str, str]: + if input: + for line in input.splitlines(keepends=True): + self.stdin.write(line) + out = "".join(self.stdout.lines) + return out, "" + + def poll(self) -> int | None: + return 0 + + def terminate(self) -> None: + pass + + def wait(self, timeout: float = 1.0) -> None: + pass + + def kill(self) -> None: + pass + + monkeypatch.setattr(subprocess, "Popen", lambda *args, **kwargs: MockProcess()) + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: "/bin/sh") + + # Peupler le cache avec un snapshot explicite + accessibility._snapshots["snap_7"] = { + "nodes": {"7": ":1.42/org/a11y/atspi/accessible/7"}, + "app_name": "test_app", + } + accessibility._last_snapshot_id = "snap_7" + accessibility._last_node_cache["7"] = ":1.42/org/a11y/atspi/accessible/7" + + # Vérifier perform_action avec résolution + ok_act = perform_action("7", "press", snapshot_id="snap_7") + assert ok_act is True + assert any('"element_identifier": ":1.42/org/a11y/atspi/accessible/7"' in line for line in written_lines) + assert not any('"element_index"' in line for line in written_lines) + + # Vérifier set_value avec résolution + written_lines.clear() + ok_val = set_value("7", "valeur_test", snapshot_id="snap_7") + assert ok_val is True + assert any('"element_identifier": ":1.42/org/a11y/atspi/accessible/7"' in line for line in written_lines) + assert not any('"element_index"' in line for line in written_lines) + assert any('"element_identifier": ":1.42/org/a11y/atspi/accessible/7"' in line for line in written_lines) + + +def test_mcp_core_sdk_facade_integration() -> None: + """Vérifie l'exposition et l'appel des outils d'accessibilité depuis mcp_core.""" + set_mock_state({"status": "success", "count": 1, "tree": [{"role": "window"}]}) + set_mock_action_handler(True) + set_mock_value_handler(True) + + state = mcp_core.get_app_state() + assert state["status"] == "success" + assert state["count"] == 1 + + act_ok = mcp_core.perform_action("1", "activate") + assert act_ok is True + + val_ok = mcp_core.set_value("1", "texte_saisi") + assert val_ok is True + + +def test_get_app_state_invalid_tree_payload(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie le rejet d'un payload AT-SPI mal formé et le nettoyage du cache.""" + accessibility._last_node_cache["1"] = ":1.1/old" + accessibility._last_snapshot_id = "old_snap" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: "/bin/sh") + + mock_proc = MagicMock() + mock_proc.returncode = 0 + mock_proc.stdout = json.dumps({"tree": "not_a_list"}) + mock_proc.stderr = "" + monkeypatch.setattr(subprocess, "run", lambda *args, **kwargs: mock_proc) + + res = get_app_state() + assert res["status"] == "error" + assert "non pris en charge" in res["error"] + assert len(accessibility._last_node_cache) == 0 + assert accessibility._last_snapshot_id is None + + +def test_perform_action_and_set_value_with_snapshot_id_mismatch(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie qu'un snapshot_id obsolète entraîne un rejet fail-closed immédiat.""" + written_lines: list[str] = [] + + class MockStdin: + def write(self, text: str) -> None: + written_lines.append(text) + + def flush(self) -> None: + pass + + class MockStdout: + def __init__(self) -> None: + self.lines = [ + json.dumps({"jsonrpc": "2.0", "id": 1, "result": {}}) + "\n", + json.dumps({"jsonrpc": "2.0", "id": 2, "result": {"structuredContent": {"ok": True}}}) + "\n", + ] + + def readline(self) -> str: + if self.lines: + return self.lines.pop(0) + return "" + + class MockProcess: + def __init__(self) -> None: + self.stdin = MockStdin() + self.stdout = MockStdout() + self.stderr = MagicMock() + + def communicate(self, input: str | None = None, timeout: float | None = None) -> tuple[str, str]: + if input: + for line in input.splitlines(keepends=True): + self.stdin.write(line) + out = "".join(self.stdout.lines) + return out, "" + + def poll(self) -> int | None: + return 0 + + def terminate(self) -> None: + pass + + def wait(self, timeout: float = 1.0) -> None: + pass + + def kill(self) -> None: + pass + + monkeypatch.setattr(subprocess, "Popen", lambda *args, **kwargs: MockProcess()) + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: "/bin/sh") + + accessibility._snapshots["current_snap"] = {"nodes": {"5": ":1.99/stale/ref"}, "app_name": "app"} + accessibility._last_node_cache["5"] = ":1.99/stale/ref" + accessibility._last_snapshot_id = "current_snap" + + # Avec snapshot_id obsolète, l'action et l'écriture doivent être immédiatement rejetées + act_ok = perform_action("5", "activate", snapshot_id="stale_snap") + assert act_ok is False + assert len(written_lines) == 0 + + val_ok = set_value("5", "texte", snapshot_id="stale_snap") + assert val_ok is False + assert len(written_lines) == 0 + + +def test_perform_action_and_set_value_with_missing_cache_index(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie qu'un index numérique absent du cache est rejeté sans appel externe.""" + mock_popen = MagicMock() + monkeypatch.setattr(subprocess, "Popen", mock_popen) + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: "/bin/sh") + accessibility._snapshots["snap_1"] = {"nodes": {}, "app_name": "app"} + accessibility._last_node_cache.clear() + accessibility._last_snapshot_id = "snap_1" + + assert perform_action("999", "activate", snapshot_id="snap_1") is False + assert set_value("999", "valeur", snapshot_id="snap_1") is False + assert mock_popen.call_count == 0 + + +def test_numeric_index_without_snapshot_id_rejected(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie qu'un index numérique sans snapshot_id est systématiquement rejeté (fail-closed).""" + mock_popen = MagicMock() + monkeypatch.setattr(subprocess, "Popen", mock_popen) + accessibility._snapshots["valid_snap"] = {"nodes": {"1": ":1.1/valid"}, "app_name": "app"} + accessibility._last_snapshot_id = "valid_snap" + accessibility._last_node_cache["1"] = ":1.1/valid" + + # Tout appel numérique sans snapshot_id doit échouer immédiatement sans appeler Popen + assert perform_action("1", "activate") is False + assert set_value("1", "texte") is False + assert mock_popen.call_count == 0 + + +def test_dbus_fallback_action_and_value(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie le fonctionnement du fallback direct busctl/D-Bus lorsque le binaire Rust est absent.""" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: None) + + executed_cmds: list[list[str]] = [] + + def mock_run(cmd: list[str], *args: Any, **kwargs: Any) -> Any: + executed_cmds.append(cmd) + mock_res = MagicMock() + mock_res.returncode = 0 + mock_res.stdout = "b true" + mock_res.stderr = "" + return mock_res + + monkeypatch.setattr(subprocess, "run", mock_run) + monkeypatch.setattr(accessibility, "_get_atspi_bus_address", lambda: "unix:path=/test/bus") + monkeypatch.setattr("shutil.which", lambda name: f"/bin/{name}" if name == "busctl" else None) + + res_act = perform_action(":1.42/org/a11y/atspi/accessible/42", "0") + assert res_act is True + assert any("DoAction" in cmd for cmd in executed_cmds) + + res_val = set_value(":1.42/org/a11y/atspi/accessible/42", "hello") + assert res_val is True + assert any("SetTextContents" in cmd for cmd in executed_cmds) + + +def test_is_elf_binary_architecture_validation(tmp_path: Any, monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie que _is_elf_binary rejette les architectures incompatibles et accepte l'hôte.""" + import struct + + # 1. Fichier non-ELF + txt_file = tmp_path / "script.sh" + txt_file.write_text("#!/bin/sh\necho hello\n") + assert accessibility._is_elf_binary(str(txt_file)) is False + + # 2. ELF valide pour x86_64 (machine 0x3E) + x86_file = tmp_path / "bin_x86" + hdr_x86 = bytearray(b"\x7fELF\x02\x01\x01\x00" + b"\x00" * 10) + hdr_x86 += struct.pack(" None: + """Vérifie que _dbus_get_app_state explore bien les contrôles descendants avec rôles et actions.""" + monkeypatch.setattr("shutil.which", lambda name: f"/bin/{name}" if name == "busctl" else None) + monkeypatch.setattr(accessibility, "_get_atspi_bus_address", lambda: "unix:path=/test/bus") + + def mock_run(cmd: list[str], *args: Any, **kwargs: Any) -> Any: + mock_res = MagicMock() + mock_res.returncode = 0 + if "GetChildren" in cmd: + if "/org/a11y/atspi/accessible/root" in cmd: + mock_res.stdout = 'a(so) 1 ":1.100" "/org/a11y/atspi/accessible/100"' + else: + mock_res.stdout = 'a(so) 1 ":1.100" "/org/a11y/atspi/accessible/101"' + elif "GetRoleName" in cmd: + mock_res.stdout = 's "push button"' + elif "GetNActions" in cmd: + mock_res.stdout = "i 1" + elif "GetName" in cmd: + mock_res.stdout = 's "click"' + elif "Name" in cmd: + if "/org/a11y/atspi/accessible/100" in cmd: + mock_res.stdout = 's "mon_app"' + else: + mock_res.stdout = 's "Bouton Valider"' + else: + mock_res.stdout = "" + mock_res.stderr = "" + return mock_res + + monkeypatch.setattr(subprocess, "run", mock_run) + + tree = accessibility._dbus_get_app_state(app_name="mon_app", max_depth=2, max_nodes=10) + assert len(tree) >= 2 + assert tree[0]["name"] == "mon_app" + assert tree[0]["depth"] == 0 + assert tree[1]["name"] == "Bouton Valider" + assert tree[1]["role"] == "push button" + assert tree[1]["depth"] == 1 + assert "click" in tree[1]["actions"] + + +def test_perform_action_and_set_value_numeric_int_type(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie que passer element_id sous forme d'entier (ex: 42) est normalisé sans erreur.""" + written_lines: list[str] = [] + + class MockStdin: + def write(self, text: str) -> None: + written_lines.append(text) + + def flush(self) -> None: + pass + + class MockStdout: + def __init__(self) -> None: + self.lines = [ + json.dumps({"jsonrpc": "2.0", "id": 1, "result": {}}) + "\n", + json.dumps({"jsonrpc": "2.0", "id": 2, "result": {"structuredContent": {"ok": True}}}) + "\n", + json.dumps({"jsonrpc": "2.0", "id": 1, "result": {}}) + "\n", + json.dumps({"jsonrpc": "2.0", "id": 2, "result": {"structuredContent": {"ok": True}}}) + "\n", + ] + + def readline(self) -> str: + if self.lines: + return self.lines.pop(0) + return "" + + class MockProcess: + def __init__(self) -> None: + self.stdin = MockStdin() + self.stdout = MockStdout() + self.stderr = MagicMock() + + def communicate(self, input: str | None = None, timeout: float | None = None) -> tuple[str, str]: + if input: + for line in input.splitlines(keepends=True): + self.stdin.write(line) + out = "".join(self.stdout.lines) + return out, "" + + def poll(self) -> int | None: + return 0 + + def terminate(self) -> None: + pass + + def wait(self, timeout: float = 1.0) -> None: + pass + + def kill(self) -> None: + pass + + monkeypatch.setattr(subprocess, "Popen", lambda *args, **kwargs: MockProcess()) + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *args, **kwargs: "/bin/sh") + + accessibility._snapshots["snap_int"] = {"nodes": {"42": ":1.42/btn/42"}, "app_name": "app"} + accessibility._last_snapshot_id = "snap_int" + accessibility._last_node_cache["42"] = ":1.42/btn/42" + + # Passer l'entier 42 au lieu de la chaîne "42" + act_ok = perform_action(42, "activate", snapshot_id="snap_int") + assert act_ok is True + assert any('"element_identifier": ":1.42/btn/42"' in line for line in written_lines) + + written_lines.clear() + val_ok = set_value(42, "nouveau", snapshot_id="snap_int") + assert val_ok is True + assert any('"element_identifier": ":1.42/btn/42"' in line for line in written_lines) + + +def test_dbus_call_action_or_value_dest_and_path_validation(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie le rejet strict d'identifiants D-Bus avec injection d'arguments ou chemin invalide.""" + monkeypatch.setattr("shutil.which", lambda name: f"/bin/{name}") + # Rejet d'une destination débutant par '-' + assert accessibility._dbus_call_action_or_value("perform_action", {"element_identifier": "--user/path"}) is False + assert accessibility._dbus_call_action_or_value("perform_action", {"element_identifier": "-help/path"}) is False + # Rejet d'un chemin d'objet ne débutant pas par '/' + assert accessibility._dbus_call_action_or_value("perform_action", {"element_identifier": ":1.14"}) is False + + +def test_dbus_set_value_b_false_rejection(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie que SetTextContents renvoie False lorsque D-Bus répond 'b false'.""" + monkeypatch.setattr("shutil.which", lambda name: f"/bin/{name}") + monkeypatch.setattr(accessibility, "_get_atspi_bus_address", lambda: "unix:path=/test/bus") + + mock_res = MagicMock() + mock_res.returncode = 0 + mock_res.stdout = "b false\n" + mock_res.stderr = "" + monkeypatch.setattr(subprocess, "run", lambda *args, **kwargs: mock_res) + + ok = accessibility._dbus_call_action_or_value("set_value", {"element_identifier": ":1.42/node/1", "value": "test"}) + assert ok is False + + +def test_python_mcp_loop_resilience_and_tools_list(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie la résilience de la boucle MCP Python face aux JSON malformés et l'appel tools/list.""" + import io + import sys + + input_stream = io.StringIO( + "MALFORMED JSON LINE\n" + + json.dumps({"jsonrpc": "2.0", "method": "notifications/initialized"}) + + "\n" + + json.dumps({"jsonrpc": "2.0", "id": 10, "method": "tools/list"}) + + "\n" + ) + output_stream = io.StringIO() + monkeypatch.setattr(sys, "stdin", input_stream) + monkeypatch.setattr(sys, "stdout", output_stream) + + accessibility._run_python_atspi_cli(["mcp"]) + + output_lines = [json.loads(line) for line in output_stream.getvalue().strip().splitlines() if line.strip()] + assert len(output_lines) == 2 + # 1. Réponse d'erreur Parse error sur la première ligne invalide + assert output_lines[0].get("error", {}).get("code") == -32700 + # 2. Réponse valide à tools/list contenant les 3 outils + tools_res = output_lines[1] + assert tools_res.get("id") == 10 + tool_names = [t["name"] for t in tools_res.get("result", {}).get("tools", [])] + assert "get_app_state" in tool_names + assert "perform_action" in tool_names + assert "set_value" in tool_names + + +def test_dbus_get_app_state_deadline_interruption(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie que la traversée D-Bus s'interrompt dès que le deadline est dépassé.""" + monkeypatch.setattr("shutil.which", lambda name: f"/bin/{name}") + monkeypatch.setattr(accessibility, "_get_atspi_bus_address", lambda: "unix:path=/tmp/test") + monkeypatch.setattr( + accessibility, + "_dbus_read_children", + lambda *args, **kwargs: [(":1.1", "/root/1"), (":1.2", "/root/2")], + ) + + # 1. Timeout immédiat / déjà expiré + nodes = accessibility._dbus_get_app_state(timeout=-1.0) + assert nodes == [] + + # 2. Expiration en cours de boucle + curr_time = 100.0 + + def mock_time() -> float: + nonlocal curr_time + curr_time += 5.0 + return curr_time + + monkeypatch.setattr("time.monotonic", mock_time) + nodes_mid = accessibility._dbus_get_app_state(timeout=2.0) + assert isinstance(nodes_mid, list) + + +def test_multi_snapshot_concurrent_validity(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie que la capture d'un second snapshot n'invalide pas les requêtes sur le premier snapshot.""" + called_refs: list[str] = [] + + def mock_mcp_call(tool_name: str, args: dict[str, Any], timeout: float = 15.0) -> bool: + called_refs.append(args.get("element_identifier", "")) + return True + + monkeypatch.setattr(accessibility, "_call_mcp_action_or_value", mock_mcp_call) + + # Créer deux snapshots distincts (simulant deux applications ou requêtes concurrentes) + accessibility._snapshots["snap_app_a"] = { + "nodes": {"1": ":1.10/node/button_a"}, + "app_name": "AppA", + } + accessibility._snapshots["snap_app_b"] = { + "nodes": {"1": ":1.20/node/button_b"}, + "app_name": "AppB", + } + + # Agir sur snap_app_a après la création de snap_app_b + ok_a = perform_action("1", "click", snapshot_id="snap_app_a") + assert ok_a is True + assert called_refs[-1] == ":1.10/node/button_a" + + # Agir sur snap_app_b + ok_b = perform_action("1", "press", snapshot_id="snap_app_b") + assert ok_b is True + assert called_refs[-1] == ":1.20/node/button_b" + + # set_value sur les deux snapshots + val_a = set_value("1", "texte_a", snapshot_id="snap_app_a") + assert val_a is True + val_b = set_value("1", "texte_b", snapshot_id="snap_app_b") + assert val_b is True + + +def test_get_app_state_dbus_fallback_when_binary_missing_or_failed(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie le repli sur D-Bus pur quand le binaire Rust est manquant ou échoue.""" + fake_tree = [{"name": "FallbackBtn", "role": "push button", "object_ref": ":1.99/node"}] + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *a, **kw: None) + monkeypatch.setattr(accessibility, "_dbus_get_app_state", lambda app_name=None: fake_tree) + + res = get_app_state(include_screenshot=False, app_name="test_app") + assert res["status"] == "success" + assert res.get("degraded") is True + assert res["count"] == 1 + assert res["tree"][0]["name"] == "FallbackBtn" + + +def test_call_mcp_action_or_value_fallback_on_popen_error(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie le repli direct sur _dbus_call_action_or_value en cas d'erreur de lancement Popen.""" + monkeypatch.setattr(accessibility, "find_atspi_mediator_binary", lambda *a, **kw: "/bin/false") + + def mock_popen_fail(*args: Any, **kwargs: Any) -> Any: + raise OSError("Processus non exécutable") + + monkeypatch.setattr(subprocess, "Popen", mock_popen_fail) + dbus_called = False + + def mock_dbus_call(tool_name: str, args: dict[str, Any], timeout: float = 10.0) -> bool: + nonlocal dbus_called + dbus_called = True + return True + + monkeypatch.setattr(accessibility, "_dbus_call_action_or_value", mock_dbus_call) + + res = accessibility._call_mcp_action_or_value( + "perform_action", {"element_identifier": ":1.1/node", "action": "click"} + ) + assert res is True + assert dbus_called is True + + +def test_get_atspi_bus_address_resolution(monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie la résolution de l'adresse AT-SPI via variable d'environnement, busctl (cité/non-cité) et dbus-send.""" + # 1. Via variable d'environnement explicite + monkeypatch.setenv("AT_SPI_BUS_ADDRESS", "unix:path=/custom/atspi") + assert accessibility._get_atspi_bus_address() == "unix:path=/custom/atspi" + + # 2. Sans variable, via busctl avec guillemets + monkeypatch.delenv("AT_SPI_BUS_ADDRESS", raising=False) + mock_run = MagicMock() + mock_run.returncode = 0 + mock_run.stdout = 's "unix:path=/run/user/1000/at-spi/bus_0"\n' + monkeypatch.setattr(subprocess, "run", lambda *a, **kw: mock_run) + assert accessibility._get_atspi_bus_address() == "unix:path=/run/user/1000/at-spi/bus_0" + + # 3. Sans variable, via busctl SANS guillemets (format brut retourné par busctl) + mock_run.stdout = "s unix:path=/run/user/1000/at-spi/bus_0\n" + assert accessibility._get_atspi_bus_address() == "unix:path=/run/user/1000/at-spi/bus_0" + + # 4. Repli dbus-send si busctl est absent + monkeypatch.setattr("shutil.which", lambda name: "/bin/dbus-send" if name == "dbus-send" else None) + mock_run.stdout = "string unix:path=/tmp/at-spi-dbus\n" + assert accessibility._get_atspi_bus_address() == "unix:path=/tmp/at-spi-dbus" + + +def test_mcp_stale_index_rejection_with_token() -> None: + """Vérifie que le serveur MCP Rust rejette un index numérique avec un snapshot_token périmé ou absent.""" + bin_path = str(Path(__file__).resolve().parent.parent / "bin" / "gui-agent-atspi") + if not os.path.isfile(bin_path) or not os.access(bin_path, os.X_OK): + pytest.skip("gui-agent-atspi binaire non présent") + + proc = subprocess.Popen( + [bin_path, "mcp"], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + init_req = {"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}} + action_no_snap = { + "jsonrpc": "2.0", + "id": 2, + "method": "tools/call", + "params": {"name": "perform_action", "arguments": {"element_index": 7}}, + } + action_stale_token = { + "jsonrpc": "2.0", + "id": 3, + "method": "tools/call", + "params": {"name": "perform_action", "arguments": {"element_index": 7, "snapshot_token": "stale-snap-999"}}, + } + payload = "\n".join([json.dumps(init_req), json.dumps(action_no_snap), json.dumps(action_stale_token)]) + "\n" + out, _ = proc.communicate(input=payload, timeout=5.0) + lines = [json.loads(line_str) for line_str in out.strip().splitlines() if line_str.strip()] + + resp_2 = next((line_item for line_item in lines if line_item.get("id") == 2), None) + assert resp_2 is not None + assert resp_2["result"]["isError"] is True + + resp_3 = next((line_item for line_item in lines if line_item.get("id") == 3), None) + assert resp_3 is not None + assert resp_3["result"]["isError"] is True + + +def test_mcp_direct_identifier_without_snapshot() -> None: + """Vérifie qu'un identifiant d'élément explicite résolu (ex: :1.42/ref) est transmis directement sans exiger de snapshot actif.""" + bin_path = str(Path(__file__).resolve().parent.parent / "bin" / "gui-agent-atspi") + if not os.path.isfile(bin_path) or not os.access(bin_path, os.X_OK): + pytest.skip("gui-agent-atspi binaire non présent") + + proc = subprocess.Popen( + [bin_path, "mcp"], + stdin=subprocess.PIPE, + stdout=subprocess.PIPE, + stderr=subprocess.PIPE, + text=True, + ) + init_req = {"jsonrpc": "2.0", "id": 1, "method": "initialize", "params": {}} + # Appel direct avec element_identifier résolu : ne doit PAS renvoyer "Aucun snapshot actif" + action_direct = { + "jsonrpc": "2.0", + "id": 2, + "method": "tools/call", + "params": { + "name": "perform_action", + "arguments": { + "action": "click", + "element_identifier": ":1.42/org/a11y/atspi/accessible/root", + }, + }, + } + payload = "\n".join([json.dumps(init_req), json.dumps(action_direct)]) + "\n" + out, _ = proc.communicate(input=payload, timeout=5.0) + lines = [json.loads(line_str) for line_str in out.strip().splitlines() if line_str.strip()] + + resp_2 = next((line_item for line_item in lines if line_item.get("id") == 2), None) + assert resp_2 is not None + # L'erreur (le cas échéant sur un bus headless absent) ne doit PAS être "Aucun snapshot actif" + content_text = resp_2["result"]["content"][0]["text"] + assert "Aucun snapshot actif" not in content_text diff --git a/linux/tests/test_linux_structure.py b/linux/tests/test_linux_structure.py new file mode 100644 index 0000000..71ce76f --- /dev/null +++ b/linux/tests/test_linux_structure.py @@ -0,0 +1,210 @@ +"""Tests pour l'architecture racine Linux et la résolution dynamique des chemins.""" + +from __future__ import annotations + +import os +from pathlib import Path + +import pytest + +from gui_agent.paths import ( + LinuxPaths, + find_platform_binary, + get_binary_install_dir, + get_cache_dir, + get_config_dir, + get_data_dir, + get_screenshots_dir, + get_videos_dir, +) + + +def test_root_directories_structure() -> None: + """Valide l'arborescence à la racine : linux contenant le code, windows et macos vides.""" + root = Path(__file__).resolve().parent.parent.parent + + # 1. Répertoire linux contenant directement le code, les tests et les exemples + linux_dir = root / "linux" + assert linux_dir.is_dir(), "Répertoire linux/ manquant à la racine" + assert not (linux_dir / "gui_agent").exists(), "linux/gui_agent ne doit pas exister (pas de nesting redondant)" + assert (linux_dir / "core").is_dir(), "linux/core manquant" + assert (linux_dir / "layers").is_dir(), "linux/layers manquant" + assert (linux_dir / "utils").is_dir(), "linux/utils manquant" + assert (linux_dir / "tests").is_dir(), "linux/tests manquant" + assert (linux_dir / "examples").is_dir(), "linux/examples manquant" + assert (linux_dir / "server.py").is_file(), "linux/server.py manquant" + assert (linux_dir / "mcp_gui_server.py").is_file(), "linux/mcp_gui_server.py manquant" + assert (linux_dir / "install.sh").is_file(), "linux/install.sh manquant" + assert (linux_dir / "uninstall.sh").is_file(), "linux/uninstall.sh manquant" + assert (linux_dir / "paths.py").is_file(), "linux/paths.py manquant" + assert (linux_dir / "accessibility.py").is_file(), "linux/accessibility.py manquant" + assert (linux_dir / "input.py").is_file(), "linux/input.py manquant" + assert (linux_dir / "window.py").is_file(), "linux/window.py manquant" + + # 2. Répertoires windows et macos + windows_dir = root / "windows" + assert windows_dir.is_dir(), "Répertoire windows/ manquant à la racine" + win_files = {f.name for f in windows_dir.iterdir() if f.name != "__pycache__"} + assert win_files.issubset({".gitkeep", "install.ps1", "uninstall.ps1"}), ( + f"windows/ ne doit contenir que .gitkeep et scripts d'installation : {win_files}" + ) + + macos_dir = root / "macos" + assert macos_dir.is_dir(), "Répertoire macos/ manquant à la racine" + mac_files = [f.name for f in macos_dir.iterdir() if f.name != "__pycache__"] + assert mac_files in ([], [".gitkeep"]), f"macos/ doit être vide pour le moment : {mac_files}" + + # 3. Absence de répertoires de code/tests, doublon win/ ou caches à la racine + assert not (root / "win").exists(), "Le doublon win/ ne doit pas exister (seul windows/ est retenu)" + assert not (root / "gui_agent").exists(), "Le répertoire gui_agent ne doit plus exister à la racine" + assert not (root / "tests").exists(), ( + "Le répertoire tests/ ne doit plus exister à la racine (déplacé dans linux/tests)" + ) + assert not (root / "examples").exists(), ( + "Le répertoire examples/ ne doit plus exister à la racine (déplacé dans linux/examples)" + ) + assert not (root / "screenshots").exists(), "Le répertoire de cache screenshots/ ne doit plus exister à la racine" + + +def test_linux_paths_default_xdg(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Valide la résolution des chemins Linux par défaut selon les spécifications XDG.""" + fake_home = tmp_path / "fake_home" + fake_home.mkdir() + monkeypatch.setenv("HOME", str(fake_home)) + monkeypatch.delenv("XDG_DATA_HOME", raising=False) + monkeypatch.delenv("XDG_CONFIG_HOME", raising=False) + monkeypatch.delenv("XDG_CACHE_HOME", raising=False) + monkeypatch.delenv("XDG_RUNTIME_DIR", raising=False) + monkeypatch.delenv("GUI_AGENT_SCREENSHOTS_DIR", raising=False) + monkeypatch.delenv("GUI_AGENT_VIDEOS_DIR", raising=False) + + paths = LinuxPaths() + assert paths.platform_name == "linux" + assert paths.get_data_dir() == fake_home / ".local" / "share" / "gui-agent" + assert paths.get_config_dir() == fake_home / ".config" / "gui-agent" + assert paths.get_cache_dir() == fake_home / ".cache" / "gui-agent" + assert paths.get_screenshots_dir() == fake_home / ".cache" / "gui-agent" / "screenshots" + assert paths.get_videos_dir() == fake_home / ".cache" / "gui-agent" / "videos" + + +def test_linux_paths_custom_env(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Valide la prise en compte prioritaire des variables d'environnement XDG sous Linux.""" + c_data = tmp_path / "custom_data" + c_conf = tmp_path / "custom_conf" + c_cache = tmp_path / "custom_cache" + c_shots = tmp_path / "custom_shots" + c_vids = tmp_path / "custom_vids" + + monkeypatch.setenv("XDG_DATA_HOME", str(c_data)) + monkeypatch.setenv("XDG_CONFIG_HOME", str(c_conf)) + monkeypatch.setenv("XDG_CACHE_HOME", str(c_cache)) + monkeypatch.setenv("GUI_AGENT_SCREENSHOTS_DIR", str(c_shots)) + monkeypatch.setenv("GUI_AGENT_VIDEOS_DIR", str(c_vids)) + + paths = LinuxPaths() + assert paths.get_data_dir() == c_data / "gui-agent" + assert paths.get_config_dir() == c_conf / "gui-agent" + assert paths.get_cache_dir() == c_cache / "gui-agent" + assert paths.get_screenshots_dir() == c_shots + assert paths.get_videos_dir() == c_vids + + +def test_high_level_helpers_and_directories_created(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Valide les fonctions utilitaires et la création automatique des répertoires de cache.""" + test_shots = tmp_path / "shots" + test_vids = tmp_path / "vids" + monkeypatch.setenv("GUI_AGENT_SCREENSHOTS_DIR", str(test_shots)) + monkeypatch.setenv("GUI_AGENT_VIDEOS_DIR", str(test_vids)) + + shots = get_screenshots_dir() + vids = get_videos_dir() + cache = get_cache_dir() + data = get_data_dir() + config = get_config_dir() + bin_dir = get_binary_install_dir() + + assert os.path.isabs(shots) + assert os.path.isabs(vids) + assert os.path.isabs(cache) + assert os.path.isabs(data) + assert os.path.isabs(config) + assert os.path.isabs(bin_dir) + assert os.path.isdir(shots) + assert os.path.isdir(vids) + + +def test_binary_candidates_and_discovery(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie la découverte dynamique des binaires exécutables natifs sous Linux.""" + fake_bin = tmp_path / "custom_atspi_bin" + fake_bin.write_text("#!/bin/sh\necho ok\n", encoding="utf-8") + fake_bin.chmod(0o755) + + monkeypatch.setenv("GUI_AGENT_ATSPI_BIN", str(fake_bin)) + found = find_platform_binary("gui-agent-atspi") + assert found == str(fake_bin) + + +def test_video_recording_tool_preserved_and_configured() -> None: + """Valide formellement que l'outil de capture vidéo est conservé et configuré.""" + import gui_agent + import gui_agent.server as s + + assert hasattr(gui_agent, "gui_start_video_recording") + assert hasattr(gui_agent, "gui_stop_video_recording") + assert "gui_start_video_recording" in gui_agent.__all__ + assert "gui_stop_video_recording" in gui_agent.__all__ + + assert hasattr(s, "VIDEOS_DIR") + assert os.path.isabs(s.VIDEOS_DIR) + assert os.path.isdir(s.VIDEOS_DIR) + + +def test_linux_paths_rejects_relative_xdg_env(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie que les variables XDG relatives sont ignorées et remplacées par les chemins par défaut.""" + fake_home = tmp_path / "home" + fake_home.mkdir() + monkeypatch.setenv("HOME", str(fake_home)) + monkeypatch.setenv("XDG_DATA_HOME", "relative/data") + monkeypatch.setenv("XDG_CONFIG_HOME", "relative/config") + monkeypatch.setenv("XDG_CACHE_HOME", "relative/cache") + + paths = LinuxPaths() + assert paths.get_data_dir() == fake_home / ".local" / "share" / "gui-agent" + assert paths.get_config_dir() == fake_home / ".config" / "gui-agent" + assert paths.get_cache_dir() == fake_home / ".cache" / "gui-agent" + + +def test_linux_paths_runtime_dir_security(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None: + """Vérifie la validation de sécurité (mode 0700 et UID) pour XDG_RUNTIME_DIR.""" + insecure_dir = tmp_path / "insecure_run" + insecure_dir.mkdir() + + fake_stat = insecure_dir.stat() + insecure_stat = os.stat_result( + ( + fake_stat.st_mode | 0o077, + fake_stat.st_ino, + fake_stat.st_dev, + fake_stat.st_nlink, + fake_stat.st_uid, + fake_stat.st_gid, + fake_stat.st_size, + int(fake_stat.st_atime), + int(fake_stat.st_mtime), + int(fake_stat.st_ctime), + ) + ) + + orig_stat = Path.stat + + def mocked_stat(self: Path, *, follow_symlinks: bool = True) -> os.stat_result: + if self == insecure_dir: + return insecure_stat + return orig_stat(self, follow_symlinks=follow_symlinks) + + monkeypatch.setattr(Path, "stat", mocked_stat) + monkeypatch.setenv("XDG_RUNTIME_DIR", str(insecure_dir)) + paths = LinuxPaths() + runtime_dir = paths.get_xdg_runtime_dir() + assert runtime_dir != insecure_dir + assert os.path.isdir(runtime_dir) diff --git a/tests/test_package.py b/linux/tests/test_package.py similarity index 76% rename from tests/test_package.py rename to linux/tests/test_package.py index 7e64c0b..f3d82c5 100644 --- a/tests/test_package.py +++ b/linux/tests/test_package.py @@ -89,14 +89,180 @@ def test_installation_and_uninstallation_scripts_presence(): """Vérifie la présence et la taille minimale des scripts d'installation/désinstallation.""" import os - assert os.path.isfile("install.sh"), "install.sh manquant" - assert os.path.isfile("install.ps1"), "install.ps1 manquant" - assert os.path.isfile("uninstall.sh"), "uninstall.sh manquant" - assert os.path.isfile("uninstall.ps1"), "uninstall.ps1 manquant" - assert os.path.getsize("install.sh") > 100 - assert os.path.getsize("install.ps1") > 100 - assert os.path.getsize("uninstall.sh") > 100 - assert os.path.getsize("uninstall.ps1") > 100 + assert os.path.isfile("linux/install.sh"), "linux/install.sh manquant" + assert os.path.isfile("windows/install.ps1"), "windows/install.ps1 manquant" + assert os.path.isfile("linux/uninstall.sh"), "linux/uninstall.sh manquant" + assert os.path.isfile("windows/uninstall.ps1"), "windows/uninstall.ps1 manquant" + assert os.path.getsize("linux/install.sh") > 100 + assert os.path.getsize("windows/install.ps1") > 100 + assert os.path.getsize("linux/uninstall.sh") > 100 + assert os.path.getsize("windows/uninstall.ps1") > 100 + + +def test_uninstall_script_cleans_standalone_atspi_mediator(tmp_path): + """Vérifie comportementalement que uninstall.sh nettoie le binaire autonome gui-agent-atspi.""" + import os + import subprocess + + isolated_home = tmp_path / "home" + local_bin = isolated_home / ".local" / "bin" + local_bin.mkdir(parents=True) + fake_bin = local_bin / "gui-agent-atspi" + fake_bin.write_text("#!/bin/sh\necho fake\n", encoding="utf-8") + fake_bin.chmod(0o755) + + env = os.environ.copy() + env["HOME"] = str(isolated_home) + + # 1. Mode Dry-run : le binaire ne doit pas être supprimé + res_dry = subprocess.run( + ["bash", "linux/uninstall.sh", "--dry-run", "-y"], + env=env, + capture_output=True, + text=True, + check=False, + ) + assert res_dry.returncode == 0 + assert "gui-agent-atspi" in res_dry.stdout + assert fake_bin.exists() + + # 2. Mode Réel : le binaire autonome doit être effectivement supprimé + res_real = subprocess.run( + ["bash", "linux/uninstall.sh", "-y"], + env=env, + capture_output=True, + text=True, + check=False, + ) + assert res_real.returncode == 0 + assert not fake_bin.exists() + + +def test_uninstall_script_purges_screenshots_securely(tmp_path): + """Vérifie que uninstall.sh --purge-data purge le cache par défaut et les répertoires personnalisés de manière sécurisée.""" + import os + import subprocess + + isolated_home = tmp_path / "home" + isolated_cache = isolated_home / ".cache" + default_screenshots = isolated_cache / "gui-agent" / "screenshots" + default_screenshots.mkdir(parents=True) + (default_screenshots / "screenshot_1.png").write_text("fake png", encoding="utf-8") + + custom_dir = tmp_path / "custom_screenshots" + custom_dir.mkdir(parents=True) + custom_screenshot = custom_dir / "screenshot_2026.png" + custom_screenshot.write_text("fake capture", encoding="utf-8") + custom_recording_ts = custom_dir / "recording_20260913_120000.mp4" + custom_recording_ts.write_text("fake video ts", encoding="utf-8") + custom_recording_uuid = custom_dir / "recording_550e8400-e29b-41d4-a716-446655440000.mp4" + custom_recording_uuid.write_text("fake video uuid", encoding="utf-8") + custom_web_screenshot = custom_dir / "web_screenshot_20260913_120000.png" + custom_web_screenshot.write_text("fake web screenshot", encoding="utf-8") + unrelated_file = custom_dir / "non-gui-agent-file.txt" + unrelated_file.write_text("important user document", encoding="utf-8") + vacation_capture = custom_dir / "capture-vacation.txt" + vacation_capture.write_text("my vacation notes", encoding="utf-8") + video_notes = custom_dir / "video-notes.txt" + video_notes.write_text("video conference notes", encoding="utf-8") + unrelated_video = custom_dir / "video_projet.mp4" + unrelated_video.write_text("user video project", encoding="utf-8") + unrelated_recording = custom_dir / "recording_interview.mp4" + unrelated_recording.write_text("user recording interview", encoding="utf-8") + unrelated_recording_num_prefix = custom_dir / "recording_1_interview.mp4" + unrelated_recording_num_prefix.write_text("user recording 1 interview", encoding="utf-8") + unrelated_screenshot = custom_dir / "screenshot_final.png" + unrelated_screenshot.write_text("user final screenshot", encoding="utf-8") + unrelated_screenshot_num_prefix = custom_dir / "screenshot_1_final.png" + unrelated_screenshot_num_prefix.write_text("user screenshot 1 final", encoding="utf-8") + + # Répertoire externe contenant 'gui-agent' dans son chemin (ex. un repo de dev) + dev_repo = tmp_path / "gui-agent-projects" / "my-repo" + dev_repo.mkdir(parents=True) + dev_code = dev_repo / "main.py" + dev_code.write_text("print('hello')", encoding="utf-8") + dev_screenshot = dev_repo / "screenshot_999.png" + dev_screenshot.write_text("fake capture", encoding="utf-8") + + env = os.environ.copy() + env["HOME"] = str(isolated_home) + env["XDG_CACHE_HOME"] = str(isolated_cache) + env["GUI_AGENT_SCREENSHOTS_DIR"] = str(custom_dir) + + # 1. Mode Dry-run : rien ne doit être supprimé + res_dry = subprocess.run( + ["bash", "linux/uninstall.sh", "--dry-run", "--purge-data", "-y"], + env=env, + capture_output=True, + text=True, + check=False, + ) + assert res_dry.returncode == 0 + assert (default_screenshots / "screenshot_1.png").exists() + assert custom_screenshot.exists() + assert custom_recording_ts.exists() + assert custom_recording_uuid.exists() + assert custom_web_screenshot.exists() + assert unrelated_file.exists() + assert vacation_capture.exists() + assert video_notes.exists() + assert unrelated_video.exists() + assert unrelated_recording.exists() + assert unrelated_recording_num_prefix.exists() + assert unrelated_screenshot.exists() + assert unrelated_screenshot_num_prefix.exists() + + # 2. Mode Purge réel : les captures doivent être supprimées, les fichiers tiers préservés + res_real = subprocess.run( + ["bash", "linux/uninstall.sh", "--purge-data", "-y"], + env=env, + capture_output=True, + text=True, + check=False, + ) + assert res_real.returncode == 0 + assert not default_screenshots.exists() + assert not custom_screenshot.exists() + assert not custom_recording_ts.exists() + assert not custom_recording_uuid.exists() + assert not custom_web_screenshot.exists() + assert unrelated_file.exists(), "Les fichiers tiers non liés ne doivent jamais être supprimés !" + assert vacation_capture.exists(), "capture-vacation.txt doit être préservé !" + assert video_notes.exists(), "video-notes.txt doit être préservé !" + assert unrelated_video.exists(), "video_projet.mp4 doit être préservé !" + assert unrelated_recording.exists(), "recording_interview.mp4 doit être préservé !" + assert unrelated_recording_num_prefix.exists(), "recording_1_interview.mp4 doit être préservé !" + assert unrelated_screenshot.exists(), "screenshot_final.png doit être préservé !" + assert unrelated_screenshot_num_prefix.exists(), "screenshot_1_final.png doit être préservé !" + assert custom_dir.exists(), "Le dossier personnalisé contenant des fichiers tiers ne doit pas être supprimé !" + + # 3. Mode purge sur répertoire externe contenant 'gui-agent' dans son chemin : + # il ne doit PAS être supprimé récursivement ! + env["GUI_AGENT_SCREENSHOTS_DIR"] = str(dev_repo) + res_repo = subprocess.run( + ["bash", "linux/uninstall.sh", "--purge-data", "-y"], + env=env, + capture_output=True, + text=True, + check=False, + ) + assert res_repo.returncode == 0 + assert not dev_screenshot.exists() + assert dev_code.exists(), "Le code source d'un repo tiers contenant 'gui-agent' ne doit pas être supprimé !" + assert dev_repo.exists(), "Le dossier du repo ne doit pas être supprimé récursivement !" + + # 4. Mode Protection racine : si GUI_AGENT_SCREENSHOTS_DIR pointe vers HOME ou racine + env["GUI_AGENT_SCREENSHOTS_DIR"] = str(isolated_home) + res_unsafe = subprocess.run( + ["bash", "linux/uninstall.sh", "--purge-data", "-y"], + env=env, + capture_output=True, + text=True, + check=False, + ) + assert res_unsafe.returncode == 0 + assert "non sécurisé détecté" in res_unsafe.stderr or "non sécurisé détecté" in res_unsafe.stdout + assert isolated_home.exists() def test_install_doc_structure(): @@ -110,7 +276,7 @@ def test_install_doc_structure(): assert "Guide d'Installation" in content assert "Microsoft Windows" in content - assert "Linux & macOS" in content + assert "Installation sous Linux" in content assert "Dépannage" in content assert "powershell" in content.lower() assert "uv tool install" in content @@ -872,7 +1038,8 @@ def test_modular_architecture_scaffolding(monkeypatch): with pytest.raises(RuntimeError, match="Aucun serveur graphique"): utils.coordinates.check_display_env() monkeypatch.undo() - assert layers.get_app_state()["status"] == "not_implemented" + assert layers.get_app_state()["status"] in ("success", "error") + assert layers.get_app_state()["layer"] == "accessibility" assert layers.mouse_click_at(100, 200)["status"] == "not_implemented" monkeypatch.setattr("os.path.exists", lambda p: p != "/proc") assert layers.process_list()[0]["status"] == "error" @@ -882,5 +1049,77 @@ def test_modular_architecture_scaffolding(monkeypatch): assert utils.validate_video_recording_params(None, fps=f, monitor_index=m, duration=d)["status"] == "error" emitted: list[list[str]] = [] - monkeypatch.setattr("subprocess.run", lambda c, **kw: (emitted.append(c), type("R", (), {"returncode": 0})())[1]) + + def mock_run(c: list[str], **kw: object) -> object: + emitted.append(c) + return type("R", (), {"returncode": 0})() + + monkeypatch.setattr("subprocess.run", mock_run) assert utils.type_char_human("a", base_delay=0.0) is True and any("type" in c and "a" in c for c in emitted) + + +def test_repl_safe_read_incremental_utf8() -> None: + """Vérifie que _safe_read gère les fragments multi-octets UTF-8 sans corrompre ni couper le flux.""" + import codecs + import os + from linux.core.repl import _safe_read + + r, w = os.pipe() + try: + os.set_blocking(r, False) + decoder = codecs.getincrementaldecoder("utf-8")(errors="replace") + + # 1. Écriture des 2 premiers octets du caractère € (\xe2\x82\xac) + os.write(w, b"\xe2\x82") + # _safe_read doit bufferiser et retourner None car le caractère est incomplet + assert _safe_read(r, decoder) is None + + # 2. Écriture du 3ème octet et d'un suffixe ASCII + os.write(w, b"\xac_valid") + assert _safe_read(r, decoder) == "€_valid" + + # 3. Écriture d'un emoji 4 octets 🚀 (\xf0\x9f\x9a\x80) scindé en 2 écritures + os.write(w, b"\xf0\x9f") + assert _safe_read(r, decoder) is None + os.write(w, b"\x9a\x80") + assert _safe_read(r, decoder) == "🚀" + + # 4. Flush sur fermeture du descripteur (EOF) + os.close(w) + assert _safe_read(r, decoder) == "" + finally: + os.close(r) + + +def test_repl_execute_script_multibyte_utf8() -> None: + """Vérifie l'exécution nominale d'un script émettant de l'UTF-8 complexe.""" + from linux.core.repl import execute_script + + res = execute_script("print('Bonjour 🚀 € 🌟 àéïôù')") + assert res["status"] == "success" + assert "Bonjour 🚀 € 🌟 àéïôù" in res["stdout"] + + +def test_windows_install_ps1_guards() -> None: + """Vérifie la présence des gardes PSScriptRoot/Get-Location, LASTEXITCODE et mcpServers dans windows/install.ps1.""" + with open("windows/install.ps1", encoding="utf-8") as f: + content = f.read() + + assert "$scriptDir = if ($PSScriptRoot) { $PSScriptRoot } else { (Get-Location).Path }" in content + assert "if ($LASTEXITCODE -eq 0) {" in content + assert "$null -eq $configData.mcpServers" in content + assert 'throw "mcpServers doit être un objet JSON valide."' in content + + +def test_pty_session_output_limit_no_unbound_error() -> None: + """Vérifie que pty_session ne lève pas UnboundLocalError lors d'un dépassement de limite de sortie.""" + import sys + + if sys.platform == "win32": + pytest.skip("PTY non supporté sous Windows") + from linux.core.pty_session import PTYSession + + session = PTYSession(timeout=2.0, max_output_chars=10) + code, out = session.execute(["python3", "-c", "print('A' * 100)"]) + assert code == -1 + assert "Taille de sortie PTY maximale dépassée" in out diff --git a/tests/test_verify_workflows.py b/linux/tests/test_verify_workflows.py similarity index 93% rename from tests/test_verify_workflows.py rename to linux/tests/test_verify_workflows.py index dadac9b..abbc3f6 100644 --- a/tests/test_verify_workflows.py +++ b/linux/tests/test_verify_workflows.py @@ -7,16 +7,16 @@ from typing import Any # Chargement dynamique du script verify_workflows.py sans dépendre du sys.path -SCRIPT_PATH = Path(__file__).resolve().parent.parent / ".github" / "scripts" / "verify_workflows.py" +SCRIPT_PATH = Path(__file__).resolve().parent.parent.parent / ".github" / "scripts" / "verify_workflows.py" spec = importlib.util.spec_from_file_location("verify_workflows", str(SCRIPT_PATH)) assert spec is not None and spec.loader is not None verify_workflows: Any = importlib.util.module_from_spec(spec) spec.loader.exec_module(verify_workflows) -def test_existing_workflows_are_all_valid(): +def test_existing_workflows_are_all_valid() -> None: """Valide que tous les workflows réels du dépôt respectent 100% des invariants.""" - workflows_dir = Path(__file__).resolve().parent.parent / ".github" / "workflows" + workflows_dir = Path(__file__).resolve().parent.parent.parent / ".github" / "workflows" workflows = sorted([*workflows_dir.glob("*.yml"), *workflows_dir.glob("*.yaml")]) assert len(workflows) > 0, "Aucun workflow trouvé dans .github/workflows" @@ -25,7 +25,7 @@ def test_existing_workflows_are_all_valid(): assert errors_count == 0, f"Erreurs trouvées dans {wf_path}: {errors}" -def test_reject_anchored_top_level_on_key(tmp_path: Path): +def test_reject_anchored_top_level_on_key(tmp_path: Path) -> None: wf = tmp_path / "anchored_top_level_on.yml" wf.write_text( """ @@ -51,7 +51,7 @@ def test_reject_anchored_top_level_on_key(tmp_path: Path): assert any("pull_request_target est formellement interdit" in e for e in errors) -def test_reject_pull_request_target(tmp_path: Path): +def test_reject_pull_request_target(tmp_path: Path) -> None: """Vérifie le rejet strict du déclencheur pull_request_target même sous clé avec guillemets.""" wf = tmp_path / "bad_pr_target.yml" wf.write_text( @@ -78,7 +78,7 @@ def test_reject_pull_request_target(tmp_path: Path): assert any("pull_request_target est formellement interdit" in e for e in errors) -def test_nested_escaped_event_keys_and_dispatch_inputs_not_confused(tmp_path: Path): +def test_nested_escaped_event_keys_and_dispatch_inputs_not_confused(tmp_path: Path) -> None: """Vérifie que les clés d'événements échappées (ex: 'pull_request_t\\u0061rget') sont bien détectées, et que le texte des inputs workflow_dispatch n'est pas confondu avec un trigger.""" # 1. Échappement imbriqué de pull_request_target wf_escaped = tmp_path / "nested_escaped_trigger.yml" @@ -132,7 +132,7 @@ def test_nested_escaped_event_keys_and_dispatch_inputs_not_confused(tmp_path: Pa assert count2 == 0, f"Erreurs inattendues pour inputs de workflow_dispatch: {errors2}" -def test_reject_missing_top_level_permissions(tmp_path: Path): +def test_reject_missing_top_level_permissions(tmp_path: Path) -> None: """Vérifie le rejet en l'absence de bloc permissions top-level.""" wf = tmp_path / "missing_perms.yml" wf.write_text( @@ -154,7 +154,7 @@ def test_reject_missing_top_level_permissions(tmp_path: Path): assert any("Bloc top-level 'permissions:' manquant" in e for e in errors) -def test_reject_write_all_and_write_permissions(tmp_path: Path): +def test_reject_write_all_and_write_permissions(tmp_path: Path) -> None: """Vérifie le rejet strict de permissions: write-all et permissions: write.""" wf1 = tmp_path / "write_all.yml" wf1.write_text( @@ -197,7 +197,7 @@ def test_reject_write_all_and_write_permissions(tmp_path: Path): assert any("write" in e for e in errors2) -def test_reject_missing_timeout_or_runs_on_and_no_nested_leak(tmp_path: Path): +def test_reject_missing_timeout_or_runs_on_and_no_nested_leak(tmp_path: Path) -> None: """Vérifie que les clés runs-on et timeout-minutes doivent être au niveau du job et non dans un script imbriqué.""" wf = tmp_path / "missing_job_keys.yml" wf.write_text( @@ -223,7 +223,7 @@ def test_reject_missing_timeout_or_runs_on_and_no_nested_leak(tmp_path: Path): assert any("timeout-minutes:' obligatoire manquante" in e for e in errors) -def test_accept_matrix_runs_on_and_commented_timeout(tmp_path: Path): +def test_accept_matrix_runs_on_and_commented_timeout(tmp_path: Path) -> None: """Vérifie que les expressions runs-on et les commentaires inline sur timeout-minutes sont acceptés.""" wf = tmp_path / "dynamic_job.yml" wf.write_text( @@ -246,7 +246,7 @@ def test_accept_matrix_runs_on_and_commented_timeout(tmp_path: Path): assert count == 0, f"Erreurs inattendues: {errors}" -def test_accept_runs_on_mapping_and_trailing_whitespace_timeout(tmp_path: Path): +def test_accept_runs_on_mapping_and_trailing_whitespace_timeout(tmp_path: Path) -> None: """Vérifie que runs-on sous forme de mapping et timeout-minutes avec espaces de fin sont acceptés.""" wf = tmp_path / "mapping_runner_trailing_timeout.yml" wf.write_text( @@ -271,7 +271,7 @@ def test_accept_runs_on_mapping_and_trailing_whitespace_timeout(tmp_path: Path): assert count == 0, f"Erreurs inattendues: {errors}" -def test_accept_indentationless_runs_on_sequence_and_yaml_aliases(tmp_path: Path): +def test_accept_indentationless_runs_on_sequence_and_yaml_aliases(tmp_path: Path) -> None: """Vérifie que les séquences runs-on sans indentation supplémentaire et les alias YAML pour on: sont correctement acceptés/résolus.""" # 1. runs-on avec séquence sans indentation supplémentaire wf_runner = tmp_path / "indentationless_runner.yml" @@ -321,7 +321,7 @@ def test_accept_indentationless_runs_on_sequence_and_yaml_aliases(tmp_path: Path assert any("pull_request_target est formellement interdit" in e for e in errors2) -def test_reject_block_anchored_trigger_alias(tmp_path: Path): +def test_reject_block_anchored_trigger_alias(tmp_path: Path) -> None: wf = tmp_path / "block_anchor_alias.yml" wf.write_text( """ @@ -348,7 +348,7 @@ def test_reject_block_anchored_trigger_alias(tmp_path: Path): assert any("pull_request_target est formellement interdit" in e for e in errors) -def test_reject_external_multiline_and_sequence_anchor_aliases(tmp_path: Path): +def test_reject_external_multiline_and_sequence_anchor_aliases(tmp_path: Path) -> None: multiline_flow = tmp_path / "external_multiline_flow_anchor.yml" multiline_flow.write_text( """ @@ -399,7 +399,7 @@ def test_reject_external_multiline_and_sequence_anchor_aliases(tmp_path: Path): assert any("pull_request_target est formellement interdit" in e for e in errors2) -def test_reject_external_multiline_flow_sequence_anchor_alias(tmp_path: Path): +def test_reject_external_multiline_flow_sequence_anchor_alias(tmp_path: Path) -> None: wf = tmp_path / "external_multiline_flow_sequence_anchor.yml" wf.write_text( """ @@ -426,7 +426,7 @@ def test_reject_external_multiline_flow_sequence_anchor_alias(tmp_path: Path): assert any("pull_request_target est formellement interdit" in e for e in errors) -def test_accept_concurrency_group_after_cancel_in_progress(tmp_path: Path): +def test_accept_concurrency_group_after_cancel_in_progress(tmp_path: Path) -> None: wf = tmp_path / "concurrency_group_after_cancel.yml" wf.write_text( """ @@ -450,7 +450,7 @@ def test_accept_concurrency_group_after_cancel_in_progress(tmp_path: Path): assert count == 0, f"Erreurs inattendues: {errors}" -def test_reject_scalar_trigger_anchor_aliases(tmp_path: Path): +def test_reject_scalar_trigger_anchor_aliases(tmp_path: Path) -> None: scalar_pr_target = tmp_path / "scalar_pr_target_anchor.yml" scalar_pr_target.write_text( """ @@ -496,7 +496,7 @@ def test_reject_scalar_trigger_anchor_aliases(tmp_path: Path): assert any("concurrency" in e for e in errors2) -def test_reject_multiline_flow_concurrency_without_group(tmp_path: Path): +def test_reject_multiline_flow_concurrency_without_group(tmp_path: Path) -> None: wf = tmp_path / "multiline_flow_concurrency_without_group.yml" wf.write_text( """ @@ -521,7 +521,7 @@ def test_reject_multiline_flow_concurrency_without_group(tmp_path: Path): assert any("concurrency" in e for e in errors) -def test_reject_flow_sequence_alias_trigger(tmp_path: Path): +def test_reject_flow_sequence_alias_trigger(tmp_path: Path) -> None: wf = tmp_path / "flow_sequence_alias_trigger.yml" wf.write_text( """ @@ -546,7 +546,7 @@ def test_reject_flow_sequence_alias_trigger(tmp_path: Path): assert any("pull_request_target est formellement interdit" in e for e in errors) -def test_reject_multiline_flow_triggers(tmp_path: Path): +def test_reject_multiline_flow_triggers(tmp_path: Path) -> None: wf = tmp_path / "multiline_flow_triggers.yml" wf.write_text( """ @@ -572,7 +572,7 @@ def test_reject_multiline_flow_triggers(tmp_path: Path): assert any("pull_request_target est formellement interdit" in e for e in errors) -def test_accept_quoted_concurrency_group(tmp_path: Path): +def test_accept_quoted_concurrency_group(tmp_path: Path) -> None: wf = tmp_path / "quoted_concurrency_group.yml" wf.write_text( """ @@ -615,7 +615,7 @@ def test_accept_quoted_concurrency_group(tmp_path: Path): assert count2 == 0, f"Erreurs inattendues: {errors2}" -def test_reject_nested_alias_inside_flow_anchor(tmp_path: Path): +def test_reject_nested_alias_inside_flow_anchor(tmp_path: Path) -> None: wf = tmp_path / "nested_flow_alias_anchor.yml" wf.write_text( """ @@ -641,7 +641,7 @@ def test_reject_nested_alias_inside_flow_anchor(tmp_path: Path): assert any("pull_request_target est formellement interdit" in e for e in errors) -def test_reject_nested_concurrency_group_without_direct_group(tmp_path: Path): +def test_reject_nested_concurrency_group_without_direct_group(tmp_path: Path) -> None: wf = tmp_path / "nested_concurrency_group.yml" wf.write_text( """ @@ -666,7 +666,7 @@ def test_reject_nested_concurrency_group_without_direct_group(tmp_path: Path): assert any("concurrency" in e for e in errors) -def test_reject_multiline_anchored_trigger_mapping(tmp_path: Path): +def test_reject_multiline_anchored_trigger_mapping(tmp_path: Path) -> None: wf = tmp_path / "multiline_anchor_mapping.yml" wf.write_text( """ @@ -692,7 +692,7 @@ def test_reject_multiline_anchored_trigger_mapping(tmp_path: Path): assert any("pull_request_target est formellement interdit" in e for e in errors) -def test_reject_anchored_trigger_mapping_and_require_concurrency(tmp_path: Path): +def test_reject_anchored_trigger_mapping_and_require_concurrency(tmp_path: Path) -> None: anchored_pr_target = tmp_path / "anchored_pr_target.yml" anchored_pr_target.write_text( """ @@ -739,7 +739,7 @@ def test_reject_anchored_trigger_mapping_and_require_concurrency(tmp_path: Path) assert any("Bloc top-level 'concurrency:' manquant" in e for e in errors2) -def test_reject_empty_concurrency_for_push(tmp_path: Path): +def test_reject_empty_concurrency_for_push(tmp_path: Path) -> None: wf = tmp_path / "empty_concurrency.yml" wf.write_text( """ @@ -762,7 +762,7 @@ def test_reject_empty_concurrency_for_push(tmp_path: Path): assert any("concurrency" in e for e in errors) -def test_reject_inline_anchored_trigger_values(tmp_path: Path): +def test_reject_inline_anchored_trigger_values(tmp_path: Path) -> None: anchored_pr_target = tmp_path / "inline_anchored_pr_target.yml" anchored_pr_target.write_text( """ @@ -806,7 +806,7 @@ def test_reject_inline_anchored_trigger_values(tmp_path: Path): assert any("Bloc top-level 'concurrency:' manquant" in e for e in errors2) -def test_reject_persist_credentials_true(tmp_path: Path): +def test_reject_persist_credentials_true(tmp_path: Path) -> None: """Vérifie le rejet de persist-credentials: true.""" wf = tmp_path / "persist_creds.yml" wf.write_text( @@ -832,7 +832,7 @@ def test_reject_persist_credentials_true(tmp_path: Path): assert any("persist-credentials: true est formellement interdit" in e for e in errors) -def test_accept_alternate_yaml_indentation(tmp_path: Path): +def test_accept_alternate_yaml_indentation(tmp_path: Path) -> None: """Vérifie que les workflows avec indentation variable (ex: 4 espaces pour jobs, 8 espaces pour properties) sont supportés.""" wf = tmp_path / "alternate_indent.yml" wf.write_text( @@ -855,7 +855,7 @@ def test_accept_alternate_yaml_indentation(tmp_path: Path): assert count == 0, f"Erreurs inattendues pour indentation alternative: {errors}" -def test_reject_missing_concurrency_flow_style_and_mapping(tmp_path: Path): +def test_reject_missing_concurrency_flow_style_and_mapping(tmp_path: Path) -> None: """Vérifie l'obligation de concurrency pour les déclencheurs mapping et flow-style list.""" wf_mapping = tmp_path / "missing_concurrency_mapping.yml" wf_mapping.write_text( diff --git a/linux/uninstall.sh b/linux/uninstall.sh new file mode 100755 index 0000000..0b9644b --- /dev/null +++ b/linux/uninstall.sh @@ -0,0 +1,300 @@ +#!/usr/bin/env bash +# ============================================================================== +# GUI Agent - Automated Uninstallation Script (Linux / macOS) +# Method: Clean removal of uv tool, MCP registrations, and optional data purge +# ============================================================================== + +set -euo pipefail + +# ANSI Color codes +RED='\033[0;31m' +GREEN='\033[0;32m' +YELLOW='\033[1;33m' +BLUE='\033[0;34m' +CYAN='\033[0;36m' +BOLD='\033[1m' +NC='\033[0m' # No Color + +# Default options +NON_INTERACTIVE=false +DRY_RUN=false +PURGE_DATA=false +PACKAGE_NAME="gui-agent" + +log_info() { + echo -e "${BLUE}[INFO]${NC} $1" +} + +log_success() { + echo -e "${GREEN}[SUCCESS]${NC} $1" +} + +log_warn() { + echo -e "${YELLOW}[WARN]${NC} $1" +} + +log_error() { + echo -e "${RED}[ERROR]${NC} $1" >&2 +} + +print_banner() { + echo -e "${CYAN}${BOLD}" + echo "============================================================" + echo " GUI Agent - Clean Uninstaller (Linux / macOS) " + echo "============================================================" + echo -e "${NC}" +} + +show_help() { + print_banner + echo "Usage: ./linux/uninstall.sh [OPTIONS]" + echo "" + echo "Options:" + echo " -y, --yes, --no-prompt Non-interactive mode (accept defaults)" + echo " -p, --purge-data Purge generated screenshots and runtime data" + echo " -d, --dry-run Show uninstallation steps without modifying system" + echo " -h, --help Show this help message and exit" + echo "" +} + +# Parse command line arguments +while [[ $# -gt 0 ]]; do + case "$1" in + -y|--yes|--no-prompt) + NON_INTERACTIVE=true + shift + ;; + -p|--purge-data) + PURGE_DATA=true + shift + ;; + -d|--dry-run) + DRY_RUN=true + shift + ;; + -h|--help) + show_help + exit 0 + ;; + *) + log_error "Option inconnue : $1" + show_help + exit 1 + ;; + esac +done + +print_banner + +# Step 1: Remove Isolated uv tool +log_info "1/3 - Désinstallation du package '$PACKAGE_NAME' via Astral 'uv'..." +UV_PATH="$(command -v uv 2>/dev/null || echo "${HOME}/.local/bin/uv")" + +if [[ -x "$UV_PATH" ]]; then + if [[ "$DRY_RUN" == "true" ]]; then + log_info "[Dry-Run] $UV_PATH tool uninstall $PACKAGE_NAME" + else + set +e + UV_OUT="$("$UV_PATH" tool uninstall "$PACKAGE_NAME" 2>&1)" + UV_STATUS=$? + set -e + if [[ $UV_STATUS -eq 0 ]]; then + log_success "Outil '$PACKAGE_NAME' désinstallé avec succès de uv." + elif echo "$UV_OUT" | grep -qiE "not installed|not found"; then + log_info "Package '$PACKAGE_NAME' n'était pas présent dans les outils uv." + else + log_warn "Échec de désinstallation de l'outil uv : $UV_OUT" + fi + fi +else + log_warn "Gestionnaire 'uv' non détecté. Aucun outil uv à désinstaller." +fi + +# Nettoyage du binaire autonome du médiateur AT-SPI Rust dans ~/.local/bin s'il existe +STANDALONE_ATSPI_BIN="${HOME}/.local/bin/gui-agent-atspi" +if [[ -f "$STANDALONE_ATSPI_BIN" ]]; then + if [[ "$DRY_RUN" == "true" ]]; then + log_info "[Dry-Run] Suppression du binaire autonome : $STANDALONE_ATSPI_BIN" + else + rm -f "$STANDALONE_ATSPI_BIN" + log_success "Binaire autonome du médiateur AT-SPI supprimé : $STANDALONE_ATSPI_BIN" + fi +fi + +# Step 2: Remove MCP Client Registrations +log_info "2/3 - Nettoyage des configurations des clients MCP..." + +if [[ "$DRY_RUN" == "true" ]]; then + log_info "[Dry-Run] claude mcp remove gui-agent" + log_info "[Dry-Run] Suppression de l'entrée 'gui-agent' dans ~/.gemini/config/mcp_config.json" +else + # 1. Claude Code CLI + if command -v claude >/dev/null 2>&1; then + log_info "Suppression du serveur MCP dans Claude Code..." + if claude mcp remove "$PACKAGE_NAME" 2>/dev/null; then + log_success "Configuration Claude Code nettoyée." + else + log_warn "Échec de suppression de la configuration Claude Code (le serveur n'était peut-être pas configuré)." + fi + fi + + # 2. Antigravity CLI Config (~/.gemini/config/mcp_config.json) + ANTIGRAVITY_CONFIG_FILE="${HOME}/.gemini/config/mcp_config.json" + if [[ -f "$ANTIGRAVITY_CONFIG_FILE" ]]; then + log_info "Nettoyage de la configuration Antigravity CLI..." + if [[ -x "$UV_PATH" ]]; then + if "$UV_PATH" run python - </dev/null && pwd -P || echo "${HOME}")" +REAL_CACHE="$(cd "${CACHE_BASE}" 2>/dev/null && pwd -P || echo "${CACHE_BASE}")" + +# Collecte des répertoires cibles : emplacement personnalisé documenté et cache par défaut +TARGET_DIRS=() +if [[ -n "${GUI_AGENT_SCREENSHOTS_DIR:-}" && -d "${GUI_AGENT_SCREENSHOTS_DIR}" ]]; then + TARGET_DIRS+=("${GUI_AGENT_SCREENSHOTS_DIR}") +fi + +DEFAULT_SCREENSHOTS_DIR="${CACHE_BASE}/gui-agent/screenshots" +if [[ -d "$DEFAULT_SCREENSHOTS_DIR" ]]; then + TARGET_DIRS+=("$DEFAULT_SCREENSHOTS_DIR") +fi + +PROCESSED_DIRS=() +for candidate in "${TARGET_DIRS[@]}"; do + [[ -d "$candidate" ]] || continue + CANONICAL_DIR="$(cd "$candidate" 2>/dev/null && pwd -P || true)" + [[ -z "$CANONICAL_DIR" ]] && continue + + # Dédoublonnage des répertoires déjà traités + ALREADY_DONE=false + for done_dir in "${PROCESSED_DIRS[@]}"; do + if [[ "$done_dir" == "$CANONICAL_DIR" ]]; then + ALREADY_DONE=true + break + fi + done + [[ "$ALREADY_DONE" == "true" ]] && continue + PROCESSED_DIRS+=("$CANONICAL_DIR") + + # Protection stricte contre la suppression accidentelle de racines système ou répertoires utilisateur critiques + if [[ "$CANONICAL_DIR" == "/" || "$CANONICAL_DIR" == "$REAL_HOME" || "$CANONICAL_DIR" == "$REAL_CACHE" || \ + "$CANONICAL_DIR" == "/tmp" || "$CANONICAL_DIR" == "/var" || "$CANONICAL_DIR" == "/etc" || \ + "$CANONICAL_DIR" == "/usr" || "$CANONICAL_DIR" == "/bin" || "$CANONICAL_DIR" == "/sbin" || \ + "$CANONICAL_DIR" == "/home" || "$CANONICAL_DIR" == "/root" || "$CANONICAL_DIR" == "/opt" || \ + "$CANONICAL_DIR" == "/boot" || "$CANONICAL_DIR" == "/dev" || "$CANONICAL_DIR" == "/proc" || \ + "$CANONICAL_DIR" == "/sys" || "$CANONICAL_DIR" == "$REAL_HOME/Desktop" || \ + "$CANONICAL_DIR" == "$REAL_HOME/Documents" || "$CANONICAL_DIR" == "$REAL_HOME/Downloads" || \ + "$CANONICAL_DIR" == "$REAL_HOME/Pictures" || "$CANONICAL_DIR" == "$REAL_HOME/Music" || \ + "$CANONICAL_DIR" == "$REAL_HOME/Videos" ]]; then + log_error "Chemin de captures d'écran non sécurisé détecté : $candidate. Purge annulée." + continue + fi + + # Validation stricte de propriété : vérification que le dossier appartient à l'utilisateur courant + dir_uid="$(stat -c '%u' "$CANONICAL_DIR" 2>/dev/null || stat -f '%u' "$CANONICAL_DIR" 2>/dev/null || true)" + user_uid="$(id -u 2>/dev/null || true)" + if [[ ! -O "$CANONICAL_DIR" && -n "$user_uid" && -n "$dir_uid" && "$dir_uid" != "$user_uid" ]]; then + log_error "Le répertoire ($CANONICAL_DIR) n'appartient pas à l'utilisateur courant (UID ${user_uid}). Purge annulée par sécurité." + continue + fi + + if [[ "$DRY_RUN" == "true" ]]; then + log_info "[Dry-Run] Purge possible du répertoire de captures : $CANONICAL_DIR" + else + DO_PURGE=false + if [[ "$PURGE_DATA" == "true" ]]; then + DO_PURGE=true + elif [[ "$NON_INTERACTIVE" != "true" ]]; then + echo -e "${YELLOW}Voulez-vous supprimer définitivement les captures dans ($CANONICAL_DIR) ? [o/N]${NC} " + response="" + read -r response || response="n" + if [[ "$response" =~ ^([oO][uU][iI]|[oO])$ ]]; then + DO_PURGE=true + fi + fi + + if [[ "$DO_PURGE" == "true" ]]; then + APPROVED_CACHE="${REAL_CACHE}/gui-agent" + APPROVED_DATA="${XDG_DATA_HOME:-${REAL_HOME}/.local/share}/gui-agent" + + if [[ "$CANONICAL_DIR" == "$APPROVED_CACHE" || "$CANONICAL_DIR" == "$APPROVED_CACHE/"* || \ + "$CANONICAL_DIR" == "$APPROVED_DATA" || "$CANONICAL_DIR" == "$APPROVED_DATA/"* ]]; then + rm -rf -- "$CANONICAL_DIR" + log_success "Répertoire de captures supprimé : $CANONICAL_DIR" + else + # Emplacement personnalisé : purge ciblée avec validation stricte des noms d'artefacts applicatifs authentiques + while IFS= read -r -d '' filepath; do + fname="$(basename "$filepath")" + is_artifact=false + if [[ "$fname" =~ ^(screenshot|raw_screenshot|web_screenshot)_[0-9]+(_[0-9]+)?(\ \([0-9]+\))?\.(png|jpg|jpeg|webp)$ ]]; then + is_artifact=true + elif [[ "$fname" =~ ^recording_([0-9]+(_[0-9]+)?|[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}|[0-9a-fA-F]{32})(\ \([0-9]+\))?\.mp4$ ]]; then + is_artifact=true + elif [[ "$fname" =~ ^_mcp_screen_tmp_[0-9a-zA-Z_]+\.png$ ]]; then + is_artifact=true + fi + + if [[ "$is_artifact" == "true" ]]; then + rm -f -- "$filepath" 2>/dev/null || true + fi + done < <(find "$CANONICAL_DIR" -maxdepth 1 -type f -print0 2>/dev/null) + rmdir "$CANONICAL_DIR" 2>/dev/null || true + log_success "Captures purgées dans l'emplacement personnalisé : $CANONICAL_DIR" + fi + else + log_info "Répertoire de captures conservé : $CANONICAL_DIR" + fi + fi +done + +if [[ ${#PROCESSED_DIRS[@]} -eq 0 ]]; then + log_info "Aucun répertoire de capture résiduel trouvé." +fi + +echo "" +echo -e "${GREEN}${BOLD}============================================================${NC}" +echo -e "${GREEN}${BOLD} Désinstallation de GUI Agent terminée avec succès ! ${NC}" +echo -e "${GREEN}${BOLD}============================================================${NC}" +echo "" diff --git a/linux/utils/__init__.py b/linux/utils/__init__.py new file mode 100644 index 0000000..317f3eb --- /dev/null +++ b/linux/utils/__init__.py @@ -0,0 +1,18 @@ +"""Utilitaires et drivers partagés pour gui_agent.""" + +from . import coordinates, human_mimic, video +from .coordinates import get_monitor_geometry, normalize_coordinates +from .human_mimic import generate_smooth_path, sleep_human, type_char_human +from .video import validate_video_recording_params + +__all__ = [ + "coordinates", + "generate_smooth_path", + "get_monitor_geometry", + "human_mimic", + "normalize_coordinates", + "sleep_human", + "type_char_human", + "validate_video_recording_params", + "video", +] diff --git a/gui_agent/utils/coordinates.py b/linux/utils/coordinates.py similarity index 100% rename from gui_agent/utils/coordinates.py rename to linux/utils/coordinates.py diff --git a/gui_agent/utils/human_mimic.py b/linux/utils/human_mimic.py similarity index 91% rename from gui_agent/utils/human_mimic.py rename to linux/utils/human_mimic.py index 3a9349e..7047f01 100644 --- a/gui_agent/utils/human_mimic.py +++ b/linux/utils/human_mimic.py @@ -47,6 +47,14 @@ def translate_key(key: str) -> str: "enter": "Return", "return": "Return", "ctrl": "control", + "control": "control", + "backspace": "BackSpace", + "tab": "Tab", + "space": "space", + "up": "Up", + "down": "Down", + "left": "Left", + "right": "Right", "escape": "Escape", "esc": "Escape", } diff --git a/gui_agent/utils/video.py b/linux/utils/video.py similarity index 86% rename from gui_agent/utils/video.py rename to linux/utils/video.py index dfd3954..117fb28 100644 --- a/gui_agent/utils/video.py +++ b/linux/utils/video.py @@ -55,7 +55,17 @@ def validate_video_recording_params( if dur_val is None: return {"status": "error", "message": "duration doit être un entier strictement positif."} - path = output_path or os.path.join(default_dir or tempfile.gettempdir(), f"recording_{uuid.uuid4().hex}.mp4") + if not default_dir: + try: + from ..paths import get_videos_dir + + default_base = get_videos_dir() + except Exception: + default_base = tempfile.gettempdir() + else: + default_base = default_dir + + path = output_path or os.path.join(default_base, f"recording_{uuid.uuid4().hex}.mp4") norm_path = os.path.abspath(os.path.expanduser(str(path))) if os.path.isdir(norm_path): return {"status": "error", "message": "output_path ne peut pas être un répertoire existant."} diff --git a/linux/window.py b/linux/window.py new file mode 100644 index 0000000..da2ed68 --- /dev/null +++ b/linux/window.py @@ -0,0 +1,15 @@ +"""Gestion et ciblage des fenêtres pour Linux (xdotool / wmctrl / EWMH).""" + +from __future__ import annotations + +from .layers.window_management import ( + activate_window, + process_list, + process_run, +) + +__all__ = [ + "activate_window", + "process_list", + "process_run", +] diff --git a/macos/.gitkeep b/macos/.gitkeep new file mode 100644 index 0000000..83c8524 --- /dev/null +++ b/macos/.gitkeep @@ -0,0 +1 @@ +# Répertoire réservé pour l'implémentation macOS (Phase 5 #135) diff --git a/maj.md b/maj.md new file mode 100644 index 0000000..7b25101 --- /dev/null +++ b/maj.md @@ -0,0 +1,85 @@ +# Directives de Restructuration & Mises à Jour (MAJ) + +Ce document répertorie les exigences architecturales majeures pour la restructuration et l'évolution multi-plateforme du projet `gui_agent`. + +--- + +## 1. Agnosticisme de l'Emplacement & Résolution Dynamique des Chemins + +Le code doit être strictement indépendant de son emplacement physique sur le système de fichiers (aucun chemin codé en dur vers un répertoire utilisateur spécifique ou une arborescence locale). + +- **Résolution Dynamique des Chemins Système** : + - Chaque système d'exploitation dispose de sa propre logique de résolution dynamique conforme aux standards natifs. + - **Linux** : Conformité stricte avec les spécifications XDG Base Directory (`XDG_CONFIG_HOME`, `XDG_DATA_HOME`, `XDG_CACHE_HOME`, `$HOME/.local/share`, `/tmp` ou `XDG_RUNTIME_DIR`). + - **macOS** : Résolution vers `~/Library/Application Support`, `~/Library/Caches`, et les répertoires standards Apple. + - **Windows** : Résolution via les variables d'environnement natives (`%APPDATA%`, `%LOCALAPPDATA%`, `%TEMP%`, `ProgramData`). +- **Gestion des Binaires Natifs & Installations** : + - Résolution dynamique de l'emplacement d'installation du binaire natif (ex. `gui-agent-atspi` ou équivalent plateforme), avec découverte dans les répertoires standards du système, les variables d'environnement prioritaires (`GUI_AGENT_*_BIN`), ou le chemin local de l'environnement virtuel. +- **Gestion des Caches & Artefacts Temporaires** : + - Les répertoires de stockage des caches (captures d'écran, logs d'exécution, enregistrements vidéo) sont résolus dynamiquement dans le dossier de cache standard de l'OS hôte avec création automatique et sécurisée. + +--- + +## 2. Restructuration Complète de l'Arborescence par Plateforme + +Le projet adopte une séparation nette par système d'exploitation afin d'éviter le couplage entre couches spécifiques : + +- **Répertoires dédiés et étanches** : + - `linux/` : Code spécifique Linux (AT-SPI / D-Bus, X11/XWayland, portail XDG RemoteDesktop, uinput/evdev), outils dédiés et scripts d'installation/désinstallation (`install.sh`, `uninstall.sh`). + - `macos/` : Code spécifique macOS (NSAccessibility, CoreGraphics / Quartz Event Taps, permissions accessibilité), outils dédiés et scripts d'installation/désinstallation (`install.sh`, `uninstall.sh`). + - `windows/` : Code spécifique Windows (UI Automation, Win32 API, SendInput), outils dédiés et scripts d'installation/désinstallation (`install.ps1`, `uninstall.ps1`). +- **Scripts d'Installation Séparés** : + - Chaque répertoire de plateforme possède son propre script d'installation automatisé assurant la vérification des prérequis, la compilation éventuelle des binaires natifs et la configuration des dépendances système nécessaires. + +--- + +## 3. Règle Formelle sur l'Outil de Capture Vidéo + +- **Conservation Impérative de la Capture Vidéo** : + - Malgré toute mention ou proposition d'élagage précédente, les outils d'enregistrement vidéo (`gui_start_video_recording`, `gui_stop_video_recording`) **ne doivent en aucun cas être supprimés**. + - L'outil de capture vidéo fait partie intégrante du périmètre fonctionnel supporté de `gui_agent`. + - La gestion de ses fichiers de sortie, tampons temporaires et répertoires de stockage s'appuie sur la résolution dynamique des chemins système propres à chaque OS via `VIDEOS_DIR` et `get_videos_dir()`. + +--- + +## 4. État d'Implémentation & Architecture Technique + +### 4.1. Structure Épurée à la Racine & Absence de Nesting Redondant +- **Arborescence Racine** : + - Seuls les trois répertoires de plateformes (`linux/`, `windows/`, `macos/`) subsistent à la racine, accompagnés des métadonnées de projet standard (`README.md`, `pyproject.toml`, `ci.sh`, `Cargo.toml`, etc.). + - **Déplacement des tests et exemples** : `tests/` et `examples/` ont été déplacés dans `linux/tests/` et `linux/examples/`. Aucun dossier de test ou d'exemple ne subsiste à la racine hors des dossiers d'OS. + - **Suppression définitive des caches de la racine et de `linux/`** : Le répertoire `screenshots/` ainsi que tout cache résiduel (anciens `target/` de crate de 1,5 Go, `__pycache__`) ont été purgés. Les captures et vidéos sont dirigées dynamiquement vers `$XDG_CACHE_HOME/gui-agent/`. + - **Aucun doublon ni symlink** : Pas de doublon `win/`, aucun symlink artificiel `gui_agent -> ...`, et aucun répertoire `gui_agent/` laissé à la racine. + - **Pas de sous-dossier imbriqué `linux/gui_agent/`** : Tout le code Linux réside directement à la racine de `linux/`. + - **Fichier `.gitignore` exhaustif et durci** : Couvre l'ensemble des environnements virtuels (`venv/`, `.venv/`), des caches d'outils et de linters (`__pycache__/`, `.pytest_cache/`, `.mypy_cache/`, `.ruff_cache/`), des médias locaux (`screenshots/`, `videos/`, `*.mp4`, `.cache/`), des artefacts de build (`build/`, `dist/`, `target/`, `**/target/`) et des exécutables natifs multi-plateformes (`bin/`, `**/bin/`, `*.exe`, `*.dll`, `*.so`, `*.dylib`). + +### 4.2. Arborescences Dédiées par OS +- `linux/` : + - `core/` : REPL, PTY, SDK `mcp_core`. + - `layers/` : Accessibilité AT-SPI / D-Bus, émulation d'entrées, perception visuelle, fenêtrage. + - `utils/` : Coordonnées, mimétisme cinématique, validation vidéo. + - `crates/atspi_mediator` : Moteur natif Rust AT-SPI2 / D-Bus. + - `tests/` : Suite complète de tests unitaires et d'intégration Linux. + - `examples/` : Exemples d'usage et scripts de validation end-to-end. + - `paths.py` : Résolution dynamique des chemins Linux XDG. + - `server.py` : Serveur FastMCP monolithique. + - `mcp_gui_server.py` : Point d'entrée rétrocompatible. + - `install.sh` / `uninstall.sh` : Scripts d'automatisation dédiés Linux. +- `macos/` : + - `.gitkeep` (réservé pour Phase 5 #135). +- `windows/` : + - `.gitkeep`, `install.ps1`, `uninstall.ps1` (réservé pour Phase 5 #134). + +### 4.3. Décision d'Architecture : Bundle Unique Natif par OS (Rust + REPL PyO3) +- **Objectif cible** : Fournir un binaire unique et directement exécutable par système d'exploitation (`gui-agent` pour Linux, `gui-agent.exe` pour Windows, `gui-agent` pour macOS). +- **Compilation sur machine hôte** : Compilation locale unifiée via Cargo (`cargo build --release`) sans dépendance à un environnement virtuel Python complexe à l'exécution. +- **Intégration du REPL CodeAct** : Intégration de l'interpréteur via PyO3 pour l'outil `execute_script`, assurant une exécution locale ultra-rapide (<5ms) et une consommation mémoire minimale (<15 Mo). + +### 4.4. Intégration dans le Serveur FastMCP (`linux/server.py`) +- Remplacement du chemin codé en dur `SCREENSHOTS_DIR` par la résolution dynamique `get_screenshots_dir()`. +- Introduction de `VIDEOS_DIR` via `get_videos_dir()` pour le stockage isolé et sécurisé des enregistrements vidéo. +- Conservation intégrale des outils `gui_start_video_recording` et `gui_stop_video_recording`. + +### 4.5. Protocole de Publication et Revue +- Les Pull Requests sont publiées sous le compte GitHub **personal agent**. +- La temporisation pour l'attente du verdict Greptile distant est fixée à **5 minutes minimum (300 secondes)** (ne pas configurer de timer court à 30 secondes). diff --git a/note-de-conception-fondements-et-architecture.md b/note-de-conception-fondements-et-architecture.md new file mode 100644 index 0000000..7eb37e9 --- /dev/null +++ b/note-de-conception-fondements-et-architecture.md @@ -0,0 +1,127 @@ +# Note de Conception : Fondements Épistémologiques et Architecture du Contrôle d'Interface Graphique (CUA) + +> *« Réduire l'interaction avec un système d'exploitation à une suite de prédictions matricielles de coordonnées sur des captures d'écran revient à confondre la lecture d'un texte avec l'analyse chimique de l'encre sur le papier. L'illusion de la performance empirique ne saurait se substituer à la compréhension de la structure ontologique de la machine. »* +(cette note est propre a linux il faudra ecrire l'equivalant pour Win & MacOs) +--- + +## 1. La Dérive Téléologique : Du Mythe du Benchmark à la Réalité du Système + +Ces derniers mois, les agents autonomes d'interaction avec l'ordinateur (*Computer Use Agents* ou CUA) ont fait l'objet d'une frénésie d'annonces spectaculaires. L'industrie de l'intelligence artificielle s'est enorgueillie de voir des modèles multimodaux résoudre des tâches sur des suites de benchmarks standardisées (OSWorld, Mind2Web, WebArena). Pourtant, cette course aux métriques brutes dissimule un désalignement fondamental entre les objectifs des concepteurs d'agents et la réalité structurelle de l'informatique bureautique. + +Pour satisfaire ces bancs d'essai, l'industrie a majoritairement embrassé une approche réductionniste : **le mimétisme servile de l'humain à l'écran**. Parce qu'un être humain regarde un moniteur, déplace une souris mécanique et frappe sur un clavier physique, on a postulé que l'agent devait être contraint à cette même phénoménologie superficielle. L'interaction homme-machine a ainsi été aplatie en une succession aveugle d'instantanés graphiques (*screenshots* PNG), d'appels de vision lourds et de clics projetés aux coordonnées $(x, y)$. + +Cette approche repose sur une erreur épistémologique majeure : **confondre le proxy avec l'essence**. L'écran n'est pas le système d'exploitation ; il n'en est que la projection rasterisée éphémère, destinée aux imperfections de la vision biologique. Un système d'exploitation n'est pas une fresque de pixels : c'est un ensemble complexe et hiérarchisé de processus en mémoire vive, d'arbres sémantiques d'accessibilité (AT-SPI / D-Bus), de gestionnaires de fenêtres topologiques (compositeurs X11 et Wayland) et de sous-systèmes d'événements du noyau (*evdev*, *uinput*). + +En forçant un grand modèle de langage à scruter le bureau à travers le trou de serrure d'une capture d'écran tous les trois dixièmes de seconde, on l'assujettit à deux tares rédhibitoires : +1. **La tyrannie du temps aller-retour (RTT) :** Chaque micro-action motrice exige un cycle d'inférence complet de 2 à 5 secondes. Une tâche élémentaire de dix gestes produit une latence absurde de 30 à 50 secondes, condamnant l'agent à l'échec face à toute interface dynamique ou évanescente. +2. **La cécité sémantique et spatiale :** L'agent devient le jouet des décalages d'échelle (fractional scaling), des fenêtres volantes sous capture exclusive de pointeur (*click-away grab*), des collisions de processus aux identifiants partagés, et des déformations typographiques. + +Ce que nous cherchons n'est pas d'optimiser artificiellement un score sur un benchmark au prix d'une fragilité opérationnelle extrême. **Nous cherchons à rétablir l'alignement entre l'intelligence décisionnelle du modèle et l'architecture réelle de la machine qu'il orchestre.** + +--- + +## 2. La Déconstruction Empirique des Dogmes + +Pour dégager l'architecture optimale, notre démarche a refusé tout a priori dogmatique. Nous avons confronté sept structures candidates fondamentales (A1 à A7) à quatorze épreuves empiriques rigoureuses, couvrant l'ensemble du spectre des pathologies graphiques Linux. + +Les résultats de cette confrontation ont agi comme un révélateur impitoyable, dynamitant successivement les chapelles techniques : + +### L'Échec du « Tout-Sémantique » (L'impasse de l'Accessibilité Pure) +L'approche formaliste (Architecture A3), qui postule qu'un système bien conçu doit être manipulable uniquement par son arbre d'accessibilité (AT-SPI) en mémoire, s'effondre avec fracas dès qu'elle rencontre la réalité du Web moderne et des applications créatives. Face à Figma, Blender ou aux moteurs de rendu WebGL compilés en WebAssembly (Cas **S-01**), l'arbre d'accessibilité s'éteint : il ne renvoie qu'un conteneur opaque sans enfants (`role="canvas"`). Dépourvue d'yeux et d'actuation spatiale, la sémantique pure sombre dans la paralysie totale. + +### L'Échec du « Tout-Pixel » (L'illusion de la Vision Pure) +À l'opposé, l'approche purement visuelle (Architecture A2), prônée par les partisans du VLM universel, se révèle d'une naïveté coûteuse. Dès lors que l'interface impose une contrainte temporelle stricte — comme un bouton d'annulation évanescent s'auto-détruisant après trois secondes (Cas **T-02**) ou un menu contextuel se refermant au moindre millimètre de déviation (Cas **T-01**) —, la latence d'inférence de la boucle multimodale dépasse le temps de vie du composant. De surcroît, le scaling fractionnel (Cas **V-02**) crée un décalage irréductible de 25% entre la grille logique du modèle et le framebuffer matériel du noyau, faisant atterrir les clics dans le vide. + +### L'Échec du « Tout-Shell » (L'incompatibilité de Confinement) +L'approche réductionniste par ligne de commande (Architecture A4), si séduisante par son déterminisme et son coût nul en images, bute contre l'isolation des environnements de bureau modernes. Elle est aveugle aux fenêtres enfants partageant le même PID (Expérience 02) et s'avère impuissante face aux portails de sécurité sandboxés Flatpak/Snap (Cas **O-02**) dont l'état interactif n'est pas négociable par un script bash distant. + +### L'Échec du « Tout-en-un OCR » (Le mirage du couplage hâtif) +L'idée séduisante de fusionner perception et action dans une boîte noire (Architecture A1) échoue systématiquement face aux pictogrammes purs (Cas **V-01**), où un croissant de lune 🌙 n'offre aucune prise typographique, et face aux cinématiques continues comme le glisser-déposer d'onglets (Cas **T-03**). + +--- + +## 3. Le Cheminement vers l'Architecture de Synthèse + +De ces décombres conceptuels émerge une vérité fondamentale : **l'environnement graphique d'un système d'exploitation n'est pas monolithique ; il est stratifié en strates de fidélité et d'abstraction.** + +L'agent ne doit pas chercher un outil universel miraculeux, mais adopter une **posture d'adéquation ontologique** : utiliser le moyen d'interaction dont le niveau d'abstraction correspond exactement à la nature de la cible. + +Ce raisonnement a gouverné la genèse de notre solution en deux piliers inséparables : + +```text +┌─────────────────────────────────────────────────────────────────────────┐ +│ PILIER 2 : LE MOTEUR REPL LOCAL │ +│ (Éradication du RTT : Programmation locale au lieu de RPC) │ +└────────────────────────────────────┬────────────────────────────────────┘ + │ orchestre dynamiquement +┌────────────────────────────────────▼────────────────────────────────────┐ +│ PILIER 1 : L'ESCALADE PROGRESSIVE (A7 REVISITÉ) │ +│ │ +│ NIVEAU L3 : Sémantique D-Bus & AT-SPI │ +│ ├── perform_action / set_value / activate_window (Window ID) │ +│ └── Idéal : Déterministe, instantané (<50ms), 0 image │ +│ │ +│ NIVEAU L2 : OCR Local Haute Précision │ +│ ├── find_text (RapidOCR local -> coordonnées) │ +│ └── Intermédiaire : Cible textuelle sans instrumentation AT-SPI │ +│ │ +│ NIVEAU L1 : Matériel & Vision Cartésienne uinput │ +│ ├── screen_capture (grille numérotée) / mouse_drag_smooth / key_tap │ +│ └── Filet de sécurité ultime : Surfaces GPU WebGL, Canvas, Dessin │ +│ │ +│ COUCHE SYSTÈME TRANSVERSALE : PTY Shell Interactif │ +│ └── process_run (Bypass sécurisé Polkit / sudo -S) │ +└─────────────────────────────────────────────────────────────────────────┘ +``` + +--- + +## 4. L'Architecture Proposée : Les Deux Piliers Fondateurs + +### Pilier I : L'Escalade Progressive et l'Hybridation Bidirectionnelle +L'architecture abandonne l'égalitarisme des outils pour une hiérarchie stricte d'efficience cognitive : + +1. **La Priorité Sémantique (Niveau L3 - Mémoire RAM) :** + L'agent tente systématiquement en premier lieu de s'interfacer avec le bus de communication D-Bus (`at-spi2-core`). S'il doit valider une modale, cliquer sur un bouton ou changer de bureau virtuel, il invoque `perform_action` ou transmet le `Window ID` au compositeur. L'action est consommée en mémoire vive en moins de cinquante millisecondes, sans déplacer le curseur, sans émettre de paquet vidéo, insensible aux déformations d'échelle et invulnérable aux disparitions subites de fenêtres. +2. **Le Décrochage Optique Local (Niveau L2 - OCR Découplé) :** + Si l'arbre d'accessibilité est incomplet mais que la cible est typographique, le serveur traite localement le framebuffer sans l'envoyer au modèle, renvoyant des coordonnées nettes sans inflation de tokens. +3. **Le Recours Spatial Matériel (Niveau L1 - Noyau & Pixels) :** + Si l'interface est un canevas graphique hermétique (Figma, jeux), l'agent bascule de façon fluide sur le niveau L1 : superposition d'une grille cartésienne absolue et émission de signaux noyau via `uinput` et `evdev`. +4. **L'Hybridation Coopérative L1 $\leftrightarrow$ L3 :** + L'architecture ne se contente pas d'escalader en cas d'échec ; elle fait dialoguer les couches. L'exemple paradigmatique est celui du sélecteur de fichiers XDG (Cas **O-02**) : l'agent constate en L3 l'absence de champ textuel libre, envoie en L1 la combinaison matérielle `Ctrl+L` pour forcer l'affichage de la barre de chemin par le toolkit GTK, puis rebascule instantanément en L3 pour injecter la chaîne `/tmp/test.txt` avec une exactitude typographique parfaite. + +### Pilier II : L'Émancipation Temporelle par le Moteur REPL (CodeAct) +L'Escalade Progressive, bien que théoriquement parfaite, restait menacée par la friction temporelle du protocole réseau MCP lui-même. Lorsque l'agent devait vérifier l'apparition d'un toast toutes les deux cents millisecondes ou exécuter un glisser-déposer sans relâcher la pression, l'interposition du protocole JSON-RPC réintroduisait la latence du réseau. + +La réponse architecturale réside dans **l'inversion du contrôle via un REPL local** : +* Plutôt que d'agir comme un marionnettiste maladroit tirant sur des ficelles distantes appel d'outil par appel d'outil, **l'agent projette son raisonnement sous forme de code Python exécutable localement**. +* Le serveur MCP expose une primitive souveraine : `execute_script`. +* L'agent exécute localement ses vérifications conditionnelles et trace ses courbes cinématiques `mouse_drag_smooth` **directement en mémoire hôte**, sans la latence des aller-retours réseau JSON-RPC. +* Le RTT s'effondre : la totalité de la cinématique se résout en **un seul aller-retour cognitif**. + +--- + +## 5. Synthèse des Invariants Système & Conclusion + +Au terme de cette recherche, treize outils chirurgicaux d'interaction directe (complétés par deux outils d'enregistrement vidéo pour la traçabilité continue) suffisent à garantir une couverture intégrale (100% de succès sur les douze cas limites réputés insolubles) : + +1. **`execute_script`** (Cerveau d'exécution locale, abolition de la latence RTT) +2. **`process_run`** (Contournement PTY des verrous Polkit/Wayland) +3. **`process_list`** (Introspection `/proc` anti-doublon) +4. **`activate_window`** (Commutation compositeur par Window ID) +5. **`get_app_state`** (Extraction textuelle AT-SPI ultra-légère) +6. **`perform_action`** (Actuation sémantique en mémoire RAM) +7. **`set_value`** (Mutation directe de variables d'entrée) +8. **`find_text`** (Localisation OCR sans transmission d'image) +9. **`screen_capture`** (Perception matricielle calibrée avec grille 100px) +10. **`mouse_click_at`** (Actuation uinput ultime) +11. **`mouse_drag_smooth`** (Flux cinématique continu d'arrachement) +12. **`mouse_scroll`** (Matérialisation active d'états virtualisés) +13. **`key_tap`** (Raccourcis conventionnels système et applicatifs) +14. **`gui_start_video_recording`** (Capture vidéo continue du flux d'écran pour audit comportemental et relecture déterministe) +15. **`gui_stop_video_recording`** (Finalisation sécurisée du conteneur vidéo et nettoyage des ressources ffmpeg) + +### Épilogue : De l'Artifice à la Maîtrise +Cette note de conception ne propose pas une énième surcouche heuristique. Elle acte le passage d'une informatique d'illusion — où l'on tentait de faire croire qu'un réseau de neurones « voyait » un bureau comme un œil humain — à une informatique de vérité mécanique. + +En respectant l'architecture interne des systèmes d'exploitation plutôt qu'en singeant leurs manifestations de surface, nous donnons à l'intelligence artificielle les moyens d'opérer non plus comme un observateur aveuglé par les reflets de l'écran, mais comme un véritable co-processeur cognitif intégré à la machine. diff --git a/pyproject.toml b/pyproject.toml index 0cffeae..0261953 100644 --- a/pyproject.toml +++ b/pyproject.toml @@ -5,7 +5,7 @@ build-backend = "hatchling.build" [project] name = "gui-agent" version = "0.1.0" -description = "Monolithic FastMCP GUI Agent Server for Linux Desktop Computer Use" +description = "Monolithic FastMCP GUI Agent Server for Desktop Computer Use" readme = "README.md" requires-python = ">=3.10" license = { text = "MIT" } @@ -60,9 +60,16 @@ dev = [ [project.scripts] gui-agent = "gui_agent.server:main" mcp-gui-server = "gui_agent.server:main" +gui-agent-atspi-py = "gui_agent.layers.accessibility:main_cli" [tool.hatch.build.targets.wheel] -packages = ["gui_agent"] +packages = ["linux"] + +[tool.hatch.build.targets.wheel.sources] +"linux" = "gui_agent" + +[tool.hatch.build.targets.wheel.hooks.custom] +path = "hatch_build.py" [tool.ruff] line-length = 120 @@ -109,6 +116,7 @@ max-complexity = 25 [tool.mypy] python_version = "3.13" +mypy_path = "." ignore_missing_imports = true check_untyped_defs = true disallow_incomplete_defs = true @@ -123,12 +131,12 @@ exclude = [ ] [tool.bandit] -exclude_dirs = ["venv", ".venv", "screenshots", ".GCC", "dist", "build"] +exclude_dirs = ["venv", ".venv", "screenshots", ".GCC", "dist", "build", "linux/tests", "linux/examples", "tests", "examples"] skips = ["B603", "B607", "B404"] [tool.pytest.ini_options] -testpaths = ["tests"] +testpaths = ["linux/tests"] python_files = ["test_*.py"] -pythonpath = ["."] +pythonpath = ["linux", "."] diff --git a/uninstall.sh b/uninstall.sh deleted file mode 100755 index 2f8133f..0000000 --- a/uninstall.sh +++ /dev/null @@ -1,180 +0,0 @@ -#!/usr/bin/env bash -# ============================================================================== -# GUI Agent - Automated Uninstallation Script (Linux / macOS) -# Method: Clean removal of uv tool, MCP registrations, and optional data purge -# ============================================================================== - -set -euo pipefail - -# ANSI Color codes -RED='\033[0;31m' -GREEN='\033[0;32m' -YELLOW='\033[1;33m' -BLUE='\033[0;34m' -CYAN='\033[0;36m' -BOLD='\033[1m' -NC='\033[0m' # No Color - -# Default options -NON_INTERACTIVE=false -DRY_RUN=false -PURGE_DATA=false -PACKAGE_NAME="gui-agent" - -log_info() { - echo -e "${BLUE}[INFO]${NC} $1" -} - -log_success() { - echo -e "${GREEN}[SUCCESS]${NC} $1" -} - -log_warn() { - echo -e "${YELLOW}[WARN]${NC} $1" -} - -log_error() { - echo -e "${RED}[ERROR]${NC} $1" >&2 -} - -print_banner() { - echo -e "${CYAN}${BOLD}" - echo "============================================================" - echo " GUI Agent - Clean Uninstaller (Linux / macOS) " - echo "============================================================" - echo -e "${NC}" -} - -show_help() { - print_banner - echo "Usage: ./uninstall.sh [OPTIONS]" - echo "" - echo "Options:" - echo " -y, --yes, --no-prompt Non-interactive mode (accept defaults)" - echo " -p, --purge-data Purge generated screenshots and runtime data" - echo " -d, --dry-run Show uninstallation steps without modifying system" - echo " -h, --help Show this help message and exit" - echo "" -} - -# Parse command line arguments -while [[ $# -gt 0 ]]; do - case "$1" in - -y|--yes|--no-prompt) - NON_INTERACTIVE=true - shift - ;; - -p|--purge-data) - PURGE_DATA=true - shift - ;; - -d|--dry-run) - DRY_RUN=true - shift - ;; - -h|--help) - show_help - exit 0 - ;; - *) - log_error "Option inconnue : $1" - show_help - exit 1 - ;; - esac -done - -print_banner - -# Step 1: Remove Isolated uv tool -log_info "1/3 - Désinstallation du package '$PACKAGE_NAME' via Astral 'uv'..." -UV_PATH="$(command -v uv 2>/dev/null || echo "${HOME}/.local/bin/uv")" - -if [[ -x "$UV_PATH" ]]; then - if [[ "$DRY_RUN" == "true" ]]; then - log_info "[Dry-Run] $UV_PATH tool uninstall $PACKAGE_NAME" - else - "$UV_PATH" tool uninstall "$PACKAGE_NAME" 2>/dev/null || log_warn "Package '$PACKAGE_NAME' n'était pas présent dans les outils uv." - log_success "Outil '$PACKAGE_NAME' désinstallé avec succès de uv." - fi -else - log_warn "Gestionnaire 'uv' non détecté. Aucun outil uv à désinstaller." -fi - -# Step 2: Remove MCP Client Registrations -log_info "2/3 - Nettoyage des configurations des clients MCP..." - -if [[ "$DRY_RUN" == "true" ]]; then - log_info "[Dry-Run] claude mcp remove gui-agent" - log_info "[Dry-Run] Suppression de l'entrée 'gui-agent' dans ~/.gemini/config/mcp_config.json" -else - # 1. Claude Code CLI - if command -v claude >/dev/null 2>&1; then - log_info "Suppression du serveur MCP dans Claude Code..." - claude mcp remove "$PACKAGE_NAME" 2>/dev/null || true - log_success "Configuration Claude Code nettoyée." - fi - - # 2. Antigravity CLI Config (~/.gemini/config/mcp_config.json) - ANTIGRAVITY_CONFIG_FILE="${HOME}/.gemini/config/mcp_config.json" - if [[ -f "$ANTIGRAVITY_CONFIG_FILE" ]]; then - log_info "Nettoyage de la configuration Antigravity CLI..." - if [[ -x "$UV_PATH" ]]; then - "$UV_PATH" run python - <