Skip to content

feat(input): add native X11 xdotool pointer clicks - #70

Merged
avifenesh merged 2 commits into
agent-sh:mainfrom
UniqueDing:feat/x11-xdotool-pointer
Aug 4, 2026
Merged

feat(input): add native X11 xdotool pointer clicks#70
avifenesh merged 2 commits into
agent-sh:mainfrom
UniqueDing:feat/x11-xdotool-pointer

Conversation

@UniqueDing

@UniqueDing UniqueDing commented Jul 31, 2026

Copy link
Copy Markdown
Contributor

Summary

  • Prefer xdotool/XTEST for coordinate clicks in explicit native X11 sessions.
  • Preserve the existing pointer fallback paths if xdotool is unavailable or a command fails.
  • Report native-X11 xdotool availability in diagnostics/readiness.

Closes #64

@UniqueDing
UniqueDing requested a review from avifenesh as a code owner July 31, 2026 18:52
@gemini-code-assist

Copy link
Copy Markdown

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.

@avifenesh avifenesh left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@UniqueDing, thank you for the careful DDE/X11 report, the real dual-display validation, and for implementing this.

I also owe you an apology: while your branch was in flight, I merged #69 and #72 through the same input and diagnostics code without first reconciling #70. That created these conflicts and independently replaced much of your diagnostics/process work. That is on me, not on your contribution.

The core #64 fix is still missing from main: native-X11 coordinate clicks still do not use xdotool. I would like to keep this PR and merge that work after a focused rebase onto 4b4126c.

Please update it as follows:

  • Drop the behavioral src/diagnostics.rs diff; main now has the broader readiness, ordering, and override handling.
  • Place native-X11 xdotool clicking after the absolute-pointer attempt and before ydotool.
  • Require explicit XDG_SESSION_TYPE=x11, nonempty DISPLAY, and no active WAYLAND_DISPLAY; honor COMPUTER_USE_LINUX_FORCE_YDOTOOL_POINTER.
  • Use one supervised command: xdotool mousemove --sync -- X Y click --repeat N BUTTON.
  • Fall back only if xdotool cannot launch. If it starts and fails, return the error rather than risking duplicate clicks.
  • Resolve button support before moving, preserving correct fallback for extended buttons.
  • Add focused backend/failure tests plus the short README and changelog updates discussed in #64.

If you would rather not handle the conflict resolution caused by my overlapping changes, I can update your branch while preserving your authorship and credit. Your original diagnosis and native-X11 backend are still the valuable part of this fix. Thank you again.

@UniqueDing

Copy link
Copy Markdown
Contributor Author

I reproduced an old-xdotool compatibility issue on Deepin’s packaged
xdotool 3.20160805.1.

The requested command:

xdotool mousemove --sync -- X Y click --repeat N BUTTON

times out when the pointer is already at (X, Y). If the pointer is moved
away first, the same command succeeds. Newer upstream xdotool versions fixed
this, but distribution packages may still carry the older behavior.

Would you accept this conditional compatibility path while retaining one
supervised xdotool process per click?

  • target differs from current pointer position:
    xdotool mousemove --sync -- X Y click --repeat N BUTTON
  • target already equals current pointer position:
    xdotool mousemove -- X Y click --repeat N BUTTON

This avoids the old same-coordinate --sync hang without relying on a
distribution-specific xdotool version threshold. Spawn failure would still
fall back; a launched nonzero command would still return an error without
replaying the click.

@avifenesh

Copy link
Copy Markdown
Collaborator

Thank you for checking this against the distro-packaged version. I reproduced the issue in an isolated native-X11 workspace using exactly xdotool 3.20160805.1 with XTEST. With the pointer already at (250, 250), the single-process mousemove --sync ... click command remained blocked until killed and emitted no button event.

I also tested the unsynchronized single-process form both from a different starting position and with the pointer already at the target:

xdotool mousemove -- X Y click --repeat N BUTTON

In both cases it completed immediately, and xev recorded the press/release at the exact requested root coordinates. I also verified a repeated right-button click (--repeat 2 3) at the requested target.

That changes my earlier --sync request: please omit --sync unconditionally rather than adding a current-position preflight and conditional path. A preflight would add process/state complexity and retain a race where the pointer reaches the target after the check. The single unsynchronized command preserves the required move-then-click ordering in this test and works on the affected old package.

The other review constraints still apply: resolve button support before launching or moving; try the absolute pointer first; gate this to explicit native X11 and honor COMPUTER_USE_LINUX_FORCE_YDOTOOL_POINTER; fall back only when xdotool cannot launch; and return a launched command failure without replaying the click. Thank you for surfacing this compatibility issue.

@UniqueDing
UniqueDing force-pushed the feat/x11-xdotool-pointer branch from 6854e06 to 7a4e2f5 Compare August 4, 2026 07:28
@avifenesh

Copy link
Copy Markdown
Collaborator

Great, thanks!

@avifenesh
avifenesh merged commit 7faa0da into agent-sh:main Aug 4, 2026
15 checks passed
@avifenesh avifenesh mentioned this pull request Aug 4, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

X11/DDE MCP click falls back to ydotool; add xdotool pointer backend

2 participants