Skip to content

fix(tao): forward Linux IME preedit into Compose - #613

Merged
kdroidFilter merged 4 commits into
NucleusFramework:mainfrom
takke:fix/558-linux-ime
Aug 29, 2026
Merged

fix(tao): forward Linux IME preedit into Compose#613
kdroidFilter merged 4 commits into
NucleusFramework:mainfrom
takke:fix/558-linux-ime

Conversation

@takke

@takke takke commented Aug 28, 2026

Copy link
Copy Markdown
Contributor

Closes #558.

Linux was the last backend without input-method support, and it was not a
matter of degree: gtk::IMContextSimple is GTK's built-in fallback for Compose
sequences and Ctrl+Shift+U, and it never talks to the system input method. So
CJK input was impossible, not merely awkward. Typing aiueo and confirming
left a single stray a behind — five keystrokes swallowed, one leaked.

This is the other half of #558, after #603 did Windows.

What was wrong

platform_impl/linux/event_loop.rs carried the upstream // TODO Add actual IME from system and four consequences of it:

  1. IMContextSimple was pinned, so ibus / fcitx5 / Mozc and the Wayland
    text-input protocol were never reached.
  2. connect_preedit_changed was never wired, so the composition was invisible.
  3. filter_keypress ran after the key had already been handed to the app,
    and its result was discarded — the Linux twin of the VK_PROCESSKEY leak
    fixed for Windows.
  4. set_ime_position was a //TODO stub, so the candidate window had no idea
    where the caret was.

What this does

IMMulticontext instead of IMContextSimple, which resolves the platform
module the same way GTK's own text widgets do — the GTK immodule on X11, the
text-input-v3 client on Wayland — and routes its signals into the
ImePreedit / ImeCommit events added for macOS in #595 and extended to
Windows in #558. commit splits on whether a composition is open, the same
distinction the Windows backend draws between GCS_RESULTSTR and a plain
WM_CHAR. The context now follows the window's focus instead of latching
focus_in once at construction.

Keys the input method consumed no longer reach Compose. filter_keypress
runs first and stops propagation, but that alone is not enough, and the reason
is worth spelling out: only the press is withheld. On Wayland the compositor
filters those keys out of the stream before the client sees them — text-input-v3
has no way to say "filtered" — while the matching release arrives regardless.
Compose fires clickable's onClick on KeyUp, so a release with no press behind
it activates whatever holds focus: the Return that merely confirmed a
conversion went on to press a button. Tracking which presses were actually
forwarded and dropping unmatched releases covers the X11 path too, where
filter_keypress withholds the press in-process for the same net effect.

Modifier state is derived from each event rather than accumulated. This one
is not IME-specific, and it is in its own commit. ModifiersChanged was only
emitted when the key itself was a modifier, which holds up only while every
release arrives — and on X11 they do not, because ibus re-injects what it
forwards. A dropped Control release left Compose reading every later plain
Return as Ctrl+Return for the rest of the session. GDK reports the live mask on
every event, so deriving the state from the event self-heals on the next
keystroke. The same traces also show a Meta_L press answered by an Alt_L
release, which pair-based bookkeeping cannot survive either.

The candidate window is kept off the caret. GTK is area-based where IMM32
is point-based: gtk_im_context_set_cursor_location receives the region the
cursor covers and the input method stays off it, so the point that
set_ime_position can carry is not enough — pass one and Mozc's "Tab to
select" hint sits on the composition it is describing. WindowExtUnix grows
set_ime_cursor_area, mirroring the winit API of that name;
set_ime_position becomes a zero-sized area at the point, so the
cross-platform call behaves as documented. Two further corrections were needed
to land it in the right place: GDK works in logical pixels (invisible at scale
1, wrong on HiDPI), and the caret arrives in client-area coordinates while GTK
wants them relative to the toplevel GdkWindow — which on a CSD window also
spans the invisible resize border and drop shadow. Measured here that offset is
(26, 23); without it the input method placed the caret a shadow's height too
high and drew its list straight over the text.

Tests

platform_impl::linux::ime::ImeState owns the decisions the signal handlers
used to make inline, so they can be replayed without a display. Ten cases, all
taken from real traces on GNOME 46 with ibus + Mozc — the withheld press with a
delivered release on both Wayland and X11, the confirming Return that activated
a control, the swallowed Control release, the asymmetric Meta_L/Alt_L
report, and a regression guard that Ctrl+C and friends still pass the gate.

cargo test --lib from src/main/native/vendor/tao. The Windows backend draws
the same line with its ImeSource trait; Linux needs no trait, because GTK
pushes everything and there is nothing to read back.

Verification

Real-device testing on GNOME 46 (Wayland session) with ibus + Mozc, on both
the Wayland path and the XWayland one (NUCLEUS_TAO_LINUX_RENDERER=x11) — the
two resolve different modules (im-wayland.so vs im-ibus.so) and consume
keys by different mechanisms, so both were exercised:

  • inline preedit while composing
  • the confirming Return inserts no newline and activates nothing
  • BackSpace during composition stays inside it
  • committed text is inserted exactly once
  • the candidate window and the suggestion hint sit below the caret and follow
    it across lines
  • no regression in ASCII input, or in Ctrl+C / Ctrl+V / Ctrl+Return

:decorated-window-tao:check passes (ktlint / detekt / test / apiCheck /
koverVerify).

One question for you

WindowExtUnix::set_ime_cursor_area adds public API. It is named after the
winit call with the same signature, and the area is genuinely required — GTK
cannot be told to avoid a region it was never given. If you would rather see
this shaped differently (widening set_ime_position, keeping it
pub(crate) and driving it from elsewhere, or another name), say so and I will
change it.

takke added 4 commits August 28, 2026 14:27
The Linux backend pinned `gtk::IMContextSimple` — GTK's built-in fallback,
which only knows Compose sequences and Ctrl+Shift+U. It never reaches the
system input method, so CJK input was not merely degraded on Linux, it was
impossible: typing `aiueo` and confirming left a single stray `a` behind.
`connect_preedit_changed` was never wired either, and `filter_keypress` ran
*after* the key had already been handed to the app, with its result discarded.

Switch to `gtk::IMMulticontext`, which resolves the platform module the same
way GTK's own text widgets do (ibus / fcitx5 through the GTK immodule on X11,
the text-input-v3 client on Wayland), and route its signals into the
`ImePreedit` / `ImeCommit` events added for macOS in NucleusFramework#595 and extended to
Windows in NucleusFramework#558:

  - `preedit-changed` -> ImePreedit, `preedit-end` -> ImePreedit("")
  - `commit` -> ImeCommit while composing, ReceivedImeText otherwise
    (the same split as the Windows backend's GCS_RESULTSTR vs. plain WM_CHAR)
  - the input context now follows the window's focus instead of latching
    `focus_in` once at construction

Keys an input method consumes must not also reach Compose, or the Return that
confirms a conversion goes on to insert a newline and the BackSpace that edits
the composition deletes committed text. `filter_keypress` now runs first and
stops propagation — but on its own that is not enough, because only the *press*
is withheld: on Wayland the compositor filters those keys out of the stream
before the client sees them (text-input-v3 has no notion of "filtered"), while
the matching release arrives regardless. Compose fires `clickable`'s onClick on
KeyUp, so a release with no press behind it activates whatever holds focus.
Track the keycodes whose press was actually forwarded and drop releases that
have no match; this covers the X11 path too, where `filter_keypress` withholds
the press in-process for the same effect.

Finally, implement `set_ime_position`, which was a `//TODO` stub, so the
candidate window follows the caret (NucleusFramework#558). The input context lives on the
event-loop side, so the caret goes through the window request channel like
every other window mutation, and `platform/linux/ime.rs` supplies the
`nativeSetImeRect` JNI entry point the JVM side already calls on macOS and
Windows — same contract, the caret's bottom edge in window-local physical
pixels.

Verified on GNOME 46 (Wayland session) with ibus + Mozc, on both the Wayland
path and the XWayland one (`NUCLEUS_TAO_LINUX_RENDERER=x11`): inline preedit,
no newline from the confirming Return, BackSpace confined to the composition,
committed text inserted exactly once, the candidate window tracking the caret,
and no regression in ASCII input or in Ctrl+C / Ctrl+V / Ctrl+Return.
`ModifiersChanged` was only ever emitted when the key *itself* was a modifier,
so Compose's view of Ctrl / Shift / Alt / Super was built up from press and
release bookkeeping. That holds only as long as every release arrives, and it
does not: put an input method in the event path on X11 and modifier releases go
missing, because ibus re-injects what it forwards rather than passing the
original events through. A dropped Control release then leaves Compose
believing Ctrl is held for the rest of the session — a plain Return reads as
Ctrl+Return, and the app fires whatever it binds to that instead of inserting a
newline. Pressing and releasing Ctrl by hand was the only way out.

GDK reports the live modifier mask on every event, so derive the state from the
event each time and publish it whenever it changed. That self-heals on the very
next keystroke no matter which release was lost, and it costs nothing when
nothing changed. It also handles the asymmetric reports seen in the same
traces, where a `Meta_L` press is answered by an `Alt_L` release.

Found while verifying NucleusFramework#558 on the XWayland path, but it is not IME-specific:
any lost modifier release had the same effect. It only stayed hidden because
the Linux backend never connected to a system input method until now.
The decisions the Linux input-method path makes were spread across GTK
closures: whether a key event reaches Compose, whether committed text replaces
a preedit or stands on its own, whether the modifier state needs republishing.
All three are about *sequences* — a press withheld but its release delivered, a
modifier whose release never arrives, a commit that lands with no composition
open — and none of them could be reproduced from a unit test while the state
lived in the closures.

Move that state into `platform_impl::linux::ime::ImeState` and leave the
signal handlers doing nothing but translating GTK callbacks into calls on it.
The Windows backend draws the same line with its `ImeSource` trait; Linux needs
no trait, because there is nothing to read back — GTK pushes everything, so
plain data suffices.

The tests replay sequences taken from real traces on GNOME 46 with ibus + Mozc:

  - `release_without_a_press_is_dropped` — Wayland withholds the press of a key
    the input method consumed and delivers the release anyway
  - `filtered_press_withholds_its_release` — the X11 shape of the same thing,
    where `filter_keypress` claims the press in-process
  - `confirming_return_does_not_reach_compose` — the observed failure: the
    Return that confirmed a conversion also activated the focused control
  - `lost_modifier_release_recovers_on_the_next_key` and
    `asymmetric_modifier_reports_settle` — a swallowed Control release, and the
    `Meta_L` press answered by an `Alt_L` release that the same traces show
  - `shortcuts_pass_through` — Ctrl+C and friends must survive the key gate

No behaviour change; this is the same logic addressed through one owner.
Run with `cargo test --lib` from `src/main/native/vendor/tao`.
The candidate window tracked the caret and covered it: Mozc's "Tab to select"
hint sat exactly on the composition it was describing, so the underlined
preedit was never visible. Three separate mistakes stacked up to land it there.

The first is that the caret was reduced to a point. IMM32 takes a point and
hangs the candidate list off it, which is all `set_ime_position` can carry, so
the Windows implementation passes the caret's bottom edge and is done. GTK is
area-based instead: `gtk_im_context_set_cursor_location` is given the region
the cursor covers and the input method keeps its own windows off that area. A
1x1 rect tells it there is nothing to avoid. `WindowExtUnix` grows
`set_ime_cursor_area`, mirroring the winit API of the same name, and
`set_ime_position` becomes a zero-sized area at the point — the documented
behaviour of the cross-platform call is unchanged.

The second is the coordinate space: GDK works in logical pixels and the caret
was handed over in physical ones. Invisible at scale 1, wrong on HiDPI.

The third is what actually put the window on top of the text. The caret arrives
in client-area coordinates — what `set_ime_position` documents — but GTK wants
it relative to the toplevel GdkWindow, which on a client-side-decorated window
also spans the invisible resize border and drop shadow. Measured here that is
(26, 23), so the input method placed the caret a shadow's height too high and
drew its list over the line being typed. Translating by the content widget's
allocation is the same correction the Wayland subsurface already applies to
position the Compose surface, so the two now agree on where the caret is.

Verified on GNOME 46 (Wayland session) with ibus + Mozc: the hint and the
candidate list both sit below the composition and follow the caret across
lines.
@kdroidFilter
kdroidFilter merged commit c00685c into NucleusFramework:main Aug 29, 2026
20 of 23 checks passed
@kdroidFilter

Copy link
Copy Markdown
Collaborator

Amazing ! Thanks !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tao/Linux: IME candidate window does not follow the caret

2 participants