fix(tao): forward Linux IME preedit into Compose - #613
Merged
Conversation
The Linux backend pinned `gtk::IMContextSimple` — GTK's built-in fallback, which only knows Compose sequences and Ctrl+Shift+U. It never reaches the system input method, so CJK input was not merely degraded on Linux, it was impossible: typing `aiueo` and confirming left a single stray `a` behind. `connect_preedit_changed` was never wired either, and `filter_keypress` ran *after* the key had already been handed to the app, with its result discarded. Switch to `gtk::IMMulticontext`, which resolves the platform module the same way GTK's own text widgets do (ibus / fcitx5 through the GTK immodule on X11, the text-input-v3 client on Wayland), and route its signals into the `ImePreedit` / `ImeCommit` events added for macOS in NucleusFramework#595 and extended to Windows in NucleusFramework#558: - `preedit-changed` -> ImePreedit, `preedit-end` -> ImePreedit("") - `commit` -> ImeCommit while composing, ReceivedImeText otherwise (the same split as the Windows backend's GCS_RESULTSTR vs. plain WM_CHAR) - the input context now follows the window's focus instead of latching `focus_in` once at construction Keys an input method consumes must not also reach Compose, or the Return that confirms a conversion goes on to insert a newline and the BackSpace that edits the composition deletes committed text. `filter_keypress` now runs first and stops propagation — but on its own that is not enough, because only the *press* is withheld: on Wayland the compositor filters those keys out of the stream before the client sees them (text-input-v3 has no notion of "filtered"), while the matching release arrives regardless. Compose fires `clickable`'s onClick on KeyUp, so a release with no press behind it activates whatever holds focus. Track the keycodes whose press was actually forwarded and drop releases that have no match; this covers the X11 path too, where `filter_keypress` withholds the press in-process for the same effect. Finally, implement `set_ime_position`, which was a `//TODO` stub, so the candidate window follows the caret (NucleusFramework#558). The input context lives on the event-loop side, so the caret goes through the window request channel like every other window mutation, and `platform/linux/ime.rs` supplies the `nativeSetImeRect` JNI entry point the JVM side already calls on macOS and Windows — same contract, the caret's bottom edge in window-local physical pixels. Verified on GNOME 46 (Wayland session) with ibus + Mozc, on both the Wayland path and the XWayland one (`NUCLEUS_TAO_LINUX_RENDERER=x11`): inline preedit, no newline from the confirming Return, BackSpace confined to the composition, committed text inserted exactly once, the candidate window tracking the caret, and no regression in ASCII input or in Ctrl+C / Ctrl+V / Ctrl+Return.
`ModifiersChanged` was only ever emitted when the key *itself* was a modifier, so Compose's view of Ctrl / Shift / Alt / Super was built up from press and release bookkeeping. That holds only as long as every release arrives, and it does not: put an input method in the event path on X11 and modifier releases go missing, because ibus re-injects what it forwards rather than passing the original events through. A dropped Control release then leaves Compose believing Ctrl is held for the rest of the session — a plain Return reads as Ctrl+Return, and the app fires whatever it binds to that instead of inserting a newline. Pressing and releasing Ctrl by hand was the only way out. GDK reports the live modifier mask on every event, so derive the state from the event each time and publish it whenever it changed. That self-heals on the very next keystroke no matter which release was lost, and it costs nothing when nothing changed. It also handles the asymmetric reports seen in the same traces, where a `Meta_L` press is answered by an `Alt_L` release. Found while verifying NucleusFramework#558 on the XWayland path, but it is not IME-specific: any lost modifier release had the same effect. It only stayed hidden because the Linux backend never connected to a system input method until now.
The decisions the Linux input-method path makes were spread across GTK
closures: whether a key event reaches Compose, whether committed text replaces
a preedit or stands on its own, whether the modifier state needs republishing.
All three are about *sequences* — a press withheld but its release delivered, a
modifier whose release never arrives, a commit that lands with no composition
open — and none of them could be reproduced from a unit test while the state
lived in the closures.
Move that state into `platform_impl::linux::ime::ImeState` and leave the
signal handlers doing nothing but translating GTK callbacks into calls on it.
The Windows backend draws the same line with its `ImeSource` trait; Linux needs
no trait, because there is nothing to read back — GTK pushes everything, so
plain data suffices.
The tests replay sequences taken from real traces on GNOME 46 with ibus + Mozc:
- `release_without_a_press_is_dropped` — Wayland withholds the press of a key
the input method consumed and delivers the release anyway
- `filtered_press_withholds_its_release` — the X11 shape of the same thing,
where `filter_keypress` claims the press in-process
- `confirming_return_does_not_reach_compose` — the observed failure: the
Return that confirmed a conversion also activated the focused control
- `lost_modifier_release_recovers_on_the_next_key` and
`asymmetric_modifier_reports_settle` — a swallowed Control release, and the
`Meta_L` press answered by an `Alt_L` release that the same traces show
- `shortcuts_pass_through` — Ctrl+C and friends must survive the key gate
No behaviour change; this is the same logic addressed through one owner.
Run with `cargo test --lib` from `src/main/native/vendor/tao`.
The candidate window tracked the caret and covered it: Mozc's "Tab to select" hint sat exactly on the composition it was describing, so the underlined preedit was never visible. Three separate mistakes stacked up to land it there. The first is that the caret was reduced to a point. IMM32 takes a point and hangs the candidate list off it, which is all `set_ime_position` can carry, so the Windows implementation passes the caret's bottom edge and is done. GTK is area-based instead: `gtk_im_context_set_cursor_location` is given the region the cursor covers and the input method keeps its own windows off that area. A 1x1 rect tells it there is nothing to avoid. `WindowExtUnix` grows `set_ime_cursor_area`, mirroring the winit API of the same name, and `set_ime_position` becomes a zero-sized area at the point — the documented behaviour of the cross-platform call is unchanged. The second is the coordinate space: GDK works in logical pixels and the caret was handed over in physical ones. Invisible at scale 1, wrong on HiDPI. The third is what actually put the window on top of the text. The caret arrives in client-area coordinates — what `set_ime_position` documents — but GTK wants it relative to the toplevel GdkWindow, which on a client-side-decorated window also spans the invisible resize border and drop shadow. Measured here that is (26, 23), so the input method placed the caret a shadow's height too high and drew its list over the line being typed. Translating by the content widget's allocation is the same correction the Wayland subsurface already applies to position the Compose surface, so the two now agree on where the caret is. Verified on GNOME 46 (Wayland session) with ibus + Mozc: the hint and the candidate list both sit below the composition and follow the caret across lines.
Collaborator
|
Amazing ! Thanks ! |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #558.
Linux was the last backend without input-method support, and it was not a
matter of degree:
gtk::IMContextSimpleis GTK's built-in fallback for Composesequences and
Ctrl+Shift+U, and it never talks to the system input method. SoCJK input was impossible, not merely awkward. Typing
aiueoand confirmingleft a single stray
abehind — five keystrokes swallowed, one leaked.This is the other half of #558, after #603 did Windows.
What was wrong
platform_impl/linux/event_loop.rscarried the upstream// TODO Add actual IME from systemand four consequences of it:IMContextSimplewas pinned, so ibus / fcitx5 / Mozc and the Waylandtext-input protocol were never reached.
connect_preedit_changedwas never wired, so the composition was invisible.filter_keypressran after the key had already been handed to the app,and its result was discarded — the Linux twin of the
VK_PROCESSKEYleakfixed for Windows.
set_ime_positionwas a//TODOstub, so the candidate window had no ideawhere the caret was.
What this does
IMMulticontextinstead ofIMContextSimple, which resolves the platformmodule the same way GTK's own text widgets do — the GTK immodule on X11, the
text-input-v3 client on Wayland — and routes its signals into the
ImePreedit/ImeCommitevents added for macOS in #595 and extended toWindows in #558.
commitsplits on whether a composition is open, the samedistinction the Windows backend draws between
GCS_RESULTSTRand a plainWM_CHAR. The context now follows the window's focus instead of latchingfocus_inonce at construction.Keys the input method consumed no longer reach Compose.
filter_keypressruns first and stops propagation, but that alone is not enough, and the reason
is worth spelling out: only the press is withheld. On Wayland the compositor
filters those keys out of the stream before the client sees them — text-input-v3
has no way to say "filtered" — while the matching release arrives regardless.
Compose fires
clickable's onClick on KeyUp, so a release with no press behindit activates whatever holds focus: the Return that merely confirmed a
conversion went on to press a button. Tracking which presses were actually
forwarded and dropping unmatched releases covers the X11 path too, where
filter_keypresswithholds the press in-process for the same net effect.Modifier state is derived from each event rather than accumulated. This one
is not IME-specific, and it is in its own commit.
ModifiersChangedwas onlyemitted when the key itself was a modifier, which holds up only while every
release arrives — and on X11 they do not, because ibus re-injects what it
forwards. A dropped Control release left Compose reading every later plain
Return as Ctrl+Return for the rest of the session. GDK reports the live mask on
every event, so deriving the state from the event self-heals on the next
keystroke. The same traces also show a
Meta_Lpress answered by anAlt_Lrelease, which pair-based bookkeeping cannot survive either.
The candidate window is kept off the caret. GTK is area-based where IMM32
is point-based:
gtk_im_context_set_cursor_locationreceives the region thecursor covers and the input method stays off it, so the point that
set_ime_positioncan carry is not enough — pass one and Mozc's "Tab toselect" hint sits on the composition it is describing.
WindowExtUnixgrowsset_ime_cursor_area, mirroring the winit API of that name;set_ime_positionbecomes a zero-sized area at the point, so thecross-platform call behaves as documented. Two further corrections were needed
to land it in the right place: GDK works in logical pixels (invisible at scale
1, wrong on HiDPI), and the caret arrives in client-area coordinates while GTK
wants them relative to the toplevel GdkWindow — which on a CSD window also
spans the invisible resize border and drop shadow. Measured here that offset is
(26, 23); without it the input method placed the caret a shadow's height too
high and drew its list straight over the text.
Tests
platform_impl::linux::ime::ImeStateowns the decisions the signal handlersused to make inline, so they can be replayed without a display. Ten cases, all
taken from real traces on GNOME 46 with ibus + Mozc — the withheld press with a
delivered release on both Wayland and X11, the confirming Return that activated
a control, the swallowed Control release, the asymmetric
Meta_L/Alt_Lreport, and a regression guard that Ctrl+C and friends still pass the gate.
cargo test --libfromsrc/main/native/vendor/tao. The Windows backend drawsthe same line with its
ImeSourcetrait; Linux needs no trait, because GTKpushes everything and there is nothing to read back.
Verification
Real-device testing on GNOME 46 (Wayland session) with ibus + Mozc, on both
the Wayland path and the XWayland one (
NUCLEUS_TAO_LINUX_RENDERER=x11) — thetwo resolve different modules (
im-wayland.sovsim-ibus.so) and consumekeys by different mechanisms, so both were exercised:
it across lines
:decorated-window-tao:checkpasses (ktlint / detekt / test / apiCheck /koverVerify).
One question for you
WindowExtUnix::set_ime_cursor_areaadds public API. It is named after thewinit call with the same signature, and the area is genuinely required — GTK
cannot be told to avoid a region it was never given. If you would rather see
this shaped differently (widening
set_ime_position, keeping itpub(crate)and driving it from elsewhere, or another name), say so and I willchange it.