KeyCmd is the companion iOS app for Openterface hardware -- a USB/Bluetooth KVM-style bridge that lets you control a target computer from your iPhone. The app sends HID keyboard, mouse, and gamepad commands over Bluetooth Low Energy (BLE) to the Openterface KVM device.
Requirements: iOS 17.0+, iPhone with BLE, Openterface KVM hardware
| Area | Capabilities |
|---|---|
| Keyboard & Mouse | Full QWERTY, OS-specific layouts (Win/mac/Linux), touchpad, scroll strips, numpad, modifier modes (sticky / momentary-chord), key repeat, Pro-mode Fn lock and long-press alternates |
| Gamepad | Preset layouts (Default, XYAB, XYAB Trackpad, Minecraft Java, Emu 6), dual analog sticks, D-pad, turbo, editable layout, custom key mapping, portrait & landscape |
| Shortcut Hub | Profiles for Standard, Blender, KiCAD, Fusion 360, Photoshop, VS Code, Nomad Sculpt; create/import/export/custom shortcuts, favorites, drag-and-drop |
| Macros | Record & replay key sequences, composite/special tokens, scheduling, repeating, nesting |
| Voice Input | Apple Speech / Whisper STT, AI text refinement (OpenAI-compatible), auto-send to target, clipboard integration, Unicode support |
| Customization | 8 accent color themes, dark/light mode, rows 2-3 strip profiles, haptic feedback, scroll sensitivity |
| Welcome | Keyboard Basic | Pro Keyboard |
|---|---|---|
| Welcome screen with mode selection | Basic QWERTY keyboard | Pro keyboard with shortcut strip |
| Touchpad | Gamepad | Numpad |
|---|---|---|
| Touchpad with mouse buttons & scroll strip | Gamepad preset layouts | Numeric keypad |
| Shortcut Hub | Macros | Voice Input |
|---|---|---|
| Shortcut profiles & editor | Macro creation & scheduler | Voice-to-text with AI refinement |
- Pair your iPhone with the Openterface KVM device via BLE (the app scans automatically on launch).
- Select a mode from the launch panel: Keyboard & Mouse, Gamepad, Shortcuts, Macros, or Voice Input.
- Set the target OS (Windows / macOS / Linux) so modifier keys and Unicode input match your machine.
- Start controlling your target computer.
- Basic mode -- dedicated full-screen keyboard with touchpad, no app header
- Pro mode -- composite layout with shortcut strips and OS-specific shortcut buttons
- Modifier behavior: choose between sticky (tap to toggle) or momentary-chord (press-and-hold, slide-to-lock)
- Chord sustain: control when HID modifier-down is sent -- immediately or only when a regular key is chorded
- Fn Lock (Pro mode): while locked,
q-ptype1-0,a-lbecome F1-F9,z/x/cbecome F10-F12, and Backspace sends Delete - Key repeat: long-press auto-repeat with a fixed 0.4 s initial delay and 0.08 s interval; Settings choose between repeat and press-and-hold behavior
- Long-press alternates (Pro mode): press and hold symbol keys to pick an alternate from a popup of up to three options
- Target OS setting: switches modifier labels (Win / Cmd / Super) and Unicode input method
- Relative mouse movement via gesture
- Left / Middle / Right buttons with hold and lock modes
- Drag mode toggle for drag-and-drop
- Double click support
- Vertical and horizontal scroll strips
- Configurable scroll sensitivity
- Pointer tip visual feedback with glow border
- Preset layouts: Default, XYAB, XYAB Trackpad, Minecraft Java, Emu 6; import and export presets as JSON
- Left analog stick sends W/A/S/D by default (each direction is remappable), pressing two keys together for diagonals, with a 0.6 activation threshold and a 0.1 dead zone
- Right analog stick controls mouse movement with dynamic acceleration
- D-Pad mapped to arrow keys
- Shoulder & trigger buttons (LB/RB/LT/RT or L1/R1/L2/R2)
- Editable layout -- drag components to reposition
- Custom key mapping -- remap any button
- Portrait and landscape orientations
- Built-in profiles: Standard, VS Code, Blender, KiCAD, Fusion 360, Photoshop, Nomad Sculpt
- Create custom profiles with categories and color-coded tabs
- Import / Export profiles via JSON and iOS share sheet
- "My Shortcuts" favorites system per profile
- Shortcut editor with modifier selection (Cmd/Ctrl/Shift/Alt) and key picker (A-Z, F1-F12, special keys)
- List and card display modes with drag-and-drop reordering
- Create macros with a label and key sequence
- Composite key syntax:
<CTRL>A</CTRL>means press Ctrl, press A, release all - Special tokens:
<ESC>,<BACK>,<ENTER>,<SPACE>,<TAB>,<INS>/<INSERT>,<DEL>/<DELETE>,<HOME>,<END>,<PGUP>/<PAGEUP>,<PGDN>/<PAGEDOWN>,<LEFT>,<RIGHT>,<UP>,<DOWN>,<F1>-<F12>,<DELAY1S>,<DELAY2S>,<DELAY3S>,<DELAY4S>,<DELAY5S>,<DELAY10S>. Tags are uppercase-only, and the quick-insert palette offers a subset (arrows, Home/End, F1-F12, the delays, and the modifier tags) - Modifier tags:
<CTRL>,<SHIFT>,<ALT>,<CMD>with matching closing tags - Macro nesting via
<Macro>label</Macro>references, or an inline<label>tag matching another macro's label - Scheduling: run at a specific date/time
- Repeating: N times or at intervals from 5 s to 1 h (slider, with 30s / 1m / 5m / 10m / 30m / 1h presets)
- Background thread execution for strict token ordering
- STT engines: Apple Speech (built-in) or Whisper, running on-device through whisper.cpp with downloadable
ggml-tiny.en,ggml-tinyandggml-base-yuemodels - Auto-pause on silence detection
- AI text refinement via OpenAI-compatible API for cleaner output
- Auto-send transcribed text directly to the target machine
- History of sent voice inputs with edit, resend, and save-as-macro actions
- Special token quick-insert buttons in the toolbar
| Category | Options |
|---|---|
| General | Theme (color family, follow-system or manual dark/light), Target System (Windows / macOS / Linux), touchpad options, clipboard monitoring on the Compose page (send clipboard content to target via Unicode), BLE key delay, haptic feedback, App Language |
| Keyboard | Modifier behavior (sticky / momentary-chord), chord sustain, long-press behavior (repeat vs. press-and-hold) -- opened from the gear button in each Keyboard & Mouse mode |
| Voice Input | STT engine selection, Whisper model management |
| AI Settings | Provider configuration (name, API base, model, API key), on-device local model downloads, system prompt roles, custom prompt editing, API connection test, request history |
- Theme system: 8 accent color families (Orange, Blue, Green, Pink, Purple, Red, Teal, Indigo) with dark/light mode (follow system or manual override)
- Rows 2-3 Strip Profiles: define slot overrides and custom shortcuts for the keyboard's rows 2-3, with a preview grid and JSON import/export in Shortcut Hub. Note that the on-screen keyboard still renders its own fixed rows 2-3, so saving a profile does not change the keyboard yet.
- Haptic feedback: Light taps for buttons, rigid ticks for scroll, medium for special actions, strong for macro execution
KeyCmd connects to the Openterface KVM device exclusively via Bluetooth Low Energy (BLE):
- Ensure the Openterface KVM device is powered on and broadcasting BLE
- The app automatically scans for devices with names starting with "openterface", "kvm" or "keymod"
- Tap the device to connect; RSSI signal strength is monitored in real time
- HID keyboard and mouse data is sent via the FFF2 BLE characteristic
| Item | Detail |
|---|---|
| Package | com.openterface.KeyCmd |
| Min OS | iOS 17.0 |
| HID Protocol | CH9329 UART over BLE |
| BLE Characteristic | FFF2 |
| STT Engine | Apple SFSpeechRecognizer / whisper.cpp (local ML) |
| AI Provider | OpenAI-compatible API (configurable base URL, model, key) |
| Unicode | Windows Alt+NumpadHex, macOS Option+Hex, Linux Ctrl+Shift+U+Hex |
| Core Library | Openterface C core (HID mapping, script tokenization) via submodule |
| Requirement | Detail |
|---|---|
| Xcode | Verified on Xcode 26.5 (Swift 6.3) |
| Submodule | KeyCmd/Openterface_Core -- the shared Openterface C core |
| Sibling packages | ../whisper.cpp, ../SwiftTerm, ../Libssh2Prebuild (see below) |
git clone --recurse-submodules https://github.com/TechxArtisanStudio/Openterface_KeyCmd_iOS.git
cd Openterface_KeyCmd_iOS
git submodule update --init --recursive # if the clone above did not fetch itKeyCmd.xcodeproj references three Swift packages by relative path, outside this
repository (packageReferences in KeyCmd.xcodeproj/project.pbxproj). They must sit
next to this project, under the same parent directory. If any is missing, package
resolution fails before a single source file is compiled.
A working layout looks like this:
workspaces/
├── whisper.cpp/ # voice input (Whisper STT)
├── SwiftTerm/ # terminal emulator view
├── Libssh2Prebuild/ # prebuilt libssh2 for SSH
└── Openterface_KeyCmd_iOS/
- Open
KeyCmd.xcodeprojin Xcode - Select your development team for code signing
- Build and run on a physical iPhone (BLE is not available on Simulator)
xcodebuild -project KeyCmd.xcodeproj -scheme KeyCmd -configuration Debug \
-destination 'platform=iOS Simulator,name=iPhone 17 Pro' \
build CODE_SIGNING_ALLOWED=NOThis produces an unsigned Simulator build, which is enough to verify that the app compiles. It is not a substitute for a signed device build, because BLE cannot be exercised on the Simulator.
KeyCmdUITests covers the Keyboard & Mouse Basic surface: the Touchpad / Keyboard /
Numpad submode tabs (including the landscape lock the Keyboard tab applies and the
portrait unlock on returning to Touchpad) and opening and dismissing the setup sheet
from the gear button.
./scripts/offline-packages.sh test # offline, iPhone 17 Pro Simulator
xcodebuild test -project KeyCmd.xcodeproj -scheme KeyCmdUITests \
-destination 'platform=iOS Simulator,name=iPhone 17 Pro' CODE_SIGNING_ALLOWED=NOThe KeyCmdUITests scheme is separate from KeyCmd on purpose: the app's own test
action keeps running only unit tests, so nothing gets slower for people who are not
touching the UI.
Two conventions worth knowing before adding cases:
- Elements are located by
accessibilityIdentifier(km-tab-*,km-submode-*,km-setup-button), not by key labels. The on-screen keyboard builds its keys from custom buttons whose labels change with the target OS, so label-based queries would break on the first layout change. - Tests launch with
-HasSeenLaunchPanel YES. The first-run mode picker is a full-screen overlay that swallows taps; the argument domain suppresses it without adding test-only branches to the app.
Cases cost a whole-tree accessibility snapshot per lookup, so their wall time tracks simulator state rather than logic: 5-17 s per case on a freshly booted device, and an order of magnitude worse after the simulator has been churned by many launches.
Beyond the sibling packages above, the project resolves 8 remote Swift packages and
clones mlx-swift's two C++ submodules (mlx, mlx-c). Every one of those is
fetched from GitHub during package resolution, and a single failed fetch aborts the
build before any source is compiled. On a slow or unreliable connection to GitHub this
makes the build effectively unbuildable.
scripts/offline-packages.sh removes that dependency by serving every one of those
URLs from a local Git mirror. It rewrites them with url.<mirror>.insteadOf injected
into the build's environment only -- no repository or global Git configuration is
touched -- and the mirror itself is gitignored.
./scripts/offline-packages.sh verify # confirm the mirror covers Package.resolved
./scripts/offline-packages.sh rules # show the URL rewrites a build applies
./scripts/offline-packages.sh build # xcodebuild, entirely offline
./scripts/offline-packages.sh test # xcodebuild test (KeyCmdUITests), offline
./scripts/offline-packages.sh refresh # mirror the 8 packages from GitHub (online)
./scripts/offline-packages.sh submodules # re-mirror mlx / mlx-c at their pinned SHA
./scripts/offline-packages.sh snapshot # pack the mirror for a teammate or CI
./scripts/offline-packages.sh restore # unpack such a snapshotFirst time, on a machine that can reach GitHub:
./scripts/offline-packages.sh refresh # packages + submodules, retried automatically
./scripts/offline-packages.sh snapshot # writes .offline-packages/mirror-snapshot.tar.gzThen, on any machine -- including one with no network at all:
./scripts/offline-packages.sh restore /path/to/mirror-snapshot.tar.gz
./scripts/offline-packages.sh buildThe mirror is about 55 MB, and the snapshot about 53 MB. mlx and mlx-c are mirrored
shallow at their pinned commits, which keeps them near 3.5 MB instead of the 72 MB their
full history would cost.
Limits worth knowing:
- Xcode GUI builds are not covered. They do not inherit the rewrite environment, so
they still contact GitHub. Use the
buildsubcommand for offline work. - The mirror follows
Package.resolved. After a dependency change, runrefresh(andsubmodulesifmlx-swiftitself moved);verifynames the exact revision that is missing. - Signing is still disabled in the scripted build, matching the command line above.
Design docs live in docs/, which indexes them and marks which are current versus archived:
| Document | What it covers |
|---|---|
| docs/terminal-architecture.md | Terminal mode: BLE-Eth transport, libssh2 SSH client, SwiftTerm view, credential storage, known gaps |
| docs/ble-eth-protocol-spec.md | The wire protocol the device speaks over GATT: frames, commands, fragmentation, and where iOS differs from Android |
KeyCmd for iOS is licensed under the GNU Affero General Public License v3.0 only (AGPL-3.0-only). The full text is in LICENSE.
Because AGPL §13 treats users who interact with the software over a network as receiving it, anyone who runs a modified copy of this app's code behind a network service must offer them the corresponding source.
Third-party code keeps its own terms: KeyCmd/Openterface_Core is a separate
repository (it carries no license file in the checkout -- see
Upstream), and the sibling Swift packages listed under
Sibling local packages -- whisper.cpp, SwiftTerm,
Libssh2Prebuild -- are resolved from their own upstreams.
Learn more about the Openterface hardware at openterface.com