diff --git a/.github/ISSUE_TEMPLATE/bug_report.yml b/.github/ISSUE_TEMPLATE/bug_report.yml index 8243767f..06a1d260 100644 --- a/.github/ISSUE_TEMPLATE/bug_report.yml +++ b/.github/ISSUE_TEMPLATE/bug_report.yml @@ -1,66 +1,68 @@ name: Bug report -description: 报告可复现的功能、布局或稳定性问题 +description: Report a problem that you can reproduce with a feature, a layout or stability title: "[Bug] " labels: [bug] body: - type: markdown attributes: value: | - 请先确认问题能在最新构建复现。不要粘贴密码、剪贴板、录音或其他真实输入内容。 + Make sure that the problem occurs in the latest build. + Do not paste passwords, clipboard content, recordings or other real typed text. - type: input id: version attributes: - label: APK / commit - placeholder: openIME-1.0-debug.apk 或 commit SHA + label: APK version or commit + placeholder: "0.0.7-beta.1 or a commit SHA" validations: required: true - type: input id: device attributes: - label: 设备与 Android 版本 - placeholder: Xiaomi 24129PN74C / Android 16 + label: Device and Android version + placeholder: "Xiaomi 15 / Android 16" validations: required: true - type: dropdown id: mode attributes: - label: 输入模式 + label: Input mode options: - - 26 键拼音 - - 9 键拼音 - - 英文 / T9 - - 数字 / 符号 - - Emoji / 剪贴板 - - 语音 - - 设置 / 浮动键盘 - - 其他 + - 26-key pinyin + - Nine-key pinyin + - Stroke + - English 26-key + - Digits and symbols + - Emoji and clipboard + - Voice + - Settings and floating keyboard + - Other validations: required: true - type: textarea id: reproduce attributes: - label: 复现步骤 - description: 请使用脱敏后的示例文字。 + label: Steps to reproduce + description: Use sample text that has no personal data. placeholder: | - 1. 打开…… - 2. 切换到…… - 3. 点击…… + 1. Open ... + 2. Switch to ... + 3. Tap ... validations: required: true - type: textarea id: expected attributes: - label: 预期行为 + label: Expected behavior validations: required: true - type: textarea id: actual attributes: - label: 实际行为 + label: Actual behavior validations: required: true - type: textarea id: evidence attributes: - label: 脱敏日志或截图 - description: 请删除个人路径、输入内容、Token、密码和录音。 + label: Logs or screenshots + description: Remove personal paths, typed text, tokens, passwords and recordings. diff --git a/.github/ISSUE_TEMPLATE/feature_request.yml b/.github/ISSUE_TEMPLATE/feature_request.yml index 1b25b912..4dcaf70d 100644 --- a/.github/ISSUE_TEMPLATE/feature_request.yml +++ b/.github/ISSUE_TEMPLATE/feature_request.yml @@ -1,28 +1,28 @@ name: Feature request -description: 提议新的输入、适配或维护能力 +description: Suggest a new input, compatibility or maintenance feature title: "[Feature] " labels: [enhancement] body: - type: textarea id: problem attributes: - label: 要解决的问题 - placeholder: 当前什么场景不方便，谁会遇到？ + label: Problem + placeholder: Which situation is difficult today? Who does it affect? validations: required: true - type: textarea id: proposal attributes: - label: 建议方案 - placeholder: 期望的交互、候选、布局或配置行为。 + label: Proposed solution + placeholder: Describe the interaction, candidates, layout or setting that you want. validations: required: true - type: textarea id: alternatives attributes: - label: 备选方案 + label: Alternatives - type: input id: device attributes: - label: 相关设备或窗口尺寸 - placeholder: 例如 390dp、平板、折叠屏、横屏 + label: Related device or window size + placeholder: "For example 390 dp, tablet, foldable, landscape" diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 1b078e9f..5bea32d3 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -1,34 +1,34 @@ -## 改动说明 +## Summary - + -## 影响范围 +## Affected areas -- [ ] 输入引擎 / 候选 / 分词 -- [ ] InputConnection / 编辑器行为 -- [ ] 键盘布局 / Insets / 浮动键盘 -- [ ] 工具面板 / 设置 -- [ ] 语音模型 / 音频权限 -- [ ] 构建、测试或文档 +- [ ] Input engine, candidates, word splitting +- [ ] InputConnection, editor behavior +- [ ] Keyboard layout, insets, floating keyboard +- [ ] Tool panels, settings +- [ ] Voice model, audio permission +- [ ] Build, tests or documents -## 验证 +## Verification - [ ] `:app:testDebugUnitTest` - [ ] `:app:lintDebug` - [ ] `:app:assembleDebug` -- [ ] 真实 IME 回归（如适用） -- [ ] 已执行 `git diff --check` +- [ ] Real input method regression (if it applies) +- [ ] `git diff --check` -测试设备、Android 版本和命令： +Test device, Android version and commands: -## 变更记录与版本 +## Change log and version -- [ ] 用户可见的改动（功能、行为、修复、权限、数据格式）已写入 `CHANGELOG.md` 的 `[Unreleased]`；纯内部改动不需要 -- [ ] 没有修改 `VERSION`（只有发布 PR 才升级版本，见 `docs/RELEASE.md`） -- [ ] PR 标题能独立说清结果：合并时它就是提交标题，描述就是提交正文 +- [ ] I added each user-visible change to `[Unreleased]` in `CHANGELOG.md`. Internal changes do not need an entry. +- [ ] I did not change `VERSION`. Only a release PR changes it. See `docs/RELEASE.md`. +- [ ] The PR title states the result. It becomes the commit title, and the description becomes the commit body. -## 隐私与交付检查 +## Privacy and delivery -- [ ] 没有提交密码、剪贴板、录音、设备日志或个人路径 -- [ ] 模型、词典或第三方文件的来源和许可证没有被破坏 -- [ ] 若改动用户数据或设置格式，已说明升级兼容性 +- [ ] The change has no passwords, clipboard content, recordings, device logs or personal paths. +- [ ] The change keeps the source and license of each model, dictionary and third-party file. +- [ ] If the change affects user data or settings formats, the description explains upgrade compatibility. diff --git a/CHANGELOG.md b/CHANGELOG.md index d1663c6b..872ee999 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,6 @@ [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)，版本号遵循 [语义化版本](https://semver.org/lang/zh-CN/)。每个 `## [版本] - 日期` 小节就是该版本 GitHub Release 的发布说明；版本与发布流程见 [docs/RELEASE.md](docs/RELEASE.md)。 -首个公开版本之前的开发期记录见 [docs/CHANGELOG_PRE_1.0.md](docs/CHANGELOG_PRE_1.0.md)。 ## [Unreleased] diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index e2f8e928..4d82cc9d 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -1,61 +1,82 @@ -# 贡献指南 +# Contributing -## 开始之前 +Thank you for your interest in openIME. +This guide tells you how to prepare, change and submit code. -1. 安装 JDK 17、Android SDK 36、NDK `27.0.12077973`、CMake `3.22.1` 和 Git LFS。 -2. 执行 `git lfs pull`，确认语音模型和 AAR 不是文本指针。 -3. 先阅读 [README.md](README.md)、[docs/ARCHITECTURE.md](docs/ARCHITECTURE.md) - 和 [docs/TEST_ARCHITECTURE.md](docs/TEST_ARCHITECTURE.md)。 -4. 修改 App 界面时按 [App 界面开发规范](docs/APP_UI_SPEC.md) 实现尺寸、间距、对齐、点击区及大字体适配；先复用现有令牌和资源，不自行引入一套数值。 +## Prepare -## 分支与提交 +1. Install JDK 17, Android SDK 36, NDK `27.0.12077973`, CMake `3.22.1` and Git LFS. +2. Run `git lfs pull`. Make sure the voice models and the AAR are not text pointers. +3. Read the [README](README.md), the [architecture](docs/ARCHITECTURE.md) and the [test architecture](docs/TEST_ARCHITECTURE.md). +4. For app screens, follow the [app UI specification](docs/APP_UI_SPEC.md). + Use the existing tokens and resources. Do not add new sizes or colors. -- 从最新的 `main` 创建短生命周期分支，命名为 `类型/主题`，类型取 `feat`、`fix`、`docs`、 - `chore`、`ci`、`refactor`、`test`，例如 `fix/pinyin-candidate`、`docs/repository`。 -- 分支里的提交保持“一个提交一个主题”，说明用简短的一句话写结果（中文或英文均可），例如 - `修复九键拼音候选提交`、`fix: keep the rest of the input composing`。合并时会被 squash， - 最终进入 `main` 的提交信息是 PR 标题和描述。 -- 不要提交 `local.properties`、Gradle/build 输出、根目录截图、UI dump、设备日志、 - 密码、录音、签名密钥或未经确认的模型文件。 +## Branches and commits -## 变更记录与版本 +- Create a short-lived branch from the latest `main`. + Name it `type/topic`. The types are `feat`, `fix`, `docs`, `chore`, `ci`, `refactor` and `test`. + Example: `fix/pinyin-candidate`. +- Keep one topic in each commit. + Write the message as one short sentence that states the result. + Example: `fix: keep the rest of the input composing`. +- We squash every PR. The PR title and description become the commit on `main`. +- Do not commit these items: + `local.properties`, build output, screenshots outside `docs/images/`, UI dumps, device logs, + passwords, recordings, signing keys and unverified model files. -- 用户可见的改动（功能、行为、修复、权限、数据格式）在同一个 PR 里写进 `CHANGELOG.md` 的 - `## [Unreleased]`；纯内部改动不需要。 -- 不要在功能 PR 里修改 `VERSION`：版本只在发布 PR 里升级。版本规则、签名和发布步骤见 - [docs/RELEASE.md](docs/RELEASE.md)，分支与合并规则见 [docs/REPOSITORY.md](docs/REPOSITORY.md)。 +## Change log and version -## 提交前检查 +- Add each user-visible change to `## [Unreleased]` in `CHANGELOG.md`, in the same PR. + User-visible changes are features, behavior, fixes, permissions and data formats. + Internal changes do not need an entry. +- Do not change `VERSION` in a feature PR. Only a release PR changes it. + See the [release process](docs/RELEASE.md). -```powershell -.\gradlew.bat :app:testDebugUnitTest :app:lintDebug :app:assembleDebug --no-daemon --console=plain -git diff --check -git status --short -``` +## Check your change -Linux / macOS / Git Bash 上再运行版本与变更记录检查（CI 也会运行）： +Run these commands before you open a PR: ```bash +./gradlew :app:testDebugUnitTest :app:lintDebug :app:assembleDebug --no-daemon --console=plain python3 -m unittest discover -s scripts -p 'test_*.py' python3 scripts/release_check.py check +git diff --check ``` -如果改动了真实输入链路，使用明确的设备 Serial 运行至少一组核心回归： +The CI runs the same checks. + +If you change the input path, run one core regression on a named device or emulator: ```powershell .\scripts\core_regression.ps1 -Serial ``` -如果改动了布局或 Insets，再运行 `visual_check.ps1`，并在 PR 中说明测试设备的 -Android 版本、窗口宽度和是否使用浮动键盘。 -App 界面同时按 `docs/APP_UI_SPEC.md` 检查普通屏、320dp 大字体和宽屏；运行 -`scripts/beta4_e2e.py` 并保存截图、UI XML、结果 JSON 和日志作为可重复的验收工件。 +If you change layout or insets, also run `scripts/visual_check.ps1`. +In the PR, state the Android version, the window width and whether you used the floating keyboard. + +If you change an app screen, run `scripts/beta4_e2e.py`. +Check normal, 320 dp wide with large fonts, and wide screens. +Keep the screenshots, UI XML, result JSON and logs. + +## Pull requests + +Describe these items in the PR: + +- the purpose of the change +- the parts of the product that it affects +- the test commands and their results +- known limitations +- any change to dictionaries, models, permissions or data formats + +Remove personal data from screenshots. +Do not upload real typed text, passwords, clipboard content or recordings. -## Pull Request +`main` is protected. +A PR needs a passing **Build and verify** check before it can merge. +The compatibility tests on API 26, 29, 31 and 34 also run on each PR. +Fix a failing compatibility test before you merge. -PR 描述应包含：改动目的、影响范围、测试命令和结果、已知限制，以及是否修改了 -词典、模型、权限或数据格式。涉及截图时请脱敏；不要在 PR 中上传真实输入内容、 -密码、剪贴板或录音。 +## Conduct -`main` 受保护：只能通过 PR 合并，且 **Build and verify** 必须通过；PR 一律 squash 合并， -合并后分支自动删除。API 26 / 29 / 31 / 34 兼容测试同样会在 PR 上运行，红了请先修再合并。 +Be respectful and keep discussions about the work. +Report security problems in private. See [SECURITY.md](SECURITY.md). diff --git a/README.md b/README.md index 5e3fdae9..a6123de6 100644 --- a/README.md +++ b/README.md @@ -1,86 +1,141 @@ -# openIME -

- openIME + openIME logo

-

本地优先的 Android 中文输入法

+

openIME

+ +

A local-first Chinese input method for Android.

- 版本 + Release Android CI - 许可证 + License: GPL-3.0-only

-> **当前处于 Beta 测试阶段（0.0.x）。** 功能和稳定性仍在验证，接口与数据格式可能调整， -> 请不要把它作为日常唯一的输入法。欢迎通过 [Issues](https://github.com/Slacker-LLC/openIME/issues) 反馈问题。 - -openIME 是一款独立的 Android 系统输入法。拼音候选、用户词库学习和语音识别全部在设备本地完成， -应用**没有 `INTERNET` 权限**，输入内容不会离开手机。 - -## 功能 - -- **键盘**：26 键拼音、九键拼音、英文 26 键、数字与符号；Emoji、符号、剪贴板、文本编辑、浮动键盘。 -- **拼音**：全拼、简拼、手动分词、候选展开、用户词库学习、简繁转换。 - 引擎为 librime，内置约 90 万条 Rime Ice 词典记录，打包时已预先编译；安装或升级后第一次打开键盘，几秒内即可用完整词典。 -- **九键**：输入时左栏列出下一个字的拼音，一个字选一个音节，选定后自动移到下一个字。 -- **语音输入**：长按空格说话，松手后识别并上屏；使用内置的中英双语模型，不联网；可去掉“嗯”“呃”等语气词，也可把标点写成空格。 -- **手势**：删除键上滑清空；空格左右滑动移动光标，滑动时底行其他按键锁定。 -- **表情联想**：选词后联想栏先给出相关表情（开心 → 😊），词表内置，不联网。 -- **自动填充**：Android 11+ 上，密码管理器的账号、验证码直接显示在键盘工具栏位置。 -- **数字和符号**：26 键字母键右上角印着数字和符号，上滑或长按直接输入。 -- **适配**：横竖屏、平板与折叠屏、深色模式、大字号；终端、远程桌面、游戏等原始按键输入框； - 外接键盘可直接打拼音。详见 [兼容性说明](docs/COMPATIBILITY.md)。 -- **自我保护**：连续崩溃或卡死后自动进入安全模式，保证仍然可以打字； - “设置 → 关于与数据”下，“关于”可复制诊断信息，“数据管理”可导出、导入用户数据。 - -## 下载与安装 - -1. 在 [Releases](https://github.com/Slacker-LLC/openIME/releases) 下载最新的 `openIME-v*-arm64-release.apk` 和 - `SHA256SUMS.txt`。发布包仅支持 `arm64-v8a` 设备（绝大多数近年的 Android 手机），系统要求 Android 8.0（API 26）及以上。 -2. 校验后再安装： +

English · 简体中文

+ +> **Beta software (0.0.x).** +> Features, settings and data formats can change between releases. +> Do not use openIME as your only keyboard. +> Report problems in [Issues](https://github.com/Slacker-LLC/openIME/issues). + +openIME is a system keyboard for Android. +Pinyin input, word learning and voice recognition all run on the device. +The app has no `INTERNET` permission, so your text cannot leave the phone. + + + + + + + + + + + + + + + + + + + + + + + + + + +
26-key pinyin keyboard with candidates for nihaoNine-key pinyin keyboard with the syllable columnStroke keyboardEmoji panel
26-key pinyinNine-key pinyinStrokeEmoji
App home screen, light themeSettings screen, light themeApp home screen with keyboard, dark themeSettings screen, dark theme
HomeSettingsDark themeDark settings
+ +## Features + +- **Keyboards:** 26-key pinyin, nine-key pinyin, stroke, English 26-key, digits and symbols. + Emoji, symbols, clipboard history, quick phrases, text editing and a floating keyboard are also available. +- **Pinyin engine:** [librime](https://github.com/rime/librime) with about 900,000 Rime Ice dictionary entries. + The dictionary is compiled at build time. + The full dictionary is ready a few seconds after the first start. +- **Pinyin input:** full pinyin, abbreviations, manual word splitting, fuzzy pinyin, user-word learning and simplified/traditional conversion. +- **Nine-key input:** a column on the left lists the pinyin for the next character. + Tap one syllable to lock it. +- **Voice input:** hold the space key and speak. + openIME inserts the text when you release the key. + The bilingual (Chinese and English) model is in the APK and works without a network. + Options remove filler words and replace punctuation with spaces. +- **Gestures:** swipe up on the delete key to clear the text. + Swipe left or right on the space key to move the cursor. +- **Emoji suggestions:** after you choose a word, the suggestion bar shows related emoji. + The word list is in the APK. +- **Autofill (Android 11 and later):** account names and codes from your password manager appear in the toolbar. +- **Numbers and symbols:** each letter key shows a number or symbol. + Swipe up or press and hold to enter it. +- **Display support:** portrait, landscape, tablets, foldables, dark theme and large font sizes. + Raw key input fields (terminals, remote desktops, games) and physical keyboards also work. + See [Compatibility](docs/COMPATIBILITY.md). +- **Self-protection:** after repeated crashes or freezes, openIME starts in safe mode so you can still type. + In the app, 设置 → 关于与数据 (Settings → About and data) copies diagnostics and exports or imports your data. + +## Install + +1. Open [Releases](https://github.com/Slacker-LLC/openIME/releases). + Download the latest `openIME-v*-arm64-release.apk`. +2. Check the SHA-256 checksum that the release notes show: ```bash - sha256sum -c SHA256SUMS.txt + sha256sum openIME-v*-arm64-release.apk ``` -3. 打开 openIME，按引导启用输入法并切换到 openIME。 -4. 如需语音输入，在引导页授权麦克风；不授权也不影响普通打字。 -5. 在引导页的输入框里试打一下，确认键盘、候选和上屏正常。 +3. Install the APK. +4. Open openIME. Follow the steps to enable it and select it as your keyboard. +5. For voice input, allow the microphone. + Typing works without it. + +Requirements: + +- Android 8.0 (API 26) or later. +- An `arm64-v8a` device. This includes most phones made in recent years. -所有发布包使用同一把固定密钥签名，证书 SHA-256 见 [docs/release-cert.sha256](docs/release-cert.sha256)， -同一把密钥签名的新版本可以直接覆盖安装。 +All releases use the same signing key. +The certificate SHA-256 is in [docs/release-cert.sha256](docs/release-cert.sha256). +You can install a new release over an old one. -### 已知限制 +### Known limitations -- 手写输入尚未接入识别引擎，入口默认隐藏。 -- 九键暂不支持与外接键盘同时使用。 -- 部分厂商系统对输入法的后台限制不同，尚未在大量真机上验证。 -- 版本号小于此前已装版本时，系统会拒绝覆盖安装；需要先卸载（卸载前可导出用户数据）。 +- Handwriting input has no recognition engine yet. The entry point is hidden. +- The nine-key keyboard does not work with a physical keyboard. +- Some vendor Android versions limit background input methods. Few devices have been tested. +- Android does not install a version that is older than the installed one. + Uninstall first. Export your data before you uninstall. -## 隐私与安全 +## Privacy and security -- 应用不声明 `INTERNET` 权限，`allowBackup` 关闭；词典、语音模型和 runtime 都随 APK 提供。 -- 语音音频只在内存中处理，结束、取消或失败时清空，不落盘。 -- 密码输入框不组合拼音、不学习词库、不读取输入框内容、不写日志；可使用剪贴板历史（来源应用标记为敏感的内容和要求关闭个性化学习的输入框除外）；语音可用，识别结果只在结束时一次性上屏。 -- 崩溃记录只保存异常类型和调用栈，不含输入内容。 -- 卸载会清除本机全部数据，包括学习到的用户词库。 +- The manifest has no `INTERNET` permission. `allowBackup` is off. +- The dictionary, the voice models and the runtime are in the APK. +- Voice audio stays in memory. openIME clears it when recognition ends, is canceled or fails. +- In password fields, openIME does not compose pinyin, learn words, read the field text or write logs. + Voice input inserts the final text only. +- Crash records contain the exception type and the stack trace. They never contain typed text. +- Uninstall removes all data on the device, including learned words. -发现安全问题请不要直接创建公开 Issue，按 [SECURITY.md](SECURITY.md) 私下报告。 +Do not report a security problem in a public issue. +Follow [SECURITY.md](SECURITY.md). -## 参与开发 +## Build from source -环境要求：JDK 17、Android SDK Platform 36、NDK `27.0.12077973`、CMake `3.22.1`、Git LFS。 +Requirements: JDK 17, Android SDK Platform 36, NDK `27.0.12077973`, CMake `3.22.1` and Git LFS. ```bash -git lfs install && git lfs pull # 语音模型与 sherpa-onnx AAR -bash scripts/fetch_rime_deps.sh # 锁定版本的 librime 原生依赖 -./gradlew :app:assembleDebug # 输出 app/build/outputs/apk/debug/app-debug.apk -bash scripts/verify_linux.sh # 单元测试、Lint、Debug APK、仪器测试 APK +git lfs install && git lfs pull # voice models and the sherpa-onnx AAR +bash scripts/fetch_rime_deps.sh # pinned librime dependencies +./gradlew :app:assembleDebug # app/build/outputs/apk/debug/app-debug.apk +bash scripts/verify_linux.sh # unit tests, lint, debug APK, test APK ``` -Debug 包仅用于开发与回归（含 `x86_64`），带调试用 Activity 和 Receiver，不会随正式包发布。 -在设备上安装与启用： +The debug APK is for development only. +It includes `x86_64` and test activities, and we do not release it. +Install it and select it as the keyboard: ```bash adb install -r app/build/outputs/apk/debug/app-debug.apk @@ -88,24 +143,26 @@ adb shell ime enable --user 0 llc.slacker.openime/.LocalVoiceImeService adb shell ime set --user 0 llc.slacker.openime/.LocalVoiceImeService ``` -仓库结构： +Repository layout: ```text -app/ Android 应用、输入法服务、Rime JNI、内置模型与词典 -scripts/ 构建、回归、发布脚本 -docs/ 架构、兼容性、测试、发布与仓库管理文档 -.github/ CI、发布流水线、Dependabot、Issue 与 PR 模板 -VERSION 版本号的唯一来源 -CHANGELOG.md 变更记录，同时是 Release 说明的来源 +app/ Android app, input method service, Rime JNI, models and dictionaries +scripts/ Build, test and release scripts +docs/ Architecture, testing, release and repository documents +.github/ CI, release workflow, Dependabot, issue and PR templates +VERSION Single source of the version number +CHANGELOG.md Change log. It is also the source of the release notes. ``` -文档入口： +## Documentation -- [文档索引](docs/README.md)　[输入法架构](docs/ARCHITECTURE.md)　[兼容性](docs/COMPATIBILITY.md) -- [测试 SOP](docs/TEST_SOP.md)　[脚本说明](scripts/README.md) -- [发布与版本管理](docs/RELEASE.md)　[仓库管理](docs/REPOSITORY.md)　[贡献指南](CONTRIBUTING.md) +- [Documentation index](docs/README.md) +- [Architecture](docs/ARCHITECTURE.md) · [Compatibility](docs/COMPATIBILITY.md) · [Test procedure](docs/TEST_SOP.md) +- [Release process](docs/RELEASE.md) · [Repository settings](docs/REPOSITORY.md) +- [Contributing](CONTRIBUTING.md) · [Security policy](SECURITY.md) · [Change log](CHANGELOG.md) -## 许可证 +## License -openIME 以 [GPL-3.0-only](LICENSE) 发布。第三方组件保留各自的许可证， -见 [docs/LICENSING.md](docs/LICENSING.md) 和 [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)。 +openIME uses the [GPL-3.0-only](LICENSE) license. +Third-party components keep their own licenses. +See [docs/LICENSING.md](docs/LICENSING.md) and [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md). diff --git a/README.zh-CN.md b/README.zh-CN.md new file mode 100644 index 00000000..d939d363 --- /dev/null +++ b/README.zh-CN.md @@ -0,0 +1,134 @@ +

+ openIME +

+ +

openIME

+ +

本地优先的 Android 中文输入法

+ +

+ 版本 + Android CI + 许可证 +

+ +

English · 简体中文

+ +> **当前处于 Beta 测试阶段（0.0.x）。** 功能和稳定性仍在验证，接口与数据格式可能调整， +> 请不要把它作为日常唯一的输入法。欢迎通过 [Issues](https://github.com/Slacker-LLC/openIME/issues) 反馈问题。 + +openIME 是一款独立的 Android 系统输入法。拼音候选、用户词库学习和语音识别全部在设备本地完成， +应用**没有 `INTERNET` 权限**，输入内容不会离开手机。 + + + + + + + + + + + + + + + + + + + + +
26 键拼音九键拼音笔画表情
26 键拼音九键拼音笔画表情
首页偏好设置深色模式深色偏好设置
首页偏好设置深色模式深色设置
+ +## 功能 + +- **键盘**：26 键拼音、九键拼音、笔画、英文 26 键、数字与符号；Emoji、符号、剪贴板、文本编辑、浮动键盘。 +- **拼音**：全拼、简拼、手动分词、候选展开、用户词库学习、简繁转换。 + 引擎为 librime，内置约 90 万条 Rime Ice 词典记录，打包时已预先编译；安装或升级后第一次打开键盘，几秒内即可用完整词典。 +- **九键**：输入时左栏列出下一个字的拼音，一个字选一个音节，选定后自动移到下一个字。 +- **语音输入**：长按空格说话，松手后识别并上屏；使用内置的中英双语模型，不联网；可去掉“嗯”“呃”等语气词，也可把标点写成空格。 +- **手势**：删除键上滑清空；空格左右滑动移动光标，滑动时底行其他按键锁定。 +- **表情联想**：选词后联想栏先给出相关表情（开心 → 😊），词表内置，不联网。 +- **自动填充**：Android 11+ 上，密码管理器的账号、验证码直接显示在键盘工具栏位置。 +- **数字和符号**：26 键字母键右上角印着数字和符号，上滑或长按直接输入。 +- **适配**：横竖屏、平板与折叠屏、深色模式、大字号；终端、远程桌面、游戏等原始按键输入框； + 外接键盘可直接打拼音。详见 [兼容性说明](docs/COMPATIBILITY.md)。 +- **自我保护**：连续崩溃或卡死后自动进入安全模式，保证仍然可以打字； + “设置 → 关于与数据”下，“关于”可复制诊断信息，“数据管理”可导出、导入用户数据。 + +## 下载与安装 + +1. 在 [Releases](https://github.com/Slacker-LLC/openIME/releases) 下载最新的 `openIME-v*-arm64-release.apk`。 + 发布包仅支持 `arm64-v8a` 设备（绝大多数近年的 Android 手机），系统要求 Android 8.0（API 26）及以上。 +2. 对照发布说明里的 SHA-256 校验后再安装： + + ```bash + sha256sum openIME-v*-arm64-release.apk + ``` + +3. 打开 openIME，按引导启用输入法并切换到 openIME。 +4. 如需语音输入，在引导页授权麦克风；不授权也不影响普通打字。 +5. 在引导页的输入框里试打一下，确认键盘、候选和上屏正常。 + +所有发布包使用同一把固定密钥签名，证书 SHA-256 见 [docs/release-cert.sha256](docs/release-cert.sha256)， +同一把密钥签名的新版本可以直接覆盖安装。 + +### 已知限制 + +- 手写输入尚未接入识别引擎，入口默认隐藏。 +- 九键暂不支持与外接键盘同时使用。 +- 部分厂商系统对输入法的后台限制不同，尚未在大量真机上验证。 +- 版本号小于此前已装版本时，系统会拒绝覆盖安装；需要先卸载（卸载前可导出用户数据）。 + +## 隐私与安全 + +- 应用不声明 `INTERNET` 权限，`allowBackup` 关闭；词典、语音模型和 runtime 都随 APK 提供。 +- 语音音频只在内存中处理，结束、取消或失败时清空，不落盘。 +- 密码输入框不组合拼音、不学习词库、不读取输入框内容、不写日志；可使用剪贴板历史（来源应用标记为敏感的内容和要求关闭个性化学习的输入框除外）；语音可用，识别结果只在结束时一次性上屏。 +- 崩溃记录只保存异常类型和调用栈，不含输入内容。 +- 卸载会清除本机全部数据，包括学习到的用户词库。 + +发现安全问题请不要直接创建公开 Issue，按 [SECURITY.md](SECURITY.md) 私下报告。 + +## 参与开发 + +环境要求：JDK 17、Android SDK Platform 36、NDK `27.0.12077973`、CMake `3.22.1`、Git LFS。 + +```bash +git lfs install && git lfs pull # 语音模型与 sherpa-onnx AAR +bash scripts/fetch_rime_deps.sh # 锁定版本的 librime 原生依赖 +./gradlew :app:assembleDebug # 输出 app/build/outputs/apk/debug/app-debug.apk +bash scripts/verify_linux.sh # 单元测试、Lint、Debug APK、仪器测试 APK +``` + +Debug 包仅用于开发与回归（含 `x86_64`），带调试用 Activity 和 Receiver，不会随正式包发布。 +在设备上安装与启用： + +```bash +adb install -r app/build/outputs/apk/debug/app-debug.apk +adb shell ime enable --user 0 llc.slacker.openime/.LocalVoiceImeService +adb shell ime set --user 0 llc.slacker.openime/.LocalVoiceImeService +``` + +仓库结构： + +```text +app/ Android 应用、输入法服务、Rime JNI、内置模型与词典 +scripts/ 构建、回归、发布脚本 +docs/ 架构、兼容性、测试、发布与仓库管理文档 +.github/ CI、发布流水线、Dependabot、Issue 与 PR 模板 +VERSION 版本号的唯一来源 +CHANGELOG.md 变更记录，同时是 Release 说明的来源 +``` + +文档入口（英文）： + +- [文档索引](docs/README.md)　[架构](docs/ARCHITECTURE.md)　[兼容性](docs/COMPATIBILITY.md) +- [测试流程](docs/TEST_SOP.md)　[脚本说明](scripts/README.md) +- [发布流程](docs/RELEASE.md)　[仓库设置](docs/REPOSITORY.md)　[贡献指南](CONTRIBUTING.md) + +## 许可证 + +openIME 以 [GPL-3.0-only](LICENSE) 发布。第三方组件保留各自的许可证， +见 [docs/LICENSING.md](docs/LICENSING.md) 和 [THIRD_PARTY_NOTICES.md](THIRD_PARTY_NOTICES.md)。 diff --git a/SECURITY.md b/SECURITY.md index a3be1220..ad056c00 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -1,26 +1,38 @@ -# 安全问题报告 +# Security policy -不要通过公开 Issue 报告可能泄露用户文字、剪贴板、密码、录音或模型文件的问题。 +Do not report a security problem in a public issue. +This applies to any problem that can expose typed text, clipboard content, passwords, recordings or model files. -## 如何报告 +## Report a problem -请使用仓库的 **Security → Report a vulnerability**（私密漏洞报告）私下提交。如果该入口暂不可用， -请先联系仓库维护者并提供最小复现信息，不要附带真实用户数据。报告中建议包含： +Use **Security → Report a vulnerability** in this repository. +This sends the report to the maintainers in private. -- 受影响的版本或 commit、APK 变体和 Android 版本； -- 复现步骤与影响范围； -- 已脱敏的日志或截图； -- 临时缓解方式（如果有）。 +If this option is not available, contact a maintainer. +Send only the minimum information that is necessary to reproduce the problem. +Do not send real user data. -输入法相关报告请特别说明是否发生在密码编辑器、语音权限拒绝、跨应用编辑器切换、 -剪贴板或模型加载路径中。维护者会先确认问题，再决定修复、发布说明和披露时间； -修复发布前请不要公开细节。 +Include these items in your report: -## 受支持的版本 +- the affected version or commit, the APK type and the Android version +- the steps to reproduce the problem and its effect +- logs or screenshots with personal data removed +- a workaround, if you know one -| 版本 | 状态 | +Tell us if the problem occurs in one of these cases: +a password field, a denied voice permission, a switch between editors, the clipboard, or model loading. + +The maintainers first confirm the problem. +They then decide the fix, the release notes and the date of disclosure. +Do not publish details before the fix is released. + +## Supported versions + +| Version | Status | |---|---| -| 最新的 1.x 正式版 | 接收安全修复，以 PATCH 版本发布并写入 CHANGELOG | -| 更早的版本、Debug 签名的开发版 | 不再维护，请升级到最新正式版 | +| The latest release (beta or stable) | Receives security fixes. We publish them as a new patch or beta version and list them in the change log. | +| Older releases and debug builds | Not maintained. Update to the latest release. | -发布 APK 的真伪可以用 `SHA256SUMS.txt` 和签名证书指纹核对，见 [docs/RELEASE.md](docs/RELEASE.md)。 +To verify an APK, compare its SHA-256 checksum with the release notes. +Compare its signing certificate with [docs/release-cert.sha256](docs/release-cert.sha256). +See the [release process](docs/RELEASE.md). diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index a16eddfe..853e7f9b 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -1,27 +1,33 @@ -# 第三方资源与分发核对 +# Third-party notices -本文件记录 openIME 实际随源码或 APK 分发的主要第三方代码、数据和模型。openIME 本身的许可证见仓库根目录 `LICENSE`（GPL-3.0-only），本文件只记录第三方组件。 +This file lists the third-party code, data and models that openIME ships in its source or in the APK. +openIME uses the GPL-3.0-only license. See `LICENSE`. +This file covers only the third-party components. -| 组件 | 仓库内位置 | 上游/来源 | 许可证 | 分发核对 | +| Component | Location in the repository | Source | License | Distribution check | |---|---|---|---|---| -| librime | `app/src/main/cpp/vendor/librime/` | https://github.com/rime/librime | BSD-3-Clause | vendored `LICENSE` 保留；APK 同时包含 `assets/licenses/librime-BSD-3-Clause.txt` | -| OpenCC | `app/src/main/cpp/vendor/OpenCC/` | https://github.com/BYVoid/OpenCC | Apache-2.0 | vendored `LICENSE` 保留；APK 同时包含 `assets/licenses/OpenCC-Apache-2.0.txt` | -| Snappy | `app/src/main/cpp/vendor/snappy/` | https://github.com/google/snappy | BSD-3-Clause | vendored `COPYING` 保留；APK 同时包含 `assets/licenses/snappy-BSD-3-Clause.txt` | -| Rime Ice 词典 | `app/src/main/assets/rime-data/openime_dicts/`、`pinyin_phrases.tsv` | https://github.com/iDvel/rime-ice，固定提交 `75e6572bebc05b49021e842949ce947882e3e4b2` | GPL-3.0-only | 已核对；完整文本随 APK/源码位于 `app/src/main/assets/licenses/rime-ice-GPL-3.0.txt` | -| Rime 基础数据 | `app/src/main/assets/rime/` | Rime / luna-pinyin / essay 等上游数据 | 以各目录 `AUTHORS` / 上游许可为准 | 已保留上游 AUTHORS；发布前不得删除这些归属文件 | -| sherpa-onnx Android runtime | `app/libs/sherpa-onnx-1.13.6.aar` | https://github.com/k2-fsa/sherpa-onnx | Apache-2.0 | APK 包含 `assets/licenses/sherpa-onnx-Apache-2.0.txt` | -| 中英双语 Streaming Paraformer INT8 语音模型 | `app/src/main/assets/models/voice/bilingual-paraformer/` | https://huggingface.co/csukuangfj/sherpa-onnx-streaming-paraformer-bilingual-zh-en | Apache-2.0 | 只打包 `encoder.int8.onnx`、`decoder.int8.onnx` 和 `tokens.txt`；APK 包含 `assets/licenses/paraformer-model-Apache-2.0.txt`，`manifest.json` 固定模型 ID/版本与文件哈希 | -| CT-Transformer 中英标点 INT8 模型 | `app/src/main/assets/models/voice/punctuation/` | https://modelscope.cn/models/iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch 、https://github.com/k2-fsa/sherpa-onnx/releases/tag/punctuation-models | Apache-2.0 | 固定 `2024-04-12-int8` 导出，清单包含文件哈希；APK 附 `ct-transformer-Apache-2.0.txt` | -| Microsoft Fluent Emoji | `app/src/main/assets/emoji/fluent/` | https://github.com/microsoft/fluentui-emoji | MIT | 仅打包当前实际使用的基础情绪表情资源；APK 包含 `assets/licenses/fluent-emoji-MIT.txt` | +| librime | `app/src/main/cpp/vendor/librime/` | https://github.com/rime/librime | BSD-3-Clause | The vendored `LICENSE` file is kept. The APK contains `assets/licenses/librime-BSD-3-Clause.txt`. | +| OpenCC | `app/src/main/cpp/vendor/OpenCC/` | https://github.com/BYVoid/OpenCC | Apache-2.0 | The vendored `LICENSE` file is kept. The APK contains `assets/licenses/OpenCC-Apache-2.0.txt`. | +| Snappy | `app/src/main/cpp/vendor/snappy/` | https://github.com/google/snappy | BSD-3-Clause | The vendored `COPYING` file is kept. The APK contains `assets/licenses/snappy-BSD-3-Clause.txt`. | +| Rime Ice dictionaries | `app/src/main/assets/rime-data/openime_dicts/`, `app/src/main/assets/pinyin_phrases.tsv` | https://github.com/iDvel/rime-ice, pinned commit `75e6572bebc05b49021e842949ce947882e3e4b2` | GPL-3.0-only | Checked. The full text is in `app/src/main/assets/licenses/rime-ice-GPL-3.0.txt`. | +| Rime base data | `app/src/main/assets/rime/` | Rime, luna-pinyin, essay and other upstream data | See the `AUTHORS` file in each directory | Upstream `AUTHORS` files are kept. Do not delete them before a release. | +| sherpa-onnx Android runtime | `app/libs/sherpa-onnx-1.13.6.aar` | https://github.com/k2-fsa/sherpa-onnx | Apache-2.0 | The APK contains `assets/licenses/sherpa-onnx-Apache-2.0.txt`. | +| Streaming Paraformer bilingual (Chinese and English) INT8 model | `app/src/main/assets/models/voice/bilingual-paraformer/` | https://huggingface.co/csukuangfj/sherpa-onnx-streaming-paraformer-bilingual-zh-en | Apache-2.0 | The APK contains only `encoder.int8.onnx`, `decoder.int8.onnx` and `tokens.txt`. `assets/licenses/paraformer-model-Apache-2.0.txt` is included. `manifest.json` pins the model ID, the version and the file hashes. | +| CT-Transformer punctuation INT8 model | `app/src/main/assets/models/voice/punctuation/` | https://modelscope.cn/models/iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch and https://github.com/k2-fsa/sherpa-onnx/releases/tag/punctuation-models | Apache-2.0 | We pin the `2024-04-12-int8` export. The manifest lists the file hashes. The APK contains `ct-transformer-Apache-2.0.txt`. | +| Microsoft Fluent Emoji | `app/src/main/assets/emoji/fluent/` | https://github.com/microsoft/fluentui-emoji | MIT | The APK has only the emoji images that the app uses. It contains `assets/licenses/fluent-emoji-MIT.txt`. | -## 发布前检查 +## Checks before a release -- 不要删除 vendored 源码目录中的 LICENSE、COPYING、AUTHORS 或 NOTICE 类文件。 -- `app/src/main/assets/licenses/` 下的第三方许可文本必须继续随 APK 打包，包括 Rime Ice、librime、OpenCC、Snappy、sherpa-onnx、Paraformer 模型和 Fluent Emoji。 -- 语音 runtime 与模型升级时，重新核对**具体版本/模型**的许可证，不要只沿用本文件旧结论。 -- 内置词库来源或固定提交变化时，同步更新本文件、`docs/LICENSING.md` 和 APK 内许可证文件。 -- 主项目 `LICENSE`（GPL-3.0-only）与 README、`docs/LICENSING.md` 保持一致；更换主项目许可证前先核对 Rime Ice 词典的 GPL-3.0-only 义务。 +- Do not delete `LICENSE`, `COPYING`, `AUTHORS` or `NOTICE` files from vendored source directories. +- Keep the license texts in `app/src/main/assets/licenses/` in the APK. + This includes Rime Ice, librime, OpenCC, Snappy, sherpa-onnx, the Paraformer model and Fluent Emoji. +- When you update the voice runtime or a model, check the license of the new version. + Do not rely on the earlier result. +- When the dictionary source or its pinned commit changes, update this file, [docs/LICENSING.md](docs/LICENSING.md) and the license files in the APK. +- Keep the main license (GPL-3.0-only) the same in `LICENSE`, the README and `docs/LICENSING.md`. + Before you change it, check the GPL-3.0-only terms of the Rime Ice dictionaries. -## 备注 +## Notes -Git LFS、GitHub Release 或 APK 打包方式不会改变第三方作品本身的许可证义务。此清单只记录仓库当前已核对的信息，不替代各上游许可证原文。 +Git LFS, GitHub Releases and APK packaging do not change the license of a third-party work. +This list records only what we have checked. It does not replace the upstream license texts. diff --git a/docs/APP_UI_SPEC.md b/docs/APP_UI_SPEC.md index e06a99a7..0a2baed7 100644 --- a/docs/APP_UI_SPEC.md +++ b/docs/APP_UI_SPEC.md @@ -1,103 +1,173 @@ -# App 界面开发规范 +# App UI specification -适用：首页（引导与就绪两种状态）、偏好设置、模糊音设置、键盘内的设置面板与工具面板，以及这些页面共用的按钮、导航和开关。键盘按键区保留既有参考设计。所有后续界面修改须先按本表选值，再检查实际布局；不随手添加新的间距、字号或固定坐标。 +The app UI text is in Chinese. This document gives the Chinese label with an English note where it is useful. -产品不提供按键皮肤（自定义强调色、圆角、不透明度、按键字号）和单手模式，界面上不再出现这两项。 +This specification applies to these screens: -## 依据与单位 +- the home screen (setup state and ready state) +- preferences and fuzzy pinyin settings +- the settings panel and the tool panel in the keyboard -参考 Apple [Color](https://developer.apple.com/design/human-interface-guidelines/color)、[Layout](https://developer.apple.com/design/human-interface-guidelines/layout)、[Toggles](https://developer.apple.com/design/human-interface-guidelines/toggles)、[Accessibility](https://developer.apple.com/design/human-interface-guidelines/accessibility) 与 Material 3 [Color roles](https://m3.material.io/styles/color/roles)。 +It also applies to the buttons, navigation and switches that these screens share. +The key area of the keyboard keeps its existing reference design. -Apple 的 pt 不是 Android 的 dp/sp。下面是本项目在 Android 上的实现值。尺寸/间距用 dp、文字用 sp；禁止用物理 px 定位界面。复用 `ImeDesignTokens.kt` 的令牌与 `values/dimens.xml`、`values/colors.xml` 同名资源；XML 用资源，动态 View 用令牌。 +Before you change a screen, choose values from this specification. Then check the real layout. +Do not add new spacing, font sizes or fixed coordinates. -## 强调色只给可交互元素 +The product has no key skins (custom accent color, corner radius, opacity, key font size) and no one-hand mode. +Do not add them to the UI. -强调色表示“可以点”和“当前/开启”状态，用得越少越醒目。 +## Basis and units -| 用强调色 | 不用强调色（中性灰） | +The design follows Apple [Color](https://developer.apple.com/design/human-interface-guidelines/color), [Layout](https://developer.apple.com/design/human-interface-guidelines/layout), [Toggles](https://developer.apple.com/design/human-interface-guidelines/toggles) and [Accessibility](https://developer.apple.com/design/human-interface-guidelines/accessibility), and Material 3 [Color roles](https://m3.material.io/styles/color/roles). + +An Apple point is not an Android dp or sp. +The values below are the Android values of this project. +Use dp for size and spacing, and sp for text. Never position the UI with physical pixels. + +Reuse the tokens in `ImeDesignTokens.kt` and the resources with the same names in `values/dimens.xml` and `values/colors.xml`. +XML uses resources. Dynamic views use tokens. + +## Accent color + +The accent color means "you can tap this" or "this is on". +Use it rarely, so that it stays easy to see. + +| Use the accent color | Use neutral grey | | --- | --- | -| 主按钮、文字按钮（如“允许”） | 设置行图标和工具图标（灰底深灰图标） | -| 开启的开关、滑杆已填充部分 | 导航箭头、分组标题、说明文字 | -| 引导页当前步骤编号和进度条 | “已允许”等状态标签 | -| 输入框聚焦边框与光标 | “已就绪”、已完成步骤用语义绿色对勾 | -| 键盘内快捷开关的开启态 | 卡片、分段选项的选中项（白/浅灰面） | +| Primary buttons and text buttons (such as 允许, "Allow") | Icons of settings rows and tools (dark grey on a grey tile) | +| Switches that are on, the filled part of a slider | Navigation arrows, group titles, description text | +| The current step number and the progress bar in setup | Status labels such as 已允许 ("Allowed") | +| The border and caret of a focused input field | "Ready" (已就绪) and finished steps use a semantic green check mark | +| The on state of quick switches in the keyboard | Selected items in cards and segmented controls (white or light grey surface) | + +## Colors -## 配色（`values/colors.xml` / `values-night/colors.xml`） +Files: `values/colors.xml` and `values-night/colors.xml`. -| 角色 | 浅色 | 暗色 | 说明 | +| Role | Light | Dark | Note | | --- | --- | --- | --- | -| 页面底 `setup_page_bg` | #F2F3F6 | #0F1012 | 状态栏、导航栏同色 | -| 卡片 `setup_surface` | #FFFFFF | #1B1C20 | 暗色不画阴影，靠三级灰分层 | -| 图标底/轨道 `setup_icon_tile`、`setup_muted` | #EEF0F3 / #ECEEF1 | #2A2C31 | | -| 主文字 `setup_title` | #15171C | #F2F3F5 | | -| 说明 `setup_body` | #5B6270 | #A2A8B3 | 在卡片上对比度 ≥ 4.5:1 | -| 分隔线 `setup_hairline` | #E7E9ED | #2C2E33 | | -| 强调色 `setup_primary` | #1668D0 | #2D6FD6 | 白字对比度 ≥ 4.5:1 | -| 强调文字 `setup_primary_text` | #1668D0 | #8AB4F8 | 暗色底上的蓝色文字 | -| 浅强调底 `setup_primary_tint` | #E3EDFA | #24324A | 次按钮背景 | -| 成功 `setup_ready` | #1F8A4C | #1F8A4C | 已就绪、已完成 | - -键盘内面板沿用键盘自身的令牌（`ImeTheme.IOS`），开关和快捷开关的开启态同样取键盘的强调色。 - -## 布局尺寸与位置 - -| 项目 | 数值 | 放置及对齐规则 | +| Page background `setup_page_bg` | #F2F3F6 | #0F1012 | Status bar and navigation bar use the same color | +| Card `setup_surface` | #FFFFFF | #1B1C20 | Dark theme has no shadow and uses three grey levels | +| Icon tile and track `setup_icon_tile`, `setup_muted` | #EEF0F3 / #ECEEF1 | #2A2C31 | | +| Primary text `setup_title` | #15171C | #F2F3F5 | | +| Description `setup_body` | #5B6270 | #A2A8B3 | Contrast on a card is at least 4.5:1 | +| Divider `setup_hairline` | #E7E9ED | #2C2E33 | | +| Accent `setup_primary` | #1668D0 | #2D6FD6 | White text has a contrast of at least 4.5:1 | +| Accent text `setup_primary_text` | #1668D0 | #8AB4F8 | Blue text on a dark background | +| Light accent tint `setup_primary_tint` | #E3EDFA | #24324A | Background of secondary buttons | +| Success `setup_ready` | #1F8A4C | #1F8A4C | Ready and finished | + +Panels in the keyboard use the keyboard tokens (`ImeTheme.IOS`). +Switches and quick switches in the keyboard use the keyboard accent color. + +## Layout + +| Item | Value | Placement and alignment | | --- | --- | --- | -| 系统安全区 | Android 实际 Insets | Activity 扣除状态栏、导航栏和屏幕切口一次 | -| 内容宽度 | 最多 600dp | 宽屏居中；窄屏取可用宽度，不套用键盘的 390dp 参考画布 | -| 页面左右边距 | 16dp | 卡片、输入框、主按钮共用左右边界 | -| 首页顶部留白 | 28dp | 从安全区起算 | -| 导航栏 | 56dp 高 | 返回按钮 48×48dp；标题 18sp 中粗，与页面同底色，不单独成灰条 | -| 首页品牌 | 48dp 图标、20sp 名称、13sp 副标题 | 就绪后右侧出现 44dp 圆形偏好设置按钮 | -| 分组标题 | 13sp，左缩进 16dp | 距上一组 28dp，距卡片 8dp | -| 分组卡片 | 圆角 16dp，无阴影 | 一组设置放在同一张卡里，行与行之间用 1dp 分隔线 | -| 分隔线缩进 | 60dp（有图标）/ 56dp（步骤）/ 14dp（键盘内） | 从行文字起点开始，不贯穿图标 | -| 设置行 | 至少 56dp 高，左右内边距都是 16dp | 图标块 32dp（圆角 9dp，图标 18dp），与文字间隔 12dp；开关、数值、按钮和箭头图形的右边缘都落在 16dp 线上（箭头图标框右移 6dp 抵掉自带留白）；滑块轨道两端对齐文字列 | -| 行文字 | 标签 15sp 中粗，说明 13sp | 标签与说明间隔 4dp；说明允许换行 | -| 文字到末端控件 | 至少 12dp | 开关、箭头、数值右对齐 | -| 分段选项（偏好设置） | 标签在上，轨道在下占满行宽 | 48dp 点击高度，34dp 可见轨道，选中项 30dp 高 | -| 滑杆（偏好设置） | 标签与数值同一行，滑杆在下 | 40dp 操作区；数值单行、等宽数字 | -| 开关 | 轨道 46×28dp、滑块 24dp | 滑块与端部间隔 2dp；点击区至少 48dp 高 | -| 导航箭头 | 18dp | 整行可点击，不作为独立按钮 | -| 主操作按钮 | 52dp 高、胶囊形、强调色底白字 | 仅引导状态显示，固定在底部 16dp 上方，执行当前步骤 | -| 次操作按钮 | 44dp 高、胶囊形、浅强调底 | 如语音“允许” | -| 输入框 | 至少 52dp 高，圆角 14dp | 默认 1dp 分隔线描边，聚焦 1.5dp 强调色描边 | - -## 首页 - -- 引导状态：进度文字“第 N 步，共 3 步”+ 三段进度条，标题“设置你的新键盘”。三个步骤放在同一张卡片：当前步骤为强调色实心圆编号，已完成为绿色对勾，未到的为灰色描边圆编号。“可选”分组放语音输入。底部主按钮随当前步骤变为“前往系统设置启用”或“切换到 openIME”。 -- 就绪状态：三个步骤收起为一张“openIME 已就绪”卡片；下面依次是“试一下”输入框和三个用法提示、“语音输入”状态、“常用设置”（偏好设置、模糊音与智能纠错）。不再显示底部主按钮。 - -## 键盘内面板 - -- 设置面板与键盘同高，首屏依次是四个快捷开关（按键音效、触感震动、按键气泡、数字提示，每个 68dp 高）、外观（分段选项与标签同行，宽 216dp）、键盘高度；其余设置在下方滚动。行高 52dp，不带图标和说明。 -- 工具面板：剪贴板、表情、符号、文本编辑、浮动键盘、设置、数据管理，4 列排布；每项为 56dp 圆角 16dp 的图标块加 12sp 名称，图标为中性色。不放语音输入（按住空格即可）和切换键盘（工具栏已有）。 -- 偏好设置与键盘内设置的最后一组标题为“关于与数据”，下面两行：“关于”（版本、隐私、诊断）和“数据管理”（导出与导入、卸载前提示）。 -- 标题行与面板同底色，返回按钮无底色，按下才显示反馈。 - -## 宽度与字体变化 - -- 偏好设置的分段选项和滑杆始终是“标签在上、控件在下”，窄屏无需再切换布局。 -- 三个外观选项按实际文字宽度判断：当前宽度和字体缩放下放不下最长的“跟随系统”时改为纵向排列（如 320dp 宽 + 1.3 倍字体），每项仍至少 48dp。 -- 宽屏仅扩展到 600dp；不把开关拉宽，不把主按钮拉到整块屏幕。 -- 2 倍字体允许页面自然变长并滚动；禁止截掉说明、压缩字号或把按钮放到屏幕外。 - -## 状态与交互 - -- 主按钮、次按钮、输入框必须有按下或聚焦反馈；设置行按下时整行变灰，卡片圆角裁切按下态。 -- 开关以强调色/灰色轨道和滑块位置共同表达状态，设置立即保存；读屏节点标记为 Switch，提供 checkable/checked 与状态说明。快捷开关标记为 ToggleButton。 -- 导航行整行可点击，装饰图标与箭头不形成第二个焦点。 -- 首页“模糊音与智能纠错”直接打开模糊音页，返回回到偏好设置。 - -## 按键触感 - -- Android 14 QPR3 起，`performHapticFeedback(KEYBOARD_TAP)` 由系统按设备配置的固定键盘振幅播放，多数机型偏弱；Gboard 也因此失去了自己的强度滑块。搜狗、小艺、HeliBoard 等都自己驱动振动马达。 -- 本项目由 `KeyHaptics` 直接调用 `Vibrator`，按“震动手感”选择波形：清脆（默认）用 `PRIMITIVE_TICK` / `EFFECT_TICK` / 8ms 脉冲，有力用 `PRIMITIVE_CLICK` / `EFFECT_CLICK` / 12ms 脉冲，系统用 `KEYBOARD_TAP`（厂商调校，不受强度影响）。用途标记为触摸反馈，仍跟随系统“触摸反馈”强度。线性马达（如小米 X 轴马达）上长脉冲会产生余震，所以不使用超过 12ms 的脉冲。没有振动马达时回退 `KEYBOARD_TAP`。 -- “震动强度”滑块 10%–100%（默认 100%）缩放这一下点击：支持原语时按比例缩放 `PRIMITIVE_CLICK`，有振幅控制时缩放振幅，否则缩短脉冲时长；拖动滑杆时最多每 80ms 试震一次。偏好设置放在“触感震动”下方，键盘内设置放在首屏“键盘高度”下方。 -- 按键按下、删除键手势到位、按住空格进入语音都用这一下点击；不使用 `LONG_PRESS`、`CLOCK_TICK` 或长时长振动，避免余震拖沓。依据 Android [Haptics design principles](https://developer.android.com/develop/ui/views/haptics/haptics-principles)。 -- 按键音效可选：系统（带音量参数的 `AudioManager.playSoundEffect(FX_KEYPRESS_STANDARD, -1)`，单参数版本在系统“触摸提示音”关闭时不出声）以及 `res/raw/key_sound_*.wav` 五种合成音（清脆、机械、木质、打字机、气泡，`SoundPool` 播放，用途为 UI 提示音）。选择后立即试听一次，不会先播旧的音效或震动。 -- 偏好设置“按键与输入”里依次是：按键音效、音效、触感震动、震动手感、震动强度；键盘内设置把“音效”“震动手感”放在“按键反馈”一组。 - -## 验收与可重复工件 - -运行 `scripts/beta4_e2e.py`：真实点击设置和按钮，核对实际 UI XML 的尺寸、左右边界、Button/Switch 语义；检查浅色、深色、320dp 的 1.3/2.0 倍字体和宽屏。保留截图、XML、JSON 和日志，再人工检查换行、控件位置与状态。测试命令和语音验收范围见 `BETA4_VOICE.md`。 +| System safe area | Real Android insets | The activity subtracts the status bar, navigation bar and display cutout once | +| Content width | At most 600 dp | Center on wide screens. On narrow screens use the available width. Do not use the 390 dp reference canvas of the keyboard. | +| Page side margin | 16 dp | Cards, input fields and the primary button share the left and right edges | +| Home top space | 28 dp | Starts at the safe area | +| Navigation bar | 56 dp high | Back button 48 × 48 dp. Title 18 sp, medium weight, same background as the page. No separate grey bar. | +| Home brand | 48 dp icon, 20 sp name, 13 sp subtitle | A 44 dp round preferences button appears on the right when setup is done | +| Group title | 13 sp, 16 dp start inset | 28 dp below the previous group, 8 dp above the card | +| Group card | 16 dp corner radius, no shadow | One group of settings in one card, rows separated by a 1 dp divider | +| Divider inset | 60 dp (with icon), 56 dp (step), 14 dp (in keyboard) | Starts at the text start. It does not cross the icon. | +| Settings row | At least 56 dp high, 16 dp padding on both sides | Icon tile 32 dp (9 dp corners, 18 dp icon), 12 dp gap to the text. Switch, value, button and arrow share the 16 dp right edge. The arrow box moves 6 dp right to cancel its built-in margin. A slider track aligns with the text column. | +| Sub-setting row | Same as the settings row | It has no icon and is indented to the text of its parent switch. Examples: sound style, haptic style, haptic strength. | +| Row text | Label 15 sp medium, description 13 sp | 4 dp between label and description. The description can wrap. | +| Text to end control | At least 12 dp | Switch, arrow and value align to the right | +| Segmented control (preferences) | Label above, track below, full row width | 48 dp touch height, 34 dp visible track, 30 dp selected item | +| Slider (preferences) | Label and value in one row, slider below | 40 dp touch area. The value uses one line and tabular digits. | +| Switch | 51 × 31 dp track, 27 dp knob | 2 dp between the knob and each end. The on and off states are symmetric. The touch area is at least 48 dp high. | +| Navigation arrow | 18 dp | The whole row is tappable. The arrow is not a separate button. | +| Primary action button | 52 dp high, pill shape, accent background, white text | Shown only in setup. Fixed 16 dp above the bottom. It runs the current step. | +| Secondary action button | 44 dp high, pill shape, light accent background | Example: the voice 允许 ("Allow") button | +| Input field | At least 52 dp high, 14 dp corners | 1 dp divider-color border. A focused field has a 1.5 dp accent border. | + +## Home screen + +- **Setup state:** + Show the text 第 N 步，共 3 步 ("Step N of 3") and a three-segment progress bar. The title is 设置你的新键盘 ("Set up your new keyboard"). + The three steps are in one card. + The current step has a filled accent circle with its number. + A finished step has a green check mark. + A future step has a grey outlined circle. + The 可选 ("Optional") group contains voice input. + The bottom primary button changes with the step: 前往系统设置启用 ("Go to system settings to enable") or 切换到 openIME ("Switch to openIME"). +- **Ready state:** + The three steps collapse into one openIME 已就绪 ("openIME is ready") card. + Below it are, in order: the 试一下 ("Try it") input field with three usage hints, the 语音输入 ("Voice input") status and 常用设置 ("Common settings": 偏好设置 preferences, and 模糊音与智能纠错 fuzzy pinyin and smart correction). + The bottom primary button is hidden. + +## Panels in the keyboard + +- The settings panel has the same height as the keyboard. + The first screen shows these items in order: + four quick switches, appearance and keyboard height. + The quick switches are 按键音效 (key sound), 触感震动 (haptics), 按键气泡 (key bubble) and 数字提示 (number hints). Each is 68 dp high. + The appearance segmented control is on the same row as its label and is 216 dp wide. + The other settings scroll below. Rows are 52 dp high with no icon and no description. +- The tool panel has seven items in four columns: clipboard, emoji, symbols, text editing, floating keyboard, settings and data management. + Each item is a 56 dp tile with 16 dp corners and a 12 sp name. Icons are neutral. + The panel has no voice input (hold space instead) and no keyboard switch (the toolbar has one). +- The last group in preferences and in the keyboard settings is 关于与数据 ("About and data"). + It has two rows: 关于 ("About": version, privacy, diagnostics) and 数据管理 ("Data management": export, import, pre-uninstall note). +- The title row has the same background as the panel. The back button has no background and shows feedback only when pressed. + +## Width and font changes + +- In preferences, segmented controls and sliders always show the label above and the control below. Narrow screens need no layout switch. +- The three appearance options follow the real text width. + Example: 320 dp wide with font scale 1.3. + If the longest label (跟随系统, "Follow system") does not fit, the options stack vertically. + Each option is still at least 48 dp high. +- Wide screens grow only to 600 dp. Do not stretch switches. Do not stretch the primary button across the screen. +- At font scale 2.0 the page may grow longer and scroll. + Never cut off a description, reduce the font size or move a button off the screen. + +## State and interaction + +- Primary buttons, secondary buttons and input fields must show pressed or focused feedback. + A pressed settings row turns grey over the whole row. The card corner radius clips the pressed state. +- A switch shows its state with the track color and the knob position. The setting saves at once. + The accessibility node is a Switch with checkable, checked and a state description. Quick switches are ToggleButtons. +- A navigation row is tappable as a whole. Its icon and arrow are decoration and take no focus. +- On the home screen, 模糊音与智能纠错 ("Fuzzy pinyin and smart correction") opens the fuzzy pinyin page. Back returns to preferences. + +## Key haptics and sound + +- From Android 14 QPR3, `performHapticFeedback(KEYBOARD_TAP)` plays at a fixed amplitude that the device defines. + On many phones it is weak. Gboard lost its own strength slider for this reason. + Sogou, Xiaoyi and HeliBoard drive the vibration motor themselves. +- `KeyHaptics` calls `Vibrator` directly. + The setting 震动手感 ("Haptic style") selects the waveform: + - **Crisp** (default): `PRIMITIVE_TICK`, `EFFECT_TICK` or an 8 ms pulse. + - **Strong**: `PRIMITIVE_CLICK`, `EFFECT_CLICK` or a 12 ms pulse. + - **System**: `KEYBOARD_TAP` (tuned by the vendor, not scaled by strength). + The usage is touch feedback, so the system touch-feedback strength still applies. + Long pulses ring on linear motors (such as the Xiaomi X-axis motor), so do not use a pulse longer than 12 ms. + Without a vibration motor, fall back to `KEYBOARD_TAP`. +- The 震动强度 ("Haptic strength") slider (10% to 100%, default 100%) scales this tap. + With primitives it scales `PRIMITIVE_CLICK`. With amplitude control it scales the amplitude. Otherwise it shortens the pulse. + While the user drags the slider, a test vibration plays at most every 80 ms. +- Key press, a delete gesture that reaches its threshold and a space hold that starts voice all use this one tap. + Do not use `LONG_PRESS`, `CLOCK_TICK` or long vibrations. They ring and feel slow. + Source: Android [Haptics design principles](https://developer.android.com/develop/ui/views/haptics/haptics-principles). +- Key sound options: + - **System:** `AudioManager.playSoundEffect(FX_KEYPRESS_STANDARD, -1)` with the volume parameter. + The version without a volume parameter is silent when the system touch sounds are off. + - Five synthetic sounds in `res/raw/key_sound_*.wav` (crisp, mechanical, wooden, typewriter, bubble), played by `SoundPool` as UI sounds. + A new choice plays once at once. It never plays the old sound or haptic first. +- In preferences, the group 按键与输入 ("Keys and input") shows these rows in order: 按键音效 key sound, 音效 sound style, 触感震动 haptics, 震动手感 haptic style, 震动强度 haptic strength. + In the keyboard settings, 音效 and 震动手感 are in the group 按键反馈 ("Key feedback"). + +## Acceptance + +Run `scripts/beta4_e2e.py`. +It taps real settings and buttons. +It checks size, left and right edges and Button and Switch roles in the real UI XML. +It covers light and dark themes, 320 dp wide with font scales 1.3 and 2.0, and a wide screen. +Keep the screenshots, XML, JSON and logs. +Then check line wrapping, control position and state by eye. diff --git a/docs/ARCHITECTURE.md b/docs/ARCHITECTURE.md index 86f01c7f..2f290a1d 100644 --- a/docs/ARCHITECTURE.md +++ b/docs/ARCHITECTURE.md @@ -1,8 +1,8 @@ -# openIME 架构 +# Architecture -本文描述当前生产运行时，而不是目标重构形态。重构计划见 [REPAIR_PLAN.md](REPAIR_PLAN.md)。 +This document describes the production runtime as it is today. -## 生产运行链 +## Runtime ```text Android InputMethodService @@ -11,19 +11,18 @@ Android InputMethodService LocalVoiceImeService ├── ImeState ├── CandidatePipeline / CandidateSnapshot - │ ├── CandidateEngine（Rime 未就绪时的本地回退 + 英文联想） + │ ├── CandidateEngine (fallback when Rime is not ready, English suggestions) │ ├── NativeCandidatePipeline - │ └── RimeEngine → RimeNative (JNI) → librime/OpenCC + │ └── RimeEngine → RimeNative (JNI) → librime / OpenCC ├── InputConnectionGateway ├── VoiceModelLifecycleManager │ ├── VoiceModelRepository │ ├── LocalAudioVoiceBackend / VoiceAudioRouteManager │ └── sherpa-onnx - └── IME Window - └── ImeKeyboardView（orchestration / geometry / editor state） + └── IME window + └── ImeKeyboardView (orchestration, geometry, editor state) ├── ImeTopZone + CandidateBarController - ├── Pinyin26KeyboardRenderer / Pinyin9KeyboardRenderer - ├── NumericKeyboardRenderer + ├── Pinyin26 / Pinyin9 / Stroke / Numeric keyboard renderers ├── ImePanelRenderer │ ├── ClipboardPanelController │ ├── TextEditorPanelController @@ -31,119 +30,159 @@ LocalVoiceImeService ├── VoicePanelController / VoicePanelView ├── InlineVoicePresenter ├── FloatingKeyboardController - ├── BackspaceGestureController / BackspaceKeyFactory - ├── SpaceVoiceGestureController / SpaceVoiceKeyFactory + ├── BackspaceGestureController / SpaceVoiceGestureController ├── KeyPopupController ├── NineKeySegmentRepairController - ├── ImeThemeApplier - ├── PanelHeaderFactory - └── EmojiCellFactory + └── ImeThemeApplier ``` -生产运行时只存在一个顶层键盘 View：`ImeKeyboardView`。它负责 WindowInsets、响应式几何、编辑器/composition 协调和各具体 UI owner 的编排；候选、键盘布局、Panel、Voice presentation、Theme traversal、Popup 与 held-key gesture 已由上图中的具体类分别持有。当前架构不使用版本化键盘 View 命名。 - -## 状态所有权 - -业务事实应由 Service/领域模块持有；View 只保留瞬时显示状态。 - -- `LocalVoiceImeService / ImeState`：编辑器、键盘模式、Panel、composition、候选、Shift、设置、隐私状态。 -- `CandidatePipeline / CandidateSnapshot / RimeEngine`：候选生成、generation、native identity、Rime session。 -- `VoiceModelLifecycleManager`：本地 ASR runtime、预热、录音与 cooldown。 -- `InputConnectionGateway`：所有目标编辑器副作用。 -- `ImeKeyboardView`：当前编辑器/composition 协调、响应式测量几何、Panel/window 编排等顶层瞬时状态。 -- 各具体 UI owner：只持有自己表面的瞬时状态，例如候选滚动、Panel tab/scroll、Voice presentation generation、held-key gesture pointer、Popup 生命周期和 floating drag。 - -`NineKeyUiState` 是 CandidatePipeline 实例内的 session-scoped 歧义路径/显式选择缓存，不持有候选排序或编辑器状态；它不是第二套候选 source of truth。继续重构时不要把这类局部状态重新提升成全局状态。 - -## 输入提交原则 - -1. 半成品拼音通过 composing 更新，不直接作为普通文本写入目标应用。 -2. 选择候选、空格、Enter 或明确提交动作才执行 commit。 -3. 删除优先处理 openIME 自己的 composition，再处理目标编辑器文本。 -4. 候选提交使用已经渲染的 `CandidateSnapshot`/native identity，避免旧异步结果提交到新 composition。 -5. 切换输入框、模式或结束会话时必须使旧 generation/session 失效。 -6. 密码编辑器不组合、不读取正文、不进入个性化学习和热词/纠错学习；剪贴板历史和语音可用，语音只一次性上屏最终结果。要求关闭个性化学习的编辑器和来源应用标记为敏感的剪贴板内容不进入持久历史。 - -## 候选与 Rime - -- librime 就绪后是生产中文候选的权威来源。 -- `CandidateEngine` 是本地回退、英文联想和部分九键辅助，不应被理解成第二套权威中文引擎。 -- `CandidateSnapshot` 保证“用户看到的候选”和“真正上屏的候选”属于同一 generation。 -- native 单次查询进入 JNI 后当前不可从 Kotlin 中断；在有真机延迟数据之前不要修改 librime 内部。 - -## UI 与窗口 - -- Docked/Floating 是 IME Window 状态，不属于 Panel。 -- 横竖屏只影响响应式几何，不自动改变用户的 Floating/Docked 选择。 -- `KeyboardLayoutMetrics` 只做纯 dp 计算；View 负责把结果应用到 LayoutParams。 -- `KeyPopupController` 负责 transient key popup 的定位、动画和生命周期。 -- `FloatingKeyboardController` 负责浮动卡片 chrome、drag handle 和本地 drag 交互；WindowManager 边界仍由 Service 持有。 -- `ImeThemeApplier` 负责把当前 tokens 递归应用到已构建的 native View 树。 -- 所有布局基于当前 IME Window 实际尺寸和 WindowInsets，不使用固定屏幕坐标。 +`ImeKeyboardView` is the only top-level keyboard view. +It owns window insets, responsive geometry, editor and composition coordination, and the orchestration of the classes in the diagram. +Each of those classes owns one part of the user interface. + +## State ownership + +The service and the domain modules own the business state. +Views keep only short-lived display state. + +| Owner | State | +|---|---| +| `LocalVoiceImeService`, `ImeState` | Editor, keyboard mode, panel, composition, candidates, shift, settings, privacy state | +| `CandidatePipeline`, `CandidateSnapshot`, `RimeEngine` | Candidate generation, generation numbers, native identity, Rime session | +| `VoiceModelLifecycleManager` | Speech runtime, warm-up, recording, cooldown | +| `InputConnectionGateway` | All side effects on the target editor | +| `ImeKeyboardView` | Editor and composition coordination, measured geometry, panel and window orchestration | +| Controllers | Their own short-lived state. Examples: candidate scroll, panel tab, held-key pointer, popup lifetime, floating drag | + +`NineKeyUiState` is a cache of ambiguous paths and explicit choices inside one `CandidatePipeline`. +It does not own candidate order or editor state. +Do not move state like this into global state. + +## Commit rules + +1. Unfinished pinyin is composing text. Never write it as normal text into the target app. +2. Commit only when the user selects a candidate or presses space, Enter or another explicit commit key. +3. Delete the openIME composition first. Then delete text in the target editor. +4. Commit a candidate with the rendered `CandidateSnapshot` or native identity. + This stops an old asynchronous result from reaching a new composition. +5. When the input field, the mode or the session changes, invalidate the old generation and session. +6. In password fields, do not compose and do not read the text. Do not learn words or corrections. + Clipboard history and voice input work. Voice inserts the final text only. + Editors that ask for no personalized learning, and clipboard items that the source app marks as sensitive, do not enter persistent history. + +## Candidates and Rime + +- When librime is ready, it is the source of Chinese candidates. +- `CandidateEngine` is a local fallback. It also gives English suggestions and some nine-key help. + It is not a second Chinese engine. +- `CandidateSnapshot` makes sure that the candidates the user sees and the candidates that are committed come from the same generation. +- A native query cannot be interrupted from Kotlin after it enters JNI. + Do not change librime internals without latency data from real devices. + +## Window and layout + +- Docked and floating are IME window states. They are not panels. +- Rotation changes only the responsive geometry. It does not change the docked or floating choice. +- `KeyboardLayoutMetrics` does pure dp calculation. The view applies the result to the layout parameters. +- All layout uses the real size of the IME window and its `WindowInsets`. It uses no fixed screen coordinates. + See [Coordinate system](COORDINATE_SYSTEM.md). ## Voice -`VoiceModelLifecycleManager` 是 ASR runtime 的唯一 owner。模型校验、预热、构建和释放不在 IME 主线程执行。`VoicePanelController` 只持有 presentation-side session/generation，`InlineVoicePresenter` 只负责顶部内联状态，`SpaceVoiceGestureController` 只负责长按/上滑取消手势。长按判定跟随 Android 配置的 touch-and-hold timeout；松手、取消和旧 session 回调必须保持 generation 隔离。 +`VoiceModelLifecycleManager` is the only owner of the speech runtime. +Model checks, warm-up, build and release never run on the main thread. +For details, see [Voice input](LOCAL_VOICE_MODEL.md). -## 模块与分包 +## Packages -代码按功能分成子包，依赖只能自下而上；方向由 `ArchitectureLayeringTest` 固定，新增依赖必须先改那张表。 +The code is split into packages. A package may depend only on packages below it. +`ArchitectureLayeringTest` enforces the direction. +To add a dependency, first change the table in that test. ```text - app（根包：Service、Activity、RimeNative） - │ - keyboard ──────┐ - ┌──────────┬──────────┼──────────┤ - panel voice candidate floating - │ │ │ │ - widget handwriting │ rime │ setup - └──────────┴──────────┴────┬─────┴───────┘ - data - editor - core - theme + app (root package: service, activities, RimeNative) + │ + keyboard ──────┐ + ┌──────────┬──────────┼──────────┤ + panel voice candidate floating + │ │ │ │ + widget handwriting │ rime │ setup + └──────────┴──────────┴────┬─────┴───────┘ + data + editor + core + theme ``` -| 包 | 职责 | 依赖 | +| Package | Purpose | Depends on | +|---|---|---| +| `theme` | Design tokens, themes, drawing helpers | none | +| `core` | `ImeState`, keyboard modes, static data, crash guard | theme | +| `editor` | Target editor boundary: `InputConnectionGateway`, `EditorInfoAdapter`, Enter and selection policy | core | +| `data` | Preference and file repositories, user data import and export | core, editor, theme | +| `setup` | Shared UI helpers for activities | data, theme | +| `widget` | Reusable controls: `ImeKeyView`, `SwipeUpDetector` | theme | +| `floating` | Floating keyboard window, drag, card style | theme | +| `handwriting` | Handwriting pad | data, theme | +| `rime` | librime wrapper, input normalization, native candidate references | core, data | +| `candidate` | Candidate pipeline, snapshots, nine-key decoder, fuzzy pinyin, lexicons | core, rime | +| `voice` | Speech recognition, model lifecycle, voice panel, text post-processing | data, editor, theme | +| `panel` | Tool, clipboard, settings and text-edit panels | core, data, handwriting, setup, theme, widget | +| `keyboard` | `ImeKeyboardView`, keyboard renderers, top zone, gestures, popups | all packages above except app | +| `app` (root) | Service and activities from the manifest, JNI class | all | + +Rules: + +- A low package must not use a high package. + Two exceptions exist, and the test lists them. + `panel` and `keyboard` start some activities by class name. + `rime` calls the JNI class `RimeNative`. Its name is tied to native symbols, so it cannot move. +- Keep only manifest entries, JNI classes and classes that scripts name in the root package. +- The voice layer talks back to the UI through two narrow interfaces: + `VoiceSessionHost` and `VoiceEditorContext`. + It does not use `ImeKeyboardView` or the service directly. +- Put a new feature in its own package. + Split it into pure logic, Android boundary and one entry object. + Pure logic files do not import `android.*`. +- Use `internal` visibility by default. + Only activities that the manifest needs are public. + +Two classes are still large: `ImeKeyboardView` (about 3,100 lines) and `LocalVoiceImeService` (about 1,900 lines). +`ImeKeyboardView.Listener` has many methods. +Split them next. Follow the package rules above and add no cross-package dependency. + +## Product capabilities + +| Capability | Implementation | Status | |---|---|---| -| `theme` | 设计 token、主题、绘制工具、外观枚举、参考尺寸 | 无 | -| `core` | `ImeState`、键盘模式、静态数据、崩溃保护 | theme | -| `editor` | 目标编辑器边界：`InputConnectionGateway`、`EditorInfoAdapter`、Enter/选区策略 | core | -| `data` | SharedPreferences 和文件仓库、用户数据导入导出 | core、editor、theme | -| `setup` | Activity 页面共用的 UI 工具 `SetupUi` | data、theme | -| `widget` | 可复用控件：`ImeKeyView`（按键）、`SwipeUpDetector` | theme | -| `floating` | 浮动键盘窗口、拖动、卡片外观 | theme | -| `handwriting` | 手写板 | data、theme | -| `rime` | librime 引擎封装、输入规范化、native 候选引用 | core、data（及 JNI 类 `RimeNative`） | -| `candidate` | 候选管线、快照、九键本地解码、模糊音、拼音词典、笔画表 | core、rime | -| `voice` | 语音识别、模型生命周期、语音面板、识别后处理 | data、editor、theme | -| `panel` | 工具、剪贴板、设置、文本编辑等面板 | core、data、handwriting、setup、theme、widget | -| `keyboard` | `ImeKeyboardView` 编排、26 键/九键/笔画/数字键盘、顶部区、手势、弹窗 | 以上除 app 外的全部 | -| `app`（根包） | Manifest 里的 Service 和 Activity、JNI 类 | 全部 | - -规则： - -- 低层包不能引用高层包。唯一的例外写在测试里：`panel` 和 `keyboard` 按类名启动几个 Activity，`rime` 调用 JNI 类 - `RimeNative`（它的包名和方法名绑定 native 符号，不能移动）。 -- 根包只放 Manifest、JNI 和测试脚本按名字引用的入口类，别的东西不要放进来。 -- 语音层通过窄接口回到界面：`VoiceSessionHost`（键盘监听器继承它）和 `VoiceEditorContext`（Service 提供编辑器信息）， - 不直接依赖 `ImeKeyboardView` 或 Service。 -- 新功能自成一个包，按“纯逻辑 / Android 边界 / 唯一入口”拆分：纯逻辑文件不 import `android.*`， - 包外只通过一个入口对象和它的 Activity 使用。 -- 可见性默认 `internal`，只有 Manifest 需要的 Activity 是 public。 - -仍然偏大的地方：`ImeKeyboardView` 约 3000 行，`LocalVoiceImeService` 约 1800 行，`ImeKeyboardView.Listener` 有数十个方法。 -它们是下一步拆分的对象，拆分时沿用上面的包边界，不要新增跨包依赖。 - -## Native 与第三方代码 - -`app/src/main/cpp/local_rime_jni.cc` 和 CMake glue 是本项目维护边界。vendored librime/OpenCC/Boost 等第三方源码不作为日常架构重构对象；除非有明确 native 缺陷和测试证据，否则不要改 vendor 源码。 - -## 测试边界 - -- JVM Unit：纯策略、候选、输入连接、状态、几何。 -- Android Instrumentation：真实 View/IME 交互、API 29/31 兼容性。 -- debug source set：E2E Receiver 和测试 Activity，只用于测试 APK。 -- PowerShell/Bash scripts：真实 IME、视觉、性能、升级、安全和 SOP 证据。 - -测试说明见 [TEST_ARCHITECTURE.md](TEST_ARCHITECTURE.md)。 +| 26-key pinyin | `ImeKeyboardView`, `CandidatePipeline`, Rime | Production | +| Nine-key pinyin | `ImeKeyboardView`, `NineKeyLocalDecoder`, Rime | Production. No English nine-key. | +| Stroke | `StrokeKeyboardRenderer`, `StrokeLexicon` | Production | +| English 26-key | `CandidatePipeline`, `CandidateEngine` | Production | +| Digits, phone, decimal | `EditorInfoAdapter`, `NumericKeyboardRenderer` | Production | +| Symbols | `SymbolCatalog`, `CustomSymbolRepository` | Production | +| Emoji | `EmojiCatalog`, `EmojiRecentRepository`, Fluent assets | Production | +| Clipboard history | `ClipboardHistoryRepository`, `InputConnectionGateway` | Production. See the privacy rules above. | +| Quick phrases | `QuickPhraseRepository`, `QuickPhraseEditActivity` | Production | +| Text editing | `InputConnectionGateway` | Production. Available actions depend on the editor. | +| Voice | `VoiceModelLifecycleManager`, sherpa-onnx | Production. Hold space to speak. | +| Handwriting | `HandwritingPadView` | Pad only. No recognition engine. The entry point is hidden. | +| Floating keyboard | `FloatingKeyboardController`, `LocalVoiceImeService` | Production | +| Settings | `ImeSettingsRepository`, settings UI | Production | + +openIME has no AI writer and no online model. + +## Native and third-party code + +The project maintains `app/src/main/cpp/local_rime_jni.cc` and the CMake glue. +Vendored librime, OpenCC and Boost are not refactoring targets. +Change vendored code only for a confirmed native defect that has a test. + +## Tests + +- JVM unit tests: pure policy, candidates, input connection, state, geometry. +- Android instrumentation tests: real views, IME lifecycle, API compatibility. +- Debug source set: E2E receiver and test activities. Only test APKs use them. +- Scripts: real-device evidence for input, visuals, performance, upgrade and security. + +See [Test architecture](TEST_ARCHITECTURE.md). diff --git a/docs/BETA4_VOICE.md b/docs/BETA4_VOICE.md deleted file mode 100644 index 9bca77dd..00000000 --- a/docs/BETA4_VOICE.md +++ /dev/null @@ -1,68 +0,0 @@ -# Beta4 语音与 App 界面验收 - -基线：`8576727`，`0.0.3-beta.1`。开发分支 `codex/beta4-voice`，本地版本 `0.0.4-beta.1`。 - -## 用户要求与实现 - -- 松开空格立即结束麦克风采集，删除原来的 300ms 实采等待。stop 不取消识别：等采集线程结束、排空 PCM，再处理模型末块、标点与上屏。主动取消才丢弃音频。 -- 丢尾字根因：sherpa-onnx v1.13.6 的 Streaming Paraformer `inputFinished()` 只结束特征提取；不足整块的最后帧还需要 `stream.setOption("is_final", "1")` 才会进入解码。已补齐。保留 300ms 全零数组作为模型计算尾垫，完全不继续访问麦克风。 -- 新增本地 CT-Transformer INT8 标点模型，最终结果恢复断句、逗号与问号，保留用户明确说出的标点。结构化输入框沿用原来的保护策略，不自动补句子标点。 -- 在标点恢复后应用“标点用空格代替”，补齐紧贴中文的 ASCII 标点；小数 `3.5` 和网址 `https://a.b` 保留。真实设置控件保存后立即影响后续结果。 -- PCM 使用批量拷贝，推理复用 320 样本块；溢出明确报错，不把已缺字的结果当成功上屏。采集完成即恢复媒体音量，模型排队与标点不继续静音。 -- App 首页、偏好设置、键盘内设置和工具面板按新设计重做：首页分“三步引导”和“已就绪”两种状态；设置改为分组卡片列表，行间用分隔线，不再是一张张带底边的按键；强调色只用于可交互元素和开启状态，开关开启为强调色；浅色与暗色各有一套 App 配色。按键皮肤和单手模式已移除。键盘按键本身保持原有绘制效果。 - -App 的具体配色、间距、尺寸、位置和适配阈值见 [App 界面开发规范](APP_UI_SPEC.md)。 - -## 编码前列出的失败路径 - -语音包含 Android 麦克风、native ASR、主线程事件与编辑器，不能靠孤立的文本后处理证明完整链路。需区分： - -1. 麦克风未 ready 就松手、权限拒绝、启动失败：不能留下媒体静音或错误聆听状态。 -2. 正常松手：最后 PCM 入队并消费，实采立即停止；最后不足 320 样本或不足模型整块都不能漏解码。 -3. 取消、切换输入框、隐藏键盘、重开录音：旧回调不能解除新会话静音或提交旧文字。 -4. 环形跨界、输入大于容量、最后余样本：不能丢失、重复、越界或错计 droppedSamples。 -5. 加载/解码落后引起溢出：必须报错并撤销 composing，不能提交截断结果。 -6. 冷启动采集先结束：恢复媒体不等待模型；已录音频仍需保留到推理完成。 -7. 空录音、长句、中英混说、密码框：空结果无残留；密码框只最终提交。 -8. 标点恢复失败、显式标点、ASCII 标点、数字网址、设置来回切换：文本不丢，设置实际落盘并通过最终上屏体现。 -9. App 浅色/深色、320dp 窄屏与大字体、模糊音二级页返回：控件可访问，键盘原有布局不受 App 缩放影响。 - -## 可重复端到端测试 - -不新增单元测试。Debug 专属 `VoiceAudioE2E` 接收器运行在真实 IME 进程内，固定 PCM 经过实际 native Paraformer、实际 CT-Transformer 和真实 Android InputConnection；设置通过实际 App 控件修改。麦克风停止由脚本单独驱动真实空格 DOWN/UP 与 Android AudioRecord，不注入假识别回调，也不使用假的编辑器。 - -```bash -./gradlew --offline :app:assembleDebug :app:lintDebug -adb -s emulator-5554 install -r app/build/outputs/apk/debug/app-debug.apk -python3 scripts/beta4_e2e.py --serial emulator-5554 --out .local/test-runs/beta4/layout-final -``` - -测试采用 API 36 / x86_64 模拟器，临时 4GiB RAM、256MiB Java heap class，关闭宿主音频；`AudioRecord` 仍实际运行并提供静音 PCM。脚本保存并恢复输入法、App 偏好、App 语言、字体大小、显示尺寸与密度、旋转。运行前需让模拟器支持模型内存需求；例如临时启动 `emulator -avd openime-review-api36 -memory 4096 -no-snapshot -no-audio`，其余显示参数依主机配置。 - -SDK 的 gRPC `injectAudio` 在该主机反复导致模拟器退出，不能算通过。因此语音语料在设备内重放到真实模型，麦克风停止另测，不能宣称验证了“宿主麦克风收音 → 全部语料正确识别”的单条链路。早期失败尝试保留在 `.local/test-runs/beta4/failed-rpc/`；真机收音、口音准确率、Bluetooth 路由与设备性能尚未复测。上述完整失败矩阵也并非全部执行通过。 - -语料：上游 [Paraformer 测试音频 0.wav](https://huggingface.co/csukuangfj/sherpa-onnx-streaming-paraformer-bilingual-zh-en/resolve/main/test_wavs/0.wav)，固定为 Debug 构建的 `app/src/debug/assets/voice-beta4.wav`，16kHz / mono / PCM16，160,850 个样本，SHA-256 `7d93384ca14702cc584a7a33fe2fed92e89e708549161cb12ea38c916882103b`。上游模型仓库标记 Apache-2.0，测试语料不进入正式 APK。 - -## 模型来源与代价 - -- [官方标点模型说明](https://k2-fsa.github.io/sherpa/onnx/punctuation/pretrained_models.html)，固定导出 `sherpa-onnx-punct-ct-transformer-zh-en-vocab272727-2024-04-12-int8`。 -- [ModelScope 上游模型卡](https://modelscope.cn/models/iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch/summary) 标记 Apache License 2.0；APK 附许可文本，模型参与内置清单 SHA-256 校验。 -- ONNX 大小 75,519,198 字节（约 72MiB），SHA-256 `65a3fb9f5ad7bfb96bf69e0dc4481df97f6ee60513c1d94ce981ba6effd524b1`。标点首次使用才加载，随识别器冷却释放。 -- 原来的内存预算 520,000,000 调整为 620,000,000 字节；设备仍按真实可用系统内存与余量准入。Java heap class 的原判据设 256MiB 上限，避免把新增 native 模型大小误当成 Java 堆限制，让之前可用的 256MiB 设备直接失去语音功能。 -- 标点模型提高固定例句的断句质量，不能保证每句话都正确；混合语料的 ASR 错词仍能出现，不能把“末块不再漏解码”描述成“识别零错字”。 - -## 本轮结果 - -最终构建与验收已通过： - -- Debug 构建、`lintDebug`，签名 arm64 Release 构建、`lintRelease` 均成功。发布前补跑全部单元测试（383 个）：首次有 3 个失败，已修复——开关颜色与 App 布局字号改用设计 token，受信模型目录与测试同步标点模型（4 个文件），修复后全部通过。 -- 发布前复验（x86_64 模拟器，真实触摸与 AudioRecord）：beta4 端到端 7/7；核心打字回归 19/19；beta3 功能回归 10/10（竖横屏）；显示矩阵 11/11；语音失败路径 50/50（不同按住时长、上滑取消、录音中隐藏键盘与切后台、连按 10 次、静音录音、麦克风权限被撤销后恢复），每项都核对媒体音量恢复、无崩溃、无残留 composing。 -- 固定语料在最终处理前的 partial 以“星期”结尾，`inputFinished` 后完整为“星期三”；真实编辑器内容与最终识别结果相同。已录音频的末块得到解码，但中英混说仍有 ASR 错词，原始结果见 `native-audio-editor.json`。 -- 通过实际 App 控件打开“标点用空格代替”后，真实上屏为“今天天气很好 我们一起去公园散步吧”；ASCII 标点转换与小数/网址保护通过。 -- 实际空格触摸和 AudioRecord：松手至采集结束 33ms；captured=decoded=16,640 样本，缓冲余量=0、丢弃=0，媒体音量恢复。33ms 是这次模拟器实测，不是所有设备的硬保证。 -- （以下界面验收针对重新设计之前的版本；新设计需按 `APP_UI_SPEC.md` 重新验收。）首页、设置、皮肤二级页的常规字体验收，以及首页/设置的浅色、深色、320dp / 1.3 倍字体通过；增加 320dp / 2.0 倍字体与 1400dp 宽屏验收。大字体滑杆数值“100%”保持单行，2 倍字体选项纵排；实际 UI XML 核对 16dp 页面边距、8dp 步骤间距、64dp 步骤高度下限、52dp 主按钮高度与 600dp 宽屏内容居中。截图已人工检查。 -- 最终截图验证了胶囊形主按钮与绿色/灰色开关、白色滑块；UI XML 进一步核对实际 Button 角色、Switch 的 checkable/checked 开关态。 -- 工件在 `.local/test-runs/beta4/layout-final/`：`results.json`、`artifact-index.json`（APK 与工件 SHA-256）、真实编辑器 XML、截图、采集日志与构建日志。重复命令见上文。 -- 本地安装包在 `build/beta4-release-files/openIME-v0.0.4-beta.1-arm64-release.apk`，签名与 Beta3 证书一致；正式包不含 Debug 音频和重放入口。APK SHA-256：`a3925715e4109d1445902ae57372763b9708f5418113c2cdb2099fda94d0dd71`。 - -上述验收在 x86_64 Debug 模拟器执行；arm64 Release 已完成构建、签名、ABI 与内容核对，尚未在真机复测。构建、验收与签名包均为本地操作，没有创建远端 tag 或公开发布。 diff --git a/docs/CHANGELOG_PRE_1.0.md b/docs/CHANGELOG_PRE_1.0.md deleted file mode 100644 index 1f9de5ad..00000000 --- a/docs/CHANGELOG_PRE_1.0.md +++ /dev/null @@ -1,36 +0,0 @@ -# 1.0.0 之前的开发期记录 - -这是 1.0.0 正式版之前的内部迭代记录，原样保留自旧版 `CHANGELOG.md`。从 1.0.0 起， -`CHANGELOG.md` 只记录面向用户和维护者的版本变更；这里的条目已经概括进 -1.0.0 的发布说明，不再更新。 - -## 开发期迭代（原 Unreleased） - -- 九键左栏改为按设计稿的「读法列表」：输入中整块面板、整条读法（`ni'hao`），长输入改列首音节；点选即锁定，锁定的音节继续打字时保持锁定，退格先解锁；预编辑边界统一为撇号。输入法的做法与取舍见 `docs/NINE_KEY_REFERENCE.md`。 -- 九键候选：数字刚好拼得出的词排在预测词之前；预编辑跟随首选词的读法，长句 `9694264244326` 得到 `wo'xiang'chi'fan` 与「我想吃饭」。 -- 选词只覆盖一部分输入时只上屏该词，剩余输入继续作为预编辑，不再丢失，也不再向用户词库写入没选过的整句。 -- 删除键上滑清空 / 下滑撤回：在自绘、Compose、Web 等没有「全选」也没有完整 ExtractedText 的输入框里也能清空和撤回（此前手势触发但文字纹丝不动）；触发距离 56dp → 32dp；清空与撤回统一为同一个手势提示，清空后顶栏显示「已清空 · 撤销」。 -- 上屏后的联想栏改为「‹ 联想词 ∨」（此前收起箭头占一半行宽，联想词被挤到右边）。 -- 预编辑文字、工具图标等的强调色改为从当前强调色推导，不再写死 `#006AB1`，换强调色后不会出现两种蓝并排。 -- 修复键盘首次显示或系统取整出的宽度变化后整块按键为空白（行尺寸 0×0）：`screenWidthDp` 的 float/int 取整差（约 0.0011）超过了 0.001 的比较容差，在 `onSizeChanged` 里重建按键行后没有再触发布局。容差改为 0.01，并在重建后补一次 `requestLayout`。 -- 设置页分段控件的每个选项触控高度由 34dp 提升到 48dp；轨道外观仍是 34dp，在主题里内缩绘制。 -- 同步设计稿改版后的测试契约：图标描边统一为 1.8、字号与颜色走 token、单选标记改为矢量图；插桩测试改为按无障碍名称查找语音语言控件，按手势阈值缩放后的 px 驱动空格滑动，并用 `isKeyPopupShown()` 判断按键预览（预览视图常驻，不能再靠子 View 数量判断）。 -- 修复九键方案未编译导致的 librime 静默降级：`luna_pinyin_simp` 与 `luna_pinyin_simp_fuzzy` 把字面代数规则写进了 `speller/algebra` 的 `__patch` 列表，librime 会把列表项当作补丁路径解析并报循环依赖，整个方案构建失败（部署日志 `4 success, 5 failure`），中文候选一直退回本地九键解码器。现改为在 `pinyin.yaml` 中定义命名的 `t9_transliteration` 节点并引用它。 -- `versionCode` 3 → 4：rime 数据部署标记随版本号变化，让已安装的设备重新拷贝修复后的 schema。 -- 持续完善输入法 UI、自适应布局和本地语音模型接入。 -- 引入固定版本的 Rime Ice 基础、扩展和 8105 字表，并增加首次部署期间即时可用的高频候选层。 -- 增强全拼、简拼、显式分词、首选命中和用户词排序，候选读取上限扩展到 96 项。 -- 统一选词、空格及回车提交后的 composition 清理，删除键不再误删残留候选状态。回车（确定）改为提交已输入的拼音原文，空格仍提交首选候选。 -- 增加连续长句、扩展候选、分词和选词后回删的真实 IME 回归脚本。 -- 建立 L0～L3 正式测试 SOP、统一证据目录、输入框实验室和隐私回归门禁。 -- 将 `DebugKeyboardActivity` 从主变体迁入 debug，release APK 不再导出测试宿主。 -- 将本地语音模型改为输入框出现时后台预热、隐藏后 10 秒热驻留并异步释放，键盘主线程不再执行模型哈希或映射。 -- 长按空格在模型预热期间先录音缓存，修复仅返回 final 时不上屏，并加入动态热词、本地语音纠错学习、音频路由隔离和无文本性能指标。 -- debug E2E 入口增加 `android.permission.DUMP` 保护，保留 adb 回归能力并阻止普通第三方 App 调用。 - -## 最初的开发版本 - -- 建立独立 `openIME` APK，包名为 `llc.slacker.openime`。 -- 接入 librime、OpenCC、中文拼音候选和多种键盘模式。 -- 接入 APK 内置的 sherpa-onnx 中英双语语音模型。 -- 建立真实 IME、生命周期、隐私和多宽度布局回归脚本。 diff --git a/docs/COMPATIBILITY.md b/docs/COMPATIBILITY.md index 9f049c70..46a7b259 100644 --- a/docs/COMPATIBILITY.md +++ b/docs/COMPATIBILITY.md @@ -1,61 +1,111 @@ -# 输入环境兼容性 +# Compatibility -键盘要在别人的应用里工作，所以「哪些环境、怎么处理、怎么验证」写在这里。改输入链路之前先看一遍。 +A keyboard must work inside other apps. +This document lists each input environment, how openIME handles it and how we verify it. +Read it before you change the input path. -## 编辑器 +## Editors -| 环境 | 处理 | 验证 | +| Environment | Handling | Verification | |---|---|---| -| 普通 / 多行 / 搜索 / 聊天（EditText、WebView、Compose） | 拼音预编辑 + 候选；回车按 IME action 或原始回车 | `core_regression.sh`、`ImeTestLabActivity` | -| 自绘 / Compose / Web，没有「全选」也没有 ExtractedText | 清空 / 撤回改用光标前后文本，答案长度等于请求长度时拒绝删除 | `InputConnectionGatewayTest`，`CustomEditorTestActivity` | -| 密码（含可见密码、网页密码、数字密码） | 不组合（字母逐个直接上屏）、不学习词库和纠错、不读取输入框正文；可以使用剪贴板历史；语音可用，只把最终结果一次性上屏（不显示中间结果） | `security_regression.ps1`；`VoiceFinalPolicyTest`、`ClipboardSensitivityPolicyTest` | -| 数字 / 电话 / 日期时间 | 起始键盘为数字 | `EditorInfoAdapterTest` | -| 邮箱 / URL | 起始键盘为英文 | `EditorInfoAdapterTest` | -| TYPE_NULL（终端、游戏、远程桌面） | 起始英文；每个字母立即以真实按键事件送出；退格 / 前删用按键事件（它们的 InputConnection 多半是 dummy 模式的 BaseInputConnection，`deleteSurroundingText` 返回 true 却什么也没删） | `InputConnectionGatewayTest` | -| 无个性化学习标志（隐身模式） | 不学习、不记录剪贴板 | `PersonalizedLearningPolicy` | -| 一次提交几十万字（大段粘贴、长语音） | 分块提交，每块不超过 32000 个字符且不拆代理对，避免超过 Binder 单次事务上限 | `CrashResilienceTest` | - -## 自动填充（Android 11+） - -`method.xml` 声明 `supportsInlineSuggestions`，键盘请求最多 5 个 48dp 高的条目并把系统渲染的条目放进工具栏位置。键盘只托管条目： -填入的内容由系统直接写入输入框，键盘既拿不到也不读取，所以密码框里同样可以显示条目。条目响应可能先于键盘视图到达（新输入框刚获得焦点）， -服务会暂存最近一次响应，视图建好后再显示，而不是拒绝（拒绝会让系统退回下拉菜单）。离开输入框时清除。 -Android 11 以下和不支持内嵌建议的提供者仍使用系统的下拉菜单。 -验证：`InlineChipTrackerTest`；`scripts/beta3_e2e.py autofill`（debug 构建自带测试提供者 `TestAutofillService` 和 `AutofillTestActivity`）。 - -## 物理键盘（平板、折叠屏键盘套、Chromebook、桌面模式、模拟器） - -中文 26 键模式下：字母组成拼音，空格选首选，1–9 选候选，回车保留已输入拼音，Esc 取消，`'` 分词，退格删拼音； -`, . ? ! ; : ( )` 输出全角标点（数字后的 `, . :` 保持 ASCII，3.14 不会变成 3。14）；Ctrl / Alt / Meta 组合键、 -大写字母和其他按键原样交给应用（大写会先结束当前预编辑）。英文 / 九键 / 数字模式、密码框、TYPE_NULL 编辑器不接管。 -需要键盘面板可见（候选显示在面板上）。验证：`HardwareKeyPolicyTest`，`core_regression.sh` 040–043。 - -## 显示环境 - -横屏（不进入全屏提取模式，键盘是底部面板）、字体 130% / 200%（按键标签最多放大到 1.3 倍，功能键标签自动缩小）、 -深色、小屏、窄屏、平板竖 / 横、折叠屏内屏。验证：`scripts/display_matrix_regression.py`（断言底部面板且每个键都在窗口内）、 -`DisplayEnvironmentInstrumentedTest`、`scripts/beta3_e2e.py`（横屏下空格滑动光标、字母键数字和符号、表情联想、语音处理、自动填充）。 - -## Android 版本 - -`minSdk` 26；CI 在 API 26（minSdk）、29、31、34 上运行全部仪器测试，本地另在 API 36 上运行；发布前必须通过的是 API 29 和 31。 - -## 崩溃、卡死与冲突 - -- 一次按键处理失败不会让键盘进程退出：记录（只含异常类型和代码位置，不含输入内容）、丢弃半成品预编辑、继续工作。 - 验证：`core_regression.sh` 038（调试命令 `fail-next` 注入一次失败）。 -- 崩溃历史：Java 崩溃、原生崩溃和 ANR（Android 11+ 的进程退出记录）。10 分钟内 3 次进入**安全模式**： - 关闭 librime 和语音预加载，用内置词库继续输入，「设置 → 关于与数据 → 诊断」可复制诊断信息或退出安全模式。 -- librime 启动前写标记，通过健康检查后清除。留下标记且上个进程确实是原生崩溃时逐级处理：清理编译产物 → - 把用户词库改名备份并重建 → 不再启动原生引擎。被用户或系统强停的启动不计为崩溃。 -- 语音输入静音媒体音量时，原音量同时写入磁盘并有两分钟看门狗；进程在录音中途死掉，下次启动恢复，音乐 / 视频不会一直没声。 - 验证：`VoiceMediaMuteRecoveryInstrumentedTest`。 -- 词库与九键解码器在后台线程构建，不再占用主线程（冷启动曾多占约 0.3 秒）。 -- 退格不再每次向应用发起三次同步 Binder 调用：编辑器已经报告光标是收起状态时，不再去问「选中了什么」。 - 应用卡住时，每次调用都会让键盘跟着等。 - -## 尚未覆盖 - -- 九键模式下的物理键盘（字母直接交给应用）； -- 物理键盘用户隐藏键盘面板后的候选显示（需要独立的候选窗口）； -- 真机上的 OEM 差异（小米、OPPO、三星）：目前只有模拟器与 CI 模拟器的结果。 +| Normal, multiline, search and chat fields (EditText, WebView, Compose) | Pinyin composition and candidates. Enter uses the IME action or a raw Enter. | `core_regression.sh`, `ImeTestLabActivity` | +| Custom-drawn, Compose or web fields with no select-all and no `ExtractedText` | Clear and restore use only the text before and after the cursor. The gateway refuses to delete when the returned length equals the requested length. | `InputConnectionGatewayTest`, `CustomEditorTestActivity` | +| Password fields (visible, web and numeric passwords) | No composition: each letter goes in directly. No word or correction learning. openIME does not read the field text. Clipboard history works. Voice works and inserts the final text only. | `security_regression.ps1`, `VoiceFinalPolicyTest`, `ClipboardSensitivityPolicyTest` | +| Number, phone, date and time fields | The start keyboard is digits. | `EditorInfoAdapterTest` | +| Email and URL fields | The start keyboard is English. | `EditorInfoAdapterTest` | +| `TYPE_NULL` (terminals, games, remote desktops) | The start keyboard is English. openIME sends each letter as a real key event. Delete uses key events, because the `InputConnection` of these apps is often a dummy that reports success and deletes nothing. | `InputConnectionGatewayTest` | +| No personalized learning flag (incognito) | openIME does not learn words and does not record the clipboard. | `PersonalizedLearningPolicy` | +| Very large commits (large paste, long voice text) | openIME commits in chunks of at most 32,000 characters. It never splits a surrogate pair. This keeps each Binder transaction below the size limit. | `CrashResilienceTest` | + +## Autofill (Android 11 and later) + +`method.xml` declares `supportsInlineSuggestions`. +The keyboard requests up to five entries that are 48 dp high. +It shows the entries that the system renders in the toolbar area. + +The keyboard only hosts the entries. +The system writes the filled text into the field. +The keyboard cannot read it, so entries also work in password fields. + +A response can arrive before the keyboard view exists. +The service keeps the latest response and shows it when the view is ready. +It must not reject the response, because the system then falls back to the dropdown menu. +The service clears the response when the user leaves the field. + +Android 10 and earlier, and providers without inline support, use the system dropdown menu. + +Verification: `InlineChipTrackerTest` and `scripts/beta3_e2e.py autofill`. +The debug build includes a test provider (`TestAutofillService`) and `AutofillTestActivity`. + +## Physical keyboards + +Physical keyboards include tablets, foldable keyboard cases, Chromebooks, desktop mode and emulators. +In 26-key Chinese mode: + +- Letters build pinyin. +- Space selects the first candidate. +- Keys 1 to 9 select a candidate. +- Enter keeps the typed pinyin. +- Esc cancels. +- `'` splits a syllable. Backspace deletes pinyin. +- `, . ? ! ; : ( )` produce full-width punctuation. + After a digit, `, . :` stay ASCII, so `3.14` does not change. +- Ctrl, Alt and Meta combinations, capital letters and other keys go to the app unchanged. + A capital letter first ends the current composition. + +openIME does not handle the physical keyboard in English, nine-key and digit modes, in password fields and in `TYPE_NULL` editors. +The keyboard panel must be visible, because the candidates appear on the panel. + +Verification: `HardwareKeyPolicyTest`, `core_regression.sh` cases 040 to 043. + +## Display environments + +openIME supports these environments: + +- landscape (the keyboard is a bottom panel and never uses fullscreen extract mode) +- font scale 130% and 200% (key labels grow to at most 1.3 times, and function key labels shrink to fit) +- dark theme +- small and narrow screens +- tablets in portrait and landscape +- the inner screen of a foldable + +Verification: + +- `scripts/display_matrix_regression.py` checks for a bottom panel and for every key inside the window. +- `DisplayEnvironmentInstrumentedTest` +- `scripts/beta3_e2e.py`: cursor swipe in landscape, letter-key hints, emoji suggestions, voice processing and autofill + +## Android versions + +`minSdk` is 26. +CI runs all instrumentation tests on API 26, 29, 31 and 34. +Developers also run them on API 36 locally. +A release needs a pass on API 29 and API 31. + +## Crashes, freezes and conflicts + +- One failed key handler does not stop the keyboard process. + openIME records the failure (only the exception type and the code location, never typed text), drops the unfinished composition and continues. + Verification: `core_regression.sh` case 038, with the debug command `fail-next`. +- openIME keeps a crash history of Java crashes, native crashes and ANRs (from the Android 11 exit records). + Three crashes in 10 minutes start **safe mode**. + Safe mode turns off librime and voice preloading and types with the built-in lexicon. + 设置 → 关于与数据 → 诊断 (Settings → About and data → Diagnostics) copies the diagnostics and leaves safe mode. +- openIME writes a marker before it starts librime and clears it after the health check. + If the marker stays and the last process died from a native crash, openIME escalates in steps: + clean the compiled files, rename and rebuild the user dictionary, and then do not start the native engine again. + A start that the user or the system force-stopped does not count as a crash. +- While voice input mutes media volume, openIME saves the old volume on disk and runs a two-minute watchdog. + If the process dies during recording, the next start restores the volume. + Verification: `VoiceMediaMuteRecoveryInstrumentedTest`. +- The lexicon and the nine-key decoder are built on background threads. + Cold start uses about 0.3 s less main-thread time. +- Backspace does not make three synchronous Binder calls each time. + When the editor reports a collapsed cursor, openIME does not ask for the selected text. + A frozen app would block every call. + +## Not covered + +- A physical keyboard in nine-key mode (letters go to the app). +- Candidates when the user hides the keyboard panel on a physical keyboard. This needs a separate candidate window. +- Vendor differences on real devices (Xiaomi, OPPO, Samsung). We have results only from emulators and CI emulators. diff --git a/docs/COORDINATE_SYSTEM.md b/docs/COORDINATE_SYSTEM.md index 33eb138f..4ad106ed 100644 --- a/docs/COORDINATE_SYSTEM.md +++ b/docs/COORDINATE_SYSTEM.md @@ -1,54 +1,49 @@ -# Keyboard Coordinate System +# Coordinate system -## 原则 +Keyboard tests and UI calculations never use screen pixels. +A pixel position such as "the Q key is at (123, 1876) on a 1080 × 2400 screen" breaks on other screens. -键盘测试和 UI 计算不再依赖具体的屏幕像素，例如： +Instead, all positions are normalized to the content area of the input method: ```text -1080×2400 上 Q 键在 (123, 1876) +Origin: top left corner of the IME content area +Range: 0.0 to 1.0 +Runtime: normalized × keyboardWidth / keyboardHeight → real px ``` -而是统一使用输入法内容区域自身的归一化坐标： +This way, the same tests work on 720p, 1080p and 2K screens, at different densities, in landscape and on screens with rounded corners. -```text -原点：IME 内容区域左上角 -范围：0.0 ~ 1.0 -运行时：normalized × keyboardWidth / keyboardHeight → 真实 px -``` - -这样 720p、1080p、2K、不同 DPI、横屏和圆角屏都不需要改测试基准。 - -## 实现 +## Implementation -`app/src/main/java/llc/slacker/openime/keyboard/KeyboardGeometry.kt` +File: `app/src/main/java/llc/slacker/openime/keyboard/KeyboardGeometry.kt` -- `NormalizedBounds(left, top, right, bottom)`：归一化矩形。 -- `NormalizedBounds.fromView(view, root)`：从真实 `View` 测量值生成归一化坐标。 -- `toPx(rootWidth, rootHeight)`：运行时转换为真实像素。 +- `NormalizedBounds(left, top, right, bottom)` is a normalized rectangle. +- `NormalizedBounds.fromView(view, root)` builds it from the measured size of a real `View`. +- `toPx(rootWidth, rootHeight)` converts it to real pixels at runtime. -`ImeKeyboardView.normalizedBoundsReport()` 输出的是： +`ImeKeyboardView.normalizedBoundsReport()` prints one line for each key: ```text key|x,y,w,h ``` -其中 x/y/w/h 全部相对于当前 IME Root，而不是屏幕。 +The values x, y, w and h are relative to the current IME root. They are not relative to the screen. -## 布局规则 +## Layout rules -- 整行：使用 `Row + Weight + Relative Insets`。 -- 按键宽度：优先使用 weight，而不是写死每个键的 XY。 -- 26 键：第一行 10 键、第二行 9 键居中、第三行 Shift/M 区、底部功能键。 -- 拼音九键 / 数字：左筛选栏、中网格、右操作栏；当前不提供英文九键。 -- 高度和字体：dp/sp 约束最小/最大值，避免平板和折叠屏爆炸。 -- Popup、动效、锚点：全部相对于 Key Bounds 计算。 +- Rows use `Row + Weight + Relative Insets`. +- Key width comes from weight. Never write fixed XY values for keys. +- 26-key layout: 10 keys in row 1, 9 centered keys in row 2, the Shift-to-M row, and the bottom function keys. +- Nine-key and digit layouts: a filter column on the left, a grid in the middle and an action column on the right. There is no English nine-key. +- Height and font: dp and sp have minimum and maximum values, so tablets and foldables stay in range. +- Popups, animations and anchors are all calculated from key bounds. -## 自动化定位 +## Test automation -`scripts/core_regression.ps1` 和 `scripts/extended_regression.ps1` 不读取绝对 XY： +`scripts/core_regression.ps1` and `scripts/extended_regression.ps1` read no absolute XY values: -- `E2ETestReceiver` 通过 `tap:` 驱动真实 `InputMethodService` 的 click listener。 -- 焦点定位通过 `uiautomator dump` 实时读取 EditText bounds。 -- 模式定位通过 `state` 命令读取 `ImeState`。 +- `E2ETestReceiver` runs `tap:` through the click listener of the real `InputMethodService`. +- The scripts find the focus with a live `uiautomator dump` of the EditText bounds. +- The scripts find the mode with the `state` command, which reads `ImeState`. -因此脚本可以运行在 emulator 和真实小米手机上。 +So the scripts run on emulators and on real phones. diff --git a/docs/DECISIONS.md b/docs/DECISIONS.md index 1651cec5..d063ccae 100644 --- a/docs/DECISIONS.md +++ b/docs/DECISIONS.md @@ -1,18 +1,95 @@ -# Visual language decisions - -- 2026-09-29｜浅色次级文字对比度｜保留键盘原始 `keySecondaryText=#6E6E73`，新增页面角色 `textSecondaryRole=#6D6D72`｜原值对 `surface=#EEF0F3` 约 4.44:1，低于规范 4.5:1；只修页面角色，避免改变键盘既有视觉。 -- 2026-09-29｜候选展开图标状态｜复用 `ic_chevron_down`，展开时旋转 180°｜保持单一矢量资产，不再用 Unicode 上下箭头。 -- 2026-09-29｜Setup XML 色板｜仅保留启动前必须使用的资源色，并由 TokenDriftTest 与 Kotlin token 对齐｜Android XML 在 Kotlin 初始化前需要资源颜色，不能完全移除。 -- 2026-09-29｜退格连删节奏｜继续保留现有 60ms 匀速重复｜视觉语言 v1 明确要求本轮不改，留作后续交互专项。 -- 2026-09-29｜语音松手尾部｜松手后保留 300ms 采集窗口，并用 generation/session 所有权阻止旧会话影响新会话｜修复 AudioRecord 尾部被立即截断，同时保持取消即时生效。 -- 2026-09-29｜九键解码线程策略｜先记录 `publishNineKeyDigits -> resolveNineKey` 的 20 次窗口 P50/P95；未取得中低端机 P95 前保持现有线程，不提前迁移｜只有 P95 超过 8ms 才按规范迁到 `CandidateQueryCoordinator`。 -- 2026-09-29｜九键首帧与 Rime 刷新｜候选按压或滚动期间暂缓应用异步 Rime 结果；同时记录从请求到结果可应用的端到端延迟｜防止手指下的候选列表重排；是否在 Rime 就绪时跳过首帧回退，等实测 P95 是否低于 40ms 后再定。 -- 2026-09-29｜Rime 用户词库导出｜vendored librime 1.17.0 已编入 levers 模块，`UserDictManager::Export/Import` 可在关闭用户库会话后导出/合并 UTF-8 快照｜用户数据 JSON 在 Rime 会话已加载时包含自动学习词库；不可用时必须明确提示，不静默遗漏。 -- 2026-09-29｜流式语音模型｜默认模型切换为 `sherpa-onnx-streaming-paraformer-bilingual-zh-en` 的 INT8 encoder/decoder；运行时使用 `OnlineParaformerModelConfig` + `greedy_search`，结束时补 300ms 静音；不再向 Paraformer stream 传 transducer-only 动态 hotwords｜优先降低模型体积和保持中英流式识别，同时遵循 sherpa-onnx v1.13.6 官方 Paraformer 配置。 -- 2026-10-03｜语音词表｜内置词表随版本发布（`assets/hotwords/`），用户可导入文本词表；二者都不联网，不新增 `INTERNET` 权限，不做在线定期更新｜Paraformer 不能把热词传进解码器，所以词表走识别后的“同音替换”：读音（取自 `8105.dict.yaml`，含多音字）相同而字不同的片段改成词表写法。游戏词表默认关闭，避免日常聊天被误改。 -- 2026-10-03｜整体分包｜99 个平铺文件按功能分进 theme/core/editor/data/setup/widget/floating/handwriting/rime/candidate/hotword/voice/panel/keyboard，根包只留 Service、Activity 和 JNI 类；依赖方向由 `ArchitectureLayeringTest` 固定｜包名只影响组织和可见性，不改 Manifest、native 符号和测试脚本引用的名字；Gradle 多模块暂不做，因为 `keyboard` 与 `panel`、`voice` 仍通过大接口耦合，需要先拆 `ImeKeyboardView.Listener`。 -- 2026-10-04｜自动填充条带｜只声明 `supportsInlineSuggestions`、自己拼装 androidx.autofill v1 的样式 Bundle（版本表 + 一个空的 v1 样式），不引入 androidx.autofill 依赖；条目按固定像素尺寸（150dp × 40dp）渲染并横向滚动｜`InlineContentView` 是远程 Surface，没有固有尺寸，`WRAP_CONTENT` 会得到 0×0；样式 Bundle 只有十几行，为它引入第一个 androidx 运行时依赖不值得。响应早于键盘视图到达时先暂存，不能返回 false（系统会退回下拉菜单）。 -- 2026-10-04｜字母键上的数字和符号｜不加一行数字，而是像搜狗、讯飞、微信键盘那样把数字和符号印在字母键右上角，上滑（复用九键的“上滑输入数字”开关）或长按输入；第一排 q–p 是 1–0，其余是标点，中文模式用全角形式（！ ￥ ？ （ ） ： ；），英文模式半角；提示可单独关闭（“数字和符号提示”，默认开启）｜多一行数字要么挤压每一行（竖屏约 43dp，横屏约 34dp），要么让键盘变高；输入法窗口高度一变，面板、浮动键盘、九键都要跟着变。提示 + 上滑不改布局，也是国内主流输入法的做法。 -- 2026-10-04｜空格滑动光标锁定底行｜进入光标模式（横向 18dp、横向分量大于纵向 1.25 倍、早于长按语音超时）后，同一行的其他按键 `touchLocked`（变灰、不响应）；拼音预编辑存在时移动的是预编辑光标｜手指一直在空格上，但第二根手指或漂移的拇指可能按到邻键；预编辑期间把方向键事件发给应用会打断组合。 -- 2026-10-04｜“标点用空格代替”｜只作用于语音识别结果：逗号、句号、问号等写成一个空格，结尾标点直接去掉，括号和 3.5、a.b 不变｜把这项需求理解为语音文本后处理的开关（与去语气词同属“语音输入”设置组），默认关闭。 -- 2026-10-05｜移出语音词表｜0.0.6-beta.1 起去掉语音词表（`hotword` 模块、内置词表、管理页、打字候选加权）｜同音替换还不成熟，先不发；去掉前的代码保存在 `archive/voice-word-lists` 分支（即 0.0.5-beta.1 的 `f3bb5cc`），以后改好再合回。 +# Design decisions + +This log records decisions that are not obvious from the code. +Each entry has a date, the topic, the decision and the reason. +Add new entries at the end. + +## 2026-09-29 + +- **Secondary text contrast (light theme).** + Keep the keyboard value `keySecondaryText=#6E6E73`. Add a page role `textSecondaryRole=#6D6D72`. + *Reason:* the old value has a contrast of about 4.44:1 on `surface=#EEF0F3`, below the 4.5:1 rule. + Only the page role changes, so the keyboard look stays the same. +- **Candidate expand icon.** + Reuse `ic_chevron_down` and rotate it 180° when expanded. + *Reason:* one vector asset. No Unicode arrows. +- **Setup XML colors.** + Keep only the resource colors that Android needs before Kotlin starts. `TokenDriftTest` keeps them equal to the Kotlin tokens. + *Reason:* XML needs resource colors before Kotlin initializes. +- **Backspace repeat.** + Keep the constant 60 ms repeat. + *Reason:* visual language v1 says not to change it in that round. A later interaction project can change it. +- **Voice release tail.** + After release, keep a 300 ms capture window. Use generation and session ownership so that an old session cannot affect a new one. + *Reason:* an immediate cut loses the tail of `AudioRecord`. A cancel still takes effect at once. + *Later change:* see `LOCAL_VOICE_MODEL.md`. The microphone now stops at release, and a 300 ms zero pad feeds the model. +- **Nine-key decoder thread.** + First record the 20-sample window P50 and P95 of `publishNineKeyDigits → resolveNineKey`. + Keep the current thread until we have a P95 from a low-end or mid-range phone. + *Reason:* move the decoder to `CandidateQueryCoordinator` only if P95 is above 8 ms. +- **Nine-key first frame and Rime refresh.** + While a finger presses or scrolls the candidate list, delay asynchronous Rime results. Record the delay from request to applicable result. + *Reason:* the list must not reorder under the finger. + Skip the fallback first frame only if the measured P95 is below 40 ms. +- **Rime user dictionary export.** + The vendored librime 1.17.0 includes the levers module. `UserDictManager::Export/Import` can export or merge a UTF-8 snapshot after the user database session closes. + *Reason:* the user data JSON includes learned words when a Rime session is loaded. When export is not possible, the app must say so. It must not skip the words silently. +- **Streaming voice model.** + Use the INT8 encoder and decoder of `sherpa-onnx-streaming-paraformer-bilingual-zh-en` with `OnlineParaformerModelConfig` and `greedy_search`. + *Reason:* smaller model and streaming Chinese and English recognition, with the official sherpa-onnx v1.13.6 configuration. + Do not send transducer-only dynamic hotwords to the Paraformer stream. + +## 2026-10-03 + +- **Voice word lists.** *(Removed on 2026-10-05.)* + The idea was a built-in word list plus a user import list, both offline, with no `INTERNET` permission. + Paraformer cannot take hotwords in the decoder, so the lists worked as homophone replacement after recognition. + See the 2026-10-05 entry. +- **Package split.** + Split the 99 flat files into packages: theme, core, editor, data, setup, widget, floating, handwriting, rime, candidate, voice, panel and keyboard. + The root package keeps only the service, activities and JNI classes. + `ArchitectureLayeringTest` enforces the dependency direction. + *Reason:* packages change organization and visibility only. They do not change the manifest, native symbols or the names that test scripts use. + We delay Gradle modules, because `keyboard`, `panel` and `voice` still connect through a large interface. + First split `ImeKeyboardView.Listener`. + +## 2026-10-04 + +- **Autofill strip.** + Declare only `supportsInlineSuggestions` and build the androidx.autofill v1 style bundle by hand (a version table and one empty v1 style). + Do not add the androidx.autofill dependency. + Render entries at a fixed size of 150 dp × 40 dp in a horizontal scroll. + If a response arrives before the keyboard view exists, hold it. Never return false, because the system then falls back to the dropdown menu. + *Reason:* `InlineContentView` is a remote surface with no intrinsic size, so `WRAP_CONTENT` gives 0×0. + The style bundle is a dozen lines. It is not worth the first androidx runtime dependency. +- **Numbers and symbols on letter keys.** + Do not add a number row. + Print the number or symbol at the top right of each letter key, as Sogou, iFlytek and WeChat keyboards do. + Swipe up (this uses the nine-key "swipe up for digits" setting) or long-press to enter it. + The first row q to p gives 1 to 0. The other rows give punctuation. + Chinese mode uses full-width forms (！ ￥ ？ （ ） ： ；). English mode uses half-width forms. + A separate setting, 数字和符号提示 ("Number and symbol hints", default on), turns the hints off. + *Reason:* an extra row would squeeze every row (about 43 dp in portrait, 34 dp in landscape) or make the keyboard taller. + A taller keyboard changes the IME window height, and the panels, the floating keyboard and the nine-key layout would all change. + Hints plus swipe do not change the layout. Major Chinese keyboards do the same. +- **Space-swipe cursor locks the bottom row.** + The cursor mode starts when three conditions are true. + The finger moved 18 dp horizontally. + The horizontal part is more than 1.25 times the vertical part. + The long-press voice timeout has not ended. + Then the other keys in the same row get `touchLocked` (grey, no response). + With a pinyin preedit, the swipe moves the preedit cursor. + *Reason:* the finger stays on space, but a second finger or a drifting thumb can hit a nearby key. + Sending arrow key events to the app during a preedit would break the composition. +- **标点用空格代替 ("Replace punctuation with spaces").** + This setting acts only on voice results. + Commas, periods and question marks become one space. Final punctuation is removed. Parentheses, `3.5` and `a.b` stay. + It belongs to the 语音输入 ("Voice input") group with the filler-word option. The default is off. + *Reason:* we read the request as a switch for voice text post-processing. + +## 2026-10-05 + +- **Voice word lists removed.** + From 0.0.6-beta.1, the `hotword` module, the built-in lists, the management page and the typing candidate weighting are gone. + *Reason:* homophone replacement is not mature. We do not ship it yet. + The earlier code is on the `archive/voice-word-lists` branch (`f3bb5cc`, the 0.0.5-beta.1 commit). We can merge it back after a fix. diff --git a/docs/LICENSING.md b/docs/LICENSING.md index 8e184fdc..e801b1e1 100644 --- a/docs/LICENSING.md +++ b/docs/LICENSING.md @@ -1,17 +1,21 @@ -# 许可证与第三方组件 +# Licensing -## 主项目 +## Main project -`openIME` 以 **GPL-3.0-only** 发布，全文见仓库根目录 `LICENSE`。 +openIME uses the **GPL-3.0-only** license. The full text is in `LICENSE`. -选择它的原因：APK 内置的 Rime Ice 词典按 GPL-3.0-only 使用（见下），主项目采用同一份许可证， -分发 APK 时整体的许可证状况没有歧义——不用争论词典数据与程序是「聚合」还是「衍生」。 -其余组件（librime、OpenCC、Snappy、sherpa-onnx、Paraformer 与 CT-Transformer 模型、Fluent Emoji）均为 BSD / Apache-2.0 / MIT， -与 GPL-3.0 兼容。想改用别的许可证需要先去掉或替换 Rime Ice 词典。 +The APK contains the Rime Ice dictionaries, which use GPL-3.0-only. +The main project uses the same license. +This way, the license of the APK as a whole is clear. +We do not need to decide if the dictionary data and the program are an aggregate or a derivative work. -## 已随仓库提供的第三方组件 +All other components use BSD, Apache-2.0 or MIT licenses, which are compatible with GPL-3.0. +These components are librime, OpenCC, Snappy, sherpa-onnx, the Paraformer and CT-Transformer models and Fluent Emoji. +To use a different main license, first remove or replace the Rime Ice dictionaries. -第三方源码和数据的原始许可证随各自目录保留，主要包括： +## Third-party components in the repository + +Each directory keeps the original license of its third-party source and data: - `app/src/main/cpp/vendor/librime/` - `app/src/main/cpp/vendor/OpenCC/` @@ -19,31 +23,38 @@ - `app/src/main/assets/rime/` - `app/src/main/assets/rime-data/` -`app/src/main/assets/rime-data/openime_dicts/` 中的 `8105`、`base`、`ext` 和 -`others` 词典来自 Rime Ice 固定提交 -`75e6572bebc05b49021e842949ce947882e3e4b2`，按 GPL-3.0-only 使用。 -`app/src/main/assets/pinyin_phrases.tsv` 是由这些词典生成的高频子集，沿用相同来源与 -许可证范围。许可证全文位于 -`app/src/main/assets/licenses/rime-ice-GPL-3.0.txt`，来源明细见根目录 -`THIRD_PARTY_NOTICES.md`。 +### Dictionaries + +The `8105`, `base`, `ext` and `others` dictionaries in `app/src/main/assets/rime-data/openime_dicts/` come from Rime Ice. +We pin commit `75e6572bebc05b49021e842949ce947882e3e4b2`. +They use GPL-3.0-only. + +`app/src/main/assets/pinyin_phrases.tsv` is a subset of frequent entries that we generate from these dictionaries. +It has the same source and license. + +The license text is in `app/src/main/assets/licenses/rime-ice-GPL-3.0.txt`. + +The APK does not contain the dictionary text files. +It contains binary dictionaries that librime compiles at build time (`assets/rime-data/build/`, made by `scripts/build_rime_prebuilt.py`). +The source files are always public in this repository. +The license files ship in the APK. -APK 不再带这些词典的文本源文件，而是带构建时由 librime 编译好的二进制词库 -（`assets/rime-data/build/`，由 `scripts/build_rime_prebuilt.py` 生成）。对应的源文件 -始终在本仓库中公开，许可证文件照常随 APK 分发。 +### Voice runtime and models -语音 runtime 以 `app/libs/sherpa-onnx-1.13.6.aar` 提供，上游 -`k2-fsa/sherpa-onnx` 使用 Apache-2.0。内置中英双语 Streaming Paraformer 模型 -`csukuangfj/sherpa-onnx-streaming-paraformer-bilingual-zh-en` -模型卡标记为 Apache-2.0；正式包只使用其 INT8 encoder/decoder。 -具体来源、文件位置和发布核对项统一记录在根目录 `THIRD_PARTY_NOTICES.md`。 -标点模型固定使用 sherpa-onnx `2024-04-12-int8` 导出，上游 ModelScope -`iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch` 的模型卡标记 Apache License 2.0； -APK 附带 `ct-transformer-Apache-2.0.txt`，其文件参加语音模型清单哈希校验。 -Git LFS 只负责文件存储，不改变文件的许可证。 +- The runtime is `app/libs/sherpa-onnx-1.13.6.aar`. The upstream project `k2-fsa/sherpa-onnx` uses Apache-2.0. +- The speech model is `csukuangfj/sherpa-onnx-streaming-paraformer-bilingual-zh-en`. + Its model card states Apache-2.0. The release APK uses only the INT8 encoder and decoder. +- The punctuation model is the sherpa-onnx `2024-04-12-int8` export of ModelScope `iic/punc_ct-transformer_zh-cn-common-vocab272727-pytorch`. + Its model card states Apache License 2.0. + The APK contains `ct-transformer-Apache-2.0.txt`. + The file hashes are part of the voice model manifest. +Git LFS only stores the files. It does not change their licenses. -## APK 内许可证 +## Licenses in the APK -正式 APK 在 `assets/licenses/` 内携带主要第三方许可证文本，包括 librime、OpenCC、Snappy、Rime Ice、sherpa-onnx runtime、当前 Paraformer / CT-Transformer 模型和 Fluent Emoji。根目录 `THIRD_PARTY_NOTICES.md` 记录组件、来源、版本或固定提交与对应文件位置。 +The release APK carries the main third-party license texts in `assets/licenses/`. +These are librime, OpenCC, Snappy, Rime Ice, the sherpa-onnx runtime, the Paraformer and CT-Transformer models and Fluent Emoji. -发布工作流同时把 `THIRD_PARTY_NOTICES.md` 作为 GitHub Release 附件上传，便于在 APK 外直接查看。 +`THIRD_PARTY_NOTICES.md` lists each component, its source, its version or pinned commit and its file location. +The release notes link to this file, so you can read it outside the APK. diff --git a/docs/LOCAL_VOICE_MODEL.md b/docs/LOCAL_VOICE_MODEL.md index 3c194522..5d7e106f 100644 --- a/docs/LOCAL_VOICE_MODEL.md +++ b/docs/LOCAL_VOICE_MODEL.md @@ -1,33 +1,71 @@ -# 本地语音模型接入边界 - -这个输入法是独立 APK，包名为 `llc.slacker.openime`，语音链路不依赖 -`minis-for-android` 的类、进程、网络或数据。 - -## 已落地的语音链路 - -1. 空格短按保留原来的空格/候选上屏行为。 -2. 空格长按开始录音，松手结束当前语音段。 -3. `AudioRecord` 固定使用 `16000 Hz / mono / PCM16`，每个模型块为 20 ms、 - 320 samples、640 bytes。 -4. 采集线程和推理线程分离，PCM 只进入当前会话的有界内存环形缓冲区。 -5. partial 结果通过 `setComposingText()` 更新输入框，最终结果结束 composing； - 语音会话结束后再调用标点模型，标点失败回退原始 ASR 文字。 -6. 会话结束、取消或失败时清空 PCM 缓冲区，不写文件、不建立录音历史、不上传 - 音频或文字。 -7. `onStartInputView()` 触发后台校验和预热；输入法隐藏后保留识别器 10 秒，期间 - 重新打开直接热复用，超时后调用 `OnlineRecognizer.release()`。 -8. 模型尚在预热时长按空格会先启动 `AudioRecord`，PCM 暂存在 30 秒有界环形缓冲区， - 模型就绪后从开头消费，避免丢失首音。 - -## APK 内置模型约定 - -内置模型必须放在： - -```text -app/src/main/assets/models/voice/ -``` - -其中 `manifest.json` 至少包含： +# Voice input + +openIME recognizes speech on the device. +The voice path does not use a network, and it does not depend on any other app. + +## How it works + +1. A short press on space types a space or selects the first candidate. +2. A long press on space starts recording. The long-press time follows the Android touch-and-hold timeout. +3. `AudioRecord` uses 16,000 Hz, mono, PCM16. Each model block is 20 ms: 320 samples or 640 bytes. +4. Separate threads capture audio and run the model. + PCM data goes only into a bounded in-memory ring buffer for the current session. +5. Partial results update the field through `setComposingText()`. The final result ends the composition. +6. After the session ends, the punctuation model adds punctuation. If it fails, openIME keeps the raw text. +7. When the session ends, is canceled or fails, openIME clears the PCM buffer. + It writes no audio or text to disk. It keeps no recording history and uploads nothing. +8. `onStartInputView()` starts a background check and warm-up of the model. + After the keyboard hides, the recognizer stays loaded for 10 seconds. + If the keyboard opens again in that time, openIME reuses it. After that, it calls `OnlineRecognizer.release()`. +9. If the user holds space while the model warms up, openIME starts `AudioRecord` first. + PCM waits in a 30-second bounded buffer, and the model reads it from the start when ready. No first syllable is lost. + +## Release behavior + +When the user releases space, openIME stops the microphone at once. +Stop does not cancel recognition. +openIME waits for the capture thread to end, drains the PCM queue, and then runs the model tail, the punctuation and the commit. +Only a cancel drops the audio. + +The Streaming Paraformer in sherpa-onnx 1.13.6 needs two steps to decode the last frames. +`inputFinished()` ends feature extraction. +`stream.setOption("is_final", "1")` lets the last partial block enter the decoder. +openIME also appends 300 ms of zero samples as a tail pad for the model. +It does not read the microphone during this pad. + +If the PCM ring buffer overflows, openIME reports an error and removes the composition. +It never commits a truncated result. +After capture ends, openIME restores the media volume at once. Queued model work does not keep media muted. + +## Text processing + +- The punctuation model (CT-Transformer INT8) restores sentence breaks, commas and question marks. + It keeps punctuation that the user says aloud. +- Structured fields keep their existing protection. openIME adds no sentence punctuation there. +- Filler words such as "嗯" and "呃" can be removed. + Words such as "额度" and "金额" stay unchanged. A result that is only "嗯" stays as it is. +- The option 标点用空格代替 ("Replace punctuation with spaces") acts on voice results only. + Commas, periods and question marks become one space. Final punctuation is removed. + Parentheses and values such as `3.5` or `a.b` stay unchanged. The default is off. +- If the user deletes and corrects a voice result right away, openIME stores the pair in the private `VoiceCorrectionRepository`. + The next identical raw result uses the corrected text. All data stays on the device. + +## Password fields + +Voice input works in password fields. +openIME shows no partial result and commits the final text once. +It does not learn words or corrections from these fields. +Logs never contain PCM data, transcripts or corrections. + +## Built-in models + +The models are in `app/src/main/assets/models/voice/`: + +- `bilingual-paraformer/`: encoder, decoder and tokens of `sherpa-onnx-streaming-paraformer-bilingual-zh-en` (INT8) +- `punctuation/`: the CT-Transformer INT8 punctuation model +- `manifest.json`: lists each file with its SHA-256 hash + +`manifest.json` contains these fields: ```json { @@ -43,54 +81,42 @@ app/src/main/assets/models/voice/ } ``` -`VoiceModelRepository` 会在后台加载前校验字段和 SHA-256。内置模型首次安装或 -版本/清单变化时执行完整哈希，成功后保存只读资源校验标记；同一版本后续只做快速 -清单检查，避免在键盘创建路径重复读取约 199 MB。内置模型是 APK 资源， -不可删除；下载模型必须先校验、后台加载成功后才能切换，失败、损坏、超时或 -运行异常时回退内置模型。切换不能发生在正在录音的会话中。 - -## Runtime 接入边界 - -真正的 sherpa/ONNX arm64 runtime 通过 `StreamingEmbeddedVoiceModelRuntime` 接入： - -- `start()` 创建一个新的 OnlineStream，不重新加载模型； -- `preload()` 在专用线程创建并映射 `OnlineRecognizer`； -- `release()` 在 10 秒冷却期结束后释放 native 识别器； -- `acceptWaveform()` 只接收新增的 float PCM； -- `inputFinished()` 结束当前流并返回原始 ASR 文字； -- `punctuate()` 只在语音段结束时运行； -- 模型加载和推理不能阻塞输入法主线程。 - -生产键盘通过服务级 `VoiceModelLifecycleManager` 调用语音后端，键盘 View 只渲染 -状态，不构造模型 Provider。`VoiceRecognitionBackendFactory` 不再回退到 Android/联网语音服务；本地模型未 -就绪时明确提示未就绪，避免把在线识别伪装成离线识别。 - -## 当前交付状态 - -当前 APK 使用官方 `sherpa-onnx v1.13.6` native runtime 和 -`sherpa-onnx-streaming-paraformer-bilingual-zh-en` 的 INT8 encoder/decoder， -模型路径为 `models/voice/bilingual-paraformer/`。运行时通过 -`OnlineParaformerModelConfig`、`modelType="paraformer"` 和 -`greedy_search` 按 16 kHz PCM 流式解码；键盘出现时异步预热，10 秒冷却期内 -中文或英文语音段复用同一个已加载识别器。结束语音段时追加 300 ms 静音尾垫， -再调用 `inputFinished()`，与 sherpa-onnx 官方流式 Paraformer 示例一致。 -当前内置的是纯识别模型，`punctuate()` 保留独立标点扩展边界；没有标点模型时 -安全回退原始识别文字。 - -模型包的每个文件都在 `manifest.json` 的 SHA-256 清单内，APK 启动时只选择校验 -通过的内置包。下载模型仍然必须走 `VoiceModelRepository` 的校验和原子切换， -不能覆盖正在使用的内置模型。 - -## 个性化与性能边界 - -- Streaming Paraformer 不走 sherpa-onnx 的 transducer hotword graph，因此不再把 - `VoiceHotwordProvider` 动态热词传给 native stream；本地 `VoiceCorrectionRepository` - 的识别后纠正仍保留。 -- `VoiceHotwordProvider` 保留给将来的 transducer 模型，目前没有调用方。 -- 用户在语音上屏后立即删除并改正的文本会形成私有 `VoiceCorrectionRepository` 对； - 后续相同 ASR 原结果先应用本地纠正，改正目标也会回流动态热词。 -- 密码框可以使用语音，但只在结束时一次性上屏最终结果，不显示中间结果，也不进入热词或纠错学习；日志不记录 PCM、转写、热词、纠错内容。 -- `VoicePerformanceTrace` 只记录模型准备、麦克风启动、首 PCM、首解码、首 partial、 - 首次上屏、final、标点、丢弃样本数和总耗时。`droppedPcmSamples > 0` 会标记 degraded。 -- `VoiceAudioRouteManager` 独立管理 Android 12+ 的 BLE/SCO/有线/USB 通信设备并在 - 会话后恢复；旧系统保持系统路由，避免强制 SCO 带来的首音延迟。 +`VoiceModelRepository` checks the fields and the SHA-256 hashes in the background before it loads a model. +On first install, or when the version or manifest changes, it hashes every file. +It then saves a read-only marker. +Later starts of the same version only do a quick manifest check. +This avoids a repeated read of about 199 MB on the keyboard creation path. + +A built-in model is an APK resource. Users cannot delete it. +A downloaded model must pass the same checks and load in the background before it can replace the built-in model. +If a downloaded model fails, is corrupt, times out or crashes, openIME falls back to the built-in model. +The model never changes during a recording. + +## Runtime + +`StreamingEmbeddedVoiceModelRuntime` connects the sherpa-onnx arm64 runtime: + +- `start()` creates a new `OnlineStream`. It does not reload the model. +- `preload()` creates and maps the `OnlineRecognizer` on a dedicated thread. +- `release()` frees the native recognizer after the 10-second cooldown. +- `acceptWaveform()` accepts only new float PCM data. +- `inputFinished()` ends the stream and returns the raw text. +- `punctuate()` runs once at the end of a voice segment. +- Model loading and inference never block the main thread. + +The runtime uses `OnlineParaformerModelConfig`, `modelType="paraformer"` and `greedy_search`. +Because Paraformer does not use transducer hotwords, openIME sends no hotwords to the stream. + +The keyboard uses the service-level `VoiceModelLifecycleManager` for all voice work. +The keyboard view only shows state. It does not build model providers. +`VoiceRecognitionBackendFactory` never falls back to an Android or online speech service. +If the local model is not ready, openIME says so. +It does not present online recognition as offline recognition. + +## Diagnostics and audio routes + +- `VoicePerformanceTrace` records the timing of each step and the count of dropped samples. + If `droppedPcmSamples > 0`, it marks the session as degraded. +- `VoiceAudioRouteManager` manages BLE, SCO, wired and USB communication devices on Android 12 and later. + It restores the route after the session. + Older systems keep the system route, to avoid the first-syllable delay of a forced SCO link. diff --git a/docs/MAPPING.md b/docs/MAPPING.md deleted file mode 100644 index 54e52f86..00000000 --- a/docs/MAPPING.md +++ /dev/null @@ -1,36 +0,0 @@ -# Android 能力映射 - -本文只记录当前 Android 产品的真实实现，不再维护已删除的 Web 原型映射。 - -| 能力 | 当前实现 | 状态 | -|---|---|---| -| 中文 26 键 | `ImeKeyboardView` + `CandidatePipeline` + Rime | 生产可用 | -| 中文 9 键 | `ImeKeyboardView` + `NineKeyLocalDecoder` + Rime | 生产可用；无 0、无英文九键 | -| 英文 26 键 | `CandidatePipeline` / `CandidateEngine` | 生产可用 | -| 数字/电话/小数 | `EditorInfoAdapter` + numeric renderer | 生产可用 | -| 候选栏/展开 | `ImeKeyboardView` + `CandidateSnapshot` | 生产可用 | -| 按键组件 | `ImeKeyView` | 主/副标签、图标、按压反馈 | -| 长按 Popup | `KeyPopupController` | 定位、边缘限制、入场动画 | -| 符号 | `ImeData.symbols` + `CustomSymbolRepository` | 生产可用 | -| Emoji | `ImeData` + `EmojiRecentRepository` + Fluent assets | 当前正式 UI 仍以表情类为主，待扩展完整分类 | -| 剪贴板 | `ClipboardHistoryRepository` + `InputConnectionGateway` | 普通和密码编辑器均可用；来源应用标记为敏感的内容、要求关闭个性化学习的编辑器不进入持久历史；密码编辑器的复制/剪切/全选/粘贴按钮仍不可用（不读取输入框正文） | -| 常用语 | `QuickPhraseRepository` / `QuickPhraseEditActivity` | 生产可用 | -| 文本编辑 | `InputConnectionGateway` | 生产可用，能力随目标 EditorInfo/选区变化 | -| 语音 | `VoiceModelLifecycleManager` + sherpa-onnx | 本地语音；长按空格跟随 Android touch-and-hold timeout | -| 手写 | `HandwritingPadView` + `UnavailableHandwritingProvider` | 只有笔迹 UI，识别引擎未接入，正式入口隐藏 | -| 浮动键盘 | `LocalVoiceImeService` WindowManager + `ImeKeyboardView` | Docked/Floating Window Mode | -| 设置 | `ImeSettingsRepository` + 键盘内/Activity 设置 UI | 持久化 | -| Rime | `RimeEngine` → `RimeNative` → librime/OpenCC | 中文生产权威候选 | -| 编辑器副作用 | `InputConnectionGateway` | 唯一 commit/delete/selection/clipboard 边界 | - -## 明确不存在的产品状态 - -- 不提供英文九键。 -- Floating 不是 Panel。 -- 手写识别尚未上线。 -- 项目无 AI Writer/联网模型功能。 -- 已删除的 Web `ui-suite` 不再是源码或数据的 source of truth。 - -## 当前临时架构 - -生产运行时只有 `ImeKeyboardView`。后续 UI 清理只从这个唯一 renderer 向具体组件/controller 机械拆分，不再引入版本化键盘 View 命名。 diff --git a/docs/NINE_KEY_REFERENCE.md b/docs/NINE_KEY_REFERENCE.md index 8486bdae..dbb17a7a 100644 --- a/docs/NINE_KEY_REFERENCE.md +++ b/docs/NINE_KEY_REFERENCE.md @@ -1,59 +1,83 @@ -# 九键输入：参考实现与 openIME 的取舍 - -这份文档记录「商业 / 开源输入法的九键到底怎么做」的调研结论，以及 openIME 据此做的决定。 -改九键、左栏、删除手势之前先读这里，避免回到「一个个补丁」的状态。 - -## 调研来源 - -| 来源 | 性质 | 采用的结论 | -| -- | -- | -- | -| [rime-t9-shiyin](https://github.com/Koishi-Neko/rime-t9-shiyin)（对标百度输入法小米版） | 开源，附引擎层实测报告 | 左栏逐音节选择：数字串 → 合法首音节（且剩余数字仍可切分）；点选只锁定、不上屏；点完继续给下一位；全程「零上屏」；候选条只留词；删除键上滑清空 | -| Trime 九键 / 仓输入法九键指南 | 开源 | 字母精确匹配、数字模糊匹配；`1` 键分词；锁定 / 解锁 / 撤销 | -| [百度输入法九键说明](https://jingyan.baidu.com/article/19020a0a7ee4ab529c284246.html) | 官方使用说明 | 拼音键左侧是精确拼音；可上下滑动拼音列表更改拼音组成；按 1 手动分词 | -| [搜狗输入法帮助](https://shouji.sogou.com/wap/feedback/faqdetail?id=2004148&click_fr=3&platform=Android) | 官方帮助 | 直接上滑删除键清空、直接下滑撤回；多次清空只保留最后一次 | -| iOS 九宫格（「简体拼音十键」） | 系统输入法 | 拼音编码区 + 文字候选区分开；放弃单独的分词键 | -| 《openIME 界面重构稿》设计稿 | 本项目设计依据 | 输入中左栏是整块面板，选中项为强调色胶囊（版式沿用；每项的内容改为一个字的拼音，见下）；联想态为「‹ 词 … ∨」 | - -（豆包输入法、搜狗、微信键盘等闭源产品没有可读的实现，只能依据其公开使用说明。） - -## 行为约定（openIME 的实现） - -1. **一个字一个拼音：左栏每项是下一个字的一个音节**（`ni`、`mi`），点选即锁定，所见即所锁；锁定后列表移到下一个字 - （百度输入法、rime-t9-shiyin 的做法，也是九键「先选拼音再选字」的通行流程）。用户不用为整句选拼音， - 整词的读法在预编辑里看，要换字就从候选里选。不提供会让剩余数字无法拼读的选项；孤立的 `a/o/e`、无元音的 `ng/m` - 不当读法。早先按设计稿列整条读法（`ni'hao`），与「一个字一个拼音」冲突，已改掉。 - 只有声母的项（单按 6 时的 `m`、`n`）点了同样锁定：候选按这个字母筛选（`m` → 没、么、们）。 -2. **锁定的音节发给 Rime 时保持字母**（`xiong'486`）。luna_pinyin 方案同时接受字母和 2–9 数字， - 所以 zhong / xiong 这类同数字的读法不会被重新混在一起，候选与所选读法一致。 -3. **锁定的音节在继续打字时保持锁定**（用边界封住），退格先解锁最近锁定的音节 / 分词边界。 -4. **预编辑里的音节边界一律是撇号**（`ni'hao`）。用户锁定的前缀以视图记录的为准， - 不能从文本里的空格 / 撇号反推（解码器自己也会插分隔，反推会把「猜测」当成「用户的决定」）。 -5. **数字刚好拼得出的词排在预测之前。** 输入 `9426`（xian）时 Rime 可能先给 `自从`（zi'cong 的预测）； - 我们按读法是否恰好耗尽已输入数字分组，组内保持 Rime 顺序，不丢任何候选。 -6. **预编辑跟随首选词的读法**（`我想吃饭` → `wo'xiang'chi'fan`），而不是本地解码器独立的猜测。 - 首选词只覆盖前半段时，按它的字取到覆盖不了为止（每个字依次试全拼、截断的读音、声母：`669` → 模型 → `mo'x`）， - 剩下的数字再拼成字母。**预编辑任何时候都只有字母，不出现数字**：没有词可依时，本地按「最少的完整音节、 - 末尾允许未打完的音节开头」拼出（`46` → `go`）。 -7. **选词只覆盖一部分输入时，只上屏该词，剩余输入继续作为预编辑**（九键 / 26 键一致）。 - 本地不对部分选词做原生学习（Rime 的整句提交会把没选过的短语写进用户词库）。 -8. **空格提交首选词；回车（确定）提交已输入的拼音原文**（Rime / fcitx / Gboard 拼音的约定）。 - -## 删除键手势 - -- 上滑 ≥ 32dp 清空；松手瞬间的坐标也算最后一次移动。清空是最终操作，不提供下滑撤回（产品决定）。 -- 反馈只有一个气泡组件：未到位时深色「上滑清空」，到位后变红「松手清空」， - 放在键的侧边而不是上方（拇指会挡住上方）。 -- 「清空全部」不能只依赖编辑器的全选动作：自绘 / Compose / Web 输入框通常既没有全选也没有完整的 - ExtractedText。网关因此有第三条路径：只用光标前后文本，抓取全部内容、循环删除直到编辑器报告为空， - 中途失败则把已删内容放回。答案长度等于请求长度时可能只是窗口，一律拒绝删除。 -- 调试用 `CustomEditorTestActivity` 复现这类编辑器（修复前手势触发但文字纹丝不动，且无提示）。 - -## 验证 +# Nine-key input design + +This document records how other input methods build a nine-key keyboard and what openIME decided. +Read it before you change the nine-key keyboard, the syllable column or the delete gesture. + +## Sources + +| Source | Type | What openIME takes from it | +|---|---|---| +| [rime-t9-shiyin](https://github.com/Koishi-Neko/rime-t9-shiyin) | Open source, with an engine report | The left column lists valid first syllables of the digit string. A tap locks the syllable and does not commit. The candidate bar shows only words. Swipe up on delete clears the text. | +| Trime and the Cangshu nine-key guide | Open source | Letters match exactly and digits match fuzzily. Key `1` splits words. Lock, unlock and undo. | +| [Baidu Input nine-key help](https://jingyan.baidu.com/article/19020a0a7ee4ab529c284246.html) | Official help | The left of the keys shows exact pinyin. You can scroll the list to change the pinyin. Key `1` splits words. | +| [Sogou Input help](https://shouji.sogou.com/wap/feedback/faqdetail?id=2004148&click_fr=3&platform=Android) | Official help | Swipe up on delete clears. Repeated clears keep only the last one. | +| iOS nine-grid keyboard | System keyboard | The pinyin code area and the text candidate area are separate. There is no separate split key. | + +Other keyboards are closed source, so we used only their public help pages. + +## Behavior in openIME + +1. **One syllable for one character.** + Each item in the left column is one syllable for the next character (`ni`, `mi`). + A tap locks it. Then the list moves to the next character. + The user does not choose pinyin for a whole sentence. + The preedit shows the reading of the whole word. To change a character, pick it from the candidates. + The list never offers a syllable that leaves the remaining digits unreadable. + A lone `a`, `o` or `e`, and a bare `ng` or `m` without a vowel, are not readings. + An item with only an initial (for example `m` or `n` after you press `6`) also locks. + The candidates then filter by that letter (`m` → 没, 么, 们). +2. **Locked syllables go to Rime as letters** (`xiong'486`). + The luna_pinyin schema accepts letters and the digits 2 to 9. + Readings such as zhong and xiong that share digits stay separate, and the candidates match the chosen reading. +3. **A locked syllable stays locked while the user types on.** + A boundary seals it. + Backspace first unlocks the latest locked syllable or split boundary. +4. **Syllable boundaries in the preedit are always apostrophes** (`ni'hao`). + The locked prefix comes from the view record. + Do not infer it from spaces or apostrophes in the text, because the decoder also inserts separators. +5. **Words that match the digits exactly come before predictions.** + For `9426` (xian), Rime can put 自从 (a prediction of zi'cong) first. + openIME groups candidates by whether the reading uses up exactly the typed digits. + The Rime order stays inside each group. No candidate is lost. +6. **The preedit follows the reading of the first candidate.** + For 我想吃饭, it shows `wo'xiang'chi'fan`. + If the first candidate covers only the first part, openIME takes characters until the candidate cannot cover more. + For each character it tries the full pinyin, a cut syllable and the initial. Example: `669` → 模型 → `mo'x`. + It spells the remaining digits as letters. + **The preedit never shows digits.** + If no word helps, openIME builds the preedit locally from the fewest full syllables, with an unfinished syllable at the end (`46` → `go`). +7. **If a selected word covers only part of the input, openIME commits only that word.** + The remaining input stays as preedit. This is the same for nine-key and 26-key. + openIME does no native learning for a partial selection, + because a whole-sentence commit would write phrases that the user never selected into the user dictionary. +8. **Space commits the first candidate.** + Enter (Confirm) commits the typed pinyin text. Rime, fcitx and Gboard pinyin work this way. + +## Delete key gesture + +- Swipe up 32 dp or more to clear. The final touch position counts as the last move. + Clear is final. There is no swipe-down undo. This is a product decision. +- One bubble gives the feedback. + Before the threshold it is dark and says "swipe up to clear" (上滑清空). + After the threshold it turns red and says "release to clear" (松手清空). + It sits beside the key, because the thumb covers the area above the key. +- "Clear all" cannot depend on the select-all action of the editor. + Custom-drawn, Compose and web fields often have no select-all and no full `ExtractedText`. + The gateway therefore has a third path. + It uses only the text before and after the cursor, reads all of it and deletes in a loop until the editor reports empty. + If a step fails, it puts the deleted text back. + If the returned length equals the requested length, the result can be only a window. The gateway then refuses to delete. +- The debug `CustomEditorTestActivity` reproduces such editors. + Before the fix, the gesture fired and the text stayed with no message. + +## Verification ```bash -bash scripts/core_regression.sh emulator-5554 # 26 键 / 九键 / 回车 / 部分选词 +bash scripts/core_regression.sh emulator-5554 # 26-key, nine-key, Enter, partial selection ./gradlew :app:testDebugUnitTest --tests '*CandidatePipeline*' --tests '*InputConnectionGateway*' ``` -手机与模拟器的差异（密度、系统手势、默认输入法）会影响手势与布局；真机验证时，调试构建会把手势事件写到 -`OpenIme` 日志标签（`bs begin / clearArmed / finish`，被系统取消时有 `touch CANCEL`）。 +Density, system gestures and the default keyboard differ between phones and emulators. +Both can change gestures and layout. +On a real device, the debug build writes gesture events to the `OpenIme` log tag +(`bs begin`, `clearArmed`, `finish`, and `touch CANCEL` if the system cancels the touch). diff --git a/docs/README.md b/docs/README.md index 236a281f..bf625719 100644 --- a/docs/README.md +++ b/docs/README.md @@ -1,26 +1,33 @@ -# openIME 文档 - -根目录 [README.md](../README.md) 负责快速开始；这里放维护、架构、测试和集成文档。 - -## 当前有效文档 - -- [ARCHITECTURE.md](ARCHITECTURE.md)：当前生产运行时、状态所有权和模块边界。 -- [REPAIR_PLAN.md](REPAIR_PLAN.md)：当前稳定化、架构债和 UI/交互清理执行基线。 -- [MAPPING.md](MAPPING.md)：Android 产品能力到真实实现的映射。 -- [COORDINATE_SYSTEM.md](COORDINATE_SYSTEM.md)：归一化坐标与窗口自适应规则。 -- [APP_UI_SPEC.md](APP_UI_SPEC.md)：App 界面尺寸、间距、位置、对齐、安全区、字体适配和实际界面验收规范。 -- [REFERENCE_IME_GUIDE.md](REFERENCE_IME_GUIDE.md)：参考输入法 UI 基线与取舍。 -- [NINE_KEY_REFERENCE.md](NINE_KEY_REFERENCE.md)：九键 / 左栏 / 删除手势：商业与开源输入法的做法及 openIME 的取舍。 -- [LOCAL_VOICE_MODEL.md](LOCAL_VOICE_MODEL.md)：本地语音模型目录、校验和运行边界。 -- [TEST_ARCHITECTURE.md](TEST_ARCHITECTURE.md)：自动化层级、debug harness 和 CI 门禁。 -- [TEST_SOP.md](TEST_SOP.md)：L0～L3 正式测试流程。 -- [TEST_SOP_CHECKLIST.md](TEST_SOP_CHECKLIST.md)：多设备与人工交互验收清单。 -- [COMPATIBILITY.md](COMPATIBILITY.md)：输入环境兼容性：编辑器类型、物理键盘、显示环境、崩溃 / 卡死 / 冲突的处理与验证。 -- [LICENSING.md](LICENSING.md)：主项目与第三方组件许可证边界。 -- [RELEASE.md](RELEASE.md)：版本号规则（`VERSION`）、CHANGELOG、固定签名、arm64 正式包、标签发布、演练与回滚。 -- [REPOSITORY.md](REPOSITORY.md)：分支、合并、`main` 与标签保护、安全与依赖更新，以及如何重新应用这些设置。 -- [CHANGELOG_PRE_1.0.md](CHANGELOG_PRE_1.0.md)：1.0.0 之前的开发期记录（只读存档）。 - -旧的按 PR/分支推进的审计状态文档已删除。当前状态以 GitHub 分支/PR/CI 为准，长期执行顺序只维护在 `REPAIR_PLAN.md`，避免两份计划互相冲突。 - -本地测试证据默认写入 `.local/test-runs/`、`docs/visual/`、`docs/perf/` 等被忽略目录；只有经过筛选、脱敏且确有长期价值的证据才提交仓库。 +# Documentation + +The [README](../README.md) in the repository root covers installation and a quick start. +This folder holds the documents for maintainers and contributors. + +## Design + +- [ARCHITECTURE.md](ARCHITECTURE.md): runtime, state ownership, packages and product capabilities. +- [DECISIONS.md](DECISIONS.md): design decisions and their reasons. +- [NINE_KEY_REFERENCE.md](NINE_KEY_REFERENCE.md): nine-key behavior, reference input methods and the delete gesture. +- [LOCAL_VOICE_MODEL.md](LOCAL_VOICE_MODEL.md): on-device voice input, models and runtime. +- [COORDINATE_SYSTEM.md](COORDINATE_SYSTEM.md): normalized coordinates and window-adaptive layout. +- [APP_UI_SPEC.md](APP_UI_SPEC.md): app screen sizes, spacing, colors, haptics and acceptance rules. +- [COMPATIBILITY.md](COMPATIBILITY.md): editors, physical keyboards, displays, and crash and freeze handling. + +## Testing + +- [TEST_ARCHITECTURE.md](TEST_ARCHITECTURE.md): test layers, the debug harness and the CI gate. +- [TEST_SOP.md](TEST_SOP.md): the L0 to L3 test procedure. +- [TEST_SOP_CHECKLIST.md](TEST_SOP_CHECKLIST.md): manual acceptance checklist for several devices. +- [../scripts/README.md](../scripts/README.md): test, release and repository scripts. + +## Release and repository + +- [RELEASE.md](RELEASE.md): version rules, change log, signing, tag release, rehearsal and rollback. +- [REPOSITORY.md](REPOSITORY.md): branches, merges, protection of `main` and tags, security and dependencies. +- [LICENSING.md](LICENSING.md): licenses of the main project and third-party components. +- [release-cert.sha256](release-cert.sha256): SHA-256 of the release signing certificate. + +Other files: [CONTRIBUTING.md](../CONTRIBUTING.md), [SECURITY.md](../SECURITY.md), [CHANGELOG.md](../CHANGELOG.md) and [THIRD_PARTY_NOTICES.md](../THIRD_PARTY_NOTICES.md). + +Local test evidence goes to ignored folders such as `.local/test-runs/`, `docs/visual/` and `docs/perf/`. +Commit only evidence that is selected, free of personal data and useful for a long time. diff --git a/docs/REFERENCE_IME_GUIDE.md b/docs/REFERENCE_IME_GUIDE.md deleted file mode 100644 index 84cfefc5..00000000 --- a/docs/REFERENCE_IME_GUIDE.md +++ /dev/null @@ -1,49 +0,0 @@ -# 参考输入法 UI 基线（用户提供截图） - -来源：用户提供的参考输入法 UI 原型。 - -## 截图内容 - -| 参考文件 | 识别内容 | -| -- | -- | -| `ref-qwerty-send.png` | 英文 QWERTY，底部 `123 / space / return`，右上 `发送`，候选/输入区 | -| `ref-qwerty-send-2.png` | 英文 QWERTY，长输入文本 + `发送` | -| `ref-qwerty-send-3.png` | 英文 QWERTY，`发送` 操作 | -| `ref-pinyin-candidates.png` | 拼音输入，候选含“哪个那个女警 / 灰姑娘 / 韩国那个女警”，底部 `123 / 中英 / 确定` | -| `ref-pinyin-google-suggest.png` | 拼音输入，候选 + Google 建议/联想 | -| `ref-voice-click.png` | 语音输入面板，`点击说话`，右上关闭 | -| `ref-toolbar-functions.png` | 工具栏/功能页：语音输入、键盘选择、常用语、表情、剪切板、定制工具栏、手写找字、繁体输入 | -| `ref-settings-tools.png` | 设置页：键盘调节、更多设置、问题反馈、单手模式 | - -## 值得学习的地方 - -1. 键盘上方保留一条 **action/toolbar**，把用户高频能力放进一屏： - - 语音输入 - - 键盘选择 - - 常用语 / 快捷短语 - - 表情 - - 剪切板 - - 手写找字 - - 繁体输入 - - 定制工具栏 -2. 拼音候选栏与输入内容保持同一行，候选不会盖住正文。 -3. 底部 action 明确：`123 / space / 发送`、`123 / 中英 / 确认`，与编辑器动作一致。 -4. 语音面板有明确 `点击说话` 状态，关闭后可视化状态恢复，不给用户“麦克风还在跑”的错觉。 -5. 设置入口提供键盘调节、单手模式、问题反馈，兼顾可用性和可发现性。 - -## 本项目的落地建议 - -- 在现有候选栏上方增加一个可滚动的 **功能工具栏**，复用 `Panel` 模型。 -- 把常用语、表情、剪切板、手写、语音入口集中展示。 -- 为 `发送 / 前往 / 确认` 提供真实 `EditorInfo` action 映射。 -- 语音面板打开/关闭时维护 `voiceActive`，关闭即停。 -- 所有布局继续遵循 `COORDINATE_SYSTEM.md`：不写死绝对像素，使用 Row/Weight/归一化坐标。 - -## 设备上的 openIME 与 Minis for Android - -| 包名 | 服务 | 标签 | -| -- | -- | -- | -| `llc.slacker.openime` | `LocalVoiceImeService` | `openIME`（本原型 APK） | -| `dev.openminispet.android` | `com.openminis.app.inputmethod.LocalVoiceInputMethodService` | `Minis 本地语音输入法`（主项目中的 IME） | - -两个 ID 是独立应用，所以“输入法管理”里会看到两个。若要只保留主项目版本，可以停用/卸载 `llc.slacker.openime`；若要只保留本原型，则停用 `dev.openminispet.android` 中的 IME 服务。 diff --git a/docs/RELEASE.md b/docs/RELEASE.md index 2c172cea..cc9879de 100644 --- a/docs/RELEASE.md +++ b/docs/RELEASE.md @@ -1,151 +1,195 @@ -# 发布与版本管理 - -仓库里的所有版本信息只有一个来源：根目录 `VERSION`。发布是一次可重复的流水线： -发布 PR → 合并到 `main` → 在合并提交上打标签 → 工作流构建、校验、发布。 -仓库规则（分支保护、标签保护、合并方式）见 [REPOSITORY.md](REPOSITORY.md)。 - -## 版本号 - -- 语义化版本：稳定版 `MAJOR.MINOR.PATCH`，测试版 `MAJOR.MINOR.PATCH-beta.N`（`N` 从 1 起）。 - 不使用 `-rc`、`-alpha` 等其他后缀。项目目前处于 `0.0.x` 测试阶段；`1.0.0` 只在功能和稳定性 - 都达到可以让用户日常依赖时才发布。 -- `VERSION` 是一行文本（例如 `0.0.1-beta.1`）。`app/build.gradle.kts` 读取它： - `versionName = VERSION`，`versionCode = (MAJOR × 10000 + MINOR × 100 + PATCH) × 100 + 阶段`， - 阶段对测试版是 `N`，对稳定版是 `99`（`0.0.1-beta.1` → `101`，`0.0.1` → `199`，`1.0.0` → `1000099`）。 - 要求 `MINOR`、`PATCH` 不超过 99，`N` 不超过 98，`MAJOR` 不超过 2000，且不能是 `0.0.0`。 - 这样 `versionCode` 不会被忘记升级，同一个 `X.Y.Z` 的稳定版永远高于它的测试版。 -- `scripts/release_check.py` 用同一公式检查仓库，并用 `aapt2` 校验构建出的 APK 里的 - 包名、`versionName`、`versionCode`；PR 的 CI 和发布工作流都会运行它。 -- 测试版以 GitHub **pre-release** 发布，不标记为 latest，发布说明顶部有 Beta 提示。 - 标签是 `vX.Y.Z-beta.N`，发布时由流水线按标签自动识别。 -- 注意：Android 不允许 `versionCode` 变小的覆盖安装。一旦公开发布过某个版本，后续版本必须更大。 - -什么时候升哪一位： - -| 升级 | 条件 | +# Release process + +The file `VERSION` in the repository root is the single source of the version. +A release is a repeatable pipeline: +release PR → merge to `main` → tag the merge commit → the workflow builds, checks and publishes. + +Branch protection, tag protection and merge rules are in [REPOSITORY.md](REPOSITORY.md). + +## Version numbers + +- The project uses semantic versions. + A stable version is `MAJOR.MINOR.PATCH`. A beta version is `MAJOR.MINOR.PATCH-beta.N`, and `N` starts at 1. + No other suffix (`-rc`, `-alpha`) is allowed. + The project is in the `0.0.x` beta phase. + We release `1.0.0` only when features and stability are good enough for daily use. +- `VERSION` is one line of text, for example `0.0.1-beta.1`. + `app/build.gradle.kts` reads it: + `versionName = VERSION` and `versionCode = (MAJOR × 10000 + MINOR × 100 + PATCH) × 100 + stage`. + The stage is `N` for a beta and `99` for a stable version. + Examples: `0.0.1-beta.1` → `101`, `0.0.1` → `199`, `1.0.0` → `1000099`. + Limits: `MINOR` and `PATCH` up to 99, `N` up to 98, `MAJOR` up to 2000. `0.0.0` is not allowed. + With this rule, `versionCode` cannot be forgotten, and a stable version is always higher than its betas. +- `scripts/release_check.py` checks the repository with the same formula. + It also uses `aapt2` to check the package name, `versionName` and `versionCode` in the built APK. + The PR CI and the release workflow both run it. +- A beta release is a GitHub **pre-release**. It is not marked as latest. The release notes start with a beta notice. + The tag is `vX.Y.Z-beta.N`. The pipeline detects the beta from the tag. +- Android does not install an APK with a lower `versionCode` over a higher one. + After a version is public, each later version must be higher. + +When to increase each part: + +| Part | Condition | |---|---| -| MAJOR | 用户数据格式不兼容或需要用户手动迁移；`minSdk` 提高；包名或签名变化 | -| MINOR | 新功能、新面板或键盘；词库、语音模型、第三方 runtime 的版本变化（需重新核对许可证） | -| PATCH | 缺陷修复、性能、文案、依赖的安全更新 | -| `-beta.N` | 同一个 `X.Y.Z` 的第 N 个测试快照，修复后递增 N；正式确认后去掉后缀发布 | +| MAJOR | User data format is incompatible or needs manual migration. `minSdk` increases. The package name or signing key changes. | +| MINOR | New feature, new panel or keyboard. A new version of the dictionary, the voice model or a third-party runtime (check the license again). | +| PATCH | Bug fix, performance, text, security update of a dependency. | +| `-beta.N` | The N-th test snapshot of the same `X.Y.Z`. Increase `N` after each fix. Remove the suffix for the stable release. | -Rime 词典在打包时由 `prebuildRimeData`（`scripts/build_rime_prebuilt.py`）预编译，APK 只带编译好的 -二进制词库，并附带这份数据的内容哈希（`rime-data.revision`）。升级后只有哈希变了才从 APK 重新拷贝词库、 -清掉旧的编译产物（用户词库在独立目录，不受影响）；词库没变的升级直接沿用手机上已有的那份。手机上不再编译。 +The Rime dictionaries are precompiled at build time by `prebuildRimeData` (`scripts/build_rime_prebuilt.py`). +The APK contains only the compiled binary dictionaries and a content hash (`rime-data.revision`). +After an upgrade, the app copies the dictionaries from the APK and deletes the old compiled files only if the hash changed. +The user dictionary is in a separate directory and is not affected. +If the dictionaries did not change, the upgrade reuses the files on the phone. +The phone never compiles dictionaries. -只维护最新的一条 MINOR 版本线；安全修复以 PATCH 版本发布。 +We maintain only the newest MINOR line. Security fixes ship as PATCH versions. -## CHANGELOG +## Change log -[CHANGELOG.md](../CHANGELOG.md) 遵循 Keep a Changelog： +[CHANGELOG.md](../CHANGELOG.md) follows Keep a Changelog. -- 日常 PR 把用户可见的改动写进 `## [Unreleased]`，**不改 `VERSION`**。 -- 最新的 `## [版本] - YYYY-MM-DD` 小节必须正好等于 `VERSION`，写法上不允许空小节、 - 版本或日期倒序。`release_check.py check` 在 CI 里强制这些规则，所以版本号与 - 变更记录只能一起变化。 -- 这一小节的正文就是 GitHub Release 的发布说明，请按用户能读懂的方式写。 -- 已撤回的版本在标题后加 ` [YANKED]`，并发布更高的版本（测试版递增 `N`）。 +- A normal PR adds user-visible changes to `## [Unreleased]`. It does **not** change `VERSION`. +- The newest section `## [version] - YYYY-MM-DD` must equal `VERSION`. + Empty sections and versions or dates in the wrong order are not allowed. + `release_check.py check` enforces this in CI, so the version and the change log change together. +- The body of this section is the GitHub release note. Write it so that users can understand it. +- For a withdrawn version, add ` [YANKED]` after the heading and publish a higher version (a beta increases `N`). -## 发布产物 +## Release assets -只生成 `arm64-v8a` 的正式 APK： +The pipeline builds only an `arm64-v8a` release APK: ```text openIME-v{VERSION}-arm64-release.apk ``` -GitHub Release 只附这一个 APK。它的 SHA-256 和签名证书指纹写在发布说明正文里；第三方许可证随 APK 打包（`assets/licenses/`），清单见仓库的 `THIRD_PARTY_NOTICES.md`。流水线内部仍生成 `SHA256SUMS.txt` 用于自检，但不上传。 -Debug 构建保留 `arm64-v8a + x86_64`，只用于真机和模拟器回归，不发布。 +The GitHub release has only this APK. +The release notes show its SHA-256 and the fingerprint of the signing certificate. +The third-party licenses are in the APK (`assets/licenses/`). +The release notes link to `THIRD_PARTY_NOTICES.md`. +The pipeline also creates `SHA256SUMS.txt` for its own check but does not upload it. -## 签名密钥 +Debug builds keep `arm64-v8a` and `x86_64`. +They are for regression on devices and emulators and are never released. -密钥是应用的永久身份：Android 只在新 APK 与已安装版本由同一把密钥签名时才允许覆盖安装， -密钥丢了就只能让所有用户卸载重装。keystore 不进入仓库、Issue、PR、Actions artifact 或 Release， -也不要从 Debug keystore 发布。 +## Signing key -一次性初始化（在你信任的机器上运行，不要让别人代跑）： +The key is the permanent identity of the app. +Android installs an update over an installed app only if the same key signed both. +If you lose the key, all users must uninstall and install again. + +Never put the keystore in the repository, an issue, a PR, an Actions artifact or a release. +Never publish with the debug keystore. + +Run the one-time setup on a machine that you trust. Do not let another person run it for you: ```bash bash scripts/setup_release_signing.sh ``` -脚本会：生成 4096 位 RSA keystore（默认放在 `~/.openime-release/`）、随机口令、 -并把 `release.yml` 读取的四个 Actions secrets 写进仓库： +The script does these steps: -- `OPENIME_KEYSTORE_B64`：keystore 的 Base64。 -- `OPENIME_KEYSTORE_PASSWORD`、`OPENIME_KEY_ALIAS`、`OPENIME_KEY_PASSWORD`。 +1. It creates a 4096-bit RSA keystore (default location `~/.openime-release/`) with random passwords. +2. It writes the four Actions secrets that `release.yml` reads: + - `OPENIME_KEYSTORE_B64`: the keystore in Base64 + - `OPENIME_KEYSTORE_PASSWORD`, `OPENIME_KEY_ALIAS`, `OPENIME_KEY_PASSWORD` -口令不会打印。运行后**立刻备份** `~/.openime-release/`：离线加密副本加密码管理器， -`credentials.txt` 里是明文口令，放进密码管理器后不要留在普通云盘。 +The script does not print the passwords. +After it runs, **back up** `~/.openime-release/` at once. +Keep an encrypted offline copy and a copy in a password manager. +`credentials.txt` contains the passwords in plain text. +Move it into the password manager and do not leave it on a normal cloud drive. -首次发布后，把证书 SHA-256 写入 [`release-cert.sha256`](release-cert.sha256)（发布说明里就有）。 -之后每次发布都会比对它：签名证书对不上就直接失败，避免换了密钥却没发现。 +After the first release, write the certificate SHA-256 to [`release-cert.sha256`](release-cert.sha256). +The release notes show it. +Each later release compares against this file and fails if the certificate does not match. +This stops an unnoticed key change. -本地也可以用自己的 keystore 构建 release：设置同名环境变量（`OPENIME_KEYSTORE_PATH` 指向文件， -不用 Base64），运行 `scripts/release_build.sh`。 +You can also build a release locally with your own keystore. +Set the same environment variables (`OPENIME_KEYSTORE_PATH` is the file path, not Base64) and run `scripts/release_build.sh`. -## 发布步骤 +## Release steps -1. `main` 上最近一次 CI 全绿（包括 API 29 / 31 兼容测试）。 -2. 发布 PR：把 `[Unreleased]` 整理成 `## [版本] - YYYY-MM-DD`，同时修改 `VERSION`。 - 本地先运行： +1. Make sure that the latest CI run on `main` passes, including the API 29 and API 31 compatibility tests. +2. Open a release PR. + Move `[Unreleased]` to `## [version] - YYYY-MM-DD` and change `VERSION`. + Run this command locally first: ```bash python3 scripts/release_check.py check ``` - 这个 PR 改到了发布相关文件，CI 会自动用一次性密钥完整演练一遍发布流水线（见下）。 -3. 合并后，在 `main` 的合并提交上打带注释的标签并推送： + This PR changes release files, so CI runs the whole release pipeline as a rehearsal with a one-time key (see below). +3. After the merge, create an annotated tag on the merge commit on `main` and push it: ```bash git switch main && git pull - git tag -a vX.Y.Z -m "openIME X.Y.Z" # 测试版：vX.Y.Z-beta.N + git tag -a vX.Y.Z -m "openIME X.Y.Z" # beta: vX.Y.Z-beta.N git push origin vX.Y.Z ``` - 标签只有管理员能创建，创建后不能被移动或删除（见 REPOSITORY.md）。 -4. `.github/workflows/release.yml` 自动执行： - - 标签在 `main` 上； - - 不等 CI，先用未签名配置编译并跑 `lintRelease`（和 main 的 CI 同时进行，不读取任何密钥）； - - 等该提交的 CI 通过（Build and verify、Compatibility API 29/31）。单元测试已在 CI 里跑过，这里不重复； - - 标签格式、`VERSION`、`CHANGELOG.md` 三者一致，然后用正式密钥 `assembleRelease`（只剩打包和签名）； - - APK 签名校验（不能是 Debug 证书）、只含 `arm64-v8a`、APK 内版本与 `VERSION` 一致、 - 签名证书与 `release-cert.sha256` 一致； - - 生成 SHA-256 和发布说明（测试版带 Beta 提示）； - - 另一个只有写权限、不接触密钥的 job 先建**草稿** Release，确认 APK 已附上后才公开（测试版标为 pre-release，不是 latest）。 -5. 发布后核对：下载 APK，`sha256sum` 与发布说明里的值对比，`apksigner verify --print-certs`， - 在真机上安装、启用、试打。 - -### 演练 - -`Android Release` 工作流在两种情况下用一把只存在于该次运行的一次性密钥完整执行同一套构建和校验， -但不发布任何东西：手动触发（Actions → Android Release → Run workflow），以及 PR 改动了 -`release.yml`、`release_build.sh`、`release_check.py`、`VERSION`、`CHANGELOG.md` 或 `app/build.gradle.kts`。 -所以发布流水线在真正发布之前就已经跑过。本地同样可以演练： + Only an administrator can create tags. Nobody can move or delete a tag after creation (see REPOSITORY.md). +4. `.github/workflows/release.yml` runs these steps: + - It checks that the tag is on `main`. + - It does not wait for CI. It builds with the unsigned configuration and runs `lintRelease` at the same time as the `main` CI. It reads no secrets. + - It waits for the CI checks of the commit: Build and verify, Compatibility API 29 and Compatibility API 31. + CI already ran the unit tests, so the workflow does not repeat them. + - It checks that the tag format, `VERSION` and `CHANGELOG.md` agree. + Then it runs `assembleRelease` with the real key. Only packaging and signing remain. + - It verifies the APK: the signature is not the debug certificate, only `arm64-v8a` is present, + the version inside the APK equals `VERSION`, and the certificate equals `release-cert.sha256`. + - It creates the SHA-256 and the release notes (a beta gets a beta notice). + - A second job has write permission and no access to the key. It creates a **draft** release, checks that the APK is attached, and only then publishes it. + A beta is a pre-release and not the latest release. +5. After the release, download the APK and compare `sha256sum` with the release notes. + Run `apksigner verify --print-certs`. + Install the APK on a real phone, enable it and type. + +### Rehearsal + +The `Android Release` workflow runs the same build and checks, with a one-time key that exists only for that run. +It publishes nothing. +It runs in two cases: + +- A manual start (Actions → Android Release → Run workflow). +- A PR changes `release.yml`, `release_build.sh`, `release_check.py`, `VERSION`, `CHANGELOG.md` or `app/build.gradle.kts`. + +So the pipeline runs before the real release. +You can also rehearse locally. You need the four environment variables from above: ```bash -OPENIME_REHEARSAL=1 OPENIME_SKIP_TESTS=1 scripts/release_build.sh # 需要上面的四个环境变量 +OPENIME_REHEARSAL=1 OPENIME_SKIP_TESTS=1 scripts/release_build.sh ``` -### 失败与回滚 - -- 发布工作流在发布前失败：修复后通过 PR 合并，管理员删除远端标签再重新打在新的提交上 - （`git push origin :refs/tags/vX.Y.Z`）；如果留下了草稿 Release，先把它删掉。 -- 已公开的版本发现问题：Android 不允许降级，不要删除或改写标签。在 CHANGELOG 里给该版本标 - `[YANKED]`，把 Release 改成 pre-release 并写明原因，然后发布更高的 PATCH 版本。 -- 密钥泄露：立即停止发布，在 Settings 里删除 secrets，更换密钥会让所有现有用户必须卸载重装， - 需要先在发布说明里写清楚迁移步骤（导出用户数据 → 卸载 → 安装 → 导入）。 - -## 发布前检查 - -- `AndroidManifest.xml` 不含 `INTERNET` 权限。 -- `android:allowBackup="false"` 保持不变。 -- `THIRD_PARTY_NOTICES.md` 与 `app/src/main/assets/licenses/` 同步。 -- 语音模型、词库或第三方 runtime 版本变化时重新核对对应许可证（见 [LICENSING.md](LICENSING.md)）。 -- 主项目许可证：`LICENSE`（GPL-3.0-only），说明见 LICENSING.md。 - -## 社交预览 - -仓库内提供 `docs/images/social-preview.png`（1280×640），由 `scripts/generate_brand_assets.py` 生成。 -GitHub 的 Social preview 不是源码文件配置项，需要仓库管理员在 **Settings → General → Social preview** -上传该 PNG；这一步不能通过提交代码完成。 +### Failure and rollback + +- **The workflow fails before publishing.** + Fix the problem through a PR. + An administrator deletes the remote tag (`git push origin :refs/tags/vX.Y.Z`) and tags again on the new commit. + If a draft release remains, delete it first. +- **A published version has a problem.** + Android does not allow a downgrade, so do not delete or rewrite the tag. + Mark the version `[YANKED]` in the change log. + Change the release to a pre-release and write the reason. + Then publish a higher PATCH version. +- **The key leaks.** + Stop releasing at once and delete the secrets in Settings. + A new key forces all users to uninstall and install again. + Before that, write the migration steps in the release notes: export user data, uninstall, install, import. + +## Checks before a release + +- `AndroidManifest.xml` has no `INTERNET` permission. +- `android:allowBackup="false"` is unchanged. +- `THIRD_PARTY_NOTICES.md` and `app/src/main/assets/licenses/` agree. +- If the voice model, the dictionary or a third-party runtime changed, check its license again. See [LICENSING.md](LICENSING.md). +- The main license is `LICENSE` (GPL-3.0-only). + +## Social preview + +The repository has `docs/images/social-preview.png` (1280 × 640). `scripts/generate_brand_assets.py` creates it. +The GitHub social preview is a setting, not a source file. +A repository administrator must upload the PNG in **Settings → General → Social preview**. +A commit cannot do this. diff --git a/docs/REPAIR_PLAN.md b/docs/REPAIR_PLAN.md deleted file mode 100644 index dea32e19..00000000 --- a/docs/REPAIR_PLAN.md +++ /dev/null @@ -1,448 +0,0 @@ -# openIME Stabilization, UI and Interaction Repair Plan - -Status: active execution baseline -Base: main @ 8c0daf696bf9a5ff7f61096180fdecf591a5753b -Branch: fix/ime-stabilization-ui - -## Goal - -Stabilize the current product before further feature growth, converge UI and interaction behavior into one coherent system, then resume visual polish and missing feature work. The project should not migrate to Compose or add abstraction layers unless a current problem requires them. - -## Execution rules - -1. Do not add unrelated features while the stabilization gates are red. -2. Fix root causes and all active callers instead of patching only the failing assertion. -3. Reuse existing helpers, policies, repositories and native Android capabilities before adding new layers. -4. Do not add interface/provider/factory abstractions for single implementations unless a real current boundary needs them. -5. Keep the production interaction contract and tests aligned. Never make a test green by preserving a dead UI state. -6. Preserve current mature interaction work (26-key input, voice-in-space, backspace swipe-to-clear, candidate pipeline) while restructuring around it. -7. Every phase must leave the branch buildable. Structural extractions are mechanical first, behavioral changes second. - -## Phase 1 — Restore a trustworthy green baseline - -Current blocking failures must be resolved before broad UI refactoring: - -- password-field clipboard history policy mismatch; -- password-field paste/control availability mismatch; -- floating keyboard tests targeting the removed legacy gaming panel; -- panel focus test expecting controls that no longer exist; -- clipboard clear-confirmation action mismatch; -- clipboard open/capture lifecycle mismatch. - -Required baseline: - -- testDebugUnitTest: pass; -- lintDebug: pass; -- assembleDebug: pass; -- API 29 Android tests: pass; -- API 31 Android tests: pass. - -## Phase 2 — One capability source of truth - -Create one small capability model derived from actual runtime state instead of repeating conditionals across View, Service, Repository and tests. - -The capability model must cover at least: - -- persistent clipboard history; -- clipboard paste; -- copy/cut/select-all; -- candidates; -- voice; -- handwriting; -- floating mode; -- floating drag; -- clear-all; -- editor action/Enter behavior. - -Inputs remain concrete and minimal: EditorInfo/editor kind, password state, IME window mode, provider availability and relevant permission/model state. - -## Phase 3 — Floating keyboard becomes a first-class window mode - -The current main branch contains a real WindowManager floating IME implementation, but its old Panel.GAMING control surface and tests were not fully retired. - -Target model: - -- Docked; -- Floating. - -Rules: - -- floating mode is not a Panel; -- remove Panel.GAMING from the product state machine; -- the Tools panel keeps a “浮动键盘” action; -- tapping it immediately enters the real floating IME window; -- floating mode exposes a visible drag handle and an explicit dock/exit action; -- drag is disabled when docked; -- drag bounds must keep the whole card inside safe screen margins; -- opening another panel must not silently destroy window-mode state; -- hiding/reopening the IME, focus changes and process/session recreation must restore a valid window state; -- accessibility text/actions must match actual availability. - -## Phase 4 — Separate orientation from floating state - -Current landscape auto-floating mixes a layout strategy with a user window-mode choice. - -Target behavior: - -- landscape uses compact responsive keyboard geometry; -- landscape does not implicitly mutate the user's Docked/Floating preference; -- manual floating remains manual; -- rotation preserves a valid current window mode; -- remove obsolete landscapeAutoFloating state once behavior is covered. - -## Phase 5 — Remove unreachable/dead product states - -Remove stale state only after migration behavior is defined. - -Primary target: - -- KeyboardMode.ENGLISH_T9; -- renderEnglish9 and T9-only UI state; -- t9Filter/lastT9Digits and obsolete accessibility labels; -- stale persisted-value migration/fallback handling; -- other unreachable panels/settings/demo compatibility code identified during extraction. - -Product contract remains: Chinese 26-key, Chinese 9-key, English 26-key, numeric/specialized numeric keyboards. No English 9-key. - -## Phase 6 — Mechanically split ImeKeyboardView - -ImeKeyboardView started this phase at approximately 6,020 lines and was the dominant UI debt. Do not rewrite the state model and do not migrate to Compose during this phase. - -Extract by real UI/interaction boundaries: - -- top toolbar and composition zone; -- candidate strip and expanded candidates; -- Pinyin 26 keyboard; -- Pinyin 9 keyboard; -- English 26 behavior; -- numeric/phone/decimal keyboard; -- panel host; -- emoji panel; -- symbol panel; -- clipboard/quick phrase panel; -- text-edit panel; -- settings/fuzzy panel; -- voice presentation; -- key popup; -- backspace gesture; -- space/voice gesture; -- floating drag/window presentation. - -Use internal concrete classes/functions first. Avoid new generic frameworks. - -Target: ImeKeyboardView becomes orchestration rather than the implementation home for every screen and gesture. - -Current extraction snapshot on `chore/architecture-debt-cleanup`: - -- `ImeKeyboardView` reduced from roughly 6,020 lines to roughly 2,500 lines without Compose migration. -- 26-key, 9-key and numeric/phone rendering have concrete renderer owners. -- candidate strip + expanded candidates are owned by `CandidateBarController`. -- Tools/keyboard selector/symbols/emoji/handwriting share `ImePanelRenderer`; clipboard, text-edit and settings/fuzzy use concrete controllers. -- Voice panel/session presentation, inline voice presentation, floating chrome/drag, popup, backspace gesture/wiring and space/voice gesture/wiring have concrete owners. -- theme traversal, shared panel header and emoji bitmap-cell construction are no longer implemented in the top-level View. -- remaining work should continue to remove only real implementation ownership from `ImeKeyboardView`; editor/composition orchestration and responsive geometry are allowed to stay there. - -## Phase 7 — Single production keyboard view — completed - -The historical compatibility wrapper around the production keyboard has been removed. - -Its surviving responsibilities now have explicit owners: - -- WindowInsets / bottom safe-area measurement -> `ImeKeyboardView`; -- Enter presentation -> `ImeKeyboardView.renderState`; -- 9-key earlier-segment repair and accessibility repair -> `NineKeySegmentRepairController`; -- handwriting availability -> the normal panel/tool capability path; -- space/voice timing -> `SpaceVoiceGestureController`; key binding -> `SpaceVoiceKeyFactory`; session presentation -> `VoicePanelController`. - -`LocalVoiceImeService` now creates the single production `ImeKeyboardView` directly. - -## Phase 8 — Converge design tokens - -Centralize geometry, typography and motion currently spread across hundreds of literal dp/sp values. - -Geometry must cover: - -- outer keyboard inset; -- row/key gaps; -- key heights; -- function-key widths; -- top-zone height; -- candidate height; -- bottom-row geometry; -- panel padding/radius; -- popup bounds; -- floating width/radius/drag-handle bounds; -- safe-edge margins. - -Typography should converge to semantic roles instead of local numeric sizes: - -- key primary; -- key secondary; -- function key; -- candidate; -- panel title; -- panel body; -- panel caption/status. - -Motion should centralize press, mode-switch, panel entrance, candidate expansion, popup and voice transitions. - -## Phase 9 — Chinese 26-key is the visual reference surface - -Polish this first and reuse its system everywhere else. - -Audit and lock: - -- total height; -- left/right outer inset; -- key width and row spacing; -- second/third-row optical centering; -- function-key weights; -- space bar actual and optical center; -- glyph baseline/visual center; -- Shift/Delete/Enter icon sizing; -- pressed state; -- long-press popup; -- bottom-row symmetry. - -Do not design English 26 as a separate geometry system. - -## Phase 10 — English 26, Chinese 9-key and numeric keyboards - -English 26 reuses 26-key geometry and only changes language-specific behavior. - -Chinese 9-key must retain the intended product contract: - -- 2–9 primary digit keys; -- no English 9-key; -- no 0 key in the Chinese 9-key layout; -- left segmentation/filter interactions; -- correct candidate editing/segment repair; -- consistent visual language with 26-key. - -Numeric layout must be validated for NUMBER, DECIMAL and PHONE EditorInfo. Phone remapping (*, +, #) must remain explicit and tested. - -## Phase 11 — Top zone and candidate system - -Define explicit visual states: - -- idle toolbar; -- composing; -- candidate expanded; -- voice inline. - -Reduce accidental simultaneous controls and layout jumps. Audit candidate width, first-candidate emphasis, long phrase handling, horizontal scrolling, expanded-grid flow, back behavior and emoji shortcut placement. - -## Phase 12 — Panels become one UI system - -Tools, keyboard selector, emoji, symbols, clipboard/quick phrase, text editor, settings and fuzzy settings share one panel contract: - -- header; -- back; -- title; -- tabs/categories; -- content; -- primary action; -- empty state; -- disabled state; -- error/status state; -- focus restoration. - -Floating keyboard is explicitly excluded from the Panel model. - -## Phase 13 — Clipboard and text-edit correctness - -Unify privacy and availability behavior across policy, repository, UI and tests. - -Validate: - -- opening clipboard capture behavior; -- persistent history eligibility; -- password/editor restrictions; -- pinned items; -- 24-hour retention for unpinned items; -- clear unpinned/all confirmation; -- paste availability; -- copy/cut/select-all availability; -- dynamic selection changes; -- disabled-state accessibility. - -Controls that cannot execute must be disabled before the user activates them. - -## Phase 14 — Gesture acceptance pass - -Normal keys: - -- down/move/up/cancel; -- slide out/in; -- multi-touch ownership. - -Backspace: - -- tap delete; -- hold repeat; -- swipe-up clear; -- hysteresis and cancel; -- one batch clear callback. - -Space/voice: - -- long-press threshold follows Android's configured touch-and-hold timeout; -- releasing before that threshold commits space; -- crossing that threshold arms voice; -- release finalizes/commits; -- upward swipe cancels; -- return from cancel zone restores; -- ACTION_CANCEL is safe. - -Popup: - -- consistent size; -- optical centering; -- edge clamping; -- no screen overflow; -- setting-enabled behavior only. - -## Phase 15 — Voice UI and lifecycle - -Keep the current local speech architecture and inline keyboard-height-preserving presentation. - -Polish and verify: - -- preparing; -- listening; -- cancel preview; -- recognizing/finalizing; -- partial transcript; -- final transcript/commit; -- failure; -- model unavailable/permission unavailable; -- session cleanup after release, field switch and IME hide. - -## Phase 16 — Emoji and symbols - -Symbols are already broad and should be visually/category audited after the panel system converges. - -Emoji is currently intentionally limited to Smileys & Emotion. Expand only after the core UI is stable: - -- recent; -- smileys/emotion; -- people/body; -- skin tones where applicable; -- animals/nature; -- food/drink; -- activities; -- travel/places; -- objects; -- symbols; -- flags. - -Category filtering/navigation must materially change the grid; recent usage must persist. - -## Phase 17 — Handwriting - -Current handwriting is a UI shell backed by UnavailableHandwritingProvider. - -Keep the entry hidden until a real recognizer is wired. Once an engine exists, validate stroke capture, undo, clear, candidates/results, commit, orientation and lifecycle before exposing the feature. - -## Phase 18 — App home and settings convergence - -Align Activity UI and in-keyboard settings with the same design system: - -- color/tokens; -- radius; -- typography; -- row geometry; -- toggles; -- sliders; -- fields/cards; -- disabled/focus states. - -Setting changes should update the active IME without requiring a full app restart. - -## Phase 19 — Accessibility, large text and responsive layout - -Validate every production surface for: - -- 48dp minimum touch/focus target where appropriate; -- TalkBack names/states/actions; -- no duplicate accessibility nodes; -- large font scale without clipped keys/labels; -- portrait/landscape; -- small phones; -- tablets/foldables; -- hardware keyboard presence; -- navigation/gesture insets. - -## Phase 20 — Final acceptance matrix - -Automated: - -- unit tests; -- lint; -- debug build; -- API 29; -- API 31; -- current target/API 34–36 coverage where runner support exists. - -Product acceptance: - -- Chinese 26; -- Chinese 9; -- English 26; -- number; -- decimal; -- phone; -- candidate compose/expand; -- symbols; -- emoji; -- clipboard; -- quick phrases; -- text editing; -- voice; -- floating keyboard; -- tools/keyboard selector; -- settings; -- light/dark/system appearance; -- large text; -- portrait/landscape; -- password; -- Enter actions; -- accessibility; -- IME hide/show; -- focus switch; -- rotation; -- process/session recreation. - -Performance sanity: - -- first IME frame; -- first key latency; -- continuous typing; -- candidate response; -- panel open; -- memory; -- voice model warm-up; -- rotation/window relayout. - -## Required execution order - -1. Green CI baseline. -2. Capability truth source. -3. Floating keyboard state/window repair. -4. Orientation/floating separation. -5. Dead state cleanup. -6. ImeKeyboardView mechanical split. -7. Single production ImeKeyboardView — completed. -8. Design-token convergence. -9. Chinese 26 visual lock. -10. English 26 / Chinese 9 / numeric polish. -11. Top zone and candidates. -12. Panel system. -13. Clipboard/text editing. -14. Gesture acceptance. -15. Voice polish. -16. Emoji/symbol completion. -17. Handwriting. -18. Home/settings convergence. -19. Accessibility/responsive pass. -20. Full acceptance and performance pass. diff --git a/docs/REPOSITORY.md b/docs/REPOSITORY.md index 76abb6a7..7653e89f 100644 --- a/docs/REPOSITORY.md +++ b/docs/REPOSITORY.md @@ -1,68 +1,86 @@ -# 仓库管理 - -这里记录 GitHub 仓库的治理规则。它们由 [`scripts/apply_repo_settings.sh`](../scripts/apply_repo_settings.sh) -应用，所以改规则就是改这个脚本和本文档，再由管理员重新运行；不要只在网页上点选。 - -## 分支模型 - -- 主干开发：`main` 始终可发布。从最新的 `main` 开短生命周期分支，命名 `类型/主题`， - 类型取 `feat`、`fix`、`docs`、`chore`、`ci`、`refactor`、`test`，例如 `fix/pinyin-candidate`。 -- 需要维护旧版本线时才从旧标签建 `release/X.Y` 分支；日常热修复直接在 `main` 上发布 PATCH。 -- 合并后分支自动删除。超过 30 天没有提交也没有 PR 的分支，维护者每月清理一次。 - -## 合并 - -- 只允许 **squash 合并**；不允许 merge commit 和 rebase merge，历史保持线性。 -- squash 提交信息取 **PR 标题 + PR 描述**，不再拼接分支里的每个提交说明。所以 PR 标题要独立说清结果 - （推荐 Conventional Commits 前缀：`feat:`、`fix:`、`docs:`、`ci:`…），描述写目的、影响、验证和已知限制。 -- 提交说明、PR 和讨论可以用中文或英文，同一个 PR 内保持一致。 - -## `main` 的保护规则 - -- 必须通过 PR 合并；必须通过检查 **Build and verify**（单元测试、Lint、构建、版本与变更记录检查）。 -- 禁止 force push、禁止删除、要求线性历史、要求解决所有评审对话。 -- 不强制管理员遵守（`enforce_admins=false`）：所有者在 `main` 出问题时仍能直接修复。 - 这是应急通道，不是日常做法。 -- 不要求他人批准（单人维护）；有第二位维护者后，把 `required_approving_review_count` 调到 1， - 并在 `.github/CODEOWNERS` 里加人。 -- API 29 / 31 兼容测试在每个 PR 上运行并真实失败（测试不通过就红），但不阻止合并， - 避免模拟器偶发问题卡住发布。想强制时把 `Compatibility API 29`、`Compatibility API 31` 加进 - `apply_repo_settings.sh` 的 `contexts`。发布工作流本身要求这三项检查都通过。 -- 如果重命名了 `android.yml` 里的 job，同步修改 `contexts`，否则 PR 会一直等待一个不存在的检查。 - -## 标签 - -规则集 `release-tags` 作用于 `v*`：只有仓库管理员能创建，创建后任何人（管理员除外）都不能移动或删除。 -发布标签格式固定为 `vX.Y.Z` 或 `vX.Y.Z-beta.N`，必须与 `VERSION` 一致，详见 [RELEASE.md](RELEASE.md)。 - -## 安全 - -- Secret scanning 与 push protection 开启；Dependabot 告警与安全更新开启。 -- 私密漏洞报告开启（Security → Report a vulnerability），说明见 [SECURITY.md](../SECURITY.md)。 -- Actions 默认令牌只读（`default_workflow_permissions=read`），Actions 不能批准 PR。 - 需要写权限的 job 在工作流里单独声明；签名密钥只在 `release.yml` 的构建 job 里读取， - 该 job 没有写权限，发布 job 有写权限但接触不到密钥。 -- 签名 secrets 的建立和备份见 RELEASE.md。keystore 不进入仓库，`.gitignore` 也会拦截 `*.jks`、`*.keystore`、`*.p12`。 - -## 依赖 - -`.github/dependabot.yml` 每周一检查 GitHub Actions 和 Gradle 依赖，次要和补丁更新合并成一个 PR； -Gradle 的主版本（AGP、Kotlin 等）与 NDK、SDK 的固定版本绑定，由人工升级。 -固定提交的资源不会被自动更新：Rime Ice 词典（见 `THIRD_PARTY_NOTICES.md`）、librime 依赖 -（`scripts/fetch_rime_deps.sh`）、sherpa-onnx AAR 和语音模型（Git LFS）。 - -## 大文件与仓库卫生 - -- 大二进制（`*.onnx`、`*.aar`）走 Git LFS；不要提交 APK（`.gitignore` 已拦截）、截图、UI dump、设备日志。 -- `output/` 是设计参考脚本的本地产物，已忽略。 -- Debug APK 约 380 MB，超过 GitHub 单文件 100 MB 限制；需要测试包时从 PR 的 CI artifact - `openIME-test-apks` 下载（保留 1 天）。 - -## 重新应用设置 +# Repository management + +This document lists the governance rules of the GitHub repository. +[`scripts/apply_repo_settings.sh`](../scripts/apply_repo_settings.sh) applies them. +To change a rule, change that script and this document. +Then an administrator runs the script again. +Do not change a rule only on the web page. + +## Branch model + +- Development happens on trunk. `main` is always releasable. +- Create a short-lived branch from the latest `main`. Name it `type/topic`. + The types are `feat`, `fix`, `docs`, `chore`, `ci`, `refactor` and `test`. Example: `fix/pinyin-candidate`. +- Create a `release/X.Y` branch from an old tag only to maintain an old version line. + Normal hotfixes go to `main` and ship as a PATCH version. +- GitHub deletes a branch after its merge. + Maintainers clean up branches that have no commit and no PR for 30 days. They do this once a month. + +## Merges + +- Only **squash merge** is allowed. Merge commits and rebase merges are not allowed. The history stays linear. +- The squash commit uses the **PR title and the PR description**. It does not list each branch commit. + A PR title must state the result by itself. + Use a Conventional Commits prefix (`feat:`, `fix:`, `docs:`, `ci:`). + The description gives the purpose, the effect, the verification and the known limits. + +## Protection of `main` + +- Every change goes through a PR. The check **Build and verify** must pass. + It runs unit tests, lint, build, and version and change log checks. +- Force pushes and deletion are not allowed. History must stay linear. All review conversations must be resolved. +- Administrators are not forced to follow the rules (`enforce_admins=false`). + The owner can then fix `main` directly in an emergency. This is not a normal workflow. +- No approval is required (single maintainer). + When a second maintainer joins, set `required_approving_review_count` to 1 and add the person to `.github/CODEOWNERS`. +- The API compatibility tests (26, 29, 31, 34) run on every PR and fail when a test fails. + They do not block the merge, so that an emulator problem cannot stop a release. + To enforce them, add `Compatibility API ` to `contexts` in `apply_repo_settings.sh`. + The release workflow requires Build and verify, Compatibility API 29 and Compatibility API 31. +- If you rename a job in `android.yml`, change `contexts` too. + Otherwise a PR waits for a check that does not exist. + +## Tags + +The ruleset `release-tags` covers `v*`. +Only a repository administrator can create such a tag. +After creation, nobody except an administrator can move or delete it. +A release tag is `vX.Y.Z` or `vX.Y.Z-beta.N` and must equal `VERSION`. See [RELEASE.md](RELEASE.md). + +## Security settings + +- Secret scanning and push protection are on. Dependabot alerts and security updates are on. +- Private vulnerability reporting is on (Security → Report a vulnerability). See [SECURITY.md](../SECURITY.md). +- The default Actions token is read-only (`default_workflow_permissions=read`). Actions cannot approve PRs. + A job that needs write access declares it in the workflow. + Only the build job in `release.yml` reads the signing secrets, and it has no write access. + The publish job has write access and cannot read the secrets. +- [RELEASE.md](RELEASE.md) describes how to create and back up the signing secrets. + The keystore never enters the repository, and `.gitignore` blocks `*.jks`, `*.keystore` and `*.p12`. + +## Dependencies + +`.github/dependabot.yml` checks GitHub Actions and Gradle dependencies every Monday. +It groups minor and patch updates into one PR. +Major Gradle versions (AGP, Kotlin and others) are tied to the pinned NDK and SDK. People upgrade them by hand. + +These pinned assets never update automatically: +the Rime Ice dictionaries (see `THIRD_PARTY_NOTICES.md`), the librime dependencies (`scripts/fetch_rime_deps.sh`), and the sherpa-onnx AAR and voice models (Git LFS). + +## Large files and hygiene + +- Large binaries (`*.onnx`, `*.aar`) use Git LFS. + Do not commit APKs (`.gitignore` blocks them), screenshots outside `docs/images/`, UI dumps or device logs. +- `output/` holds local output of the design reference scripts. Git ignores it. +- The debug APK is about 380 MB. This is more than the 100 MB GitHub file limit. + Download test APKs from the CI artifact `openIME-test-apks` of a PR. It is kept for 1 day. + +## Apply the settings again ```bash -bash scripts/apply_repo_settings.sh --dry-run # 只打印请求 -bash scripts/apply_repo_settings.sh # 应用，需要仓库管理员权限和已登录的 gh +bash scripts/apply_repo_settings.sh --dry-run # print the requests only +bash scripts/apply_repo_settings.sh # apply; needs repository admin rights and a logged-in gh ``` -仓库没有 Wiki（文档都在 `docs/`），已关闭。Social preview 图片只能在网页上传，见 RELEASE.md。 +The wiki is off, because all documents are in `docs/`. +Only the web page can upload the social preview image. See RELEASE.md. diff --git a/docs/TEST_ARCHITECTURE.md b/docs/TEST_ARCHITECTURE.md index 5ce46a33..c2e074b3 100644 --- a/docs/TEST_ARCHITECTURE.md +++ b/docs/TEST_ARCHITECTURE.md @@ -1,86 +1,86 @@ -# Test Architecture +# Test architecture -openIME 的测试分四层。文档只记录真实存在的入口；具体当前结果以 GitHub Actions 和本地 SOP 证据为准，不在文档里长期写死 PASS。 +The tests have four layers. +This document lists only entry points that exist. +For current results, see GitHub Actions and the local SOP evidence. Do not write PASS in this document. -## 1. JVM Unit +## 1. JVM unit tests -覆盖纯 Kotlin/可隔离策略，例如： +These tests cover pure Kotlin code and policies that you can isolate. Examples: -- `CandidateEngineTest.kt` -- `CandidatePipelineTest.kt` -- `CandidateSnapshotTest.kt` -- `InputConnectionGatewayTest.kt` -- `EditorInfoAdapterTest.kt` -- `ImeStateTest.kt` -- `KeyboardLayoutMetricsTest.kt` -- Voice / Rime policy 与生命周期相关单测 +- `CandidateEngineTest.kt`, `CandidatePipelineTest.kt`, `CandidateSnapshotTest.kt` +- `InputConnectionGatewayTest.kt`, `EditorInfoAdapterTest.kt` +- `ImeStateTest.kt`, `KeyboardLayoutMetricsTest.kt` +- tests for voice and Rime policies and lifecycles +- `ArchitectureLayeringTest.kt`, which enforces the package dependencies -已经没有生产调用方的历史 helper 及其“自测自己”的测试应直接删除，而不是为了测试数量保留。 +Delete a legacy helper that has no production caller, together with the tests that only test it. +Do not keep it to raise the test count. -## 2. Android Instrumentation +## 2. Android instrumentation tests -`app/src/androidTest` 覆盖真实 View、InputConnection、IME 生命周期和 API 兼容： +`app/src/androidTest` covers real views, `InputConnection`, the IME lifecycle and API compatibility. Examples: -- `AuditInteractionInstrumentedTest.kt` -- `ClipboardRetentionInstrumentedTest.kt` -- `TextEditControlsInstrumentedTest.kt` -- `NineKeyChineseInstrumentedTest.kt` -- `CandidatePresentationInstrumentedTest.kt` -- `CompatibilityApiInstrumentedTest.kt` -- voice lifecycle / ownership tests +- `AuditInteractionInstrumentedTest.kt`, `ClipboardRetentionInstrumentedTest.kt` +- `TextEditControlsInstrumentedTest.kt`, `NineKeyChineseInstrumentedTest.kt` +- `CandidatePresentationInstrumentedTest.kt`, `CompatibilityApiInstrumentedTest.kt` +- voice lifecycle and ownership tests -GitHub Actions 当前兼容矩阵运行 API 29 和 API 31。更高 API 和真机属于额外验收，不应冒充 CI 覆盖。 +GitHub Actions runs these tests on API 26, 29, 31 and 34. +Other API levels and real devices are extra acceptance. They are not CI coverage. -## 3. Debug-only E2E Harness +## 3. Debug-only E2E harness -`app/src/debug` 包含： +`app/src/debug` contains `DebugKeyboardActivity`, `E2ETestReceiver`, `ImeTestLabActivity`, `LifecycleTestActivity` and `SecurityTestActivity`. +Only debug builds contain them. The release APK must not depend on them. -- `DebugKeyboardActivity` -- `E2ETestReceiver` -- `ImeTestLabActivity` -- `LifecycleTestActivity` -- `SecurityTestActivity` +## 4. Scripts and device SOP -这些入口只属于 debug 构建，release APK 不应依赖它们。 +`scripts/test_sop.ps1` is the single entry point for device tests. +The scripts cover: -## 4. Script / Device SOP - -`scripts/test_sop.ps1` 是统一设备测试入口。现有脚本覆盖： - -- core/extended typing -- 9-key -- clear/delete/voice -- field matrix +- core and extended typing +- nine-key input +- clear, delete and voice +- field-type matrix - lifecycle - panel data - security -- visual -- performance/stress +- visual checks +- performance and stress - upgrade -脚本需要真实设备或 emulator 时必须明确 serial，避免命中错误设备。 +A script that needs a real device or an emulator requires an explicit serial. +This prevents it from using the wrong device. -## CI Gate +## CI gate -`.github/workflows/android.yml` 当前执行： +`.github/workflows/android.yml` runs these steps: -1. 版本与变更记录检查：`scripts/test_release_check.py`、`scripts/release_check.py check` +1. Version and change log checks: `scripts/test_release_check.py` and `scripts/release_check.py check` 2. `:app:testDebugUnitTest` 3. `:app:lintDebug` -4. `:app:assembleDebug`、`:app:assembleDebugAndroidTest`，并校验 APK 内的包名与版本 -5. API 29 全部仪器测试 -6. API 31 全部仪器测试 - -`am instrument` 即使有测试失败也以 0 退出，所以兼容矩阵会检查输出末尾的 `OK (N tests)`， -否则让 job 失败；报告保存在 `compatibility-api-*` artifact 里。 - -`main` 要求 **Build and verify** 通过才能合并；兼容矩阵在 PR 上同样运行并会真实变红， -但不阻止合并（原因与如何改成必须通过见 [REPOSITORY.md](REPOSITORY.md)）。 -发布工作流额外要求这三项检查都通过，并在 PR 改到发布流水线时用一次性密钥演练 -`lintRelease` + `assembleRelease`（见 [RELEASE.md](RELEASE.md)）。 - -PR 或分支上的“最新 HEAD”必须对应最新 CI；旧 SHA 的绿色结果不能证明新提交通过。 - -## 坐标与视觉 - -`KeyboardGeometry.kt` 的 normalized bounds 用于测试和测量；正式布局仍由当前 IME Window 的实际宽高、Insets 和语义几何 token 决定。视觉证据由 `scripts/visual_check.ps1` 生成，不把本地截图当作长期源码资产提交。 +4. `:app:assembleDebug` and `:app:assembleDebugAndroidTest`, then a check of the package name and version in the APKs +5. All instrumentation tests on API 26, 29, 31 and 34 + +`am instrument` exits with 0 even when a test fails. +So the compatibility job checks that the end of the output contains `OK (N tests)`, and fails otherwise. +The reports are in the `compatibility-api-*` artifacts. + +`main` requires **Build and verify** before a merge. +The compatibility tests also run on PRs and turn red when a test fails, but they do not block the merge. +[REPOSITORY.md](REPOSITORY.md) explains the reason and how to make them required. +The release workflow requires Build and verify, Compatibility API 29 and Compatibility API 31. +When a PR changes the release pipeline, CI also rehearses `lintRelease` and `assembleRelease` with a one-time key. +See [RELEASE.md](RELEASE.md). + +The latest CI result must belong to the latest HEAD of the PR or branch. +A green result for an old SHA does not prove that a new commit passes. + +## Coordinates and visuals + +Tests and measurements use the normalized bounds in `KeyboardGeometry.kt`. +The real layout comes from the real size of the IME window, its insets and the geometry tokens. +See [Coordinate system](COORDINATE_SYSTEM.md). +`scripts/visual_check.ps1` creates visual evidence. +Do not commit local screenshots as long-term source assets. diff --git a/docs/TEST_SOP.md b/docs/TEST_SOP.md index 015bd1eb..baa66a17 100644 --- a/docs/TEST_SOP.md +++ b/docs/TEST_SOP.md @@ -1,18 +1,20 @@ -# openIME 全量测试 SOP +# Full test procedure (SOP) -本 SOP 是 openIME 的固定版本门禁。目标不是“发现一个 Bug 测一个 Bug”，而是每个版本 -按相同级别执行、失败即停，并自动保存可复查的日志、截图、UI 树、录屏和性能数据。 +This procedure is the fixed gate for each version. +Its goal is not to find and test one bug at a time. +For each version, run the same levels and stop at the first failure. +Save logs, screenshots, UI trees, recordings and performance data, so that others can review them. -## 1. 分级与命令 +## 1. Levels and commands -| 级别 | 触发时机 | 目标耗时 | 自动化范围 | +| Level | When | Target time | Automated scope | |---|---|---:|---| -| L0 构建门禁 | 每次编译 | 5～10 分钟 | 构建、单元测试、安装、26 键、拼音九键、数字、基本删除 | -| L1 核心回归 | 每次功能修改 | 30～45 分钟 | L0、完整拼音引擎、输入框矩阵、候选、面板、生命周期、当前窗口视觉 | -| L2 完整回归 | 推送手机或 GitHub 前 | 2～4 小时 | L1、lint、升级、性能及人工设备/应用/尺寸矩阵 | -| L3 发布验收 | 正式发布 APK | 半天以上 | L2、隐私、权限、压力、多设备和 30 分钟稳定性 | +| L0 Build gate | Each build | 5 to 10 minutes | Build, unit tests, install, 26-key, nine-key pinyin, digits, basic delete | +| L1 Core regression | Each feature change | 30 to 45 minutes | L0, full pinyin engine, field matrix, candidates, panels, lifecycle, visuals of the current window | +| L2 Full regression | Before you push to a phone or GitHub | 2 to 4 hours | L1, lint, upgrade, performance, and the manual device, app and size matrix | +| L3 Release acceptance | Before a release APK | Half a day or more | L2, privacy, permissions, stress, several devices and 30 minutes of stability | -统一入口： +Single entry point: ```powershell .\scripts\test_sop.ps1 -Level L0 -Serial @@ -21,83 +23,111 @@ .\scripts\test_sop.ps1 -Level L3 -Serial -FreshInstall ``` -只查看将执行的步骤，不连接设备： +To see the steps without a device: ```powershell .\scripts\test_sop.ps1 -Level L3 -ListOnly ``` -`-FreshInstall` 会卸载现有 openIME 并清除它的设置、词频、剪贴板和常用语数据，只能在 -允许清空数据的测试设备上使用。升级保留测试不等同于全新安装测试，两项都要留下证据。 +`-FreshInstall` uninstalls openIME and deletes its settings, word frequencies, clipboard and quick phrases. +Use it only on a test device that you may reset. +An upgrade-keeps-data test does not replace a fresh-install test. Keep evidence for both. -## 2. 失败即停 +## 2. Stop at the first failure -出现以下任一现象，当前级别立即失败，不继续执行后续步骤： +The level fails at once, and the next steps do not run, if any of these occurs: -- 无法输入、九键直接输出数字、拼音或候选异常消失。 -- 删除错误对象、上滑清空误触、文字丢失/重复/乱序。 -- 闪退、ANR、黑屏、输入法进程死亡。 -- UI 溢出窗口、最后一列被裁切、关键按钮无法点击。 -- 自动断言失败或证据采集发现无效 PNG/缺失测试宿主。 +- You cannot type, the nine-key keyboard outputs digits directly, or pinyin or candidates disappear. +- Delete removes the wrong object, a swipe-up clear fires by mistake, or text is lost, repeated or reordered. +- A crash, an ANR, a black screen or a death of the input method process. +- The UI overflows the window, the last column is cut off or a key button cannot be tapped. +- An automatic assertion fails, or evidence collection finds an invalid PNG or a missing test host. -修复后从失败级别的第一步重新执行，不能只补跑失败用例后宣称整级通过。 +After a fix, run the failed level again from its first step. +Do not run only the failed case and then claim that the whole level passed. -## 3. 证据目录 +## 3. Evidence folder -统一入口将证据写入被 Git 忽略的目录： +The single entry point writes evidence to a folder that Git ignores: ```text -.local/test-runs/<时间>-<级别>-<设备序列号哈希>/ +.local/test-runs/