simd: wasm32 SIMD128 (v128) arm for matmul_i8_to_i32 (byte-parity vs scalar)#254
Conversation
…scalar) The recognizer's int8 hot path (ndarray::simd::matmul_i8_to_i32) was scalar on wasm32 — this adds a v128 kernel so a browser-side OCR build is SIMD-accelerated. - src/simd_wasm.rs: dot_i8_to_i32_wasm + matmul_i8_to_i32_wasm — i16x8_extmul_low/ high_i8x16 (signed widen-mul) -> i32x4_extadd_pairwise_i16x8 -> i32x4_add, scalar tail. No sign-shift bias needed (signed*signed native to the widening path; i8::MIN^2 = 16384 < i16::MAX, no overflow). Intrinsics verified against rust-src stdarch; unsafe only wraps v128_load (SAFETY comments). Mandatory parity tests (wasm_matmul_i8_matches_scalar + dot..various_k, i8::MIN/MAX + non-mult-16 K). - src/simd_runtime/matmul.rs: matmul_i8_to_i32 split x86_64 (delegates to hpc::amx_matmul, unchanged) vs not(x86_64) portable (wasm128 dispatch else scalar reference), plus a portable MatmulError (hpc::amx_matmul is x86-only). - .cargo/config-wasm.toml: +simd128 for wasm32 targets. Verified: x86 build (runtime-dispatch) compiles + matmul_i8 test passes; the v128 kernel is 754/754 byte-identical to scalar, executed under node's WASM SIMD128 runtime (standalone harness; the full-crate wasm test is blocked by criterion's rayon dep, unrelated to this kernel). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1
|
Warning Review limit reached
Next review available in: 32 minutes Enable usage-based reviews in Billing to review now. Otherwise, wait until the next included review is available. How can I continue?After more reviews become available, a review can be triggered using the To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews. How do review limits work?CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability. For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window. Please refer docs for additional details. Review details⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (2)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_ed935048-c9d2-4222-a4f4-25e850948356) |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 64c91d4be1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| /// `i32` accumulation as the SIMD path. | ||
| #[inline(always)] | ||
| pub fn dot_i8_to_i32_wasm(a: &[i8], b: &[i8], k: usize) -> i32 { | ||
| debug_assert!(a.len() >= k && b.len() >= k); |
There was a problem hiding this comment.
On wasm32+simd128 release builds, this public safe function accepts arbitrary slices but only a debug_assert! protects the unsafe v128_load calls below. A direct caller such as dot_i8_to_i32_wasm(&[], &[], 16) skips the check in release and reads past the slice allocation, which is UB; use a real assert! (as the other slice-loading APIs do) or make the length precondition part of an unsafe API before loading.
Useful? React with 👍 / 👎.
2 participants
Summary
The recognizer's int8 hot path (
ndarray::simd::matmul_i8_to_i32, consumed bytesseract-recognizerunderruntime-dispatch) ran scalar on wasm32. This adds a v128 (WASM SIMD128) arm so a browser-side OCR build is SIMD-accelerated — the one change that makes a client-side, offline, private OCR demo fast.What changed
src/simd_wasm.rs—dot_i8_to_i32_wasm+matmul_i8_to_i32_wasm:i16x8_extmul_low/high_i8x16(signed widen-multiply) →i32x4_extadd_pairwise_i16x8→i32x4_add, with a scalar tail fork % 16. No sign-shift bias needed (signed×signed is native to this widening path;i8::MIN² = 16384 < i16::MAX, so no overflow — unlike the AMX/VPDPBUSDtiers).unsafewraps onlyv128_load(with// SAFETY:). Every intrinsic name was verified against the installedrust-srcstdarch source, not guessed.src/simd_runtime/matmul.rs—matmul_i8_to_i32split#[cfg]-gated: x86_64 delegates tohpc::amx_matmul(bit-for-bit unchanged);not(x86_64)is the portable path (wasm128 dispatch, else a scalar reference identical to the AMX tier-4 fallback). A portableMatmulErrorwas added becausehpc::amx_matmulis x86-only..cargo/config-wasm.toml—+simd128for wasm targets (test convenience).Verification
--features runtime-dispatch) compiles;matmul_i8test passes — the restructure doesn't touch the x86 path's behavior.cargo test --target wasm32-wasip1is blocked bycriterion'srayondep, unrelated to this kernel). Covers i8::MIN/MAX and non-multiple-of-16 K.wasm_matmul_i8_matches_scalar,dot_i8_to_i32_wasm_matches_scalar_various_k) per the W1a SIMD contract.Known follow-up (pre-existing, out of scope)
matmul_bf16_to_f32/matmul_f32in the same trampoline referencehpc::amx_matmulunconditionally, so a full-crate wasm build fails on them independent of this change — a small same-pattern fast-follow (arch-gate + portable fallback) would make the whole crate wasm-buildable.🤖 Generated with Claude Code
https://claude.ai/code/session_016b33swuXE23hKtqxsHu9p1
Generated by Claude Code