feat(kernels): NEON bitnet_gemv with a capability-gated pack and a reference fallback (SKEEP-003 P6, S2.9)

[michalharakal]

Closes #1041 · Phase P6 · Milestone M2 · PRD M2-F4, M2-A2 · Proposal §5.2, §5.3

The kernel

bitnet_gemv.c: int8 activations (one absmax scale per token) against canonical TQ2_0 ternary weights. A ternary weight is an add, a subtract or nothing, so the vector work is the unpacking — a whole vector of 2-bit codes becomes int8 signs with a shift and a mask, after which sdot accumulates sixteen products per instruction where the core has ARMv8.2 dot-product.

Two things in it worth a look:

• The codes stay in {0,1,2} through the inner loop. Converting them to {-1,0,+1} costs a subtract per sixteen weights; instead the bias comes off once per block, because Σ(c−1)·a = Σc·a − Σa and Σa is the same for every output row.
• The strips take pre-shifted vectors rather than a shift amount. NDK clang rejects vshrq_n_u8(x, 0) (valid range [1,8]) where gcc accepts it — worth knowing before the next intrinsics kernel.

TQ1_0's base-3 packing and BitNet's own per-tensor packing stay on the scalar path, with the reason in the source: they are storage formats, and a checkpoint headed for this kernel ships as TQ2_0.

The pack contract

TernaryKernelPacks.install(native, capabilities, warn) registers the portable reference first, lets a pack override it, and treats the artifact's absence as a configuration, not a failure: one warning, a slower kernel, no crash. NativeBitNetGemvKernel unwraps the views once and falls back to the reference for anything the native kernel does not take — a prefill row batch, non-heap storage — so the fast path stays an optimization.

The kernel is registered under both a capability-carrying key and the operand-only key the dispatcher actually builds, because operands say nothing about the CPU and the pack only installs itself on a device that has the capability.

Unlike the Q4_0…Q6_K SPI kernels, this one is bridged into the view registry: it was written against TernaryCodec's canonical row-major order, which is what a GGUF holds, so the block-order ambiguity of #973 does not arise — and the parity test pins it rather than assuming it.

Measured on an ARMv8.2 Cortex-A55 reference board (asimddp)

gcc -O3 -ffast-math -march=armv8.2-a+dotprod:

shape	kernel	worst relative error
k=1024, n=256	0.091 ms/call	0
k=2560, n=2560	2.329 ms/call	0

Parity is exact — the arithmetic is integer until the block scale, so there is nothing to round.

Against the fallback, on the same core, k=1024 n=256: the portable Kotlin kernel takes 21.3 ms/call, the NEON kernel 0.091 ms/call. M2-A2's "≥ 3×" is met with room to spare — but that number came from a debug Kotlin/Native build of the fallback, so treat it as an upper bound rather than the figure.

The honest measure of what the intrinsics themselves buy: 1.08–1.24× over compiler-vectorized C at -O3 -ffast-math on this A55. Hand-written NEON is not a large win over what clang/gcc already do here; the large win is being native at all. Worth knowing before investing in an i8mm variant.

The seven ternary kernel tests and the timing test were also run on that board from the Kotlin/Native binary — all pass. The pack contract (registration, fallback, capability, shape refusal) is tested in common code on every target.

Gate

scripts/pr-gate.sh — all legs passed. The AAR builds for arm64-v8a and x86_64 under NDK clang, and both library variants export the new symbol.

Keeps develop green by

The kernel is an optional artifact by construction: with no pack installed, TernaryKernelPacks.install() registers exactly what #1040 registered and warns. Nothing outside the ternary formats changes path, and no existing kernel, key or provider is touched.

🤖 Generated with Claude Code