Native fused LayerNorm / GELU / softmax / conv1d kernels for the JNI and FFM tiers

[michalharakal]

Context

Companion to the dense FP32 JNI matmul issue. #949 measured that ~83% of Android eager decode time was non-matmul overhead and fixed the boxed per-element paths by hoisting the primitive FloatArray loops into shared code. That closed the boxing problem; the ops are still scalar Kotlin loops on ART, where there is no Vector API.

For a Whisper-class encoder that matters: per layer there are two LayerNorms over [1500,1280], a GELU over [1500,5120], 20 softmaxes over [1500,1500], and the stem is two conv1d layers (k=3, stride 1 and 2) over [128,3000]. skainet-backend-native-cpu/native/src/ contains matmul kernels only.

Scope

• Native C kernels (NEON bodies gated on __ARM_NEON, scalar fallback) for: layerNorm (fused mean/var/normalize/affine, FP32 accumulation), gelu (tanh approximation and erf variants, matching Fp32 reference to 1 ULP where the DSL promises exactness), row softmax (max-subtract, exp, normalize), and conv1d (direct or im2col + the FP32 matmul kernel).
• JNI entries + JniKernelProvider registrations; FFM bindings on the JVM tier for parity with the existing pattern.
• Parity tests against the scalar Kotlin implementations on device.
• simpleperf before/after on a whisper-tiny encoder step, the way Android eager decode is ~83% non-matmul overhead: base DefaultCpuOps runs boxed per-element ops — port the JVM primitive fast paths (#920 follow-up) #949 did for decode, so the remaining overhead is named.

Acceptance

• Results match the scalar path within 1 ULP (GELU/softmax) or bit-identically (LayerNorm with FP32 accumulation, conv1d).
• Combined with the dense FP32 matmul issue: whisper-tiny encoder ≥ 5× faster than 0.54.0 on a Pixel-class phone.

Related

• Android eager decode is ~83% non-matmul overhead: base DefaultCpuOps runs boxed per-element ops — port the JVM primitive fast paths (#920 follow-up) #949 — boxed element access on Android (closed; this is the SIMD follow-up)
• Vectorized eager CPU kernels for Kotlin/Native targets (native RTF 2.7–5.8x JVM on identical workloads) #910 — vectorized eager kernels for Kotlin/Native targets (same kernels, different binding)

[michalharakal]

Tracked in #1282.