From f3c327c47174ab1f729edd3786684ad188a63076 Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Mon, 31 Aug 2026 19:13:03 +0200 Subject: [PATCH] fix(dispatch): ternary kernel packs join the self-healing SPI (#1240) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 0.52.0 self-healing dispatch discovered the Q-series row-major packs but not the ternary ones: a consumer loading BITNET_B1_58 / BITNET_PLANES weights silently fell to the int8-requantize or decoding-reference path (~120x slower per the #1141 bench) unless it called the two installs explicitly — exactly the failure mode ensureInstalled() exists to eliminate, and why SKaiNET-transformers' CLI still carries explicit NativeTernaryF32GemvKernel.install() lines. - FfmTernaryKernelPackFactory (jvm jar) and JniTernaryKernelPackFactory (Android AAR), listed in the ViewKernelPack service files: install the exact FP32xB1.58 LUT gemv and the fused BITNET_PLANES lm_head; both degrade to a no-op lookup when the bundled native library is missing. - Kotlin/Native note: no ServiceLoader there — NativeKnTernaryF32Gemv / NativeKnTernaryLmhead stay explicit, now said in ViewKernelPack.other.kt and the ternary tutorial's install table (automatic vs explicit). - KernelDispatchSelfHealTest: cold dispatch must resolve matmul(FP32 x BITNET_B1_58) to the LUT kernel with zero explicit installs (skips cleanly where the native library is absent). Closes #1240. Co-Authored-By: Claude Fable 5 Claude-Session: https://claude.ai/code/session_01YKeDSK4JF295y53Uvez954 --- CHANGELOG.md | 14 +++++++++ .../tutorials/ternary-getting-started.adoc | 14 +++++---- .../api/kernel/ViewKernelPack.other.kt | 5 +++- .../kernel/jni/JniTernaryKernelPackFactory.kt | 24 +++++++++++++++ ...sk.ainet.backend.api.kernel.ViewKernelPack | 1 + .../kernel/FfmTernaryKernelPackFactory.kt | 26 ++++++++++++++++ ...sk.ainet.backend.api.kernel.ViewKernelPack | 1 + .../exec/kernel/KernelDispatchSelfHealTest.kt | 30 +++++++++++++++++++ 8 files changed, 109 insertions(+), 6 deletions(-) create mode 100644 skainet-backends/skainet-backend-jni-cpu/src/main/kotlin/sk/ainet/exec/kernel/jni/JniTernaryKernelPackFactory.kt create mode 100644 skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/FfmTernaryKernelPackFactory.kt diff --git a/CHANGELOG.md b/CHANGELOG.md index 7980d427..615f5686 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,20 @@ ## [Unreleased] +### Fixed + +- **Ternary kernel packs join the self-healing dispatch SPI** + ([#1240](https://github.com/SKaiNET-developers/SKaiNET/issues/1240)): the `ServiceLoader` + service files now list `FfmTernaryKernelPackFactory` (JVM jar) and + `JniTernaryKernelPackFactory` (Android AAR), so `KernelDispatch.ensureInstalled()` wires the + exact FP32×`BITNET_B1_58` LUT gemv and the fused `BITNET_PLANES` lm_head with no bootstrap + call — previously a consumer loading ternary weights silently got the int8-requantize or + decoding-reference path (~120× slower per the #1141 bench) unless it called + `NativeTernaryF32GemvKernel.install()` / `NativeTernaryLmheadKernel.install()` explicitly, + exactly the failure mode the 0.52.0 self-healing dispatch was released to eliminate for the + Q-series formats. Kotlin/Native still installs explicitly (no `ServiceLoader` there); the + ternary tutorial's install table now says which targets are automatic. + ## [0.51.0] - 2026-08-29 Headline: **ternary/BitNet weights join the memory-mapped weight story 0.50.0 started for every diff --git a/docs/modules/ROOT/pages/tutorials/ternary-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/ternary-getting-started.adoc index 5a869893..6d67bc7d 100644 --- a/docs/modules/ROOT/pages/tutorials/ternary-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/ternary-getting-started.adoc @@ -178,15 +178,19 @@ kotlin { } ---- -The same kernel, same C file, reaches every other deployment shape through its -own one-line install: +The same kernel, same C file, reaches every other deployment shape. On the JVM +and Android the install is automatic: the ternary packs are `ServiceLoader` +entries (`FfmTernaryKernelPackFactory` / `JniTernaryKernelPackFactory`), so +`KernelDispatch.ensureInstalled()` — which runs on the first dispatch — wires +them with no bootstrap call. Kotlin/Native has no `ServiceLoader`, so the +explicit call remains: [cols="1,2,2",options="header"] |=== | Target | Bridge | Install -| Raspberry Pi / linuxArm64, iOS, macOS | Kotlin/Native cinterop | `NativeKnTernaryF32Gemv.install()` -| Android | JNI (baseline `libskainet_jni.so` — every arm64 device, A72 included) | `JniTernaryF32Gemv.install()` -| Desktop/server JVM | FFM (`java.lang.foreign`) | `NativeTernaryF32GemvKernel.install()` +| Raspberry Pi / linuxArm64, iOS, macOS | Kotlin/Native cinterop | `NativeKnTernaryF32Gemv.install()` (explicit) +| Android | JNI (baseline `libskainet_jni.so` — every arm64 device, A72 included) | automatic (ServiceLoader; `JniTernaryF32Gemv.install()` still works) +| Desktop/server JVM | FFM (`java.lang.foreign`) | automatic (ServiceLoader; `NativeTernaryF32GemvKernel.install()` still works) |=== Removing a native artifact is never an error: the pack warns and dispatch diff --git a/skainet-backends/skainet-backend-api/src/nativeMain/kotlin/sk/ainet/backend/api/kernel/ViewKernelPack.other.kt b/skainet-backends/skainet-backend-api/src/nativeMain/kotlin/sk/ainet/backend/api/kernel/ViewKernelPack.other.kt index 071dde84..8610da61 100644 --- a/skainet-backends/skainet-backend-api/src/nativeMain/kotlin/sk/ainet/backend/api/kernel/ViewKernelPack.other.kt +++ b/skainet-backends/skainet-backend-api/src/nativeMain/kotlin/sk/ainet/backend/api/kernel/ViewKernelPack.other.kt @@ -5,7 +5,10 @@ import sk.ainet.lang.memory.ExperimentalMemoryApi /** * No `ServiceLoader` on this platform, so there is nothing to discover: packs are installed * explicitly by the consumer (the same split [KernelProvider] documents — see - * `NativeKnKernelProvider`, which is registered by hand on Kotlin/Native). + * `NativeKnKernelProvider`, which is registered by hand on Kotlin/Native). That includes the + * ternary packs (#1240): a Kotlin/Native consumer of `BITNET_B1_58` / `BITNET_PLANES` weights + * calls `NativeKnTernaryF32Gemv.install()` / `NativeKnTernaryLmhead.install()` itself, or + * dispatch serves the decoding reference kernels. */ @ExperimentalMemoryApi internal actual fun installPlatformKernelPacks(): List = emptyList() diff --git a/skainet-backends/skainet-backend-jni-cpu/src/main/kotlin/sk/ainet/exec/kernel/jni/JniTernaryKernelPackFactory.kt b/skainet-backends/skainet-backend-jni-cpu/src/main/kotlin/sk/ainet/exec/kernel/jni/JniTernaryKernelPackFactory.kt new file mode 100644 index 00000000..c4a7a17f --- /dev/null +++ b/skainet-backends/skainet-backend-jni-cpu/src/main/kotlin/sk/ainet/exec/kernel/jni/JniTernaryKernelPackFactory.kt @@ -0,0 +1,24 @@ +package sk.ainet.exec.kernel.jni + +import sk.ainet.backend.api.kernel.ViewKernelPack +import sk.ainet.lang.memory.ExperimentalMemoryApi + +/** + * `ServiceLoader` entry for the ternary kernels on Android (#1240) — the JNI counterpart of + * `FfmTernaryKernelPackFactory`: lets `KernelDispatch.ensureInstalled()` reach the exact + * FP32×`BITNET_B1_58` gemv ([JniTernaryF32Gemv]) and the fused `BITNET_PLANES` lm_head + * ([JniTernaryLmhead]) without an explicit bootstrap call. + * + * Listed in `META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack`. Both installs + * register nothing when the bundled `.so` is unavailable; the same packaging caveat as + * [JniMappedKernelPackFactory] applies — a build that strips `META-INF/services` can still + * call the installs directly. + */ +@OptIn(ExperimentalMemoryApi::class) +public class JniTernaryKernelPackFactory : ViewKernelPack { + override val name: String get() = "jni-ternary" + override fun install() { + JniTernaryF32Gemv.install(warn = {}) + JniTernaryLmhead.install(warn = {}) + } +} diff --git a/skainet-backends/skainet-backend-jni-cpu/src/main/resources/META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack b/skainet-backends/skainet-backend-jni-cpu/src/main/resources/META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack index f396ddd2..07445695 100644 --- a/skainet-backends/skainet-backend-jni-cpu/src/main/resources/META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack +++ b/skainet-backends/skainet-backend-jni-cpu/src/main/resources/META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack @@ -1 +1,2 @@ sk.ainet.exec.kernel.jni.JniMappedKernelPackFactory +sk.ainet.exec.kernel.jni.JniTernaryKernelPackFactory diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/FfmTernaryKernelPackFactory.kt b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/FfmTernaryKernelPackFactory.kt new file mode 100644 index 00000000..f6f72e34 --- /dev/null +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/kotlin/sk/ainet/exec/kernel/FfmTernaryKernelPackFactory.kt @@ -0,0 +1,26 @@ +package sk.ainet.exec.kernel + +import sk.ainet.backend.api.kernel.ViewKernelPack +import sk.ainet.lang.memory.ExperimentalMemoryApi + +/** + * `ServiceLoader` entry for the ternary kernels (#1240) — the missing sibling of + * [FfmRowMajorKernelPackFactory]: `KernelDispatch.ensureInstalled()` discovered the Q-series + * row-major pack but not the ternary ones, so a consumer loading `BITNET_B1_58` / + * `BITNET_PLANES` weights silently fell to the int8-requantize or decoding-reference path + * (~120× slower per the #1141 bench) unless it called the two installs below explicitly — + * exactly the failure mode the self-healing dispatch exists to eliminate. + * + * Installs the exact FP32×`BITNET_B1_58` LUT gemv ([NativeTernaryF32GemvKernel]) and the fused + * `BITNET_PLANES` lm_head ([NativeTernaryLmheadKernel]). Both delegate to their packs' + * `install(native?)`, which registers nothing when the bundled native library is missing — + * discovery on such a machine costs a lookup and leaves dispatch to the reference kernels. + */ +@OptIn(ExperimentalMemoryApi::class) +public class FfmTernaryKernelPackFactory : ViewKernelPack { + override val name: String get() = "ffm-ternary" + override fun install() { + NativeTernaryF32GemvKernel.install() + NativeTernaryLmheadKernel.install() + } +} diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmMain/resources/META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/resources/META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack index 29450fd1..6ec79675 100644 --- a/skainet-backends/skainet-backend-native-cpu/src/jvmMain/resources/META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmMain/resources/META-INF/services/sk.ainet.backend.api.kernel.ViewKernelPack @@ -1 +1,2 @@ sk.ainet.exec.kernel.FfmRowMajorKernelPackFactory +sk.ainet.exec.kernel.FfmTernaryKernelPackFactory diff --git a/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/KernelDispatchSelfHealTest.kt b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/KernelDispatchSelfHealTest.kt index 0753351c..9393ebf5 100644 --- a/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/KernelDispatchSelfHealTest.kt +++ b/skainet-backends/skainet-backend-native-cpu/src/jvmTest/kotlin/sk/ainet/exec/kernel/KernelDispatchSelfHealTest.kt @@ -47,6 +47,36 @@ class KernelDispatchSelfHealTest { println("SELFHEAL n=${names.size} providers=${KernelRegistry.availableNames()} kernels=${names.sorted()}") } + @Test + fun cold_dispatch_discovers_the_ternary_packs() { + KernelDispatch.clearForTesting() + KernelRegistry.clearForTesting() + + // #1240's acceptance: the ternary packs must arrive through discovery alone — no + // NativeTernaryF32GemvKernel.install() / NativeTernaryLmheadKernel.install() anywhere. + KernelDispatch.ensureInstalled() + + // The packs register only when the bundled native library resolves; on a machine + // without it, discovery must cost a lookup and register nothing (no crash, no entry). + if (!NativeTernaryF32GemvKernel.isAvailable()) { + println("SELFHEAL-TERNARY skipped: bundled native library unavailable") + return + } + val names = KernelDispatch.kernels().map { it.name } + assertTrue( + names.any { it.startsWith("ternary_f32_gemv/") }, + "exact FP32×BITNET_B1_58 gemv expected after self-heal; got $names", + ) + assertTrue( + names.any { it.startsWith("ternary_planes_matmul/") }, + "fused BITNET_PLANES lm_head expected after self-heal; got $names", + ) + assertTrue( + KernelDispatch.find(sk.ainet.backend.api.kernel.TernaryF32GemvKernel.keyFor()) != null, + "matmul(FP32 × BITNET_B1_58) must resolve to the LUT kernel with zero explicit installs", + ) + } + @Test fun explicit_registration_suppresses_auto_install() { KernelDispatch.clearForTesting()