[bitnet] T1: BitNet architecture — ModelFamily, network def, packed-weight loader

[michalharakal]

Part of the BitNet tracking issue (T1). Baseline BitNet support through the standard (unfused) path — correctness first, fusion in T2.

Tasks

• ModelRegistry: add ModelFamily.BITNET + detect() branch for GGUF general.architecture values "bitnet" / "bitnet-25" / "bitnet-b1.58"; wire into the skainet-cli/Main.kt dispatch chain
• New llm-inference/bitnet/: network def bitnetNetwork(...) via decoderTransformerNetwork (BitNet-2B4T is llama-like with sub-layernorm — verify layer structure against the GGUF metadata), BitNetGgufTensorNames, weight loader accepting packed BITNET_B1_58 tensors from the SKaiNET I2_S loader (SKaiNET#1140)
• Name resolver entries (output.weight / lm_head.weight) in LLMWeightNameResolvers
• Baseline correctness: decode through the standard VoidDense lm_head path — the packed ternary weights dispatch to SKaiNET's exact ternary f32 kernel (or its reference fallback) via linearProject → ops.matmul; no fusion yet

Tests

• Weight-loading test against a small synthetic BitNet GGUF (packed tensors arrive packed, not FP32-widened)
• Logits parity vs FP32-widened load of the same model
• CLI smoke: architecture auto-detected, coherent generation

[michalharakal]

Groundwork landed on feat/336-bitnet-architecture (branch pushed, no PR until a real SKaiNET release exists — it depends on 0.41.0-SNAPSHOT from the new -PskainetMavenLocal dev lane):

• SKaiNET published locally as 0.41.0-SNAPSHOT (deliberately not 0.40.1 — Central has an Aug-12 0.40.1 without the ternary work; a colliding local publish shadows it silently).
• Engine 0.25→0.41 migration complete: QuantPolicy (removed upstream in SKaiNET@9ada6f88) adopted as transformers-owned plumbing — it never crossed an engine API; llm-core exposed as api() where loader signatures surface it; the PreTransposed* encoding delegation diamond resolved. Full suite green: 703 tests, 0 failures against the snapshot.
• Pre-existing (also on develop, unrelated): :llm-apps:kllama-cli fails resolution — versionless skainet-lang-core with no BOM constraint on its path. Worth its own fix.

Verified BitNet-2B4T block structure (from NeoGPU's working hs_ml_infer.c driver, the port's ground truth):

x → rmsNorm(attn_norm) → MHA(RoPE, causal)
  → rmsNorm(attn_sub_norm)  ← applied to attention output BEFORE o_proj
  → o_proj → residual
→ rmsNorm(ffn_norm) → gate_proj ∥ up_proj
  → relu(gate)² * up → rmsNorm(ffn_sub_norm, over F) → down_proj → residual

(eps 1e-5 throughout.)

DSL gap for the network def — decoderTransformerNetwork hardcodes SwiGLU and has no sub-norm hooks; its own KDoc says to add an ffnKind parameter when this day came. Next unit of work:

1. ffnKind knob + a relu²·up → subNorm → down FFN module in the DSL
2. an attnSubNorm hook inside multiHeadAttention (norm between attention output and o_proj)
3. then the thin bitnetNetwork() def, ModelFamily.BITNET, tensor names, and the packed-weight loader path

[michalharakal]

T1 implemented on feat/336-bitnet-architecture (commit 3bea80c, on top of the 0.41-migration commit). Full repo sweep: 708 tests, 0 failures.

What landed, exactly along the anticipated extension seams — no new abstractions:

• DecoderFfnKind — the ffnKind parameter decoderTransformerNetwork's own KDoc promised for the first non-SwiGLU model. RELU2_SUBLN builds the new BitNetFFN: down(subNorm(relu(gate(x))² * up(x))).
• attnSubNorm on MultiHeadAttention, mirroring the existing qkNorm pattern: an RMSNorm child on the merged attention output before o_proj, applied on both the fused-decode and general SDPA paths.
• llm-inference/bitnet: bitnetNetwork() (thin, ≤10 lines), BitNetGGUFNameResolver (delegates to the engine Llama resolver + the two BitNet-only tensors blk.N.attn_sub_norm.weight / ffn_sub_norm.weight), BitNetNetworkLoader (GGUF; tied output.weight covered by the decoder loader's existing fallback).
• ModelFamily.BITNET (bitnet / bitnet-25 / bitnet-b1.58) + skainet-cli routing through the generic DSL branch.

Both structural deviations are pinned against hand-computed references (the seqLen-1 trick makes the attention path closed-form: softmax over one key = 1, so expected = W_o · rmsNorm(W_v·x, w_sub)), and the pipeline test proves every parameter — sub-norms included — maps by name.

Remaining before this issue closes: a real-GGUF smoke (F32/F16 conversion of 2B4T) once a model file is at hand. The packed-ternary load path (engine StreamingGgufParametersLoader(i2sLayout=…) → BITNET_B1_58 → vendored NEON kernels) and the BITNET_PLANES lm_head are #337.

[michalharakal]

The real model speaks. 🎉 (commit 9ad22c2, on the SKaiNET 0.49 line)

$ skainet-cli -m bitnet-2b4t-i2s.gguf --temperature 0 'The capital of France is'
The capital of France is Paris. Paris is the capital of France. Paris is known
for its beautiful architecture, art, and cuisine. The Eiffel Tower is a famous
landmark in Paris. ...

microsoft/bitnet-b1.58-2B-4T (official BitNet.cpp I2_S GGUF, 1.19 GB) loads packed — 0.25 B/weight ternary projections — and generates coherent, factually correct text through the vendored NeoGPU NEON kernels, exact f32 math end to end. One run proves the whole chain against the official file: GROUP_128 I2_S layout + trailer scales (SKaiNET#1140's verification caveat — closed), sub-norm placement, squared-ReLU FFN, tied lm_head, tokenizer.

What it took beyond the branch as it stood:

• CLI BITNET branch → BitNetPackedGgufLoader + kernel-pack installs; loadWithMetadata for bos; tied-embeddings fallback in applyWeightsToNetwork (2B4T ships no output.weight — the packed path bypasses the decoder loader that used to provide it).
• One engine fix (SKaiNET#1186, targeted at release/0.49.0): SKEEP-004 puts JVM activations in MemorySegments, which the common ternary dispatch path can't element-read — bridged with a per-call heap copy of the activation (k floats), with the trap that the JVM dataFactory adopts float arrays back into segments, so the copy constructs the heap TensorData directly.

First-run throughput: ~0.45 tok/s on an Apple-arm64 dev host — dominated by the FP32-widened tied lm_head (128256×2560 dense matmul/token), not the ternary path. Perf follow-ups (serve token_embd packed for the lm_head; cheaper activation bridging) are deliberately after correctness. Full sweep: 713 tests, 0 failures.

[michalharakal]

Re-scope note (2026-08-31): everything in the task list here shipped via PR #353 and is on engine BOM 0.52.0 — module llm-inference/bitnet/, ModelFamily.BITNET + detect for all three arch strings, packed BITNET_B1_58 loading (no FP32 widening; widened path kept as opt-in parity baseline), CLI dispatch, synthetic-GGUF + parity tests. Naming deviations from the issue text (BitNetGGUFNameResolver instead of BitNetGgufTensorNames, BitNetPackedGgufLoader) are being converged in #359 (full #346 conformance).

What was NOT delivered from this issue's test list: the real-model gated smoke (no smoke-models.json row — the 2B4T run was manual) — now tracked in #360. Suggest closing this in favor of #359/#360, maintainer's call.

[michalharakal]

Closing — everything here shipped and the residuals landed too: the module (PR #353), the model-gated smoke row (smoke-models.json, PR #364), and the #346-conformant shape (PR #363: BitNetWeightLoader, BitNetRuntimeWeights/BitNetTensorNames, llm-runtime/kbitnet). Baseline correctness is now held to a stronger standard than this issue asked for: greedy decode is token-for-token equal to bitnet.cpp and the HF BF16 reference (PR #365).