feat(gemma3n): StableHLO export harness (PLE as graph input) + gemma3n/hybrid-AI docs

.github/workflows/smoke-reference.yml

@@ -39,6 +39,10 @@ on:
         description: "Direct URL to Apertus-8B-Instruct-2509-Q4_K_S.gguf (~4.6 GB). Enables the Apertus golden-token parity gate (QK-norm + xIELU + ungated FFN). Leave blank to skip."
         required: false
         default: ""
+      gemma3n_gguf_url:
+        description: "Direct URL to gemma-3n-E2B-it-Q4_K_M.gguf (~3.0 GB). Enables the Gemma 3n golden-token parity gate on the DSL lane (AltUp + Laurel + sparsity + PLE + shared KV; needs a large-memory runner: 20g test heap). Leave blank to skip."
+        required: false
+        default: ""
       gemma4_safetensors_dir_url:
         description: "Direct URL to a tar.gz containing the Gemma-4 E2B SafeTensors checkpoint directory. Leave blank to skip the kgemma test."
         required: false
@@ -110,6 +114,18 @@ jobs:
           echo "APERTUS_GGUF_PATH=$RUNNER_TEMP/models/apertus/Apertus-8B-Instruct-2509-Q4_K_S.gguf" >> "$GITHUB_ENV"
           # The 8B parity gate needs more than the module's 6g default test heap.
           echo "APERTUS_HEAP_ARG=-PapertusTestMaxHeap=12g" >> "$GITHUB_ENV"
+
+      - name: Stage Gemma 3n E2B GGUF
+        if: inputs.gemma3n_gguf_url != ''
+        env:
+          URL: ${{ inputs.gemma3n_gguf_url }}
+        run: |
+          set -euo pipefail
+          mkdir -p "$RUNNER_TEMP/models/gemma3n"
+          curl -fsSL "$URL" -o "$RUNNER_TEMP/models/gemma3n/gemma-3n-E2B-it-Q4_K_M.gguf"
+          echo "GEMMA3N_E2B_GGUF=$RUNNER_TEMP/models/gemma3n/gemma-3n-E2B-it-Q4_K_M.gguf" >> "$GITHUB_ENV"
+          # The E2B parity gate self-skips below 16 GB test heap.
+          echo "GEMMA3N_HEAP_ARG=-PgemmaTestMaxHeap=20g" >> "$GITHUB_ENV"
         if: inputs.gemma4_safetensors_dir_url != ''
         env:
           URL: ${{ inputs.gemma4_safetensors_dir_url }}
@@ -149,6 +165,7 @@ jobs:
             -Dorg.gradle.configuration-cache=true \
             -PsmokeReference -PincludeIntegration \
             ${APERTUS_HEAP_ARG:-} \
+            ${GEMMA3N_HEAP_ARG:-} \
             test
 
       - name: Disk space (after run)

CHANGELOG.md

@@ -9,6 +9,50 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added — Gemma 3n StableHLO/IREE export harness + hybrid-AI design note
+
+- **`exportGemma3n`** (`Gemma3nExportHarness`, SmolLM2/FunctionGemma redecode pattern):
+  traces `gemma3nNetwork()` to StableHLO with external bf16 params and an in-graph argMax
+  tail. Mobile-honest contract: **`per_layer_inputs` is a graph INPUT** computed on the
+  CPU from the packed PLE table at runtime (PLE's design point — those parameters stay
+  off the accelerator), so the parameter archive carries the trunk + token embedding
+  only. `PerLayerEmbedding` gained a traceable `indexSelect` path while recording;
+  `GEMMA3N_LAYERS` truncates the trunk for pipeline verification on smaller hosts. Full
+  E2B emission is **blocked on engine SKaiNET#1247** (trace memory co-residency + an HLO
+  converter operand-linkage defect) — the harness hard-fails on both signatures instead
+  of shipping a silently-unservable module.
+- New antora explanation page `explanation/gemma3n.adoc` (why Gemma 3n's mobile-first
+  architecture and why SKaiNET fits it) and pre-PRD design note
+  `docs/specs/matformer-hybrid-on-device-ai.md` (MatFormer elasticity in SKaiNET +
+  hybrid on-device/cloud routing: draft-first, escalate-on-evidence).
+
+### Added — Gemma 3n runs on the DSL path, parity-gated (#377)
+
+- **`gemma3nNetwork()` + `Gemma3nModel`** — the full Gemma 3n text architecture declared
+  in the DSL, faithful to HF `modeling_gemma3n.py`: **AltUp** (four parallel hidden
+  streams with the tanh modality router; `Gemma3nAltUpBlock` per layer,
+  `Gemma3nAltUpGlobals` for the magnitude-renormed stream init/merge), **Laurel**,
+  **Gaussian-top-k activation sparsity** on the first ten layers (driven by the GGUF's
+  precomputed per-layer std multipliers; `-inf` = off), **PLE feeding the non-active
+  streams** (reusing the gemma-4 lane's `PerLayerEmbedding` — the math is identical),
+  per-type **shared KV** for the last ten layers, hybrid sliding/global attention with
+  dual RoPE bases, q/k-norm + parameterless v-norm, attention scale 1.0. All math goes
+  through `ctx.ops`, so the model is traceable for the StableHLO → IREE mobile path.
+- **The hand-rolled `Gemma3nRuntime` was never faithful to real checkpoints**: it loaded
+  the PLE tensors but never applied them, had no Laurel, ignored the AltUp router, and
+  its `E2B_DEFAULT` config claimed AltUp/sparsity were E4B-only — the real E2B GGUF has
+  `altup.num_inputs=4` and first-10-layer sparsity. The GGUF CLI paths (kgemma, unified
+  skainet-cli) now route gemma3n through the DSL lane; SafeTensors stays on the legacy
+  runtime until the DSL grows that leg.
+- **`Gemma3nGoldenTokenParityTest`** (#346 gate, the last ungated generative family):
+  full 32-step greedy text equality vs mainline llama.cpp b10621 on
+  `gemma-3n-E2B-it-Q4_K_M.gguf`, on the exact CLI path — engine loading stays
+  packed/MAPPED (the PLE table row-dequants on demand). Wired into the smoke-reference
+  tier (`gemma3n_gguf_url` + 20g heap arg); `smoke-models.json` gains a Gemma3n-E2B row.
+  Metadata parsing now reads the real llama.cpp GGUF keys (`sliding_window_pattern`
+  booleans, per-layer `activation_sparsity_scale`, `rope.freq_base` fallback,
+  `rms_norm_eps`, per-layer `feed_forward_length`).
+
 ### Fixed — Qwen tool calling follows the official Qwen3 chat template
 
 - **`QwenChatTemplate` rewritten against the official Qwen3 `chat_template`** (verified

README.md

@@ -46,7 +46,7 @@ Use the version shown in this README as the source of truth for first-run snippe
 > The list below describes the project's **intended** scope. Maturity varies
 > widely per item and many paths are unverified — see the project-status note above.
 
-- **Multi-model support.** Llama / Mistral, Qwen 2 / 2.5 / 3, Gemma 3 / 4, Apertus (Swiss AI) and BitNet b1.58 are each **verified token-for-token against a reference implementation** (mainline llama.cpp; bitnet.cpp + the HF BF16 reference for BitNet) by model-gated golden-token parity tests on the DSL path the CLIs ship. BERT (vs sentence-transformers) and T5/GTR (real-weights round-trip) are verified on the embedding side. Gemma 3n and Voxtral are the remaining unverified families — see the status table below.
+- **Multi-model support.** Llama / Mistral, Qwen 2 / 2.5 / 3, Gemma 3 / 4, Apertus (Swiss AI) and BitNet b1.58 are each **verified token-for-token against a reference implementation** (mainline llama.cpp; bitnet.cpp + the HF BF16 reference for BitNet) by model-gated golden-token parity tests on the DSL path the CLIs ship. BERT (vs sentence-transformers) and T5/GTR (real-weights round-trip) are verified on the embedding side. Voxtral is the remaining unverified family — see the status table below.
 - **Native CPU performance.** Auto-discovers SKaiNET's priority-100 FFM (Foreign Function & Memory) native kernel provider when present (4–6× faster Q4_K matmul, 1.5–1.8× faster FP32 SGEMM vs the priority-50 Panama Vector path; Linux x86_64 / macOS ARM64 / Windows x86_64 in the published JAR — no manual setup). On **Android**, the runtime facades ship the engine's JNI NEON backend the same way — native kernels out of the box, ~6.4× measured on SmolLM2-135M Q8_0 (see the [supported-targets matrix](#supported-targets)).
 - **Tool calling (experimental).** Family-specific chat templates and tool-call parsers (Llama 3, Qwen, Gemma, Apertus, ChatML/Hermes) and a Java surface (`KLlamaJava`, `JavaTools.definition`, `JavaAgentLoop`) exist, but tool calling is **not reliable yet** — it may fail to trigger or parse even when plain generation works.
 - **GGUF + SafeTensors loading.** Streaming reader for any model size; `NATIVE_OPTIMIZED` quant policy keeps weights in their packed SIMD-friendly form.
@@ -89,7 +89,7 @@ Honest status — see the project-status note at the top of this README.
 | **Llama / Mistral** | Verified: `LlamaGoldenTokenParityTest` asserts **full greedy text equality against mainline llama.cpp** on Llama-3.2-1B-Instruct Q8_0, on the DSL path the CLIs ship; smoke rows exercise the same path. |
 | **Qwen 2 / 2.5 / 3** | Verified: `QwenGoldenTokenParityTest` asserts **full greedy text equality against mainline llama.cpp** for both family variants — Qwen2.5-0.5B-Instruct Q8_0 (attention projection biases) and Qwen3-1.7B Q8_0 (QK-norm); smoke rows for both. |
 | **Gemma 3 / 4** | Verified: `Gemma4ChatGoldenTokenTest` asserts golden-token parity against llama.cpp on Gemma-4 E2B GGUF; gemma3 (FunctionGemma) and gemma4 checkpoints run the same DSL lane (`gemmaNetwork()`), GGUF and SafeTensors. **Gemma 2 has no supported path** (the CLI refuses it loudly). |
-| **Gemma 3n** | Own family module (`llm-inference/gemma3n` + `kgemma3n`), hand-rolled runtime (AltUp / per-layer embeddings / activation sparsity), dense-FP32 loading. Works, but **no parity gate yet** — the remainder is tracked in #377. |
+| **Gemma 3n** | Verified: `Gemma3nGoldenTokenParityTest` asserts **full greedy text equality against mainline llama.cpp** on gemma-3n-E2B-it Q4_K_M — the DSL lane (`gemma3nNetwork()`: AltUp × 4 streams, Laurel, activation sparsity, PLE, shared KV) with packed/MAPPED loading. Exports StableHLO for the IREE mobile path (`exportGemma3n`); see `docs/…/explanation/gemma3n.adoc`. |
 | **Apertus** | Verified: `ApertusGoldenTokenParityTest` asserts **full greedy text equality against mainline llama.cpp** on Apertus-8B-Instruct Q4_K_S — QK-norm, xIELU per-layer activations and the ungated FFN exercised end-to-end. |
 | **BitNet b1.58** | Packed I2_S path end-to-end on the eager JVM path: 2-bit ternary weights (0.25 B/weight), fused `BITNET_PLANES` lm_head, two-stage candidate decode. Greedy decode verified **token-for-token against bitnet.cpp and the HF BF16 reference** on 2B4T; model-gated parity + smoke tests. See `docs/modules/ROOT/pages/explanation/bitnet.adoc`. |
 | **BERT** | Sentence embeddings on the DSL path (`bertNetwork()` + `BertEncoderRuntime`, eager or traced/fused) — verified against sentence-transformers on MongoDB/mdbr-leaf. One-call `BertEmbeddingModel.fromHuggingFace(...)` with built-in Hub download; MEAN or CLS pooling and retrieval prefixes cover LEAF, BGE and E5-style models. No text generation, no tool calling. |

docs/modules/ROOT/nav.adoc

@@ -35,5 +35,6 @@
 * xref:explanation/tokenizer-internals.adoc[Tokenizer Internals]
 * xref:explanation/weight-quantization.adoc[Weight Quantization and Numeric Representation]
 * xref:explanation/bitnet.adoc[BitNet b1.58 — Ternary Inference End to End]
+* xref:explanation/gemma3n.adoc[Gemma 3n — Mobile-First Architecture]
 * xref:explanation/embeddings.adoc[How Embeddings Work]
 * xref:explanation/android-eager-vs-compiled.adoc[Eager vs. Compiled on Android]

docs/modules/ROOT/pages/explanation/dsl-vs-handcoded.adoc

@@ -82,7 +82,6 @@ If the architecture uses standard building blocks (MHA, RMSNorm, FFN), the DSL a
 Some architectures have components the DSL cannot express:
 
 * *Qwen3.5 DeltaNet* -- hybrid DeltaNet (linear attention + SSM) layers with causal 1D convolution
-* *Gemma3n* -- variable FFN dimensions per layer (MatFormer), per-layer embeddings
 * *Voxtral* -- ODE flow matching for audio codec
 
 These use `DecoderRuntime` directly.
@@ -126,9 +125,9 @@ The goal is to extend the DSL to support these patterns over time.
 |`voxtralBackboneNetwork()`
 |Partial DSL
 
-|Gemma3n
-|_none_
-|Hand-coded only
+|Gemma 3n
+|`gemma3nNetwork()`
+|DSL lane for GGUF (AltUp/Laurel/sparsity/PLE as DSL modules, parity-gated); legacy runtime remains for SafeTensors only
 
 |Qwen3.5
 |_none_

docs/modules/ROOT/pages/explanation/gemma3n.adoc

@@ -0,0 +1,93 @@
+= Gemma 3n — Mobile-First Architecture, and Why SKaiNET Fits It
+:description: What makes Gemma 3n different, and why SKaiNET's DSL → DAG → StableHLO / eager design serves it well.
+
+Gemma 3n is Google's mobile-first generation of open models
+(https://developers.googleblog.com/en/introducing-gemma-3n-developer-guide/[developer guide]):
+it is engineered so that a model with 5B raw parameters (E2B) or 8B (E4B) runs in the
+memory footprint of a much smaller one (~2 GB / ~3 GB respectively). Every one of its
+architectural tricks is a memory-or-latency trade tailored to phones — and each maps
+naturally onto a SKaiNET mechanism.
+
+== What Gemma 3n actually is
+
+*MatFormer (Matryoshka Transformer)*::
+A nested transformer built for elastic inference: the E4B model contains a fully
+functional E2B sub-model, co-trained. Beyond the two pre-extracted sizes, *Mix-n-Match*
+slices custom-sized models between them by adjusting per-layer feed-forward widths
+(8192 → 16384) and skipping layers. This is why the GGUF stores
+`feed_forward_length` as a *per-layer array* — and why `gemma3nNetwork()` builds each
+layer's FFN width independently.
+
+*Per-Layer Embeddings (PLE)*::
+A large per-layer embedding table (262 144 × layers·256) that improves quality without
+growing the accelerator-resident core: only the ~2B (E2B) / ~4B (E4B) trunk weights need
+fast memory — the PLE parameters can stay on CPU and be *gathered per token*.
+
+*KV-cache sharing*::
+The last block of layers reuses the K/V of the last non-shared layer of the same
+attention type — Google reports a *2× prefill improvement* over Gemma 3 4B.
+
+*Hybrid attention + AltUp + activation sparsity*::
+4-out-of-5 layers use a 512-token sliding window (10k RoPE base); every 5th sees the
+full context (1M base). AltUp keeps four parallel hidden streams but routes only one
+through the expensive layers; the first ten layers apply Gaussian-top-k activation
+sparsity (95%) to their FFN gates.
+
+== Why SKaiNET is a natural fit
+
+*One declarative definition, two execution paths.*::
+`gemma3nNetwork()` declares the architecture once, through the transformer DSL. The same
+module tree runs **eagerly** on the JVM (`OptimizedLLMRuntime` — the path the
+golden-token parity gate certifies token-for-token against mainline llama.cpp) and
+**traces to a compute graph** for StableHLO emission and `iree-compile` to a mobile
+`vmfb` (`exportGemma3n`). No hand-written second implementation to drift.
+
+*PLE lands exactly where Google designed it to.*::
+SKaiNET's loader keeps the PLE table *packed* (its stored quantization, ~2 GB instead of
+8 GB dense) and gathers rows on demand on the CPU — for eager decode via a row-dequant
+wrapper, and for the compiled path by making `per_layer_inputs` a *graph input* computed
+host-side, so the accelerator archive carries only the trunk. That is the PLE
+memory-split from the Gemma 3n paper, realized by the engine's `WeightForm` machinery
+rather than bespoke code.
+
+*Packed, memory-mapped weights.*::
+The engine's `StreamingGgufParametersLoader` serves quantized tensors zero-copy from
+file-backed pages (`MAPPED` residency). A phone-class memory budget is the entire point
+of Gemma 3n; the loading path honors it instead of inflating everything to FP32.
+
+*Per-layer heterogeneity is free in the DSL.*::
+MatFormer's variable FFN widths, the 4+1 sliding/global pattern, dual RoPE bases and
+per-type shared KV caches (`OwnerReadOnlyKVCache`) are all per-layer decisions in a
+plain Kotlin loop — the DSL builds a different stage per layer, no special casing.
+
+*Kotlin Multiplatform reach.*::
+The same codebase targets JVM/desktop, Android (eager via the JNI NEON backend, or
+compiled via `llm-runtime/iree-android`), and Kotlin/Native — matching Gemma 3n's
+"everywhere on-device" distribution story.
+
+*Faithfulness is gated, not claimed.*::
+`Gemma3nGoldenTokenParityTest` asserts full 32-step greedy text equality against
+mainline llama.cpp on a real E2B checkpoint — AltUp router, Laurel, sparsity, PLE and
+KV-sharing all exercised. See the
+xref:../index.adoc#_supported_model_families[verified-model matrix].
+
+== Where the pieces live
+
+|===
+|Concern |Code
+
+|DSL definition
+|`llm-inference/gemma3n` — `gemma3nNetwork()`, `Gemma3nModel`, `Gemma3nAltUpBlock`,
+ `Gemma3nLaurelBlock`, `Gemma3nSparseGeGluFFN`, `Gemma3nPerLayerApply`
+
+|PLE machinery (shared with Gemma 4)
+|`llm-inference/gemma` — `PerLayerEmbedding` (packed row-dequant + traceable
+ `indexSelect` path)
+
+|Compiled export
+|`Gemma3nExportHarness` / `exportGemma3n` gradle task → `gemma3n-gen.mlir` +
+ `gemma3n.safetensors` (bf16) + `manifest.json`
+
+|Parity gate
+|`Gemma3nGoldenTokenParityTest` (model-gated, smoke-reference tier)
+|===

docs/modules/ROOT/pages/index.adoc

@@ -43,9 +43,9 @@ headers record the exact oracle build and commands.
 
 |Gemma 3n
 |Gemma 3n E2B/E4B
-|Not yet — hand-rolled runtime (AltUp, per-layer embeddings); gate tracked in issue #377
+|mainline llama.cpp — full greedy text equality (gemma-3n-E2B-it Q4_K_M; AltUp, Laurel, sparsity, PLE, shared KV)
 |No
-|Hand-coded
+|`gemma3nNetwork()`
 
 |Apertus
 |Apertus 8B