Problem
`llm-runtime/kllama`'s `Main.kt` (kllama-cli) branches on GGUF architecture: Qwen GGUFs go through the modern DSL path (`DecoderGgufWeightLoader` → `QwenNetworkLoader` → `OptimizedLLMRuntime` DIRECT — packed/MAPPED weights, kernel-dispatch-routed matmuls), but Llama/Mistral GGUFs still fall to the legacy `LlamaRuntime` path via `LlamaIngestion`/`LlamaLoadConfig` — dense FP32, no `KernelDispatch`/`WeightForm` involvement at all. See the comment at the branch point:
```kotlin
// --- Llama / SafeTensors / BIN: legacy LlamaRuntime path
// (dense FP32; the eager kernels assume dense data).
// Qwen GGUF stays on the DSL branch above.
```
This is the exact machinery the #338 engine-adoption arc (PR #353) migrated everything else off of — `LlamaRuntime` predates `WeightForm`/MAPPED residency/the row-major kernel packs entirely, and (being dense-FP32-only) can't benefit from any of it.
Impact (measured)
Loading `Llama-3.2-1B-Instruct-Q4_K_M.gguf` (762 MB) via kllama-cli's legacy branch: still running 16-token generation after 5+ minutes at 190%+ CPU, never completing. The exact same file through the DSL path (`KLlamaJava.loadGGUF`, used by `skainet-cli` and by SKaiNET-EdgeTranslator's SkaiNet engine integration) loads in 1.9s and generates 16 tokens in 661ms (~24 tok/s) — coherent output.
This is a correctness-adjacent trap, not just a perf gap: anyone hitting kllama-cli with a plain Llama/Mistral GGUF (the common case — Qwen is the exception, not the rule) gets a runtime that appears to hang, with the working DSL path sitting right there in the same module used one branch away.
Ask
- Route the Llama/Mistral GGUF branch in kllama-cli's `Main.kt` through the same DSL path Qwen already uses (`DecoderGgufWeightLoader` → `LlamaNetworkLoader.fromWeights` → `OptimizedLLMRuntime` DIRECT — exactly what `KLlamaJava.loadGGUF` already does).
- Once nothing exercises it, delete `LlamaRuntime`, `LlamaRuntimeInterface`, `LlamaIngestion`/`LlamaLoadConfig`'s eager-dense path, and any other FP32-only/no-DSL runtime left over from before the engine-loader migration. (`LlamaRuntime` also still serves the llama2.c `.bin` lane via `Llama2DotCWeightLoader` — confirm that lane's fate first; either port it onto the DSL too or keep a narrowly-scoped legacy runtime for `.bin` only, not GGUF.)
- Audit other non-CLI callers of `LlamaIngestion`/`LlamaRuntime` for the same trap (kllama's Android/native test targets, samples) while this is being cleaned up.
Context
Found while wiring SKaiNET-EdgeTranslator's SkaiNet engine option (PR #353's downstream consumer) — `KLlamaJava.loadGGUF` (the facade EdgeTranslator calls) was already correctly on the DSL path, but was separately missing the `KernelPacks.install()`/`FfmRowMajorKernelPack.install()` calls skainet-cli has (fixed alongside this investigation). kllama-cli's own Main.kt, investigated as a comparison baseline, is what surfaced this separate, deeper issue.
🤖 Generated with Claude Code
Problem
`llm-runtime/kllama`'s `Main.kt` (kllama-cli) branches on GGUF architecture: Qwen GGUFs go through the modern DSL path (`DecoderGgufWeightLoader` → `QwenNetworkLoader` → `OptimizedLLMRuntime` DIRECT — packed/MAPPED weights, kernel-dispatch-routed matmuls), but Llama/Mistral GGUFs still fall to the legacy `LlamaRuntime` path via `LlamaIngestion`/`LlamaLoadConfig` — dense FP32, no `KernelDispatch`/`WeightForm` involvement at all. See the comment at the branch point:
```kotlin
// --- Llama / SafeTensors / BIN: legacy LlamaRuntime path
// (dense FP32; the eager kernels assume dense data).
// Qwen GGUF stays on the DSL branch above.
```
This is the exact machinery the #338 engine-adoption arc (PR #353) migrated everything else off of — `LlamaRuntime` predates `WeightForm`/MAPPED residency/the row-major kernel packs entirely, and (being dense-FP32-only) can't benefit from any of it.
Impact (measured)
Loading `Llama-3.2-1B-Instruct-Q4_K_M.gguf` (762 MB) via kllama-cli's legacy branch: still running 16-token generation after 5+ minutes at 190%+ CPU, never completing. The exact same file through the DSL path (`KLlamaJava.loadGGUF`, used by `skainet-cli` and by SKaiNET-EdgeTranslator's SkaiNet engine integration) loads in 1.9s and generates 16 tokens in 661ms (~24 tok/s) — coherent output.
This is a correctness-adjacent trap, not just a perf gap: anyone hitting kllama-cli with a plain Llama/Mistral GGUF (the common case — Qwen is the exception, not the rule) gets a runtime that appears to hang, with the working DSL path sitting right there in the same module used one branch away.
Ask
Context
Found while wiring SKaiNET-EdgeTranslator's SkaiNet engine option (PR #353's downstream consumer) — `KLlamaJava.loadGGUF` (the facade EdgeTranslator calls) was already correctly on the DSL path, but was separately missing the `KernelPacks.install()`/`FfmRowMajorKernelPack.install()` calls skainet-cli has (fixed alongside this investigation). kllama-cli's own Main.kt, investigated as a comparison baseline, is what surfaced this separate, deeper issue.
🤖 Generated with Claude Code