Skip to content

Retire kllama-cli's legacy dense-FP32 LlamaRuntime path; route Llama/Mistral GGUF through the DSL like Qwen already does #354

Description

@michalharakal

Problem

`llm-runtime/kllama`'s `Main.kt` (kllama-cli) branches on GGUF architecture: Qwen GGUFs go through the modern DSL path (`DecoderGgufWeightLoader` → `QwenNetworkLoader` → `OptimizedLLMRuntime` DIRECT — packed/MAPPED weights, kernel-dispatch-routed matmuls), but Llama/Mistral GGUFs still fall to the legacy `LlamaRuntime` path via `LlamaIngestion`/`LlamaLoadConfig` — dense FP32, no `KernelDispatch`/`WeightForm` involvement at all. See the comment at the branch point:

```kotlin
// --- Llama / SafeTensors / BIN: legacy LlamaRuntime path
// (dense FP32; the eager kernels assume dense data).
// Qwen GGUF stays on the DSL branch above.
```

This is the exact machinery the #338 engine-adoption arc (PR #353) migrated everything else off of — `LlamaRuntime` predates `WeightForm`/MAPPED residency/the row-major kernel packs entirely, and (being dense-FP32-only) can't benefit from any of it.

Impact (measured)

Loading `Llama-3.2-1B-Instruct-Q4_K_M.gguf` (762 MB) via kllama-cli's legacy branch: still running 16-token generation after 5+ minutes at 190%+ CPU, never completing. The exact same file through the DSL path (`KLlamaJava.loadGGUF`, used by `skainet-cli` and by SKaiNET-EdgeTranslator's SkaiNet engine integration) loads in 1.9s and generates 16 tokens in 661ms (~24 tok/s) — coherent output.

This is a correctness-adjacent trap, not just a perf gap: anyone hitting kllama-cli with a plain Llama/Mistral GGUF (the common case — Qwen is the exception, not the rule) gets a runtime that appears to hang, with the working DSL path sitting right there in the same module used one branch away.

Ask

  • Route the Llama/Mistral GGUF branch in kllama-cli's `Main.kt` through the same DSL path Qwen already uses (`DecoderGgufWeightLoader` → `LlamaNetworkLoader.fromWeights` → `OptimizedLLMRuntime` DIRECT — exactly what `KLlamaJava.loadGGUF` already does).
  • Once nothing exercises it, delete `LlamaRuntime`, `LlamaRuntimeInterface`, `LlamaIngestion`/`LlamaLoadConfig`'s eager-dense path, and any other FP32-only/no-DSL runtime left over from before the engine-loader migration. (`LlamaRuntime` also still serves the llama2.c `.bin` lane via `Llama2DotCWeightLoader` — confirm that lane's fate first; either port it onto the DSL too or keep a narrowly-scoped legacy runtime for `.bin` only, not GGUF.)
  • Audit other non-CLI callers of `LlamaIngestion`/`LlamaRuntime` for the same trap (kllama's Android/native test targets, samples) while this is being cleaned up.

Context

Found while wiring SKaiNET-EdgeTranslator's SkaiNet engine option (PR #353's downstream consumer) — `KLlamaJava.loadGGUF` (the facade EdgeTranslator calls) was already correctly on the DSL path, but was separately missing the `KernelPacks.install()`/`FfmRowMajorKernelPack.install()` calls skainet-cli has (fixed alongside this investigation). kllama-cli's own Main.kt, investigated as a comparison baseline, is what surfaced this separate, deeper issue.

🤖 Generated with Claude Code

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions