Deprecate LEAP SDK and add native llama.cpp developer guides

INTERACTIVE_QUICKSTART_WIDGET.md

@@ -5,8 +5,8 @@
 - Laptop with Ollama
 - Laptop with llama.cpp
 - Macbook with MLX
-- iOS with LEAP SDK
-- Android with LEAP SDK
+- iOS with llama.cpp
+- Android with llama.cpp
 - Cloud with vLLM
 - Browser with Transformers.js
 

README.md

@@ -6,15 +6,14 @@
   />
   <div style="display: flex; justify-content: center; gap: 0.5em;">
     <a href="https://playground.liquid.ai/"><strong>Try LFM</strong></a> •
-    <a href="https://docs.liquid.ai/lfm"><strong>Documentation</strong></a> •
-    <a href="https://leap.liquid.ai/"><strong>LEAP</strong></a>
+    <a href="https://docs.liquid.ai/lfm"><strong>Documentation</strong></a>
   </div>
   <br/>
   <a href="https://discord.com/invite/liquid-ai"><img src="https://img.shields.io/discord/1385439864920739850?style=for-the-badge&logo=discord&logoColor=white&label=Discord&color=5865F2" alt="Join Discord"></a>
 </div>
 </br>
 
-This is the **official documentation repository** for Liquid AI. It contains comprehensive guides, API references, and tutorials for building with our open-weight [LFMs](https://huggingface.co/LiquidAI) and the [LEAP SDK](https://leap.liquid.ai/) on laptops, mobile, and edge devices. The documentation is hosted at [docs.liquid.ai](https://docs.liquid.ai).
+This is the **official documentation repository** for Liquid AI. It contains comprehensive guides, API references, and tutorials for building with our open-weight [LFMs](https://huggingface.co/LiquidAI) on laptops, mobile, and edge devices. The documentation is hosted at [docs.liquid.ai](https://docs.liquid.ai).
 
 ## Table of Contents
 
@@ -73,7 +72,7 @@ You should see `liquid-docs` in the list of MCP servers.
 Then, in a Claude Code session, ask a question about Liquid AI:
 
 ```
-How do I deploy an LFM model to Android using the LEAP SDK?
+How do I run an LFM model on Android with llama.cpp?
 ```
 
 Claude Code will query your MCP server and return an answer grounded in Liquid docs.

deployment/on-device/leap-sdk-changelog.mdx

@@ -1,15 +1,20 @@
 ---
 title: "Changelog"
 description: "Release notes for the LEAP SDK, including the 0.9.x → 0.10.x Kotlin Multiplatform transition."
+noindex: true
 ---
 
+<Warning>
+The LEAP SDK is deprecated and this page is no longer maintained. Build on llama.cpp directly instead — see [Migrating from LEAP SDK](/deployment/on-device/llama-cpp/migrating-from-leap-sdk).
+</Warning>
+
 Latest release: **v0.10.7** ([GitHub](https://github.com/Liquid4All/leap-sdk/releases/tag/v0.10.7)).
 
 This page covers user-visible changes in the LEAP SDK across releases. For per-build commit detail, see the release notes on [`Liquid4All/leap-sdk`](https://github.com/Liquid4All/leap-sdk/releases).
 
 ## 0.9.x → 0.10.x: Kotlin Multiplatform unification
 
 Starting with v0.10.0, the LEAP SDK ships from a single Kotlin Multiplatform codebase. The two previously separate distributions (the Android-only `ai.liquid.leap:*` Maven artifacts and the iOS-only `Liquid4All/leap-ios` Swift package) were collapsed into one source tree that publishes to:
 
 - **Swift Package Manager** — [`Liquid4All/leap-sdk`](https://github.com/Liquid4All/leap-sdk) (new repo, for iOS / macOS consumers).
 - **Maven Central** — `ai.liquid.leap:*` (Android, JVM, and Kotlin/Native targets).
@@ -25,7 +30,7 @@
 | `LeapSDK` | `ai.liquid.leap:leap-sdk` | Core inference + conversation API |
 | `LeapModelDownloader` | `ai.liquid.leap:leap-model-downloader` | Hosted / manifest-based model fetch |
 | `LeapOpenAIClient` | `ai.liquid.leap:leap-openai-client` | OpenAI-compatible cloud chat client (new in 0.10.0) |
 | `LeapUI` | `ai.liquid.leap:leap-ui` | Voice assistant widget — Compose Multiplatform (new in 0.10.0) |
 | `LeapSDKMacros` | _(Swift only)_ | `@Generatable` / `@Guide` constrained-generation macros |
 
 ### Breaking changes for iOS consumers
@@ -36,8 +41,8 @@
 
 - **SPM URL change.** Point your Swift Package Manager dependency at `https://github.com/Liquid4All/leap-sdk.git` (not the deprecated `leap-ios` repo).
 - **CocoaPods removed.** The SDK ships exclusively through SPM in v0.10.0 onward.
 - **Toolchain bump.** Xcode 16 and Swift 6.0 are required.
 - **Swift downloader name.** In current 0.10.x, Swift code instantiates `ModelDownloader` from the `LeapModelDownloader` SPM product. Android code still uses the Kotlin class `ai.liquid.leap.downloader.LeapModelDownloader`. See [Model Loading](/deployment/on-device/sdk/model-loading) for the constructor signatures.
 
 ## Major additions since 0.9.x
 
@@ -51,11 +56,11 @@
 
 ### Voice assistant widget
 
 `LeapUI` / `leap-ui` is a Compose Multiplatform module that ships a drop-in voice assistant widget — an animated orb, mic button, and status label — backed by a state machine that handles recording, generation, and audio playback. Stable on iOS, macOS, Android, and JVM; Wasm/Web is present in the source tree as preview.
 
 See [Voice Assistant Widget](/deployment/on-device/sdk/voice-assistant).
 
 ### Sideloading models from explicit paths
 
 `LeapDownloader.loadSimpleModel` and `LeapModelDownloader.loadSimpleModel` load a model from explicit resource paths or URLs without going through the LEAP Model Library manifest. Useful for ADB-pushed bundles, app-bundled models, or any setup where you've already placed the model files on disk.
 
@@ -63,16 +68,16 @@
 
 ### iOS background downloads
 
 The iOS / macOS Swift `ModelDownloader(sessionConfiguration:)` initializer accepts an optional `URLSessionConfiguration?` so downloads can continue when the app is suspended or killed. See [Model Loading → Constructing the downloader](/deployment/on-device/sdk/model-loading#constructing-the-downloader).
 
 ### `autoDetectCompanionFiles`
 
 `Leap.load(url:options:)` on iOS gained an `autoDetectCompanionFiles: Bool = true` parameter that picks up companion files sitting next to the model file (e.g. multimodal projection weights).
 
 ### Swift ergonomics
 
 - **Compatibility layer** keeps 0.9.x call sites (`Leap.load(...)`, `Conversation.generateResponse(...)`) compiling on top of the unified KMP surface.
 - **`onEnum(of:)`** — SKIE-bridged sealed-class switching for Kotlin enums and sealed hierarchies (e.g. `MessageResponse`).
 - **`ChatMessageContent` static factories** — `.text(...)`, `.image(...)`, `.audio(...)` helpers instead of constructor calls.
 - **Builder-style options** — `LiquidInferenceEngineOptions.with(cacheOptions:)`, etc.
 
@@ -80,11 +85,11 @@
 
 Starting in **v0.10.4** and on by default through the current release, the inference engine loads model weights via `mmap` (`use_mmap=true`). It's the default behavior for every loaded model. On mobile this is the most user-visible runtime change in the 0.10.x line. A public opt-out arrived in **v0.10.5** as `ModelLoadingOptions.useMmap: Boolean?` (Kotlin) / `LiquidInferenceEngineOptions(useMmap:)` (Swift) — leave it `null`/`nil` to keep the default, or set `false` for filesystems where `mmap` misbehaves (some Android scoped-storage paths, certain network mounts).
 
 **What changed.** Previously the engine `read(2)`-ed the entire model file into a heap-allocated buffer before running prefill. Now it memory-maps the file: the kernel maps the on-disk weights into the process's virtual address space and loads pages lazily as they're accessed.
 
 **Performance implications on mobile:**
 
 - **Lower private RSS.** mmap'd weights are file-backed pages, not "anonymous private" RSS. iOS's jetsam and Android's low-memory killer both score apps primarily by anonymous RSS, so a 1.2B-Q4 model that previously counted as ~700 MB of dirty heap now shows as backing pages the OS may evict for free. Foreground apps are significantly less likely to be terminated under memory pressure.
 - **Faster cold load.** The constructor returns as soon as the file is mapped — typically tens of milliseconds — instead of waiting for the entire model to be read into RAM. The first inference pays the page-in cost incrementally as the engine touches weights.
 - **Faster warm reloads.** After the first load, the kernel's page cache holds the model's hot pages. Re-creating a runner on the same model (e.g. after a background termination and relaunch within the same boot) is near-instant — pages stream from the page cache, not disk.
 - **Multi-model sharing.** Two processes (or two runners in one process) loading the same model file share physical pages via the page cache, with no extra RAM cost.
@@ -92,7 +97,7 @@
 
 **Trade-offs:**
 
 - **First-token latency on cold pages.** The first generation against a freshly-mapped model triggers page faults as the engine walks the weights. This adds disk-I/O latency to TTFT on the first call after process start. The KV cache reuse documented [above](#kv-cache-reuse-across-generations) compounds well here: cached prefixes skip both prefill compute *and* the page-fault cost for weights touched during prefill.
 - **Storage type matters.** On devices with slow eMMC / external SD storage, lazy page-in can be noticeably slower than the old eager-read flow that loaded the whole file once. Internal flash on every shipped iOS device and any modern Android device is fast enough that this isn't visible in practice.
 - **Opt-out available since v0.10.5.** Pass `useMmap = false` on `ModelLoadingOptions` (Kotlin) or `LiquidInferenceEngineOptions(useMmap: false)` (Swift) to force the legacy full-read loader. Use only when `mmap` misbehaves on the target filesystem; the default of `null`/`nil` keeps the engine default.
 
@@ -100,13 +105,13 @@
 
 ### KV cache reuse across generations
 
 `CacheOptions` (new in v0.10.4, ergonomic Swift surface in v0.10.4.3) lets the engine persist KV-cache data between `generateResponse` calls so requests that share a prompt prefix skip the prefill work for the shared tokens.
 
 <Warning>
 **Disabled by default.** `cacheOptions` is `nil` (Swift) / `null` (Kotlin) until you explicitly pass `LiquidCacheOptions.enabled(path:)` / `ModelLoadingOptions.cacheOptions(path:)`. Apps that don't opt in see no prefix reuse and no on-disk cache directory created — same behavior as 0.9.x and pre-0.10.4 builds.
 </Warning>
 
 **Why it matters.** Transformer inference has two phases: **prefill** (compute keys and values for every prompt token) and **decode** (generate one new token at a time, reusing those K/V vectors). On mobile, prefill dominates time-to-first-token for any prompt longer than a few hundred tokens. With `CacheOptions` enabled, a previously seen prefix is read from disk instead of recomputed — TTFT can drop from seconds to under a hundred milliseconds on cache hits. Per-token decode cost is unchanged.
 
 **When it speeds things up.** Anywhere the same tokens appear at the start of many requests:
 
@@ -114,7 +119,7 @@
 - **RAG / retrieval-augmented generation.** Many queries share the retrieved-document preamble.
 - **Few-shot prompting.** A fixed set of examples precedes every request.
 - **Agent loops.** Tool definitions, role instructions, and task scaffold are stable across iterations.
 - **Voice assistant continuations.** Conversation history grows; everything before the latest user turn is cacheable.
 
 The cache is a bounded LRU — the engine caps cache size and evicts the least-recently-used entries automatically; you do not need to manage the directory yourself. See [Model Loading → KV cache reuse](/deployment/on-device/sdk/model-loading#kv-cache-reuse) for the per-platform configuration.
 
@@ -153,14 +158,14 @@
 
 **New targets on `leap-openai-client`** ([PR #256](https://github.com/Liquid4All/leap-android-sdk/pull/256)):
 
 - **`jvm`** (Ktor CIO engine) — Maven Central now publishes `ai.liquid.leap:leap-openai-client-jvm:0.10.7`. Pure-JVM desktop / server apps can route OpenAI-compatible chat completions without dragging in Android or KMP targets. (The 0.10.0 — 0.10.6 SPM cascade only shipped Android + Apple + Linux/MinGW K/N + wasmJs metadata; the JVM slice was absent.)
 - **`wasmJs`** (Ktor Js engine) — browser-side chat-completions client matching what `leap-sdk` already targets.
 
 The Apple slice (`LeapOpenAIClient.xcframework`) ships unchanged — same SSE-stream surface, same `OpenAiClientConfig`, same OpenRouter extra-headers support. SKIE is still not applied to this module in v0.10.7, so the Kotlin/Native exports remain the same as v0.10.6: `Flow<ChatCompletionEvent>` is not bridged to Swift `AsyncSequence`, and `onEnum(of:)` is not generated for `ChatCompletionEvent`. **The next release will enable SKIE on `leap-sdk-openai-client`**, bringing `for try await` over the stream, exhaustive `onEnum(of:)` switching, and SKIE-bundled Swift convenience inits — see the [OpenAI client page](/deployment/on-device/sdk/openai-client) for the current pinning guidance.
 
 **Bytecode hardening:**
 
 - The `leap-sdk-jvm`, `leap-openai-client-jvm`, `leap-ui-jvm`, and `leap-ui-android` artifacts had been silently shipping Java 17 / Java 21 bytecode against the project's stated JVM-target-11 stance. All ten published JVM / Android slices now consistently emit class-file major version `0x37` (Java 11). Consumers running on JDK 11 — particularly long-running services and JDK-11-pinned Android Gradle builds — are no longer at risk of `UnsupportedClassVersionError`.
 
 **Internal: KMP build centralization** (no consumer-visible API change):
 
@@ -173,7 +178,7 @@
 
 **iOS surface (unchanged from v0.10.6):**
 
 The four XCFrameworks (`LeapSDK`, `LeapModelDownloader`, `LeapOpenAIClient`, `LeapUi`) ship the same Swift APIs as v0.10.6. The v0.10.6 ObjC class rename to `ModelDownloader`, the dual-import guard, the dynamic `LeapModelDownloader` framework, and the `LeapDownloaderConfig()` parameterless init all remain in place.
 
 ### v0.10.6 — 2026-05-12
 
@@ -181,9 +186,9 @@
 
 **New iOS API on `ModelDownloader`:**
 
 - **`loadModel(modelName:, quantizationType:, options:, generationTimeParameters:, forceDownload:, downloadProgress:)`** — downloads (when needed) and loads in one call. The transfer registers in `queryStatus`, is cancellable via `requestStopDownload`, and continues across backgrounding when constructed with `sessionConfiguration: .backgroundSessionConfiguration(withIdentifier:)`.
 - **`loadModel(manifestUrl:, options:, generationTimeParameters:, forceDownload:, downloadProgress:)`** — same flow keyed by a manifest URL.
 - **`loadSimpleModel(model: ModelSource, options:, generationTimeParameters:, downloadProgress:)`** — sideload from explicit paths or URLs; HTTPS sources stream through `URLSession`, local paths pass straight through.
 - **`forceDownload: Bool = false`** on all three load methods. Resolves the manifest first, then deletes the local cache, then re-downloads — a registry failure on resolve leaves the previously-working cached copy intact.
 - **Resource-lookup helpers** that previously lived only on the cross-platform `LeapDownloader`: `getModelResourceFolder(...)`, `getCachedManifest(...)`, `getCachedFilePath(...)`, `resolve(...)`, `deleteModelFile(...)`.
 - **`requestDownloadModel(manifestUrl:, forceDownload:)`** overload for symmetry with `removeModel(manifestUrl:)` / `queryStatus(manifestUrl:)`.
@@ -197,18 +202,18 @@
   ```
 - **Parameter labels renamed across the iOS `ModelDownloader` surface** — `model:` / `quantization:` → `modelName:` / `quantizationType:` on every method that already existed: `downloadModel(...)`, `requestDownloadModel(...)`, `requestStopDownload(...)`, `queryStatus(...)`, `removeModel(...)`, `getModelSize(...)`. Every loader now uses the same labels across Swift and Kotlin — `ModelDownloader` (iOS, macOS), `LeapModelDownloader` (Android), and `LeapDownloader` (cross-platform) all share `modelName:` / `quantizationType:`.
 - **`LeapModelDownloader` SPM library product is now single-target.** It no longer bundles the `LeapSDK` target. Apps depending on this product must drop any direct `LeapSDK` SPM dependency from the same target — `import LeapModelDownloader` re-exports every LeapSDK Kotlin type (`Conversation`, `ModelRunner`, `ChatMessage`, `Leap`, the convenience extensions, …). Keeping both library products on the same target double-bundles the inference engine dylibs and triggers a build-time `#error` from the LMD umbrella header (see "dual-import guard" below); the `LeapUI` library product still bundles `LeapSDK` because LeapUI does not re-emit those types in its ObjC binding.
 - **`LeapModelDownloader.xcframework` is now a dynamic framework.** It was a static archive in 0.10.5. SPM applies Embed & Sign automatically; manual integrators must add the framework with "Embed & Sign" instead of "Do Not Embed". The shipped XCFramework now also bundles the inference engine dylibs (`libinference_engine.dylib`, `libinference_engine_llamacpp_backend.dylib`, `libie_zip.dylib`) under `Frameworks/` with an `@loader_path/Frameworks` LC_RPATH — consumers using LMD on its own no longer need `LeapSDK.framework/Frameworks` on their search path.
 - **Dual-import build-time guard.** LMD's umbrella header carries a `__has_include(<LeapSDK/LeapSDK.h>) && !defined(LEAP_DUAL_IMPORT_ALLOW)` check that fires `#error` at the consumer's preprocessing time when both `LeapSDK` and `LeapModelDownloader` frameworks are reachable in the same target. To opt out for legitimate combinations (e.g. transitive linkage via `LeapUI`), add `LEAP_DUAL_IMPORT_ALLOW=1` to `OTHER_CFLAGS`.
 
 **New Swift conveniences:**
 
 - **`ModelDownloader()`**, **`ModelDownloader(sessionConfiguration:)`**, **`ModelDownloader(config:)`** — Kotlin/Native ObjC export strips default-argument metadata, so 0.10.5 forced Swift callers to pass every parameter (and `LeapDownloaderConfig` has seven). These new SKIE-bundled convenience inits restore the parameterless / single-arg forms.
 - **`LeapDownloaderConfig()`** parameterless convenience init mirroring the Kotlin defaults (`saveDir = "leap_models"`, `validateSha256 = true`, etc.). Same rationale — `LeapDownloaderConfig` is a Kotlin `data class` with seven defaulted fields that the ObjC export couldn't carry through.
 
 **Behavior changes:**
 
 - **`requestDownloadModel(forceDownload: false)`** now short-circuits when a cached manifest already exists *and* every resource referenced by that manifest is present on disk — matches both the Android downloader's idempotent-call semantics and what `queryStatus(...)` already reports. Earlier 0.10.5 builds would short-circuit on the manifest alone, leaving the caller stuck if any resource file had been removed. Pass `forceDownload: true` to re-download on top of a cache.
 - **Cached-file lookup** uses Ktor URL parsing instead of substring slicing, so URLs with fragments or query strings now produce the same filename the loader expects (`getCachedFilePath` was previously brittle for those shapes).
 
 **Fixes:**
 
@@ -228,11 +233,11 @@
 
 - **`ModelLoadingOptions.cacheDir: String?` → `cacheOptions: EngineOptions.CacheOptions?`** — KV cache configuration moves to a bounded-LRU `CacheOptions` value with explicit `enabled` master switch, per-tier caps (`maxEntriesDisk`, `maxEntriesMemory`, `maxBytesMemory`), and optional `diskDisabled = true` for memory-only mode. Migrate via the `ModelLoadingOptions.cacheOptions(path = ...)` factory (preserves the historical 40-entry disk budget and sets `enabled = true`). Constructing a raw `CacheOptions` requires `enabled = true` to enable the cache — a positive `maxEntries` alone is no longer sufficient.
 - **`LeapDownloader.loadModel(modelName, quantizationSlug, modelLoadingOptions, …)` → `loadModel(modelName, quantizationType, options, …)`** — parameter renames bring `LeapDownloader` in line with `LeapModelDownloader`. The same rename applies to `loadSimpleModel(model, modelLoadingOptions, …)` → `loadSimpleModel(model, options, …)` and `loadModelFromManifestUrl(…)`. Swift sites that called `downloader.loadModel(modelName:, quantizationSlug:, modelLoadingOptions:)` need to swap to `quantizationType:` / `options:` after upgrading.
 - **`progress` is now nullable** (`progress: ((ProgressData) -> Unit)? = null`) — pass `null` to opt out (was an empty-lambda default).
 
 **New features:**
 
 - **`ModelLoadingOptions.useMmap: Boolean? = null`** — exposes the engine's `use_mmap` toggle to Kotlin/Swift callers. `null` (default) defers to the engine default of `true`. Set `false` only on filesystems where `mmap` misbehaves (some Android scoped-storage paths, certain network mounts). On Swift, `LiquidInferenceEngineOptions` gained a matching `.with(useMmap:)` builder. Previously mmap could not be disabled from the SDK.
 - **Leap Model Service (Android)** — `leap-model-service` is a new optional Android service that hosts loaded models in its own process and lets multiple client apps share them. Apps using `LeapModelDownloader.loadModel(...)` route through the service transparently when it's installed on the device; otherwise they fall back to in-process loading. Per-UID session quotas, persistent foreground notification, disk-backed KV cache reuse across cold starts, and AIDL-routed `registerFunction(s)`. Pass `forceLocal = true` on `LeapModelDownloader.loadModel(...)` to bypass the service for testing. See [Model Loading](/deployment/on-device/sdk/model-loading) for the routing model.
 - **Service-side load progress** — when routing through the model service, `LeapModelDownloader.loadModel`'s `progress` callback now fires for service-side downloads too (was previously local-path-only).
 
@@ -261,7 +266,7 @@
 - New `with(cacheOptions:)` builders on `LiquidInferenceEngineOptions` and `LiquidInferenceEngineManifestOptions`.
 - New `LiquidCacheOptions.enabled(path:)` static factory — Swift analog of `ModelLoadingOptions.cacheOptions(path:)`.
 
 (v0.10.4.2 was staged to Sonatype but never released; superseded by 0.10.4.3.)
 
 ### v0.10.4.1 — 2026-05-07
 
@@ -269,10 +274,10 @@
 
 ### v0.10.4 — 2026-05-06
 
 - **Bounded-LRU `CacheOptions` API** across JVM, Android, Kotlin/Native, Apple, and wasmJs.
 - **`use_mmap=true` is now the engine default** (via vendored IE pin `v26.02.1-79+`). Model weights are memory-mapped instead of `read(2)`-ed into a heap buffer. See [Memory-mapped model loading by default](#memory-mapped-model-loading-by-default) above for the mobile performance impact.
 - K/N Linux link fix (`--allow-shlib-undefined` for `libinference_engine.so` against modern glibc).
 - Dynamic vendor pipeline + `DT_NEEDED`-based shipped-libs verify; `inference_engine` RUNPATH=`$ORIGIN` cascade for Linux/Windows shared vendor libs.
 - `NativeLibLoader` cross-platform load fixes (resource extraction + Windows pre-load topo-retry).
 - Three release-gate smokes (Linux K/N, Apple SwiftPM consumer, Windows JVM) wired into CI.
 
@@ -281,9 +286,9 @@
 Additive fix release for Linux/MinGW Kotlin/Native consumers. Apple/SPM consumers see no API or behavior changes vs v0.10.0.
 
 - `leap-sdk` Linux/MinGW K/N artifacts on Maven Central now publish a `-natives.zip` classifier containing the runtime `.so`/`.dll` libraries.
 - New `ai.liquid.leap.nativelibs` Gradle plugin auto-wires the natives ZIP into consumer K/N executables.
 - `leap-openai-client` now publishes Linux/MinGW K/N klibs.
 
 ### v0.10.0 — 2026-04-28
 
 Initial Kotlin Multiplatform unification release. See [the 0.9.x → 0.10.x section above](#0-9-x-0-10-x-kotlin-multiplatform-unification) for the full migration story.

deployment/on-device/llama-cpp.mdx

@@ -7,11 +7,15 @@
   Use llama.cpp for CPU-only environments, local development, or edge deployment and on-device inference.
 </Tip>
 
+<Note>
+  Building an application with llama.cpp? Jump to [Build with llama.cpp](#building-applications) for mobile, desktop, chat, tool-calling, structured-output, and multimodal guides.
+</Note>
+
 For GPU-accelerated inference at scale, consider using [vLLM](/deployment/gpu-inference/vllm) instead.
 
 <div className="colab-link">
   <a href="https://colab.research.google.com/github/Liquid4All/docs/blob/main/notebooks/LFM2_Inference_with_llama_cpp.ipynb" target="_blank">
     <img src="https://colab.research.google.com/assets/colab-badge.svg" alt="Open in Colab" />
   </a>
 </div>
 
@@ -88,8 +92,8 @@
     ```bash
     git clone https://github.com/ggml-org/llama.cpp
     cd llama.cpp
     cmake -B build
     cmake --build build --config Release -j 8
     ```
 
     The compiled programs will be in `./build/bin/`.
@@ -211,7 +215,7 @@
 Control text generation behavior using parameters in the OpenAI-compatible API or command-line flags. Key parameters:
 
 * **`temperature`** (`float`): Controls randomness (0.0 = deterministic, higher = more random). Typical range: 0.1-2.0
 * **`top_p`** (`float`): Nucleus sampling - limits to tokens with cumulative probability ≤ top\_p. Typical range: 0.1-1.0
 * **`top_k`** (`int`): Limits to top-k most probable tokens. Typical range: 1-100
 * **`min_p`** (`float`): Filters tokens below `min_p * max_probability`. Typical range: 0.05-0.3
 * **`max_tokens`** / **`--n-predict`** (`int`): Maximum number of tokens to generate
@@ -243,7 +247,7 @@
 
 ## Vision Models
 
 LFM2-VL GGUF models can be used for multimodal inference with llama.cpp.
 
 ### Quick Start with llama-cli
 
@@ -303,7 +307,7 @@
 </Accordion>
 
 <Accordion title="Using llama-server">
   Start a vision model server with both the model and mmproj files:
 
   ```bash
   llama-server \
@@ -349,12 +353,12 @@
 </Accordion>
 
 <Info>
 For a complete working example with step-by-step instructions, see the [llama.cpp Vision Model Colab notebook](https://colab.research.google.com/drive/1q2PjE6O_AahakRlkTNJGYL32MsdUcj7b?usp=sharing).
 </Info>
 
 ## Converting Custom Models
 
 If you have a finetuned model or need to create a GGUF from a Hugging Face model:
 
 ```bash
 # Clone llama.cpp if you haven't already
@@ -367,6 +371,31 @@
 
 Use `--outtype` to specify the quantization level (e.g., `q4_0`, `q4_k_m`, `q5_k_m`, `q6_k`, `q8_0`, `f16`).
 
+## Building Applications
+
+The guides below cover embedding llama.cpp in your own software — from a mobile app calling the C API to a desktop app driving `llama-server`:
+
+<CardGroup cols={2}>
+  <Card title="iOS & Android" icon="mobile" href="/deployment/on-device/llama-cpp/mobile">
+    Link the XCFramework or NDK build and run LFM GGUFs in-process.
+  </Card>
+  <Card title="Desktop & Server Apps" icon="desktop" href="/deployment/on-device/llama-cpp/desktop">
+    llama-server as a sidecar, or Python / Node.js / .NET bindings.
+  </Card>
+  <Card title="Chat & Streaming" icon="comments" href="/deployment/on-device/llama-cpp/chat">
+    Multi-turn conversations, sampling parameters, prompt caching.
+  </Card>
+  <Card title="Function Calling & Agents" icon="wrench" href="/deployment/on-device/llama-cpp/function-calling">
+    OpenAI-style tools with native LFM2.5 tool-call parsing.
+  </Card>
+  <Card title="Structured Output" icon="brackets-curly" href="/deployment/on-device/llama-cpp/structured-output">
+    JSON schema and GBNF grammar constrained generation.
+  </Card>
+  <Card title="Vision & Audio" icon="image" href="/deployment/on-device/llama-cpp/multimodal">
+    LFM2.5-VL and LFM2.5-Audio on llama.cpp.
+  </Card>
+</CardGroup>
+
 ## Example Applications
 
 For more comprehensive example applications using llama.cpp with LFM models, check out these repositories:

deployment/on-device/llama-cpp/chat.mdx

@@ -0,0 +1,210 @@
+---
+title: "Chat & Streaming"
+description: "Multi-turn conversations on llama.cpp with streaming, the correct sampling parameters for each LFM family, and prompt caching."
+---
+
+The quickest way to build a chat experience on llama.cpp is `llama-server`. It applies the model's chat template, exposes an OpenAI-compatible `/v1/chat/completions` endpoint, streams tokens over server-sent events, and reuses the KV cache across turns. Any OpenAI client library becomes your app-side API. For in-process use (mobile, or a desktop app that must not spawn a helper process) the same loop is a few dozen lines of the C API — see [Native C API](#native-c-api).
+
+## Start the server
+
+```bash
+llama-server -hf LiquidAI/LFM2.5-1.2B-Instruct-GGUF:Q4_K_M -c 4096 --port 8080 --jinja
+```
+
+- `-hf` downloads the GGUF from Hugging Face on first run; `:Q4_K_M` selects the quantization. Use `-m path/to/model.gguf` for a local file.
+- `-c` is the context length. Larger contexts cost memory linearly; check the model page for the supported maximum.
+- `--jinja` uses the chat template embedded in the GGUF. It is required for [tool calling](/deployment/on-device/llama-cpp/function-calling) and recommended for everything else.
+- `-ngl 99` offloads all layers to the GPU (Metal, CUDA, Vulkan) when one is available.
+- `-np 4` serves up to four requests concurrently; each slot gets `-c / 4` tokens of context.
+
+`GET /health` returns `{"status":"ok"}` once the model is loaded.
+
+## Sampling parameters
+
+Every LFM checkpoint has validated sampling defaults. Use them; placeholder values such as `temperature=0.7` degrade output quality.
+
+| Model family | `temperature` | `top_k` | `top_p` | `min_p` | `repeat_penalty` |
+|---|---|---|---|---|---|
+| LFM2.5-1.2B-Instruct | 0.1 | 50 | — | — | 1.05 |
+| LFM2.5-1.2B-Thinking | 0.1 | 50 | 0.1 | — | 1.05 |
+| LFM2.5-8B-A1B | 0.2 | 80 | — | — | 1.05 |
+| LFM2-24B-A2B | 0.1 | 50 | — | — | 1.05 |
+| LFM2 text, LFM2.5-JP | 0.3 | — | — | 0.15 | 1.05 |
+| LFM2-VL, LFM2.5-VL | 0.1 | — | — | 0.15 | 1.05 |
+
+<Note>
+`llama-server` reads the penalty as **`repeat_penalty`** in the request body (matching the `--repeat-penalty` CLI flag). `top_k`, `min_p`, and `repeat_penalty` are not part of the OpenAI schema, so pass them through `extra_body` in the OpenAI Python client. The exact values for any checkpoint are on its Hugging Face model card.
+</Note>
+
+## Send a message
+
+<Tabs>
+  <Tab title="Python (OpenAI client)">
+    ```python
+    from openai import OpenAI
+
+    client = OpenAI(base_url="http://localhost:8080/v1", api_key="not-needed")
+
+    response = client.chat.completions.create(
+        model="lfm2.5-1.2b-instruct",   # any string; llama-server serves one model
+        messages=[
+            {"role": "system", "content": "You are a helpful assistant."},
+            {"role": "user", "content": "What is machine learning?"},
+        ],
+        temperature=0.1,
+        max_tokens=512,
+        extra_body={"top_k": 50, "repeat_penalty": 1.05},
+    )
+    print(response.choices[0].message.content)
+    ```
+  </Tab>
+  <Tab title="curl">
+    ```bash
+    curl http://localhost:8080/v1/chat/completions \
+      -H "Content-Type: application/json" \
+      -d '{
+        "messages": [
+          {"role": "system", "content": "You are a helpful assistant."},
+          {"role": "user", "content": "What is machine learning?"}
+        ],
+        "temperature": 0.1,
+        "top_k": 50,
+        "repeat_penalty": 1.05,
+        "max_tokens": 512
+      }'
+    ```
+  </Tab>
+  <Tab title="JavaScript (fetch)">
+    ```javascript
+    const res = await fetch("http://localhost:8080/v1/chat/completions", {
+      method: "POST",
+      headers: { "Content-Type": "application/json" },
+      body: JSON.stringify({
+        messages: [
+          { role: "system", content: "You are a helpful assistant." },
+          { role: "user", content: "What is machine learning?" },
+        ],
+        temperature: 0.1,
+        top_k: 50,
+        repeat_penalty: 1.05,
+        max_tokens: 512,
+      }),
+    });
+    const data = await res.json();
+    console.log(data.choices[0].message.content);
+    ```
+  </Tab>
+</Tabs>
+
+## Stream tokens
+
+Set `stream: true` and consume the `delta.content` chunks as they arrive.
+
+<Tabs>
+  <Tab title="Python (OpenAI client)">
+    ```python
+    stream = client.chat.completions.create(
+        model="lfm2.5-1.2b-instruct",
+        messages=[{"role": "user", "content": "Write a haiku about the ocean."}],
+        temperature=0.1,
+        max_tokens=256,
+        extra_body={"top_k": 50, "repeat_penalty": 1.05},
+        stream=True,
+    )
+    for chunk in stream:
+        delta = chunk.choices[0].delta.content
+        if delta:
+            print(delta, end="", flush=True)
+    ```
+  </Tab>
+  <Tab title="JavaScript (fetch + SSE)">
+    ```javascript
+    const res = await fetch("http://localhost:8080/v1/chat/completions", {
+      method: "POST",
+      headers: { "Content-Type": "application/json" },
+      body: JSON.stringify({
+        messages: [{ role: "user", content: "Write a haiku about the ocean." }],
+        temperature: 0.1, top_k: 50, repeat_penalty: 1.05, max_tokens: 256,
+        stream: true,
+      }),
+    });
+
+    const reader = res.body.getReader();
+    const decoder = new TextDecoder();
+    let buffer = "";
+    while (true) {
+      const { value, done } = await reader.read();
+      if (done) break;
+      buffer += decoder.decode(value, { stream: true });
+      const lines = buffer.split("\n");
+      buffer = lines.pop();
+      for (const line of lines) {
+        if (!line.startsWith("data: ") || line === "data: [DONE]") continue;
+        const delta = JSON.parse(line.slice(6)).choices[0].delta.content;
+        if (delta) process.stdout.write(delta);
+      }
+    }
+    ```
+  </Tab>
+</Tabs>
+
+## Multi-turn conversations
+
+The API is stateless: send the whole `messages` history on every request and append the assistant's reply before the next turn.
+
+```python
+messages = [{"role": "system", "content": "You are a helpful assistant."}]
+
+def ask(user_text: str) -> str:
+    messages.append({"role": "user", "content": user_text})
+    response = client.chat.completions.create(
+        model="lfm2.5-1.2b-instruct",
+        messages=messages,
+        temperature=0.1,
+        max_tokens=512,
+        extra_body={"top_k": 50, "repeat_penalty": 1.05},
+    )
+    reply = response.choices[0].message.content
+    messages.append({"role": "assistant", "content": reply})
+    return reply
+
+print(ask("My name is Ada."))
+print(ask("What is my name?"))
+```
+
+Re-sending the history is cheap: `llama-server` keeps the KV cache of the previous request in its slot and only prefills the new suffix (`cache_prompt` defaults to `true`). Two flags make this more effective:
+
+- `--cache-reuse 256` — reuse cached chunks even when an earlier part of the prompt changed (for example, a trimmed history), by shifting KV entries instead of recomputing them.
+- `-np N` with `--slot-prompt-similarity` — with multiple slots, route each request to the slot whose cached prompt matches best; useful when several users share one server.
+
+To trim history, drop the oldest user/assistant pairs but keep the system message first. Long system prompts and RAG preambles are exactly what the prompt cache is for — keep them byte-identical across requests so the prefix stays cached.
+
+## Generation controls
+
+| Setting | Request field | CLI flag | Notes |
+|---|---|---|---|
+| Max new tokens | `max_tokens` | `-n` | Also `n_predict` on `/completion`. `-1` = until end-of-generation. |
+| Stop strings | `stop` | `-r` | Array of strings that end generation. |
+| Deterministic runs | `seed` | `-s` | Fixed seed + `temperature: 0` gives reproducible output on the same build/hardware. |
+| Per-request cache | `cache_prompt` | `--cache-prompt` | Default `true`. |
+| Timing info | `timings_per_token` | — | Adds prompt/decode timings to streamed chunks. |
+
+## Native C API
+
+In-process, the conversation loop is the same as in [iOS & Android](/deployment/on-device/llama-cpp/mobile#3-load-the-model-and-stream-a-response), with three additions:
+
+1. **Format only the new suffix.** Keep the message history and the formatted prompt string. For each turn, format the full history with the [LFM2 chat template](/lfm/key-concepts/chat-template) and tokenize only the part that was not already decoded. The KV cache still holds the earlier tokens, so prefill cost is proportional to the new turn.
+2. **Close the assistant turn.** The loop stops when `llama_vocab_is_eog()` fires, *before* the end token is decoded. Feed the tokens for `<|im_end|>\n` (with `parse_special = true`) after each reply so the cache matches what the template produces on the next turn.
+3. **Reset with `llama_memory_clear(llama_get_memory(ctx), true)`** to start a new conversation without reloading the model.
+
+```cpp
+// Per turn, after appending the user message to `history`:
+std::string suffix = format_lfm2(history) /* full template */ .substr(n_chars_already_decoded);
+std::vector<llama_token> toks = tokenize(vocab, suffix, /*add_special=*/false, /*parse_special=*/true);
+llama_decode(ctx, llama_batch_get_one(toks.data(), toks.size()));
+// ... sample until llama_vocab_is_eog(vocab, tok), streaming pieces ...
+std::vector<llama_token> close = tokenize(vocab, "<|im_end|>\n", false, true);
+llama_decode(ctx, llama_batch_get_one(close.data(), close.size()));
+```
+
+[`examples/simple-chat/simple-chat.cpp`](https://github.com/ggml-org/llama.cpp/blob/master/examples/simple-chat/simple-chat.cpp) is the reference implementation of this pattern. If you need the full Jinja template (for example, tool definitions), link the `common` library and use `common_chat_templates_init()` / `common_chat_templates_apply()` from [`common/chat.h`](https://github.com/ggml-org/llama.cpp/blob/master/common/chat.h) instead of formatting by hand.