From 06f814008197cbc176c85a29eb4946666609002f Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Sun, 15 Nov 2026 06:31:57 -0400 Subject: [PATCH] llama-context: a DFlash draft with no target context fails to create, not aborts (engine 27B --dflash) llama-server's memory fit measures the speculative draft as an extra model (common_fit_params, from the #2138 merge). That probe builds the draft's context with no ctx_other. #681 lets EAGLE3/DFLASH/QWEN4EXP drafts through with a warning so fitting can measure them, but the DFlash graph takes the target's tok_embd/output through ctx_other and asserts on it (src/models/dflash.cpp:944) while sched_reserve builds it. GGML_ASSERT aborts, so the fitter's catch-and-fit-alone never runs, and `1bit serve --dflash` on Qwen3.8-27B + DFlash2 q8_0 exits before it is ready. A DFlash draft without its own tok_embd/output now throws in the context constructor, as Gemma4Assistant already does. llama_init_from_model returns nullptr and the fitter fits the target alone. The real draft context (common_speculative_init) passes ctx_other and is unchanged. EAGLE3 and QWEN4EXP keep #681's warn-and-continue. Co-Authored-By: Claude Opus 5.5 --- src/llama-context.cpp | 7 +++++++ 1 file changed, 7 insertions(+) diff --git a/src/llama-context.cpp b/src/llama-context.cpp index ba0a26639c80..77a2e4f97a8e 100644 --- a/src/llama-context.cpp +++ b/src/llama-context.cpp @@ -249,6 +249,13 @@ llama_context::llama_context( if (model.arch == LLM_ARCH_EAGLE3 || model.arch == LLM_ARCH_DFLASH || model.arch == LLM_ARCH_QWEN4EXP) { if (model.tok_embd == nullptr || model.output == nullptr) { + if (params.ctx_other == nullptr && model.arch == LLM_ARCH_DFLASH) { + // A DFlash draft without its own tok_embd/output builds its graph from the target's tensors + // (dflash.cpp asserts ctx_other), so reserving it here would abort. The memory-fitting probe + // (common_fit_params' extra model) creates this context with no target; refuse it so the fitter + // falls back to fitting the target alone, as it does for any extra model it cannot measure. + throw std::runtime_error("DFlash draft without tok_embd/output needs ctx_other (normal during memory fitting)"); + } if (params.ctx_other == nullptr) { // engine #681: the memory-fitting probe has no target context. The real draft context sets ctx_other // in common_speculative_init; warn here so fitting can measure instead of refusing the draft head.