From 7aa4c7dcdca340df55d9ca4d756010fc554a95f2 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Thu, 5 Nov 2026 15:36:00 -0400 Subject: [PATCH 1/2] serve: pin the two chat models the stick actually ships, from their measured templates MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `config/template-pins.json` pins 7 names, and none of them is a row the appliance itself serves: the stick's staged marker is `Qwen3-4B-Instruct-2507-1bit` (unsloth/Qwen3-4B-Instruct-2507-GGUF, `Qwen3-4B-Instruct-2507-Q4_K_M.gguf`) and `Qwen3-0.6B-NPU-1bit` (the FastFlowLM/Qwen3-0.6B-NPU2 container) — the last row of ci.yml's appliance table and the one a stick image boots with. So on the appliance's own default chat row, `--template-pin-strict` checked nothing: an unpinned model is logged, not refused (docs/serve.md), and the owner's open decision "flip the serve rows to strict" would have bought the appliance's model nothing. Four rows added, keyed on both names a row can be served under (the catalog id that becomes `--alias` and the file/dir name), with digests measured rather than guessed: qwen3-4b-instruct-2507-1bit c979e0e71a3e21b8… (the staged qwen3-4b-instruct-2507-q4_k_m.gguf c979e0e71a3e21b8… Qwen3-4B-Instruct-2507-Q4_K_M.gguf on the c26 staging tree; equal to the digest models/pins.json records for it) qwen3-0.6b-npu-1bit a136047ba140347b… (the container's tokenizer_config.json qwen3-0.6b-npu2 a136047ba140347b… `chat_template` string — the digest #1095's serve line printed on a real container, not the JSON file's own) `tests/template_pin.sh` gains the case that makes the claim checkable: a fixture named `qwen3-4b-instruct-2507-q4_k_m.gguf` whose template is not the pinned one must warn with that digest. Verified in both directions — against the current main build (wt/onebp-zyphra/build-main-merged/1bit, built 11:56 today from the merged #1089/#1095) every existing case passes and **the new case fails**, so it detects the missing row; `cmake -S . -B /tmp/tp-cfg` then generates `generated/template_pins_default.h` containing the new names, which is what the binary compiles in. CI runs the suite (`add_test(NAME template_pin ...)`, ci.yml `ctest`), so the pass is proven there. --- config/template-pins.json | 16 ++++++++++++++++ docs/serve.md | 6 ++++++ tests/template_pin.sh | 10 +++++++++- 3 files changed, 31 insertions(+), 1 deletion(-) diff --git a/config/template-pins.json b/config/template-pins.json index 26273a27b..84a6ac282 100644 --- a/config/template-pins.json +++ b/config/template-pins.json @@ -1,5 +1,21 @@ { "pins": { + "qwen3-0.6b-npu-1bit": { + "sha256": "a136047ba140347b359d17b6ff8fa1bf8cb29d23f0c07bef7e0bf479b8680793", + "source": "FastFlowLM/Qwen3-0.6B-NPU2" + }, + "qwen3-0.6b-npu2": { + "sha256": "a136047ba140347b359d17b6ff8fa1bf8cb29d23f0c07bef7e0bf479b8680793", + "source": "FastFlowLM/Qwen3-0.6B-NPU2" + }, + "qwen3-4b-instruct-2507-1bit": { + "sha256": "c979e0e71a3e21b8f208e6ab120d5cb29327885f29d2a8b18fda67a723798e18", + "source": "unsloth/Qwen3-4B-Instruct-2507-GGUF" + }, + "qwen3-4b-instruct-2507-q4_k_m.gguf": { + "sha256": "c979e0e71a3e21b8f208e6ab120d5cb29327885f29d2a8b18fda67a723798e18", + "source": "unsloth/Qwen3-4B-Instruct-2507-GGUF" + }, "qwen3-coder-30b-a3b-instruct-q4_k_m.gguf": { "sha256": "87710339d25b4e789c1d723f93c91ee861a86d305bb3d20a845536f251d6ea8a", "source": "Qwen/Qwen3-Coder-30B-A3B-Instruct" diff --git a/docs/serve.md b/docs/serve.md index ff49c003a..a185d7e3b 100644 --- a/docs/serve.md +++ b/docs/serve.md @@ -796,6 +796,12 @@ says nothing, because nothing runs (this keeps the header-only route fixtures on {"pins": {"zaya1-8b-q4_k_m.gguf": {"sha256": "b55ecfbbbcbb238a…", "source": "Zyphra/ZAYA1-8B"}}} ``` +The table carries the rows the appliance itself ships — the stick's `Qwen3-4B-Instruct-2507-1bit` GGUF and +its `Qwen3-0.6B-NPU-1bit` container, whose digest is the `chat_template` string from +`tokenizer_config.json` ([npu-chat-template.md](npu-chat-template.md)) — plus the demo and scoreboard rows. +A model with no row is logged, not checked, which is why a row is added with its measured digest rather +than guessed (`models/pins.json` records that digest for every file we serve and publish). + A pin is matched against the model id serve serves (`--alias`, i.e. Lemonade's catalog entry) first, then the file's own name, case-insensitively; `"none"` pins a model that must carry no template. A model whose pin does not match prints a second line and **still serves**: diff --git a/tests/template_pin.sh b/tests/template_pin.sh index 3a4563862..77982e46d 100755 --- a/tests/template_pin.sh +++ b/tests/template_pin.sh @@ -53,7 +53,9 @@ base = [st("general.architecture", "qwen3"), u32("qwen3.block_count", 28), u32(" u32("qwen3.embedding_length", 1024), u32("qwen3.attention.head_count", 16), u32("qwen3.attention.head_count_kv", 8)] for name, kv in (("tpl", [st("tokenizer.chat_template", sys.argv[2])]), ("bare", []), - ("zaya1-8b-q4_k_m", [st("tokenizer.chat_template", "{% for m in messages %}{{ m }}{% endfor %}")])): + ("zaya1-8b-q4_k_m", [st("tokenizer.chat_template", "{% for m in messages %}{{ m }}{% endfor %}")]), + ("qwen3-4b-instruct-2507-q4_k_m", + [st("tokenizer.chat_template", "{% for m in messages %}{{ m }}{% endfor %}")])): all_kv = base + kv with open(f"{sys.argv[1]}/{name}.gguf", "wb") as f: f.write(b"GGUF" + struct.pack(" Date: Thu, 5 Nov 2026 15:37:23 -0400 Subject: [PATCH 2/2] serve: pin the laya-first `short` row too, so all four routed rows carry an expectation MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The router's four classes are the one place a model is pulled and then served by name (docs/laya-first.md). Three of them were pinned by file name; `short` — Qwen3-0.6B-Q8_0.gguf, digest 57f1fd00f0013a2b… in models/pins.json — was not, so the smallest model in the routing table was the one a strict pin could not refuse. All four now agree with models/pins.json's measured digests. Doc: the coverage sentence says so, instead of listing the stick plus demo/scoreboard. --- config/template-pins.json | 4 ++++ docs/serve.md | 2 +- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/config/template-pins.json b/config/template-pins.json index 84a6ac282..62f5ad5b6 100644 --- a/config/template-pins.json +++ b/config/template-pins.json @@ -8,6 +8,10 @@ "sha256": "a136047ba140347b359d17b6ff8fa1bf8cb29d23f0c07bef7e0bf479b8680793", "source": "FastFlowLM/Qwen3-0.6B-NPU2" }, + "qwen3-0.6b-q8_0.gguf": { + "sha256": "57f1fd00f0013a2be96aa79b857391f27e23df5b5f847072b524c897e24d0361", + "source": "1bit-MONSTER/Qwen3-0.6B-GGUF" + }, "qwen3-4b-instruct-2507-1bit": { "sha256": "c979e0e71a3e21b8f208e6ab120d5cb29327885f29d2a8b18fda67a723798e18", "source": "unsloth/Qwen3-4B-Instruct-2507-GGUF" diff --git a/docs/serve.md b/docs/serve.md index a185d7e3b..9ec19fab9 100644 --- a/docs/serve.md +++ b/docs/serve.md @@ -798,7 +798,7 @@ says nothing, because nothing runs (this keeps the header-only route fixtures on The table carries the rows the appliance itself ships — the stick's `Qwen3-4B-Instruct-2507-1bit` GGUF and its `Qwen3-0.6B-NPU-1bit` container, whose digest is the `chat_template` string from -`tokenizer_config.json` ([npu-chat-template.md](npu-chat-template.md)) — plus the demo and scoreboard rows. +`tokenizer_config.json` ([npu-chat-template.md](npu-chat-template.md)) — plus the demo and scoreboard rows and all four `laya-first` routing rows. A model with no row is logged, not checked, which is why a row is added with its measured digest rather than guessed (`models/pins.json` records that digest for every file we serve and publish).