From b34b41e0768d4d37f01f26bbeeda1dcc85f1570f Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Wed, 11 Nov 2026 00:04:46 -0400 Subject: [PATCH 1/3] tools: npu_serve_smoke, the release gate's check that npu serve is on the fast route at speed (refs #2106, #2102) A serve that falls back to the per-op forward still answers, so a check that asks 'did it answer' passes on the wrong route. This starts 1bit serve --device npu, sends three prompts with thinking off, and asserts the route's own log line, a decode floor (60 tok/s), the first token (600 ms) and non-empty, non-degenerate content. Measured, Qwen3-0.6B, one npu-hold ticket each: the 2026-11-07 release candidate PASSes (median 84.4 tok/s, 277 ms); a main build without the fast route FAILs route, rate (1.9 tok/s) and first-token while its answers are fine. --self-test drives the judging on 15 shapes (the silent fallback, thinking left on, a degenerate reply, the median, missing timings); registered in ctest as npu_serve_smoke_self. Co-Authored-By: Claude Fable 5.2 --- CMakeLists.txt | 4 + docs/hw-gate.md | 38 ++++++ tools/npu_serve_smoke.py | 276 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 318 insertions(+) create mode 100755 tools/npu_serve_smoke.py diff --git a/CMakeLists.txt b/CMakeLists.txt index b6b578068..afad6a10f 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -363,6 +363,10 @@ add_python_test(hf_tensor_self ${CMAKE_SOURCE_DIR}/tools/hf_tensor.py --self-tes # `tools/ci/ctest_skips.txt`, and `(Not Run)` (a missing binary) as a build-list mistake that can never be baselined. # Self-test only here: wiring it into ci.yml's Test step needs the log plus a `workflow`-scoped token (engine#1817). add_python_test(ctest_skips_self ${CMAKE_SOURCE_DIR}/tools/ci/ctest_skips.py --self-test) +# engine#2106 / #2102: the release gate's NPU serve check asserts the fast route, a decode floor, the first token and a +# non-empty answer, because a serve on the slow per-op route still answers. --self-test drives the judging on the shape +# of each failure (the silent fallback at 1.9 tok/s, thinking left on, a degenerate reply); no device, no binary. +add_python_test(npu_serve_smoke_self ${CMAKE_SOURCE_DIR}/tools/npu_serve_smoke.py --self-test) add_python_test(moe_placement ${CMAKE_SOURCE_DIR}/tools/moe_placement.py --self-test) # engine#856: one HRX process's GTT split into the loader's copy and HRX's packed second copy, from its own -v log. add_python_test(hrx_gtt_split_self ${CMAKE_SOURCE_DIR}/tools/hrx_gtt_split.py --self-test) diff --git a/docs/hw-gate.md b/docs/hw-gate.md index 67402e213..fb79f073c 100644 --- a/docs/hw-gate.md +++ b/docs/hw-gate.md @@ -252,6 +252,44 @@ The settings are environment variables at the top of the script: `HW_GATE_STORE` `$HW_GATE_OUT` (default `~/.cache/hw-gate/runs/