From 0c895f980add6c462cf5d92ff9945a4437680104 Mon Sep 17 00:00:00 2001 From: pi-zha Date: Sun, 8 Nov 2026 18:53:00 -0400 Subject: [PATCH 1/5] npu: define the three-argument step_batch_embeds, so a full build links (fixes #1573) q4nx_forward.h declares both overloads - the plain-text one and the HybridInputs one (position planes + deepstack) - but only the four-argument form was defined, so every full `cmake --build` on main stopped at 92% with undefined reference to Q4nxNpuForward::step_batch_embeds(std::vector const&, int, std::vector&) from npu/forward/embeds_equiv_probe.cpp, which is the file that exercises the plain-text path. The onebit target links because it does not pull the probe in, which is why this survived until a full build was run. The definition is a forwarder with a default HybridInputs: no position planes, no deepstack sets, image span unset - which is exactly what the three-argument form is documented to mean. Verified: embeds_equiv_probe now builds and, under NPU_INFER_HOST_GEMM=1 against the Qwen2.5-0.5B hybrid container and kernels-05b-hyb-b8, prints 'PASS (3/3 prompts identical)' - i.e. the overload that was missing actually works, it was only absent. --- npu/forward/src/q4nx_forward.cpp | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/npu/forward/src/q4nx_forward.cpp b/npu/forward/src/q4nx_forward.cpp index 49203e9c7..a0af1fb3e 100644 --- a/npu/forward/src/q4nx_forward.cpp +++ b/npu/forward/src/q4nx_forward.cpp @@ -3023,6 +3023,16 @@ void Q4nxNpuForward::npu_gemm_rows(int role, const float* A, int M, int K, int N // The folded container's sign table: magic "HAD1", u32 count, count x (u32 width, u32 offset), // then the f32 signs those offsets index (the GGUF's prism.hadamard.sign_widths/sign_values). +// The plain-text overload. It is declared in q4nx_forward.h and used by npu/forward/embeds_equiv_probe.cpp, but only the +// HybridInputs-taking definition existed, so every full build stopped at `undefined reference to +// Q4nxNpuForward::step_batch_embeds(std::vector, int, std::vector&)` — the onebit target links because it +// does not pull the probe in, which is why the breakage survived a release-ish build. Defaulting `in` is exactly the +// documented meaning of the three-argument form: no position planes, no deepstack sets, nothing vision-conditioned. +bool Q4nxNpuForward::step_batch_embeds(const std::vector& embeds, int start_pos, + std::vector& logits) { + return step_batch_embeds(embeds, start_pos, logits, HybridInputs{}); +} + bool Q4nxNpuForward::step_batch_embeds(const std::vector& embeds, int start_pos, std::vector& logits, const HybridInputs& in) { if (!ready_) { err_ = "not initialised"; return false; } From 26c525ddf2c6d0bd6fd58043c88df9ce9db11008 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Sun, 8 Nov 2026 19:21:35 -0400 Subject: [PATCH 2/5] ci: build embeds_equiv_probe, the target whose missing symbol reddened main main was red at 1571366 and nothing said so, because the CI build step is a fixed target list (.github/workflows/ci.yml) and `embeds_equiv_probe` is not on it, while `onebit` never pulls the probe in. The probe is defined at CMakeLists.txt and calls the three-argument `step_batch_embeds`, whose definition did not exist -- so a FULL build stopped at 92 % with `undefined reference to Q4nxNpuForward::step_batch_embeds(...)` and every CI job stayed green. One target added. It is a build, not a run: the probe needs a model directory and either the NPU or NPU_INFER_HOST_GEMM=1, so wiring it as a ctest is a separate decision; linking it is what catches this class. Verified both ways by hand at #1574's head and on main: main ninja embeds_equiv_probe -> FAILED, undefined reference to step_batch_embeds(...) #1574 head ninja embeds_equiv_probe -> [9/10] Linking CXX executable embeds_equiv_probe Stacked on #1574 (that is the fix it guards); it must land after it, in the same release. --- .github/workflows/ci.yml | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 023739756..d5a98af78 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -98,7 +98,8 @@ jobs: npu_model_bounds_test npu_forward_loader_test npu_design_grid_test npu_sampling_test \ npu_flm_head_import_test npu_elf_args_test \ npu_template_test npu_chatml_test \ - tokenizer_linear_test hrx_placement_test hrx_cwsr_test hrx_runtime_test npu_device_lock_test -j + tokenizer_linear_test hrx_placement_test hrx_cwsr_test hrx_runtime_test npu_device_lock_test \ + embeds_equiv_probe -j - name: Test # One hung test must not hold the job for hours: each test gets 10 minutes. run: ctest --test-dir build --output-on-failure --timeout 600 From a12ec041f10bb43fd71d95b5dc451c36f22b69d7 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Sun, 8 Nov 2026 20:41:37 -0400 Subject: [PATCH 3/5] ci: check the forward defines every overload it declares, instead of adding a target CI cannot build Review finding, and it was right in a way that invalidates the first approach: `embeds_equiv_probe` lives under `if(ONEBIT_NPU)` and this workflow configures without it (the NPU runtime is private), so adding it to `cmake --build --target ...` was a hard ninja error that stopped the whole build step -- `onebit` included. My READY was about the code and not about the configuration CI uses; third time this session that distinction has bitten someone, and this time it was me. The guard that works in THIS configuration is a symbol check, because the job already builds the right object: `onebit_npu_forward_syntax` is an OBJECT library that compiles `q4nx_forward.cpp` and links nothing (CMakeLists.txt notes it exists so CI compiles the file "without a workflow change"). So after the build: decl = declarations of `bool step_batch_embeds(` in npu/forward/include/q4nx_forward.h def = ` T Q4nxNpuForward::step_batch_embeds` entries in that object, via nm -C fail when def < decl Both directions measured, not argued: this tree, without #1574 decl=2 def=1 -> FAILS (the 2026-11-08 break, caught) main, with #1574 merged decl=2 def=2 -> PASSES The count is self-maintaining (it compares against the header rather than hard-coding two), and it names what it is: a declaration-without-definition check on one class, not a link of the probe. Also added the reviewer's step: `embeds_equiv_probe.cpp` compiled `-fsyntax-only` against the in-tree headers, which catches the probe drifting from the forward's interface -- a failure the symbol check cannot see. Both steps are cheap and neither can break the job. --- .github/workflows/ci.yml | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d5a98af78..1785bb7dd 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -85,6 +85,32 @@ jobs: g++-16 -std=c++26 -fsyntax-only -DONEBIT_NPU_RUNTIME_OWN \ -Inpu -Inpu/forward/include -Ibuild/_deps/json-src/include \ tools/mtp_acceptance.cpp + - name: The forward defines every step_batch_embeds overload the header declares + # `embeds_equiv_probe` cannot be built in this job: it lives under `if(ONEBIT_NPU)` (CMakeLists.txt), + # and this workflow configures without it because the NPU runtime is private. Adding an unknown target + # to `cmake --build --target ...` is a hard ninja error, so that is not the way to guard it. What this + # job DOES build is `onebit_npu_forward_syntax`, an OBJECT library that compiles q4nx_forward.cpp and + # links nothing -- so the object is here, and the invariant that broke on 2026-11-08 (the three-argument + # `step_batch_embeds` was declared in the header and never defined, so `embeds_equiv_probe` failed to + # link while every CI job stayed green) is checkable against it without the NPU configuration. + run: | + OBJ=$(find build -name 'q4nx_forward.cpp.o' -path '*onebit_npu_forward_syntax*' | head -1) + test -n "$OBJ" || { echo "the forward syntax object was not built"; exit 1; } + decl=$(grep -c 'bool step_batch_embeds(' npu/forward/include/q4nx_forward.h) + def=$(nm -C "$OBJ" | grep -c ' T Q4nxNpuForward::step_batch_embeds') + echo "$OBJ: $def definition(s) of Q4nxNpuForward::step_batch_embeds, $decl declared" + if [ "$def" -lt "$decl" ]; then + echo "a declared overload has no definition: anything that calls it fails to link, and this job's" + echo "own targets never do (that is how a red main survived). Counted above." + exit 1 + fi + - name: The embedding probe still compiles against the in-tree headers + # The probe itself cannot be built here (see above), but it must keep compiling: this catches the probe + # drifting from the forward's interface, which the symbol check above would not. + run: | + g++-16 -std=c++26 -fsyntax-only -DONEBIT_NPU_RUNTIME_OWN \ + -Inpu -Inpu/forward/include -Ibuild/_deps/json-src/include \ + npu/forward/embeds_equiv_probe.cpp - name: Build # Every target a ctest test invokes must be listed here: `ctest` runs whatever # binaries exist, so a test whose target is missing reports "Not Run" and never @@ -98,8 +124,7 @@ jobs: npu_model_bounds_test npu_forward_loader_test npu_design_grid_test npu_sampling_test \ npu_flm_head_import_test npu_elf_args_test \ npu_template_test npu_chatml_test \ - tokenizer_linear_test hrx_placement_test hrx_cwsr_test hrx_runtime_test npu_device_lock_test \ - embeds_equiv_probe -j + tokenizer_linear_test hrx_placement_test hrx_cwsr_test hrx_runtime_test npu_device_lock_test -j - name: Test # One hung test must not hold the job for hours: each test gets 10 minutes. run: ctest --test-dir build --output-on-failure --timeout 600 From 919e0e25d231ec5bfa33161e71e12a4f46bb0e06 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Sun, 8 Nov 2026 21:00:36 -0400 Subject: [PATCH 4/5] ci: the symbol check builds the object it looks at The step relied on `onebit` having pulled `onebit_npu_forward_syntax` in a previous step -- true today (`CMakeLists.txt:542`: `add_dependencies(onebit onebit_npu_forward_syntax)`), but a check whose precondition is another step's target list is the kind that goes quiet when that list is edited, and it fails as "the object was not built" rather than as what it means. One line inside the step fixes that: `cmake --build build --target onebit_npu_forward_syntax` before the `find`. Keeping the `test -n "$OBJ"` guard as well, so a future configuration that cannot build the target says so at the precondition rather than at the count. The `def < decl` comparison is unchanged: measured at 1-of-2 on a tree without #1574 (fails) and 2-of-2 on main with it (passes). --- .github/workflows/ci.yml | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 1785bb7dd..2831bd3ac 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -90,10 +90,15 @@ jobs: # and this workflow configures without it because the NPU runtime is private. Adding an unknown target # to `cmake --build --target ...` is a hard ninja error, so that is not the way to guard it. What this # job DOES build is `onebit_npu_forward_syntax`, an OBJECT library that compiles q4nx_forward.cpp and - # links nothing -- so the object is here, and the invariant that broke on 2026-11-08 (the three-argument + # links nothing -- so the object can be built here, and the invariant that broke on 2026-11-08 (the three-argument # `step_batch_embeds` was declared in the header and never defined, so `embeds_equiv_probe` failed to # link while every CI job stayed green) is checkable against it without the NPU configuration. run: | + # Built explicitly rather than relied on transitively: it IS built by the step above, because + # CMakeLists.txt:542 has `add_dependencies(onebit onebit_npu_forward_syntax)` -- but a check that + # depends on the order of another step's target list is the kind that goes quiet when that list is + # edited. One line here makes the object's existence this step's own precondition. + cmake --build build --target onebit_npu_forward_syntax OBJ=$(find build -name 'q4nx_forward.cpp.o' -path '*onebit_npu_forward_syntax*' | head -1) test -n "$OBJ" || { echo "the forward syntax object was not built"; exit 1; } decl=$(grep -c 'bool step_batch_embeds(' npu/forward/include/q4nx_forward.h) From 38debef63dec03fd6fc4ef627a54094196c3ced0 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Mon, 9 Nov 2026 04:34:37 -0400 Subject: [PATCH 5/5] ci: keep the target list out of the prose, so ci_targets_check can read it The build step went red on `ci_targets`, not on this PR's own logic: tools/ci/ci_targets_check.py reads ci.yml for target names and my comment explaining the guard quoted the command line, so it counted 'is', 'a', 'hard', 'ninja', 'error,' and a dozen more words as targets: ci_targets_check: ci.yml builds 'error,', which no add_executable/add_library defines Reworded so the comment describes the guard without reproducing the command line. The step itself is unchanged and its target resolves: `ci_targets_check --check` now reports "all 29 target(s) ci.yml builds exist in the configuration it creates". Worth noting for the checker's owner: it does not skip YAML comments, so any future prose that quotes a target-list command is a build failure. Stripping comments before scanning would remove that trap; I have not touched the checker here. --- .github/workflows/ci.yml | 15 ++++++++------- 1 file changed, 8 insertions(+), 7 deletions(-) diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 2831bd3ac..d4da5be53 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -86,13 +86,14 @@ jobs: -Inpu -Inpu/forward/include -Ibuild/_deps/json-src/include \ tools/mtp_acceptance.cpp - name: The forward defines every step_batch_embeds overload the header declares - # `embeds_equiv_probe` cannot be built in this job: it lives under `if(ONEBIT_NPU)` (CMakeLists.txt), - # and this workflow configures without it because the NPU runtime is private. Adding an unknown target - # to `cmake --build --target ...` is a hard ninja error, so that is not the way to guard it. What this - # job DOES build is `onebit_npu_forward_syntax`, an OBJECT library that compiles q4nx_forward.cpp and - # links nothing -- so the object can be built here, and the invariant that broke on 2026-11-08 (the three-argument - # `step_batch_embeds` was declared in the header and never defined, so `embeds_equiv_probe` failed to - # link while every CI job stayed green) is checkable against it without the NPU configuration. + # `embeds_equiv_probe` cannot be built in this job: it lives under `if(ONEBIT_NPU)` (CMakeLists.txt), and + # this workflow configures without it because the NPU runtime is private. Naming a target the configuration + # does not define is a hard ninja error that stops the whole step (tools/ci/ci_targets_check.py guards that + # now), so the probe is guarded by its symbol instead. What this job DOES build is + # `onebit_npu_forward_syntax`, an OBJECT library that compiles q4nx_forward.cpp and links nothing -- so the + # object can be built here, and the invariant that broke on 2026-11-08 (the three-argument + # `step_batch_embeds` was declared in the header and never defined, so `embeds_equiv_probe` failed to link + # while every CI job stayed green) is checkable against it without the NPU configuration. run: | # Built explicitly rather than relied on transitively: it IS built by the step above, because # CMakeLists.txt:542 has `add_dependencies(onebit onebit_npu_forward_syntax)` -- but a check that