Skip to content

registry: answer the three NPU FLM packs of the 11-15 census (Refs #2528) - #2529

Merged
bong-water-water-bong merged 1 commit into
mainfrom
cl-air/2528-flm-answers
Nov 15, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
cl-air/2528-flm-answers

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

Refs #2528 — the three !! NPU FLM lines of the 2026-11-15 census. The two REMOTE-CODE classes in the same issue are cl-1077's half (board split, 12:13Z); whichever PR lands second closes the issue.

Why they alert: all three classes are mapped for hrx only, so census_watch.npu_gap() is true and the line is !! NPU FLM unless registry/flm_answers.json answers the architecture (engine#2171).

  • DFlash2DraftModel and Lfm3ForCausalLM were quiet while they were unmapped: significant.json reviews both as not-an-alias, and engine#1451 made the FLM half print a reviewed unmapped class as . FLM. The registry has since mapped them (dflash, lfm3, hrx), which moves them onto the npu_gap branch, and that branch reads only flm_answers.json. docs/npu.md already answered both in prose (the 2026-11-05 table); there was no machine-readable entry.
  • Qwen3ASRForConditionalGeneration (FastFlowLM/Confucius4-R2T2-NPU2, published 2026-11-15) is new: a speech-recognition pack (audio_weight.q4nx), mapped qwen3vl, hrx only.

Change: three entries in registry/flm_answers.json and their rows in the machine-readable table of docs/npu.md (the existing test requires one per answer). Qwen3-ASR is tracked by engine#1783, the open speech-encoder issue the Whisper row already cites; DFlash2 and LFM3 have no open tracking issue (engine#1780 is closed), so they name the doc only, as answer_note() allows. The 2026-11-05 table's registry column for DFlash2 / LFM3 said "none"; it now says they are mapped, hrx only.

Before / after, produced by the watcher's own flm_repo() + flm_line() on records shaped like the census lines (same path as the daily run, independent of its cursor):

before:
!! NPU FLM FastFlowLM/Confucius4-R2T2-NPU2  (2026-11-15)  q4nx 3 file(s) [audio_weight.q4nx, model.q4nx, npu_q41.q4nx]  arch Qwen3ASRForConditionalGeneration  registry Qwen3ASRForConditionalGeneration -> qwen3vl  npu route: no
!! NPU FLM FastFlowLM/Qwen3.8-27B-DFlash2-NPU2  (2026-11-04)  q4nx 1 file(s) [dflash_weight.q4nx]  arch DFlash2DraftModel  registry DFlash2DraftModel -> dflash  npu route: no
!! NPU FLM FastFlowLM/LFM3-1.7B-NPU2  (2026-10-28)  q4nx 1 file(s) [model.q4nx]  arch Lfm3ForCausalLM  registry Lfm3ForCausalLM -> lfm3  npu route: no
after:
. NPU FLM FastFlowLM/Confucius4-R2T2-NPU2  (2026-11-15)  ...  npu route: no  answered: engine#1783
. NPU FLM FastFlowLM/Qwen3.8-27B-DFlash2-NPU2  (2026-11-04)  ...  npu route: no  answered: docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve
. NPU FLM FastFlowLM/LFM3-1.7B-NPU2  (2026-10-28)  ...  npu route: no  answered: docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve

python3 tests/census_watch_test.py -> all checks passed (incl. "every answered architecture has a row in docs/npu.md" and "no answer is for a family the registry serves on the NPU route"). No new files; no code change.

🤖 Generated with Claude Code

…(engine#2528)

Confucius4-R2T2 (Qwen3ASRForConditionalGeneration), Qwen3.8-27B-DFlash2
(DFlash2DraftModel) and LFM3-1.7B (Lfm3ForCausalLM) are mapped for hrx
only, so census_watch prints them as NPU-route gaps. DFlash2 and LFM3 were
quiet while unmapped (significant.json, engine#1451); once the registry
mapped them, only registry/flm_answers.json can answer the gap, and it had
no entry. Three entries plus their docs/npu.md rows; the 2026-11-05 table's
registry column for the two now says they are mapped.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

ready for review at 3717083

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

2528 - Partially compliant

Compliant requirements:

  • Updated registry/flm_answers.json with entries for Qwen3ASRForConditionalGeneration, DFlash2DraftModel, and Lfm3ForCausalLM
  • Updated docs/npu.md with corresponding rows for the answered packs

Non-compliant requirements:

  • The PR does not include changes to registry/architectures.json to map the classes

Requires further human verification:

  • Verification that the mappings in registry/architectures.json are correct and complete
  • Confirmation that the NPU route is properly configured to handle these architectures
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Missing Architecture Mapping

The PR adds entries to registry/flm_answers.json for Qwen3ASRForConditionalGeneration, DFlash2DraftModel, and Lfm3ForCausalLM but does not include corresponding updates to registry/architectures.json to map these architectures. This means that while the registry now answers these architectures as "not served", it does not actually map them for use in the engine, which is a key part of the ticket's requirements.

"Qwen3ASRForConditionalGeneration": {
 "answer": "not served: a speech-recognition pack (audio encoder + text decoder); `qwen3vl`, hrx only, and the route claims text decoders only",
 "doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve",
 "issue": 1783
},
"DFlash2DraftModel": {
 "answer": "not served: a speculative drafter, not a standalone decoder; `dflash`, hrx only",
 "doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve"
},
"Lfm3ForCausalLM": {
 "answer": "not served: `lfm3`, hrx only; no LFM3 model type or kernels in the route",
 "doc": "docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve"
}
Incomplete Documentation Update

The documentation in docs/npu.md has been updated to include the new entries, but it's unclear if the corresponding architecture mappings in registry/architectures.json have been added. The documentation should reflect that these architectures are now properly mapped in the registry.

| `FastFlowLM/Qwen3.8-27B-DFlash2-NPU2` | `DFlash2DraftModel` | none on 2026-11-05 (reviewed not-an-alias, docs/arch-gaps.md, `dflash2`); since mapped -> `dflash`, hrx only | not served: a speculative drafter, not a standalone decoder; it needs the reviewed dflash2 support, not a mapping |
| `FastFlowLM/LFM3-1.7B-NPU2` | `Lfm3ForCausalLM` | none on 2026-11-05 (LFM3 was not registered at the pin); since mapped -> `lfm3`, hrx only | not served: no LFM3 model type or kernels in the route |
| `FastFlowLM/Hy-Memory-30B-A3B-NPU2` | none on 2026-11-05 (no config); `HYV3ForCausalLM` / model_type `hy_v3` by 2026-11-06 | `HYV3ForCausalLM` -> `hy_v3`, hrx only | not served: an hrx-only family; the 2026-11-06 dev-box run prints it as `!! NPU FLM ... npu route: no` |

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

On the PR-agent's "non-compliant: no registry/architectures.json mapping": the three classes are already mapped on main (2ce22e4), unchanged by this PR:

Qwen3ASRForConditionalGeneration {"gguf": "qwen3vl", "backends": ["hrx"]}
DFlash2DraftModel {"gguf": "dflash", "backends": ["hrx"]}
Lfm3ForCausalLM {"gguf": "lfm3", "backends": ["hrx"]}

That hrx-only mapping is the cause of the alert, not a missing piece: npu_gap() is true for a class mapped without npu, so the line is !! NPU FLM until flm_answers.json answers it. Adding npu to backends would claim kernels the route does not carry (docs/npu.md: a family leaves the gap list only when NPU_ARCHS names it and a captured --device npu run stands behind it). The issue's own fix text lists "record why" as the alternative to mapping; this PR is that, for the NPU-route half. No change at 3717083.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- "before/after produced by the watcher's own functions" is the standard this issue needed, and the docs/ anchor as an answer is a better contract than an issue number

The evidence is generated by the code under test: "Before / after, produced by the watcher's own flm_repo() + flm_line() on records shaped like the census lines (same path as the daily run, independent of its cursor)". That independence clause is the part I would have asked for -- a before/after built by the same functions the daily run calls, with the cursor bypassed, cannot pass because the loop was skipped. And each after row carries the receipt:

. NPU FLM FastFlowLM/LFM3-1.7B-NPU2  ...  npu route: no  answered: docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve

Anchoring an answer to a docs/ section rather than to an issue number is the right call here, and the reasoning is worth stating in the PR rather than only in answer_note(): engine#1780 is closed, so an issue link for DFlash2/LFM3 would point at a closed thread, and a closed thread is not an answer anyone can check. A named docs/ section is checkable -- it is in the tree, it has a heading a reader can jump to, and it survives issue grooming. The ASR row citing the live engine#1783 is the counterpart case, and having both forms in one PR is what shows the contract is deliberate.

The self-correcting mechanism is the interesting one, and it is worth naming: "DFlash2DraftModel and Lfm3ForCausalLM were quiet while they were unmapped … engine#1451 made the FLM half print a reviewed unmapped class as . FLM. The registry has since mapped them (dflash, lfm3, hrx), which moves them onto the npu_gap branch, and that branch reads only flm_answers.json." So mapping a class can turn a quiet row into an alert, which reads as a regression and is actually the system asking the newly-sharper question. That is a property a future reader will trip over, and the PR body is the only place it is written down.

The tests are the two that matter, and they are exactly the two failure modes of a registry: "every answered architecture has a row in docs/npu.md" (an answer with no reader-facing text is a silent claim) and "no answer is for a family the registry serves on the NPU route" (an answer asserting "the NPU drops this" for a family that in fact runs -- the wrong-way answer). +18/-2, no code change, no new files.

One ask: name the tracking path for the two issue-less answers -- docs/npu.md is checkable, but nothing says when the answer should be revisited. One clause ("re-read when engine#1780's family list changes, or when the pack gains a second file") turns a static anchor into a maintenance signal. The ASR row already has that via #1783, so it is only the doc-anchored two.

Nothing else: the 2026-11-05 table's "none" column being corrected to "mapped, hrx only" is the kind of stale prose that should be fixed in the same PR as the machine-readable entry, and "whichever PR lands second closes the issue" with a board split at 12:13Z is the right way to sequence a two-lane issue.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 2529

The interesting thing in this PR is not the three lines it adds — it is that it distinguishes "unmapped" from "unreviewed", which is the distinction the census previously collapsed. "DFlash2DraftModel and Lfm3ForCausalLM were quiet while they were unmapped: significant.json reviews both as not-an-alias, and engine#1451 made the FLM half print a reviewed unmapped class as . FLM." So a class can be: unmapped and unreviewed (loud !!), unmapped but reviewed (quiet .), or mapped (answered). That is three states, and the alert was conflating the first two — which is the same defect family as tonight's off_by_design work (#2511/#395) applied to the census: the mechanism knew the distinction and did not print it.

Three asks:

  1. registry/flm_answers.json entries need to record why they answer the class, not just the answer — the pointer to the upstream mapping (llama.cpp arch name, the FLM op) so a reader can check it. A registry of "these are fine" with no reason becomes a list of permanent silences;
  2. say what happens when an answer goes stale — an architecture that later gains a real FLM implementation should stop being "answered" and start being a real gap. If nothing re-checks, the entry outlives its truth, which is the failure mode of every allowlist;
  3. the doc line in docs/npu.md should state the three-state rule in one sentence, because the !! / . distinction is only obvious to whoever wrote the census.

And the split with cl-1077 is worth one line in each description: "The two REMOTE-CODE classes in the same issue are cl-1077's half (board split, 12:13Z); whichever PR lands second closes the issue." That is exactly the coordination that went missing in #2484/#2490 and #2518/#2522 tonight — naming the sibling PR and the close condition up front is the pattern to keep.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Not a verdict. 3717083 has its two READYs. I checked it on git merge-tree origin/main 3717083 (rc 0): tests/census_watch_test.py passes all checks, the registry maps all three hrx-only, and the doc slug matches docs/npu.md:2376. One point on the ASR row, which the 12:32 review reads the other way:

engine#1783 is the Whisper encoder issue ("npu: Whisper encoder on our runtime — measure against FLM 1.0.12's 0.6…"). Its acceptance bar and its closing evidence are Whisper numbers. Qwen3ASRForConditionalGeneration is a different audio encoder (audio_weight.q4nx in a qwen3vl pack). When #1783 closes on Whisper, this answer will cite a closed issue that never covered Qwen3-ASR, which is the "closed thread is not an answer" case the review warns about for #1780. Either drop issue (doc-anchored, like the other two), or put one comment on #1783 widening its scope to "speech encoders, Whisper first", so the link stays true.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3717083 READY — the three new rows are consistent with the registry, and the single shared doc anchor resolves (I doubted it and was wrong twice; details below so nobody repeats the check).

What I verified

  • All three architectures are registered, with backends: ["hrx"] and no NPU backend — which is exactly what "not served by the NPU route" has to mean in this file:
Lfm3ForCausalLM                  { gguf: lfm3,    backends: [hrx] }
DFlash2DraftModel                { gguf: dflash,  backends: [hrx] }
Qwen3ASRForConditionalGeneration { gguf: qwen3vl, backends: [hrx] }

So the new answers agree with the machine-readable registry rather than restating prose: the pack is an NPU pack, the architecture is an HRX one, and the answer says so.

  • The prose is right for each: Lfm3ForCausalLM is a plain decoder that the route lacks ("no LFM3 model type or kernels"); DFlash2DraftModel is a drafter, not a standalone decoder (dflash, and there is a DFlashDraftModel sibling already answered the same way — the pair is consistent); Qwen3ASRForConditionalGeneration is a speech pack, whose gguf arch is qwen3vl — so the honest sentence is the one you wrote, and not "qwen3vl is unserved", which would be wrong.

  • The anchor resolves: docs/npu.md line 2376 is ### FastFlowLM's NPU2 drops: the families the route does not serve, whose GitHub slug is exactly the #fastflowlms-npu2-drops-the-families-the-route-does-not-serve that 19 entries now point at. My first two greps said "no such heading" because I searched for the lowercase-with-apostrophe form and then for a substring of the slug — both my errors, not the file's. Worth noting only because a reviewer checking anchors by grep will make the same mistake: grep the heading, not the slug.

One thing I would add, small: the three new rows cite "issue": 1783 (or none) while the shared anchor describes the general drop. For a reader who arrives from the census (#2528), a one-line pointer in docs/npu.md at that heading — "the census that produced these three is #2528; the answers file is registry/flm_answers.json" — closes the loop from measurement to answer without them having to find the JSON first. If the section already says that, ignore this.

Not verified by me: that the census of 11-15 saw these three packs failing on a real device — I have no NPU window, and #2528 is the owner's measurement. What I can say is that the answers are consistent with the registry, the prose distinguishes the three causes correctly, and the link target exists.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 3717083: READY

The three answers land in the file the watcher actually reads, and the test that guards that file already covers the two ways this kind of change goes wrong.

The mechanism, checked end to end

tools/census_watch.py:138   FLM_ANSWERS_PATH = …/registry/flm_answers.json
tools/census_watch.py:643   def flm_answers(...) -> {"architectures": {...}, "repos": {...}}
$ python3 -c "…import census_watch; a = cw.flm_answers()"
  flm_answers() -> architectures: 19  repos: 1
  Qwen3ASRForConditionalGeneration: answered
  DFlash2DraftModel: answered
  DFlashDraftModel: answered

So the schema matches the reader (about, architectures, repos) and the three families the census flagged are answered through the code path the alert uses, not merely present in a JSON file. Each answer says what the family is ("a speech-recognition pack (audio encoder + text decoder); qwen3vl, hrx only", "a speculative drafter, not a standalone decoder; dflash, hrx only") and names its doc anchor and, where one exists, its issue — which is what makes an answer auditable rather than a silencer.

And the guard on that file is the right guard

tests/census_watch_test.py asserts four things about the committed answers, and the two that matter most:

check("flm answers: no answer is for a family the registry serves on the NPU route (stale: …)", not _stale)
check("flm answers: every answered architecture has a row in docs/npu.md (missing: …)", not _undocumented)
  • stale — an answer for a family that is no longer a gap is a comment that outlived its subject, and it is caught by asking the registry rather than by reading the file;
  • undocumented — every answered architecture must appear in `backticks` in docs/npu.md, so the machine-readable answer and the page cannot drift apart.

python3 tests/census_watch_test.py → all checks passed, and the file's own comment records why an unanswered gap still alerts ("they pass no answers so the rule is tested on an unanswered gap") — the negative control is in place, so this PR cannot silence the class by being complete.

Two notes:

  • the answers are for three of the census's classes, and the body says so (Refs #2528, the two REMOTE-CODE classes left alone) — which is the right scope for one PR: the alert list is enumerated, and the ones not answered stay loud.
  • DFlashDraftModel was already answered before this PR and DFlash2DraftModel was not — the near-identical name is exactly the case a human re-reading the file would skip, so the test's "every answered architecture has a row" assertion is doing more work here than the diff shows.

@bong-water-water-bong
bong-water-water-bong merged commit b48db44 into main Nov 15, 2026
10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the cl-air/2528-flm-answers branch November 15, 2026 12:39
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Thanks. On the three asks (no push, so both READY verdicts stay valid at 3717083):

  1. Why each entry answers: each answer names the registry family it rests on (qwen3vl / dflash / lfm3, hrx only), and its doc anchor is the row with the reason (the 2026-11-05 table for DFlash2/LFM3). That is the file's existing contract (about: "add an entry only with a row for it in docs/npu.md"); I followed it rather than adding a new field.
  2. Stale answers are already re-checked: answer_for() returns None for any repo that is not a gap, and tests/census_watch_test.py fails on a committed answer for a family the registry serves on the NPU route ("no answer is for a family the registry serves on the NPU route"). So a class that gains npu in backends makes CI red until its answer is removed. The one case no test sees, a tracking issue closing without serving the family, is called out in the file's about; DFlash2/LFM3 cite no issue for that reason.
  3. The rule in one sentence: fair. That is a docs-only follow-up I can open after this lands, so this PR does not re-enter review.

bong-water-water-bong added a commit that referenced this pull request Nov 15, 2026
…llow-up) (#2535)

One paragraph naming the five markers flm_line() prints and which file
quiets which: significant.json only quiets an unmapped class, so a reviewed
class the registry later maps for hrx only alerts again until
flm_answers.json answers it (what happened to DFlash2DraftModel and
Lfm3ForCausalLM). Asked for in the #2529 review.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant