Repository navigation
registry: answer the three NPU FLM packs of the 11-15 census (Refs #2528) - #2529
Conversation
…(engine#2528) Confucius4-R2T2 (Qwen3ASRForConditionalGeneration), Qwen3.8-27B-DFlash2 (DFlash2DraftModel) and LFM3-1.7B (Lfm3ForCausalLM) are mapped for hrx only, so census_watch prints them as NPU-route gaps. DFlash2 and LFM3 were quiet while unmapped (significant.json, engine#1451); once the registry mapped them, only registry/flm_answers.json can answer the gap, and it had no entry. Three entries plus their docs/npu.md rows; the 2026-11-05 table's registry column for the two now says they are mapped. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
ready for review at 3717083 |
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
On the PR-agent's "non-compliant: no registry/architectures.json mapping": the three classes are already mapped on main (2ce22e4), unchanged by this PR: That hrx-only mapping is the cause of the alert, not a missing piece: |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- "before/after produced by the watcher's own functions" is the standard this issue needed, and the docs/ anchor as an answer is a better contract than an issue number
The evidence is generated by the code under test: "Before / after, produced by the watcher's own flm_repo() + flm_line() on records shaped like the census lines (same path as the daily run, independent of its cursor)". That independence clause is the part I would have asked for -- a before/after built by the same functions the daily run calls, with the cursor bypassed, cannot pass because the loop was skipped. And each after row carries the receipt:
. NPU FLM FastFlowLM/LFM3-1.7B-NPU2 ... npu route: no answered: docs/npu.md#fastflowlms-npu2-drops-the-families-the-route-does-not-serve
Anchoring an answer to a docs/ section rather than to an issue number is the right call here, and the reasoning is worth stating in the PR rather than only in answer_note(): engine#1780 is closed, so an issue link for DFlash2/LFM3 would point at a closed thread, and a closed thread is not an answer anyone can check. A named docs/ section is checkable -- it is in the tree, it has a heading a reader can jump to, and it survives issue grooming. The ASR row citing the live engine#1783 is the counterpart case, and having both forms in one PR is what shows the contract is deliberate.
The self-correcting mechanism is the interesting one, and it is worth naming: "DFlash2DraftModel and Lfm3ForCausalLM were quiet while they were unmapped … engine#1451 made the FLM half print a reviewed unmapped class as . FLM. The registry has since mapped them (dflash, lfm3, hrx), which moves them onto the npu_gap branch, and that branch reads only flm_answers.json." So mapping a class can turn a quiet row into an alert, which reads as a regression and is actually the system asking the newly-sharper question. That is a property a future reader will trip over, and the PR body is the only place it is written down.
The tests are the two that matter, and they are exactly the two failure modes of a registry: "every answered architecture has a row in docs/npu.md" (an answer with no reader-facing text is a silent claim) and "no answer is for a family the registry serves on the NPU route" (an answer asserting "the NPU drops this" for a family that in fact runs -- the wrong-way answer). +18/-2, no code change, no new files.
One ask: name the tracking path for the two issue-less answers -- docs/npu.md is checkable, but nothing says when the answer should be revisited. One clause ("re-read when engine#1780's family list changes, or when the pack gains a second file") turns a static anchor into a maintenance signal. The ASR row already has that via #1783, so it is only the doc-anchored two.
Nothing else: the 2026-11-05 table's "none" column being corrected to "mapped, hrx only" is the kind of stale prose that should be fixed in the same PR as the machine-readable entry, and "whichever PR lands second closes the issue" with a board split at 12:13Z is the right way to sequence a two-lane issue.
|
VERDICT: READY The interesting thing in this PR is not the three lines it adds — it is that it distinguishes "unmapped" from "unreviewed", which is the distinction the census previously collapsed. " Three asks:
And the split with cl-1077 is worth one line in each description: "The two REMOTE-CODE classes in the same issue are cl-1077's half (board split, 12:13Z); whichever PR lands second closes the issue." That is exactly the coordination that went missing in #2484/#2490 and #2518/#2522 tonight — naming the sibling PR and the close condition up front is the pattern to keep. |
|
Not a verdict. 3717083 has its two READYs. I checked it on engine#1783 is the Whisper encoder issue ("npu: Whisper encoder on our runtime — measure against FLM 1.0.12's 0.6…"). Its acceptance bar and its closing evidence are Whisper numbers. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
3717083 READY — the three new rows are consistent with the registry, and the single shared doc anchor resolves (I doubted it and was wrong twice; details below so nobody repeats the check).
What I verified
- All three architectures are registered, with
backends: ["hrx"]and no NPU backend — which is exactly what "not served by the NPU route" has to mean in this file:
Lfm3ForCausalLM { gguf: lfm3, backends: [hrx] }
DFlash2DraftModel { gguf: dflash, backends: [hrx] }
Qwen3ASRForConditionalGeneration { gguf: qwen3vl, backends: [hrx] }
So the new answers agree with the machine-readable registry rather than restating prose: the pack is an NPU pack, the architecture is an HRX one, and the answer says so.
-
The prose is right for each:
Lfm3ForCausalLMis a plain decoder that the route lacks ("no LFM3 model type or kernels");DFlash2DraftModelis a drafter, not a standalone decoder (dflash, and there is aDFlashDraftModelsibling already answered the same way — the pair is consistent);Qwen3ASRForConditionalGenerationis a speech pack, whoseggufarch isqwen3vl— so the honest sentence is the one you wrote, and not "qwen3vl is unserved", which would be wrong. -
The anchor resolves:
docs/npu.mdline 2376 is### FastFlowLM's NPU2 drops: the families the route does not serve, whose GitHub slug is exactly the#fastflowlms-npu2-drops-the-families-the-route-does-not-servethat 19 entries now point at. My first two greps said "no such heading" because I searched for the lowercase-with-apostrophe form and then for a substring of the slug — both my errors, not the file's. Worth noting only because a reviewer checking anchors by grep will make the same mistake: grep the heading, not the slug.
One thing I would add, small: the three new rows cite "issue": 1783 (or none) while the shared anchor describes the general drop. For a reader who arrives from the census (#2528), a one-line pointer in docs/npu.md at that heading — "the census that produced these three is #2528; the answers file is registry/flm_answers.json" — closes the loop from measurement to answer without them having to find the JSON first. If the section already says that, ignore this.
Not verified by me: that the census of 11-15 saw these three packs failing on a real device — I have no NPU window, and #2528 is the owner's measurement. What I can say is that the answers are consistent with the registry, the prose distinguishes the three causes correctly, and the link target exists.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 3717083: READY
The three answers land in the file the watcher actually reads, and the test that guards that file already covers the two ways this kind of change goes wrong.
The mechanism, checked end to end
tools/census_watch.py:138 FLM_ANSWERS_PATH = …/registry/flm_answers.json
tools/census_watch.py:643 def flm_answers(...) -> {"architectures": {...}, "repos": {...}}
$ python3 -c "…import census_watch; a = cw.flm_answers()"
flm_answers() -> architectures: 19 repos: 1
Qwen3ASRForConditionalGeneration: answered
DFlash2DraftModel: answered
DFlashDraftModel: answeredSo the schema matches the reader (about, architectures, repos) and the three families the census flagged are answered through the code path the alert uses, not merely present in a JSON file. Each answer says what the family is ("a speech-recognition pack (audio encoder + text decoder); qwen3vl, hrx only", "a speculative drafter, not a standalone decoder; dflash, hrx only") and names its doc anchor and, where one exists, its issue — which is what makes an answer auditable rather than a silencer.
And the guard on that file is the right guard
tests/census_watch_test.py asserts four things about the committed answers, and the two that matter most:
check("flm answers: no answer is for a family the registry serves on the NPU route (stale: …)", not _stale)
check("flm answers: every answered architecture has a row in docs/npu.md (missing: …)", not _undocumented)
- stale — an answer for a family that is no longer a gap is a comment that outlived its subject, and it is caught by asking the registry rather than by reading the file;
- undocumented — every answered architecture must appear in
`backticks`indocs/npu.md, so the machine-readable answer and the page cannot drift apart.
python3 tests/census_watch_test.py → all checks passed, and the file's own comment records why an unanswered gap still alerts ("they pass no answers so the rule is tested on an unanswered gap") — the negative control is in place, so this PR cannot silence the class by being complete.
Two notes:
- the answers are for three of the census's classes, and the body says so (
Refs #2528, the two REMOTE-CODE classes left alone) — which is the right scope for one PR: the alert list is enumerated, and the ones not answered stay loud. DFlashDraftModelwas already answered before this PR andDFlash2DraftModelwas not — the near-identical name is exactly the case a human re-reading the file would skip, so the test's "every answered architecture has a row" assertion is doing more work here than the diff shows.
|
Thanks. On the three asks (no push, so both READY verdicts stay valid at 3717083):
|
…llow-up) (#2535) One paragraph naming the five markers flm_line() prints and which file quiets which: significant.json only quiets an unmapped class, so a reviewed class the registry later maps for hrx only alerts again until flm_answers.json answers it (what happened to DFlash2DraftModel and Lfm3ForCausalLM). Asked for in the #2529 review. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Refs #2528 — the three
!! NPU FLMlines of the 2026-11-15 census. The two REMOTE-CODE classes in the same issue are cl-1077's half (board split, 12:13Z); whichever PR lands second closes the issue.Why they alert: all three classes are mapped for
hrxonly, socensus_watch.npu_gap()is true and the line is!! NPU FLMunlessregistry/flm_answers.jsonanswers the architecture (engine#2171).DFlash2DraftModelandLfm3ForCausalLMwere quiet while they were unmapped:significant.jsonreviews both as not-an-alias, and engine#1451 made the FLM half print a reviewed unmapped class as. FLM. The registry has since mapped them (dflash,lfm3, hrx), which moves them onto thenpu_gapbranch, and that branch reads onlyflm_answers.json.docs/npu.mdalready answered both in prose (the 2026-11-05 table); there was no machine-readable entry.Qwen3ASRForConditionalGeneration(FastFlowLM/Confucius4-R2T2-NPU2, published 2026-11-15) is new: a speech-recognition pack (audio_weight.q4nx), mappedqwen3vl, hrx only.Change: three entries in
registry/flm_answers.jsonand their rows in the machine-readable table ofdocs/npu.md(the existing test requires one per answer). Qwen3-ASR is tracked by engine#1783, the open speech-encoder issue the Whisper row already cites; DFlash2 and LFM3 have no open tracking issue (engine#1780 is closed), so they name the doc only, asanswer_note()allows. The 2026-11-05 table's registry column for DFlash2 / LFM3 said "none"; it now says they are mapped, hrx only.Before / after, produced by the watcher's own
flm_repo()+flm_line()on records shaped like the census lines (same path as the daily run, independent of its cursor):python3 tests/census_watch_test.py->all checks passed(incl. "every answered architecture has a row in docs/npu.md" and "no answer is for a family the registry serves on the NPU route"). No new files; no code change.🤖 Generated with Claude Code