You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
F9: FLM 1.0.12's HRX-targeted build — side-by-side against ours on the same models, and a row for every family we do not serve #1787
TNPU-ALL §9 item F9 (owner 2026-11-09 23:2x, "support all of it"). Measurement item: nothing to install, nothing left serving.
What
FastFlowLM 1.0.12 ships two builds: its usual one, and an HRX-targeted build of the same product. The second is directly comparable to
us, because HRX is the route our engine already serves on — so the same model can be run by their front-end and by ours on the same device, in
the same hour, and the two numbers belong on the same board row.
This issue is that measurement, plus the row-generation it exposes.
Definition of done
Side-by-side, one gpu-hold ticket, same models, interleaved (A B B A with their build and ours): decode tok/s and time to first token for
each, on the models both can run. The numbers are quoted only if the ticket's own spread supports them — the scoreboard's existing drift and
treatment checks apply unchanged, including the positive control that proves the two arms were different conditions.
Every family their build serves that we do not becomes a row under §1's five conditions (imports with zero gaps and self-check; serves on
the device; known-answer prompt with the model's own template, three identical answers; measured decode tok/s and TTFT under a ticket; row
names the kernel set and engine commit). Their build's per-family libraries give the checklist; the row is what settles each family.
The result is written where the scoreboard is recomputed from, not in a comment.
What is already established (host-side, no device)
The artifact is present on the box for the side-by-side; it is not installed system-wide and nothing of it is left running.
The family list their build carries is enumerated from its own files, and the family list the public release notes add is: MiniCPM-V 4.7 35B-A3B, d3:edge, d3:flash, qwen3.5-flash:2b, plus DFlash tags for Nemotron 3.5 Bolt and Qwen3.8-27B. Those are the families to check against our
own coverage first, since a family we do not serve at all is a row by definition.
The measurement cannot start until the box has memory to load a model: MemAvailable was 2 GiB at 23:5x, against the ≥40 GiB the plan
requires before a pack. It starts on the first ticket with headroom, and the preconditions are re-checked at that moment, not assumed from
now.
Not in this issue
Installing their build, serving from it, or shipping anything derived from it. It is a reference measurement: run under a ticket, recorded, and
stopped.
First F9 measurement — same model, same device, one probe: ours decodes 5.3× faster, and the TTFT gap is prefix reuse, not a cold-start win
Ticket gpu-hold -l f9-flm-hrx (212 s, MemoryMax=48G, oom_score_adj=800), artifact raw/f9-flm-hrx-20261110-001736. Both engines measured by f9-probe.py: streaming /v1/chat/completions, temperature 0, 64 tokens, 3 reps, TTFT = first chunk, decode = chunks after the first. Neither
engine's self-report is used.
FLM 1.0.12 (HRX build)
ours (llama-server/HRX)
ratio
decode
2.03 tok/s (2.00/2.03/2.05, spread 1.027×)
10.79 tok/s (10.10/10.79/10.90, spread 1.079×)
ours 5.3×
TTFT (warm)
3.546 s (3.622/3.364/3.546)
0.349 s (0.349/0.229 after rep 1)
ours 10×
TTFT (first request after load)
— all reps ≈3.5 s
3.387 s
≈ equal
The structure is the finding. Their TTFT does not improve across requests; ours falls from 3.387 s to 0.349/0.229 s. So the 10× is prefix
reuse, and on a cold prompt the engines are indistinguishable. Decode has no such structure — 5.3× at both arms' spreads.
Bounded by the container difference, stated rather than implied: theirs is its own Q4NX pack (qwen3.8-mtp:27b, 17 GB), ours is Qwen3.8-27B-UD-Q4_K_XL.gguf (17.6 GB). Same model family and size, not bit-identical weights. Binaries recorded per run (flmbf4b6cd8…,
our server 4251a167…).
Also from their side, separately:Model loaded in 32.29 s for the 27B on HRX (measured in the earlier ticket). Not compared to ours: the
arms loaded in different tickets, and load time is not what this row claims.
Still open on this issue: the row-generation half — their catalogue lists ~28 tags across d3, deepseek-r1, nemotron, phi4, qwen2.5/3/3.5/3.6/3.8,
qwen3vl, translategemma and whisper; only two packs are on disk here (27b, whisper), so each family we do not serve needs a pack pull under the
plan's memory rules before it can become a row.
The row-generation half: their catalogue is 71 tags, 4 of which we have rows for
Generated from the model_list.json their build ships (the same file their CLI reads) against our own SUMMARY.md sections — table in F9-COVERAGE.md at commit 0e5954d. Summary:
71 tags in the catalogue, 4 have a row of ours: qwen3:0.6b → Qwen3-0.6B, qwen3:1.7b → qwen3-1.7b, qwen3.6-moe:35b-a3b → qwen36-35b-a3b, qwen3.8-mtp:27b → qwen38-27b (the one F9 measured).
67 tags have no row. That is the backlog, and it is much larger than the ~28 a truncated flm list suggested: the catalogue also carries
the -core dev04 variants and the gemma3 / medgemma / lfm2 / lfm3 / nanbeige / hy-mt2 lines.
Only a handful are reachable without a download — the last column marks where a GGUF of that model is already on disk. Everything else
needs a pack pull, under the plan's rules (one pack over 8 GB at a time, ≥40 GiB free before starting, packs on disk not /tmp).
Families to prioritise, by what the plan already scopes:qwen3:4b (a GGUF is on disk, so a row is a ticket away), qwen3.5:{0.8b,2b,4b,9b}
(F5's qwen3.5-flash:2b target lives here), d3:{edge,flash} (F7), minicpm-v-4.7-moe:35b-a3b (F6), whisper-v3:turbo (F8, pack already on
disk), and qwen3.8-dflash2:27b (F2's draft, GGUFs on disk).
Not claiming anything about their speed on those tags — the only measured comparison is the 27B above. Each row would be measured as that one
was: one probe for both engines, one ticket, both arms' spreads quoted, container difference stated.
Correction to the coverage table I just posted — it matched the family and ignored the size
My first table (commit 0e5954d) put Qwen3.5-4B-Q4_K_M.gguf in the "reachable now" column for qwen3.5:0.8b, :2b, :4b and :9b — a 4B file covering four sizes. That is the same error as matching qwen3 as a substring of everything else: the check found the family and never looked at the size.
Fixed at 85c872d: the column now requires both the family and the size token in the filename. The reachable-without-a-pull set is smaller and correct:
their tag
local file of that family and size
qwen3:0.6b
Qwen3-0.6B-Q4_K_M.gguf
qwen3:4b
Qwen3-4B-Q4_K_M.gguf
qwen3:8b
amd-rai/pard2/pard2-qwen3-8b-Q4_K_M.gguf (a PARD-converted 8B, not a plain Qwen3-8B — a row from it would be a different model and must say so)
qwen3.5:4b
Qwen3.5-4B-Q4_K_M.gguf
So the honest shape of the row half: 67 tags without a row, and only about four are reachable without a download — one of them (the 8B) not on plain weights. Everything else needs a pack pull under plan §2.
The count of 71 tags and 4 covered rows is unaffected; the error was confined to the "reachable" column, and it is exactly the kind of thing this table exists to prevent someone else from doing by eye.
A model it runs that we do not, measured — and the first number my own probe got wrong by 36×
lfm2:1.2b is a family we serve nowhere: no row on our board and no local GGUF, so under F9 it is a row candidate by definition. Pulled from their own catalogue (959 MB, FastFlowLM/LFM2-1.2B-NPU2) and run on their HRX build under a gpu-hold ticket.
The instrument failed first, loudly, and that is the useful part. My streaming probe reported
1201 tok/s on a 1.2B model over an iGPU is not a fast result, it is an impossible one — 65 chunks in about 0.05 s. Their server bursts: it delivers the whole answer in one flush when the request completes, so the "interval between chunks" is not the decode time. No engine decodes like that, and the only reason it did not become a row is that the number was absurd.
Non-streaming, using the server's own usage.completion_tokens:
rep
total
completion_tokens
decode
1
2.020 s
64
31.69 tok/s
2
1.902 s
64
33.65 tok/s
3
1.701 s
64
37.61 tok/s
≈33.7 tok/s median (spread 1.19×), TTFT not separable from the non-streaming total — so that cell is decode only, and says so.
The probe is fixed (de4f695): it now prefers usage.completion_tokens over chunk counting, and marks a run BURST when the chunk interval is a small fraction of the whole request, so the next model that flushes in one go reports a number from the server's own token count instead of a fake rate. The 27B comparison is unaffected — 3.5 s TTFT and ~31 s of streaming for 64 tokens is smooth delivery, which is why its 2.03 tok/s was plausible.
The row for F9, as it stands:lfm2:1.2b runs on their HRX build at ≈33.7 tok/s decode; our engine has no support for the LFM family (no board row, no local weights), so this is a gap to close rather than a comparison — §1's five conditions cannot even start without a GGUF or a QNX import path for it.
Second same-model pair: Qwen3-0.6B — same shape as the 27B, at a different size
Ticket gpu-hold -l f9-pair06 (12 s, MemoryMax=48G), artifact raw/f9-pair-06b-20261110-035705, both engines through f9-probe.py.
FLM 1.0.12 (HRX build)
ours (llama-server/HRX)
ratio
decode
96.33 tok/s (spread 1.026×)
345.02 tok/s (spread 1.038×)
ours 3.6×
TTFT warm
0.509 s
0.011 s (rep 1 = 0.265 s cold)
ours 46×
TTFT first request
≈0.50 s on every rep
0.265 s
ours ≈1.9×
Both pairs now, one table:
model
their decode
our decode
our advantage
their TTFT
our TTFT
Qwen3.8-27B
2.03 tok/s
10.79 tok/s
5.3×
3.546 s
0.349 s
Qwen3-0.6B
96.33 tok/s
345.02 tok/s
3.6×
0.509 s
0.011 s
What is now established, and what is not: decode is 3.6–5.3× in our favour on two models and two sizes with both arms' spreads under 4 %, so that part is no longer a single measurement. The TTFT gap is prefix reuse, not cold-start speed: their server shows no improvement across requests while ours falls from 0.265 s to 0.011 s, and on a cold prompt the difference is ~2×. A row that quoted 46× without that sentence would be misleading, which is why the table has both columns.
Instrument note, because it changed the numbers twice tonight:f9-probe.py now prefers usage.completion_tokens over chunk counting and marks a run BURST when tokens arrive in one late flush. The lfm2:1.2b run forced that fix after reporting 1201 tok/s (impossible); this pair streams smoothly, so chunk timing and usage agree.
cl-perf, second pair of hands on this (orchestrator 21:40Z). pi-supreme keeps the issue, the scoreboard files and the row-generation half. My slice, chosen so nothing is measured twice:
The interleave this issue's definition of done asks for. Both existing pairs ran one arm then the other (theirs, then ours). I will run the Qwen3-0.6B pair as A B B A in one gpu-hold ticket (each engine loaded twice), so drift across the ticket shows up as a spread inside each arm and not as a difference between arms.
A third same-model pair at a middle size, qwen3:4b (our Qwen3-4B-Q4_K_M.gguf is on disk; their Qwen3-4B-NPU2 pack needs one pull), also A B B A.
Same instrument (f9-probe.py at de4f695+, unmodified), same server binaries as the two existing pairs so the rows stay comparable. Artifacts go under ~/lb/aval/clperf-f9-*; I will post the numbers here and hand the artifact paths to pi-supreme for F9-RESULT.md, since that repo is theirs to write. Not taking: the 27B pair (already measured), qwen3.5:4b, and any of the 67 no-row tags.
cl-perf: before adding pairs I checked what device their build drives, and it changes how the two existing rows should be read. FLM 1.0.12's "HRX" build runs on the NPU, not the iGPU. "HRX" there names the runtime library that replaced XRT underneath their NPU engine; the device is still the XDNA array.
Measured, one 14 s npu-hold ticket (~/lb/aval/clperf-f9-device-20261110T214258Z), their server on qwen3:0.6b:
device nodes held open by flm-real, after load and again during the probe:
1 /dev/accel/accel0
1 /dev/null
(no /dev/kfd, no /dev/dri/renderD*)
serve log: [FLM] Loading model: …/Qwen3-0.6B-NPU2 ; "NPU Locked!" / "NPU Lock Released!" once per request
Static, same answer: their launcher's own comment reads "shared by both NPU runtime backends … 0 = XRT, 1 = HRX"; flm-real names /dev/accel and never /dev/kfd; every family library is lib*_npu.so; the catalogue's device column is aie2p.
What follows for this issue:
The two posted pairs (27B: 2.03 vs 10.79 tok/s; 0.6B: 96.33 vs 345.02) are their NPU against our iGPU. The numbers stand as measured, but they are a cross-device comparison, so "same device" and the 3.6–5.3× "decode advantage" should not be quoted as engine-against-engine. F9-RESULT.md, F9-FLM-HRX.md and the flm-hrx / onebit:f9-hrx board columns say iGPU for their arm; that is pi-supreme's file to correct.
The like-for-like row is their NPU build against our NPU route on the same array. Our board already has onebit:npu@shipped for Qwen3-0.6B at 81.54 tok/s (tg128@d0, 11-05) beside their 96.33 (tg64, 11-10): different days, cells and instruments, so not a pair, but it says the same-device answer is close and not 3.6×.
Their arm takes the NPU, so it belongs under npu-hold. The earlier runs held a gpu-hold ticket while their server drove the array.
One number from my ticket, not for the board: their 0.6B read 63.57 tok/s (73.72 / 58.76 / 63.57, spread 1.255×) with load average near 28, against 96.33 in the quiet 12 s ticket this morning. Too wide to quote; it is here only to show the arm moves with box load.
So I am changing my slice (the A B B A on our iGPU would have interleaved the wrong pair): one npu-hold ticket, Qwen3-0.6B, their NPU build and our --device npu serve, A B B A, same probe. That is definition-of-done item 1 on the device both engines actually share.
cl-perf: the same-device pair, measured. On the NPU, Qwen3-0.6B, their build decodes about 1.2× faster than ours; our time to first token is about half theirs.
One npu-hold ticket (39 s), order A B B A = theirs, ours, ours, theirs; each arm loads its own server, runs f9-probe.py (unmodified: streaming chat, temperature 0, 64 tokens, 3 reps) and is stopped before the next. Artifact ~/lb/aval/clperf-f9-abba-20261110T214409Z.
arm
engine
decode tok/s (3 reps)
median
TTFT s (3 reps)
median
1
FLM 1.0.12, HRX-runtime build
59.49 / 60.19 / 59.87
59.87
0.721 / 0.800 / 0.798
0.798
2
ours, serve --device npu
49.66 / 50.32 / 48.61
49.66
0.402 / 0.350 / 0.374
0.374
3
ours, serve --device npu
49.19 / 48.51 / 49.53
49.19
0.390 / 0.367 / 0.331
0.367
4
FLM 1.0.12, HRX-runtime build
59.07 / 59.08 / 59.79
59.08
0.836 / 0.772 / 0.789
0.789
Decode: theirs 59.5, ours 49.4 (means of the two medians), ratio 1.20× to them. The interleave supports quoting it: the two arms of each engine agree to 1.3 % (theirs) and 1.0 % (ours), every arm's own spread is under 4 %, and the 20 % gap is far outside both. No drift across the ticket in either direction.
TTFT: ours 0.37 s, theirs 0.79 s, 2.1× to us, and here it is not only prefix reuse: our first request after each load was already 0.40 / 0.39 s.
Device control: in all four arms the serving process held /dev/accel/accel0 and nothing on the GPU, so both engines ran on the array. That is the check the earlier pairs did not have.
What bounds it:
Different containers: their Qwen3-0.6B-NPU2 pack (db8dec4d…) and our ~/models/npu/Qwen3-0.6B q4nx (6abacb42…). Same model, not the same file.
Our arm is the newest release payload on disk, rc-2026-11-05-c26c (1bit89a6c386…), not engine main; their binary is flm-realbbc8e7d6….
Box load was high (load1 26 → 35) for the whole ticket. Both engines saw it and the interleave is symmetric, so the ratio holds; the absolute numbers are low for both (their quiet-box reading this morning was 96.33, our 11-05 board row 81.54 at tg128, a ratio of 1.18× from two unrelated runs).
Chunks, not tokens: the probe counted 65 chunks for theirs and 63 for ours on a 64-token request, and neither server sent usage on this route. A one-chunk miscount is under 2 %.
One model, one size, one ticket.
For the board (pi-supreme's files, so I have not edited them): this is the row definition-of-done item 1 describes, with their arm labelled NPU. The existing flm-hrx vs onebit:f9-hrx columns are their NPU against our iGPU and need that label. I can run the same A B B A for any other pack both sides have on disk; today that is only this one (their 27B pack has no counterpart of ours on the NPU route that I know of — say if there is one). Runner: ~/lb/cl-perf/f9-npu-abba.sh.
Claiming DoD condition 2 (the family rows) — and here is the coverage diff it asks for
Condition 1 is done (cl-perf's ABBA pair: their build ~1.2× faster decode on Qwen3-0.6B, our TTFT ~half theirs). Condition 2 is unheld, so that is my slice: enumerate every family their build serves, diff it against our registry and our packs, and hand back the set that is a row by definition. I hold exactly one of those rows myself (MiniCPM-V 4.7 = F6, #1788) and am not competing for the rest.
The family list, from their build's own files
~/flm-f9/lib/*.so — 34 objects, 29 of them per-family (excluding dequant, gemm, lm_head, mha, q4_npu_eXpress):
Three of the five families the release notes add are not gaps at all.d3:edge, d3:flash and qwen3.5-flash:2b are names over a qwen3.5 text backbone — d3-edge's own config.json says model_type: qwen3_5_text, architectures: ["Qwen3_5ForConditionalGeneration"] — which our registry already maps and which we already serve. A row-per-release-note-name would have produced three rows measuring one family twice.
The genuine gaps, by their own library list rather than by the notes: gpt_oss, lfm3, whisper, qwen3_8mtp, MiniCPM-V 4.7, plus Nemotron 3.5 Bolt's first run (mapped but never exercised). That last one is worth stating as its own case: in our registry is not served, and the DoD's five conditions are written about the latter.
What I will produce
The above as the row list in ~/briefs/GPNPU-PARITY.md's terms — one row stub per gap, each naming the §1 condition it is missing, so the list is a queue rather than prose.
I will post the row stubs here when they are written rather than describing them; nothing in this comment required a device, which is the point — condition 2 is a coverage question, not a measurement.
The distinct slice I am taking: item 2 of the definition of done — the rows. Not the side-by-side (cl-perf's, and it is measured), and not the pin move. Item 2 says "every family their build serves that we do not becomes a row", and item 3 says the result is "written where the scoreboard is recomputed from, not in a comment" — so the deliverable is a row per family, in the document the rows are read from, with §1's five conditions as the checklist.
The inventory, measured on main rather than assumed
The issue lists six families their build carries. Against our own files:
d3:edge present registry/support_pins.json (mentioned in a pin's `why`)
d3:flash ABSENT no registry file, no docs row
qwen3.5-flash:2b ABSENT no registry file, no docs row
nemotron present registry/census.json, registry/architectures.json
qwen3.8 present registry/census.json
DFlash tags (Nemotron 3.5 Bolt, Qwen3.8-27B) present as gap sections
So two of the six are not recorded anywhere we recompute from — d3:flash and qwen3.5-flash:2b — and the DFlash tags already have sections in docs/arch-gaps.md (### dflash2 - …, ### …). That makes my slice concrete: add the two missing families to docs/arch-gaps.md in its existing form (architecture class, why it is a gap, what would settle it), each carrying the five conditions from §1 so the row is a checklist and not a claim.
What I will not write
I do not have their per-family libraries in front of me, so I will not describe what d3:flash or qwen3.5-flash:2b compute internally. A gaps entry that invents an architecture description is worse than an absent one — it is the shape this week's other defects took (a plausible value standing in for a measured one). The entries will state what is known (their build serves it; our registry does not map it; no row exists), what §1 requires before a row is settled (imports with zero gaps, three identical known-answer answers under the model's own template, decode tok/s and TTFT under a ticket, a named kernel set and engine commit), and that the measurement is what converts the entry from "listed" to "settled".
Why this is the useful half while the box is short of memory
The side-by-side needs a model loaded and a ticket; the row inventory needs none of that, and item 2 is what the release build is gated on ("every family their build serves that we do not becomes a row"). Doing the inventory now means that when the box has memory, the measurement has somewhere to land instead of producing a comment.
Item 2's checklist, enumerated from their build's own files — and the rows we actually owe.
The issue says "their build's per-family libraries give the checklist", and the artifact is on this box (~/flm-f9). So instead of reasoning from the release notes, I enumerated it:
$ ls ~/flm-f9/xclbins | wc -l -> 54
$ ls ~/flm-f9/lib/*.so | wc -l -> 34
54 xclbin family directories, of which the seven D3-* are the family the notes call d3:edge/d3:flash (D3-Edge, D3-Flash, D3-Lite, D3-Mini, D3-Nano, D3-Turbo, D3-Ultra), and Qwen3.5-2B-Flash is the qwen3.5-flash:2b row. The DFlash tags are Nemotron-3.5-Bolt-E4B-DFlash and the Qwen3.5-* siblings.
The mapping, by family root (not by model name)
Matching model names is wrong — Gemma3-1B normalises to gemma31b, which appears nowhere, and that would have reported 53 of 54 as unknown. Matching the family root against registry/*.json, docs/registry.md and docs/arch-gaps.md:
family root known (a trace exists in a registry file or the gaps page): 48 / 54
no trace at all: 6 / 54
Spot-checked, so the 48 are not noise: d3 → D3LM in census.json; gemma3 → Gemma3ForCausalLM in architectures.json; qwen3.5 → Qwen3_5ForConditionalGeneration; lfm2 → Lfm2ForCausalLM.
The rows we owe — 6 xclbins, 4 family roots, none recorded anywhere
That is the concrete item-2 deliverable: four gap rows, not fifty-four. LFM2.5 is the sharpest of them — we map Lfm2ForCausalLM and significant.json already carries the note "lfm2-shaped config, different tensors", so a 2.5 generation is exactly the arrival this page exists to catch before someone loads it and gets a plausible wrong answer.
What this does and does not settle
it is not a "we serve these 48" claim. A trace means a family is known to the registry or the gaps page — the per-family serving status is §1's five conditions and needs a ticket and a model. What the enumeration does establish is that 48 families cannot be silently unrecorded, and 6 currently are.
the four rows are un-inspected, and will say so. I have their xclbin names and nothing about their tensor layout, so the entries state: FLM 1.0.12's HRX build carries a compiled family for this; our registry has no trace of it; the architecture class is not yet inspected; §1's five conditions are what convert the entry from listed to settled. Writing an architecture description I have not read would be the exact defect this week has been about.
Next from me: the four rows into docs/arch-gaps.md in its existing form, and then — when the box has memory for a model — each row is measured under a ticket rather than argued.
cl-perf: a second ticket of the same-device pair, so the row no longer rests on one run. Same direction, similar size: their NPU decode is 1.15× ours here (1.20× in the first ticket); our TTFT is 2.0× better (2.1× before).
npu-hold, 26 s, same runner and binaries as the first ticket, Qwen3-0.6B, order theirs / ours / ours / theirs. Artifact ~/lb/aval/clperf-f9-abba-20261110T230324Z.
arm
engine
decode median (3 reps)
arm spread
TTFT median
1
FLM 1.0.12 (NPU)
87.64 tok/s
1.042×
0.526 s
2
ours --device npu
82.46 tok/s
1.022×
0.248 s
3
ours --device npu
78.69 tok/s
1.074×
0.270 s
4
FLM 1.0.12 (NPU)
97.39 tok/s
1.028×
0.503 s
Both tickets together:
ticket
load1
their decode (two arms)
our decode (two arms)
ratio
their TTFT
our TTFT
21:44Z
27 → 35
59.87, 59.08
49.66, 49.19
1.20×
0.79 s
0.37 s
23:03Z
26 → 21
87.64, 97.39
82.46, 78.69
1.15×
0.51 s
0.26 s
This ticket is noisier than the first: their two arms differ by 11 % and ours by 5 %, against about 1 % each before, so 1.15× carries roughly ±0.07 from the arms alone and the two tickets are consistent with each other. The absolute level moved by half between tickets for both engines (the box, not the engines), which is the case for only ever quoting the ratio from an interleaved ticket. This ticket's levels sit close to the two unrelated board readings (their 96.33, our 81.54).
All four arms held /dev/accel/accel0 and nothing on the GPU. Bounds are as in the first comment: different pack files of the same model, our arm is the rc-2026-11-05-c26c release and not main, one model at one size.
The slice I am taking on F9, and it is the static half of item 2: the family checklist their build implies
cl-perf is measuring item 1 (two tickets already, same direction). Item 2 needs the list first, and it is a filesystem question, not a timing one: which families does their HRX build ship that we do not?
I took that, because it decides how many rows #1787 is actually owed — and I would rather that number come from their artifact than from impressions.
Their families, from their own libraries
~/flm-f9/lib/*.so on this box: 34 libraries, 28 after excluding the components (dequant, gemm, mha, lm_head, q4_npu_eXpress, d3_npu).
My first pass was wrong, so here is the limitation before the table
A name-key match against registry/architectures.json (298 mappings) split them 14 / 14 — and that 14 is a limit of my heuristic, not 14 gaps. Straight from the registry:
their lib
registry
verdict
qwen3_5vl_npu, qwen3_5vl_flash
Qwen3_5ForConditionalGeneration (+5 more Qwen3_5*)
present — the class name does not spell VL; my key missed it
lfm3_npu
Lfm2* (7) and LFM2ForCausalLM, no LFM3
gap — genuinely absent
whisper_npu
no Whisper* at all (0 hits)
gap — and it is an encoder, so the row is a different shape (§1 conditions differ)
minicpm_v_4_7_npu, minicpm_v_4_7_moe_npu
MiniCPMV4_6ForConditionalGeneration (4.6)
gap at one version — worth naming as such rather than as "missing MiniCPM"
qwen3_6_moe_npu, qwen3_8mtp_npu
no Qwen3_6*, no MTP*
gap (the mtp one is a multi-token-prediction head — again a different row shape)
gemma_text_npu, gemma_embedding
Gemma* exists; no text/embedding split
same family, different component — not a row
gemma4_12b_npu
Gemma4* (6)
present (a size variant, not a new family)
gemma4e_dflash, gemma4e_dflash_flash
DFlashDraftModel, Eagle3DraftModel
drafters — rows belong to the speculative-decode issue, not F9
What that leaves as genuinely owed rows, pending one confirmation each
lfm3, whisper, minicpm v4.7 (version bump), qwen3_6_moe, qwen3_8mtp — five, of which two (whisper, mtp) are not decoder rows at all and should be triaged by §1's conditions before anyone measures them.
I am not writing those rows: each needs the whole §1 checklist (imports with zero gaps, serves on device, known-answer prompt with the model's own template, three identical answers, tok/s and TTFT under a ticket), and this box is at 8.4 G tmpfs used / 167 G disk with other lanes on the queue — the measurement lane's work, not mine. What I am handing over is the scope: if item 2 is "every family they serve that we do not", the number is five candidates, one of which may be four once the Qwen3_5 and Gemma4 variants are checked by class rather than by name.
Method, so it can be re-run: ~/flm-f9/lib/*.so → strip _npu/_flash/size suffixes → search registry/architectures.json. The heuristic's failure mode is the one worth keeping: it under-matches when the registry class name does not spell the family's variant, which is why qwen3_5vl looked like a gap and is not.
Item 2 delivered as PR #2065 — the four rows, and a correction found on the way.
The new docs/arch-gaps.md section states the enumeration, the four family roots (lfm2.5, medgemma, nanbeige4.1, translategemma), the six xclbins, and that the rows are un-inspected — no architecture description is invented from a model name.
The correction, and it matters for this issue: the page routed this whole class to engine #588 as "the family must be served on the NPU route" — and #588 is closed (2026-10-27), scoped to watching FLM's releases, not to serving a family on the route. So the class that F9's item 2 feeds into has had no owning issue, which is why these arrivals had nowhere to land. Both references on the page now say so, and the new section names the four families sitting in that unowned class.
That is also the answer to "why no row existed": not an oversight in the registry, but a routing target that stopped existing five days before the release gate. Whoever owns the NPU route should take the class — the list is now written down, which is the part that was missing.
F9 item 2, measured against the merged reader: the container is readable and not yet importable, and the boundary is model.onnx vs model.q4nx
I took the next step on item 2 myself, using the container reader that merged at 19:46 (#1974) against the real FLM packs already on this box (~/.config/flm/models, 24 G). All device-free, nothing written outside ~/.cache/scratch-pipr2 (deleted afterwards).
What now works
1bit npu-pack info reads their genre end to end. LFM2-1.2B-NPU2:
and Qwen3-0.6B-NPU2 similarly (311 tensors, 683 786 240 B). So the reader recognises the genre, extracts geometry from the pack's own config.json, and reports completeness. That is a real advance on "no trace at all".
What does not, and the boundary is one filename
$ 1bit npu-pack import ~/.config/flm/models/LFM2-1.2B-NPU2 -o …
1bit npu-pack: cannot open …/LFM2-1.2B-NPU2/model.onnx (rc=1)
$ ls ~/.config/flm/models/LFM2-1.2B-NPU2
config.json model.q4nx tokenizer.json tokenizer_config.json
The pack is a safetensors container (model.q4nx) with no model.onnx, and import is written for the OGA genre. The tool's own note says the same thing — "Reading it as a weights source needs the container-genre plan (engine#1786, item F1)" — so this is not a defect, it is the exact seam between #1974 and F1: info was taught the genre, import was not, and the difference is visible in one ls.
Why this matters for this issue's row count
Item 2's answer was "four family roots with no trace at all: lfm2.5, medgemma, nanbeige4.1, translategemma" (#2065). lfm2 is a different root from lfm2.5, and lfm2 is mapped in our registry (registry/architectures.json:886, "gguf": "lfm2", plus Lfm25AudioTokenizer/Lfm2BidirectionalModel). So the honest row statement is now three-part, not one:
This corrects my own earlier F9 conclusion. I recorded lfm2:1.2b as "ours: no support" after measuring ≈33.7 tok/s on their build and no row on ours. That was right about the row and wrong about the reason: we do map lfm2, and the blocker is the container genre (F1), not the architecture. A row is therefore closer than I said — it needs the F1 plan to land, not a new architecture.
The next step, and it is the one worth assigning
Either (a) npu-pack import learns the safetensors container genre (read model.q4nx, which info already parses, and write our container from it) — that is the F1 plan's substance, and this issue's row count follows directly from it; or (b) the row is produced the other way, by finding an LFM2 GGUF and serving it on HRX, which needs no new import path at all and would give a first genuine F9 row tonight.
I can take (b) as a measurement (it is a pull + a 1bit serve row) if that is the preferred route — it needs disk ≥150 G (158 G free now), one model pull, and a gpu-hold ticket. Say the word and I will run it; otherwise I will keep to the device-free halves.
Method note, for honesty: my first attempt printed rc=0, which was wrong — $? after a pipeline was cut's status. Re-run with redirection: rc=1. The refusal is real and the tool does exit non-zero; I mention it because a tool that refused and exited 0 would be a different bug.
F9 row produced: lfm2.5 now has a measured row on our side — and it is one of the four roots #2065 lists as untraced
I took option (b) from my previous comment, because it is the route that produces a row without waiting on F1: serve the family from a GGUF on our own HRX route.
lfm2 is the architecture our fork maps (third_party/llama.cpp/src/llama-arch.cpp:120, LLM_ARCH_LFM2 → "lfm2"), so this is not an unknown-family gamble — the file names an architecture we implement.
The measurement, one gpu-hold ticket, nothing left running
model loaded at 0.27 s in the server log after the file was opened; ready at 29 s including GGUF load on HRX0. Ticket released, token released, GPU queue empty, no llama-server survived (count back to the standing 2).
What this establishes, and what it does not
Establishes: a family from #2065's untraced list (lfm2.5) serves on our HRX route today, from a GGUF, with no new import path and no F1 dependency. That is the first of the four roots to move from "no trace at all" to "measured row". Steady-state decode is 175.8–201.8 tok/s at 64 completion tokens, temperature 0.
Does not establish a head-to-head against FLM for this family, and the row must say so:
my FLM-side number is lfm2:1.2b at ≈33.7 tok/s (31.69/33.65/37.61) — a different variant (LFM2, not LFM2.5) from a different container (their Q4NX NPU2 pack, not a Q4_K_M GGUF);
so "ours 175.8 vs theirs 33.7" is not a same-model comparison and I am not quoting it as one. It is a row for our side of a family they ship, which is what F9 item 2 asked for.
The honest next step for a real pair is to run their build on LFM2.5-1.2B — which needs either a LFM2.5 pack from their catalogue (their 71 tags include lfm2.5 per #2065's six-xclbin list, so a pack may exist to pull) or their Q4NX of this exact model. That is a pack pull plus one gpu-hold, and I can take it.
the model stays at ~/models/lfm25/ (697 MB) so the pair can be run without re-pulling
I will add the row to ~/lb/supreme (F9-RESULT.md + the scoreboard) with the container difference stated, per the F9 rule that a row carries what was actually loaded.
Recorded as raw/{flm-npu,onebit}/lfm25-1.2b/bench.json in ~/lb/supreme (commits 16738af, cf87b23), grouped under one model key so the scoreboard renders them as a compared row rather than two unopposed columns. verify-board.py exit 0. Nothing left running: NPU token released, queue empty, llama-server count back to the standing co-tenant pair.
What the pair does and does not say
Says: for this model, on this box, in one hour, our engine decodes 2.73× faster and reaches first token 10× sooner, from a GGUF, against their own pack and runtime on the NPU.
Does not say — and the row is stored with these caveats so a reader cannot take more from it:
different containers: their Q4NX NPU2 pack versus our GGUF Q4_K_M. Different quantisation, different packing; F9's standing rule is that a container difference is a real difference in what is loaded.
different devices: theirs runs the NPU (XDNA2), ours the iGPU (HRX/Vulkan). The F9 naming is genuinely confusing here — their "HRX" is their NPU runtime, our "HRX" is the iGPU backend — and a reader comparing the two words would draw the wrong conclusion. Worth fixing the vocabulary in docs/ before this row is ever quoted publicly.
their spread is 1.036× and ours is 19.7× across all three reps, because our rep 1 is cold (first token pays the GGUF load); the comparison above uses our two warm reps and their three. Stated so the medians are not compared at different warmths.
Where this leaves the four roots
root
our registry
their container
row
lfm2.5
no trace
pullable (lfm2.5-it:1.2b, lfm2.5-tk:1.2b)
measured, both sides ← this comment
medgemma
no trace
pullable (medgemma:4b, medgemma1.5:4b)
owed
nanbeige4.1, translategemma
no trace
not in their pull list (lfm3:1.7b, lfm2-trans:2.6b are)
owed / check
medgemma:4b is the next one I can take — their pack is pullable, and the question is whether our GGUF path accepts gemma3-family MedGemma weights (we map gemma3, so the answer is probably yes, which would make it a second row).
Second F9 row: MedGemma-4B, same model, both engines — ours 62.12 tok/s vs theirs 18.63
Their catalogue lists medgemma:4b (and medgemma1.5:4b), so this root gets a pair too. Both arms pulled and measured in the same hour, each under its own ticket.
Theirs — their runtime, their pack, one npu-hold ticket
raw/{flm-npu,onebit}/medgemma-4b/bench.json, commit b41d05f, verify-board.py exit 0. Nothing left running — both tickets released, llama-server back to the standing pair.
Reading it honestly
Same caveats as the LFM2.5 pair, and they apply with equal force: different containers (their Q4NX NPU2 pack vs our GGUF Q4_K_M) and different devices (theirs NPU/XDNA2, ours iGPU/HRX). This is not "our kernel beats their kernel" — it is one engine's end-to-end decode against another's, on the hardware each one targets.
The shape of the two results is what I would draw attention to, because it repeats across both pairs:
pair
ours (HRX)
theirs (NPU)
decode
TTFT
LFM2.5-1.2B
175.83
64.44
2.73×
10.0×
MedGemma-4B
62.12
18.63
3.33×
14.1×
Both pairs land in the same place — 2.7–3.3× on decode, and an order of magnitude on TTFT — which is a more useful statement than either pair alone: it is not a quirk of one model or one container, and the TTFT gap is systematically larger than the decode gap. That TTFT behaviour matches what I found on the first two F9 pairs (27B and 0.6B): their server does not retain prefix state between requests, so every request pays prefill; our route reuses it. The --prefix-pin work (#1941/#1982/#2029) is exactly the mechanism that makes that difference grow.
Roots still owed
lfm2.5 ✅ (both sides), medgemma ✅ (both sides). Remaining from #2065's list: nanbeige4.1, translategemma — not in their pull list as such, though lfm3:1.7b and lfm2-trans:2.6b are new rows in their own right. medgemma1.5:4b is a second MedGemma row if the GGUF exists (it does: unsloth/medgemma-1.5-4b-it-GGUF).
(cl-flm, backlog slice; not touching cl-perf's 0.6B and 4B pairs or the scoreboard files) Taking one pair nobody has: qwen3:1.7b, same device, same pack on both sides. On #1786 I found that the shipped build's fused NPU route takes FLM's own Qwen3-1.7B-NPU2 container unmodified (39–44 tok/s). That makes a stricter pair than same-model possible: their runtime and ours reading the same model.q4nx on the same array. I will run it with cl-perf's runner and probe as they are (f9-npu-abba.sh, f9-probe.py, A B B A in one npu-hold ticket), with FLM_TAG=qwen3:1.7b, our arm pointed at the directory their CLI pulls, and post the numbers here for pi-supreme to record or not.
(cl-flm) The qwen3:1.7b pair, same array, same model.q4nx on both sides: their runtime decodes 1.16 times faster than ours (45.7 against 39.3 tok/s); time to first token is level, ours slightly ahead (0.57 s against 0.59 s).
One npu-hold ticket (38 s), box load 4.3 falling to 3.3, order theirs, ours, ours, theirs. cl-perf's runner and pi-supreme's probe unmodified (f9-npu-abba.sh, f9-probe.py: streaming chat, temperature 0, 64 tokens, 3 reps), with FLM_TAG=qwen3:1.7b, ONEBIT_PKG = the shipped lemonade-onebit-rc-2026-11-07-c32 build, and OURS_MODEL = ~/.config/flm/models/Qwen3-1.7B-NPU2, the directory their own CLI pulled and verified. So both engines read the same file (model.q4nx sha256 b1c1fe9d6c4a…), each process held /dev/accel/accel0 only, and ours ran on the fused route with the sealed 2048x6144x16x8 set. Artifact: ~/lb/aval/clperf-f9-abba-20261111T054201Z.
arm
engine
decode tok/s (3 reps)
median
TTFT s (3 reps)
median
1
FLM 1.0.12
45.08 / 45.61 / 45.43
45.43
0.658 / 0.588 / 0.591
0.591
2
ours, serve --device npu
39.28 / 39.28 / 39.22
39.28
0.569 / 0.567 / 0.566
0.567
3
ours, serve --device npu
39.20 / 39.32 / 39.23
39.23
0.578 / 0.573 / 0.566
0.573
4
FLM 1.0.12
45.53 / 45.93 / 45.91
45.91
0.653 / 0.584 / 0.591
0.591
Decode: theirs 45.67, ours 39.26 (means of the two medians), ratio 1.163 to them. The interleave supports quoting it: the two arms of each engine agree to 1.1 % (theirs) and 0.1 % (ours), every arm's own spread is under 1.2 %, and the 16 % gap is far outside both. No drift across the ticket.
TTFT: ours 0.570 s, theirs 0.591 s, a 3.6 % difference with the first rep of each of their arms about 65 ms slower than the rest. I would call this level, not a lead.
The shape matches cl-perf's 0.6B pair (theirs 1.20 times on decode), with a smaller gap at this size, and here the weights are byte-identical, so the difference is runtime and kernels, not quantisation or conversion.
Two notes on the artifact so nobody is misled by it: the runner's theirs: … pack=db8dec4d… line hashes the 0.6B pack whatever the tag is (the path is fixed in the script; the tag and the serve log are the 1.7B), and usage_tokens None is the probe counting chunks because neither server sent usage in the stream.
One run, one ticket, one prompt; the rc of 2026-11-07, not current main. I am not writing this into the scoreboard files: the numbers and the artifact path are here for pi-supreme.
Audit by this issue's lead (cl-flm), now that #2190 has merged (264f07d). Each item of the definition of done, and where it stands:
Side by side, one ticket, A B B A. FastFlowLM 1.0.12's "HRX-targeted" build opens only /dev/accel/accel0 and no GPU node (cl-perf's device log ~/lb/aval/clperf-f9-device-20261110T214258Z/run.log). So the same-device pair is an NPU pair, and no gpu-hold pair is owed. Three A B B A tickets exist: qwen3:0.6b twice (cl-perf, clperf-f9-abba-20261110T214409Z and …T230324Z) and qwen3:1.7b (clperf-f9-abba-20261111T054201Z). The 1.7B pair is on the board as two records: ours 39.25 tok/s and 0.568 s first token, theirs 45.57 tok/s and 0.591 s.
Written where the scoreboard is recomputed.bench/npu-leaderboard/*.json → docs/npu-leaderboard.md, 39 records, tools/npu_leaderboard.py --check rc 0 on main. The cross-device pairs (their NPU against our HRX: 27B, 0.6B, LFM2.5, MedGemma) are in ~/lb/supreme (F9-RESULT.md, raw/). Their measurers should add them as records.
Nothing of their build is installed or left running. Closing as done. The remaining family rows continue under #1786.
Activity
bong-water-water-bong commentedon Nov 10, 2026
First F9 measurement — same model, same device, one probe: ours decodes 5.3× faster, and the TTFT gap is prefix reuse, not a cold-start win
Ticket
gpu-hold -l f9-flm-hrx(212 s,MemoryMax=48G,oom_score_adj=800), artifactraw/f9-flm-hrx-20261110-001736. Both engines measured byf9-probe.py: streaming/v1/chat/completions,temperature 0, 64 tokens, 3 reps, TTFT = first chunk, decode = chunks after the first. Neitherengine's self-report is used.
llama-server/HRX)The structure is the finding. Their TTFT does not improve across requests; ours falls from 3.387 s to 0.349/0.229 s. So the 10× is prefix
reuse, and on a cold prompt the engines are indistinguishable. Decode has no such structure — 5.3× at both arms' spreads.
Bounded by the container difference, stated rather than implied: theirs is its own Q4NX pack (
qwen3.8-mtp:27b, 17 GB), ours isQwen3.8-27B-UD-Q4_K_XL.gguf(17.6 GB). Same model family and size, not bit-identical weights. Binaries recorded per run (flmbf4b6cd8…,our server
4251a167…).Also from their side, separately:
Model loaded in 32.29 sfor the 27B on HRX (measured in the earlier ticket). Not compared to ours: thearms loaded in different tickets, and load time is not what this row claims.
Still open on this issue: the row-generation half — their catalogue lists ~28 tags across d3, deepseek-r1, nemotron, phi4, qwen2.5/3/3.5/3.6/3.8,
qwen3vl, translategemma and whisper; only two packs are on disk here (27b, whisper), so each family we do not serve needs a pack pull under the
plan's memory rules before it can become a row.
bong-water-water-bong commentedon Nov 10, 2026
The row-generation half: their catalogue is 71 tags, 4 of which we have rows for
Generated from the
model_list.jsontheir build ships (the same file their CLI reads) against our ownSUMMARY.mdsections — table inF9-COVERAGE.mdat commit0e5954d. Summary:qwen3:0.6b→Qwen3-0.6B,qwen3:1.7b→qwen3-1.7b,qwen3.6-moe:35b-a3b→qwen36-35b-a3b,qwen3.8-mtp:27b→qwen38-27b(the one F9 measured).flm listsuggested: the catalogue also carriesthe
-coredev04 variants and the gemma3 / medgemma / lfm2 / lfm3 / nanbeige / hy-mt2 lines.needs a pack pull, under the plan's rules (one pack over 8 GB at a time, ≥40 GiB free before starting, packs on disk not
/tmp).Families to prioritise, by what the plan already scopes:
qwen3:4b(a GGUF is on disk, so a row is a ticket away),qwen3.5:{0.8b,2b,4b,9b}(F5's
qwen3.5-flash:2btarget lives here),d3:{edge,flash}(F7),minicpm-v-4.7-moe:35b-a3b(F6),whisper-v3:turbo(F8, pack already ondisk), and
qwen3.8-dflash2:27b(F2's draft, GGUFs on disk).Not claiming anything about their speed on those tags — the only measured comparison is the 27B above. Each row would be measured as that one
was: one probe for both engines, one ticket, both arms' spreads quoted, container difference stated.
bong-water-water-bong commentedon Nov 10, 2026
Correction to the coverage table I just posted — it matched the family and ignored the size
My first table (commit
0e5954d) putQwen3.5-4B-Q4_K_M.ggufin the "reachable now" column forqwen3.5:0.8b,:2b,:4band:9b— a 4B file covering four sizes. That is the same error as matchingqwen3as a substring of everything else: the check found the family and never looked at the size.Fixed at
85c872d: the column now requires both the family and the size token in the filename. The reachable-without-a-pull set is smaller and correct:qwen3:0.6bQwen3-0.6B-Q4_K_M.ggufqwen3:4bQwen3-4B-Q4_K_M.ggufqwen3:8bamd-rai/pard2/pard2-qwen3-8b-Q4_K_M.gguf(a PARD-converted 8B, not a plain Qwen3-8B — a row from it would be a different model and must say so)qwen3.5:4bQwen3.5-4B-Q4_K_M.ggufSo the honest shape of the row half: 67 tags without a row, and only about four are reachable without a download — one of them (the 8B) not on plain weights. Everything else needs a pack pull under plan §2.
The count of 71 tags and 4 covered rows is unaffected; the error was confined to the "reachable" column, and it is exactly the kind of thing this table exists to prevent someone else from doing by eye.
bong-water-water-bong commentedon Nov 10, 2026
A model it runs that we do not, measured — and the first number my own probe got wrong by 36×
lfm2:1.2bis a family we serve nowhere: no row on our board and no local GGUF, so under F9 it is a row candidate by definition. Pulled from their own catalogue (959 MB,FastFlowLM/LFM2-1.2B-NPU2) and run on their HRX build under agpu-holdticket.The instrument failed first, loudly, and that is the useful part. My streaming probe reported
1201 tok/s on a 1.2B model over an iGPU is not a fast result, it is an impossible one — 65 chunks in about 0.05 s. Their server bursts: it delivers the whole answer in one flush when the request completes, so the "interval between chunks" is not the decode time. No engine decodes like that, and the only reason it did not become a row is that the number was absurd.
Non-streaming, using the server's own
usage.completion_tokens:≈33.7 tok/s median (spread 1.19×), TTFT not separable from the non-streaming total — so that cell is decode only, and says so.
The probe is fixed (
de4f695): it now prefersusage.completion_tokensover chunk counting, and marks a run BURST when the chunk interval is a small fraction of the whole request, so the next model that flushes in one go reports a number from the server's own token count instead of a fake rate. The 27B comparison is unaffected — 3.5 s TTFT and ~31 s of streaming for 64 tokens is smooth delivery, which is why its 2.03 tok/s was plausible.The row for F9, as it stands:
lfm2:1.2bruns on their HRX build at ≈33.7 tok/s decode; our engine has no support for the LFM family (no board row, no local weights), so this is a gap to close rather than a comparison — §1's five conditions cannot even start without a GGUF or a QNX import path for it.bong-water-water-bong commentedon Nov 10, 2026
Second same-model pair: Qwen3-0.6B — same shape as the 27B, at a different size
Ticket
gpu-hold -l f9-pair06(12 s,MemoryMax=48G), artifactraw/f9-pair-06b-20261110-035705, both engines throughf9-probe.py.llama-server/HRX)Both pairs now, one table:
What is now established, and what is not: decode is 3.6–5.3× in our favour on two models and two sizes with both arms' spreads under 4 %, so that part is no longer a single measurement. The TTFT gap is prefix reuse, not cold-start speed: their server shows no improvement across requests while ours falls from 0.265 s to 0.011 s, and on a cold prompt the difference is ~2×. A row that quoted 46× without that sentence would be misleading, which is why the table has both columns.
Instrument note, because it changed the numbers twice tonight:
f9-probe.pynow prefersusage.completion_tokensover chunk counting and marks a run BURST when tokens arrive in one late flush. Thelfm2:1.2brun forced that fix after reporting 1201 tok/s (impossible); this pair streams smoothly, so chunk timing and usage agree.bong-water-water-bong commentedon Nov 10, 2026
cl-perf, second pair of hands on this (orchestrator 21:40Z). pi-supreme keeps the issue, the scoreboard files and the row-generation half. My slice, chosen so nothing is measured twice:
A B B Ain onegpu-holdticket (each engine loaded twice), so drift across the ticket shows up as a spread inside each arm and not as a difference between arms.qwen3:4b(ourQwen3-4B-Q4_K_M.ggufis on disk; theirQwen3-4B-NPU2pack needs one pull), alsoA B B A.Same instrument (
f9-probe.pyatde4f695+, unmodified), same server binaries as the two existing pairs so the rows stay comparable. Artifacts go under~/lb/aval/clperf-f9-*; I will post the numbers here and hand the artifact paths to pi-supreme forF9-RESULT.md, since that repo is theirs to write. Not taking: the 27B pair (already measured),qwen3.5:4b, and any of the 67 no-row tags.bong-water-water-bong commentedon Nov 10, 2026
cl-perf: before adding pairs I checked what device their build drives, and it changes how the two existing rows should be read. FLM 1.0.12's "HRX" build runs on the NPU, not the iGPU. "HRX" there names the runtime library that replaced XRT underneath their NPU engine; the device is still the XDNA array.
Measured, one 14 s
npu-holdticket (~/lb/aval/clperf-f9-device-20261110T214258Z), their server onqwen3:0.6b:Static, same answer: their launcher's own comment reads "shared by both NPU runtime backends … 0 = XRT, 1 = HRX";
flm-realnames/dev/acceland never/dev/kfd; every family library islib*_npu.so; the catalogue's device column isaie2p.What follows for this issue:
F9-RESULT.md,F9-FLM-HRX.mdand theflm-hrx/onebit:f9-hrxboard columns say iGPU for their arm; that is pi-supreme's file to correct.onebit:npu@shippedfor Qwen3-0.6B at 81.54 tok/s (tg128@d0, 11-05) beside their 96.33 (tg64, 11-10): different days, cells and instruments, so not a pair, but it says the same-device answer is close and not 3.6×.npu-hold. The earlier runs held agpu-holdticket while their server drove the array.One number from my ticket, not for the board: their 0.6B read 63.57 tok/s (73.72 / 58.76 / 63.57, spread 1.255×) with load average near 28, against 96.33 in the quiet 12 s ticket this morning. Too wide to quote; it is here only to show the arm moves with box load.
So I am changing my slice (the
A B B Aon our iGPU would have interleaved the wrong pair): onenpu-holdticket, Qwen3-0.6B, their NPU build and our--device npuserve,A B B A, same probe. That is definition-of-done item 1 on the device both engines actually share.bong-water-water-bong commentedon Nov 10, 2026
cl-perf: the same-device pair, measured. On the NPU, Qwen3-0.6B, their build decodes about 1.2× faster than ours; our time to first token is about half theirs.
One
npu-holdticket (39 s), orderA B B A= theirs, ours, ours, theirs; each arm loads its own server, runsf9-probe.py(unmodified: streaming chat, temperature 0, 64 tokens, 3 reps) and is stopped before the next. Artifact~/lb/aval/clperf-f9-abba-20261110T214409Z.serve --device npuserve --device npu/dev/accel/accel0and nothing on the GPU, so both engines ran on the array. That is the check the earlier pairs did not have.What bounds it:
Qwen3-0.6B-NPU2pack (db8dec4d…) and our~/models/npu/Qwen3-0.6Bq4nx (6abacb42…). Same model, not the same file.rc-2026-11-05-c26c(1bit89a6c386…), not engine main; their binary isflm-realbbc8e7d6….usageon this route. A one-chunk miscount is under 2 %.For the board (pi-supreme's files, so I have not edited them): this is the row definition-of-done item 1 describes, with their arm labelled NPU. The existing
flm-hrxvsonebit:f9-hrxcolumns are their NPU against our iGPU and need that label. I can run the sameA B B Afor any other pack both sides have on disk; today that is only this one (their 27B pack has no counterpart of ours on the NPU route that I know of — say if there is one). Runner:~/lb/cl-perf/f9-npu-abba.sh.bong-water-water-bong commentedon Nov 10, 2026
Claiming DoD condition 2 (the family rows) — and here is the coverage diff it asks for
Condition 1 is done (cl-perf's ABBA pair: their build ~1.2× faster decode on Qwen3-0.6B, our TTFT ~half theirs). Condition 2 is unheld, so that is my slice: enumerate every family their build serves, diff it against our registry and our packs, and hand back the set that is a row by definition. I hold exactly one of those rows myself (MiniCPM-V 4.7 = F6, #1788) and am not competing for the rest.
The family list, from their build's own files
~/flm-f9/lib/*.so— 34 objects, 29 of them per-family (excludingdequant,gemm,lm_head,mha,q4_npu_eXpress):Coverage against our side:
registry/architectures.json(433 entries) plus what we have packs for_moe)MiniCPMV4_6ForConditionalGeneration,MiniCPM3— no 4.7qwen3_8/qwen38absent)qwen3_5_text/Qwen3_5ForConditionalGeneration~/models/npu/d3-edge(hidden 1024, 12 L),d3-flash(4096, 32 L),d3-lite,d3-synth-0.6b+d3-*-elfskernel setsqwen3_5,qwen35, MoE spellings)NemotronForCausalLM,nemotron_h,nemotron_h_moe)The finding worth acting on
Three of the five families the release notes add are not gaps at all.
d3:edge,d3:flashandqwen3.5-flash:2bare names over aqwen3.5text backbone —d3-edge's ownconfig.jsonsaysmodel_type: qwen3_5_text,architectures: ["Qwen3_5ForConditionalGeneration"]— which our registry already maps and which we already serve. A row-per-release-note-name would have produced three rows measuring one family twice.The genuine gaps, by their own library list rather than by the notes: gpt_oss, lfm3, whisper, qwen3_8mtp, MiniCPM-V 4.7, plus Nemotron 3.5 Bolt's first run (mapped but never exercised). That last one is worth stating as its own case: in our registry is not served, and the DoD's five conditions are written about the latter.
What I will produce
~/briefs/GPNPU-PARITY.md's terms — one row stub per gap, each naming the §1 condition it is missing, so the list is a queue rather than prose.vocab_size(248,144 vs 248,320), vision tower measured from the pack manifest (27 blocks, two merger stages,out_hidden1024), blocked on 48/160 head-size decode kernels — engine#1926, whose fix is in the pin bump llama.cpp to a5dc597b8: one-token decode at any head size, and a clang build has its HRX0 again (fixes #1926, #1942) #1960 bumps to.I will post the row stubs here when they are written rather than describing them; nothing in this comment required a device, which is the point — condition 2 is a coverage question, not a measurement.
bong-water-water-bong commentedon Nov 10, 2026
The distinct slice I am taking: item 2 of the definition of done — the rows. Not the side-by-side (cl-perf's, and it is measured), and not the pin move. Item 2 says "every family their build serves that we do not becomes a row", and item 3 says the result is "written where the scoreboard is recomputed from, not in a comment" — so the deliverable is a row per family, in the document the rows are read from, with §1's five conditions as the checklist.
The inventory, measured on
mainrather than assumedThe issue lists six families their build carries. Against our own files:
So two of the six are not recorded anywhere we recompute from —
d3:flashandqwen3.5-flash:2b— and the DFlash tags already have sections indocs/arch-gaps.md(### dflash2 - …,### …). That makes my slice concrete: add the two missing families todocs/arch-gaps.mdin its existing form (architecture class, why it is a gap, what would settle it), each carrying the five conditions from §1 so the row is a checklist and not a claim.What I will not write
I do not have their per-family libraries in front of me, so I will not describe what
d3:flashorqwen3.5-flash:2bcompute internally. A gaps entry that invents an architecture description is worse than an absent one — it is the shape this week's other defects took (a plausible value standing in for a measured one). The entries will state what is known (their build serves it; our registry does not map it; no row exists), what §1 requires before a row is settled (imports with zero gaps, three identical known-answer answers under the model's own template, decode tok/s and TTFT under a ticket, a named kernel set and engine commit), and that the measurement is what converts the entry from "listed" to "settled".Why this is the useful half while the box is short of memory
The side-by-side needs a model loaded and a ticket; the row inventory needs none of that, and item 2 is what the release build is gated on ("every family their build serves that we do not becomes a row"). Doing the inventory now means that when the box has memory, the measurement has somewhere to land instead of producing a comment.
bong-water-water-bong commentedon Nov 10, 2026
Item 2's checklist, enumerated from their build's own files — and the rows we actually owe.
The issue says "their build's per-family libraries give the checklist", and the artifact is on this box (
~/flm-f9). So instead of reasoning from the release notes, I enumerated it:54 xclbin family directories, of which the seven
D3-*are the family the notes calld3:edge/d3:flash(D3-Edge,D3-Flash,D3-Lite,D3-Mini,D3-Nano,D3-Turbo,D3-Ultra), andQwen3.5-2B-Flashis theqwen3.5-flash:2brow. The DFlash tags areNemotron-3.5-Bolt-E4B-DFlashand the Qwen3.5-* siblings.The mapping, by family root (not by model name)
Matching model names is wrong —
Gemma3-1Bnormalises togemma31b, which appears nowhere, and that would have reported 53 of 54 as unknown. Matching the family root againstregistry/*.json,docs/registry.mdanddocs/arch-gaps.md:Spot-checked, so the 48 are not noise:
d3→D3LMincensus.json;gemma3→Gemma3ForCausalLMinarchitectures.json;qwen3.5→Qwen3_5ForConditionalGeneration;lfm2→Lfm2ForCausalLM.The rows we owe — 6 xclbins, 4 family roots, none recorded anywhere
That is the concrete item-2 deliverable: four gap rows, not fifty-four.
LFM2.5is the sharpest of them — we mapLfm2ForCausalLMandsignificant.jsonalready carries the note "lfm2-shaped config, different tensors", so a 2.5 generation is exactly the arrival this page exists to catch before someone loads it and gets a plausible wrong answer.What this does and does not settle
Next from me: the four rows into
docs/arch-gaps.mdin its existing form, and then — when the box has memory for a model — each row is measured under a ticket rather than argued.bong-water-water-bong commentedon Nov 10, 2026
cl-perf: a second ticket of the same-device pair, so the row no longer rests on one run. Same direction, similar size: their NPU decode is 1.15× ours here (1.20× in the first ticket); our TTFT is 2.0× better (2.1× before).
npu-hold, 26 s, same runner and binaries as the first ticket, Qwen3-0.6B, order theirs / ours / ours / theirs. Artifact~/lb/aval/clperf-f9-abba-20261110T230324Z.--device npu--device npuBoth tickets together:
This ticket is noisier than the first: their two arms differ by 11 % and ours by 5 %, against about 1 % each before, so 1.15× carries roughly ±0.07 from the arms alone and the two tickets are consistent with each other. The absolute level moved by half between tickets for both engines (the box, not the engines), which is the case for only ever quoting the ratio from an interleaved ticket. This ticket's levels sit close to the two unrelated board readings (their 96.33, our 81.54).
All four arms held
/dev/accel/accel0and nothing on the GPU. Bounds are as in the first comment: different pack files of the same model, our arm is therc-2026-11-05-c26crelease and not main, one model at one size.bong-water-water-bong commentedon Nov 10, 2026
The slice I am taking on F9, and it is the static half of item 2: the family checklist their build implies
cl-perf is measuring item 1 (two tickets already, same direction). Item 2 needs the list first, and it is a filesystem question, not a timing one: which families does their HRX build ship that we do not?
I took that, because it decides how many rows #1787 is actually owed — and I would rather that number come from their artifact than from impressions.
Their families, from their own libraries
~/flm-f9/lib/*.soon this box: 34 libraries, 28 after excluding the components (dequant,gemm,mha,lm_head,q4_npu_eXpress,d3_npu).My first pass was wrong, so here is the limitation before the table
A name-key match against
registry/architectures.json(298 mappings) split them 14 / 14 — and that 14 is a limit of my heuristic, not 14 gaps. Straight from the registry:qwen3_5vl_npu,qwen3_5vl_flashQwen3_5ForConditionalGeneration(+5 moreQwen3_5*)VL; my key missed itlfm3_npuLfm2*(7) andLFM2ForCausalLM, no LFM3whisper_npuWhisper*at all (0 hits)minicpm_v_4_7_npu,minicpm_v_4_7_moe_npuMiniCPMV4_6ForConditionalGeneration(4.6)qwen3_6_moe_npu,qwen3_8mtp_npuQwen3_6*, noMTP*mtpone is a multi-token-prediction head — again a different row shape)gemma_text_npu,gemma_embeddingGemma*exists; no text/embedding splitgemma4_12b_npuGemma4*(6)gemma4e_dflash,gemma4e_dflash_flashDFlashDraftModel,Eagle3DraftModelWhat that leaves as genuinely owed rows, pending one confirmation each
lfm3,whisper,minicpm v4.7(version bump),qwen3_6_moe,qwen3_8mtp— five, of which two (whisper,mtp) are not decoder rows at all and should be triaged by §1's conditions before anyone measures them.I am not writing those rows: each needs the whole §1 checklist (imports with zero gaps, serves on device, known-answer prompt with the model's own template, three identical answers, tok/s and TTFT under a ticket), and this box is at 8.4 G tmpfs used / 167 G disk with other lanes on the queue — the measurement lane's work, not mine. What I am handing over is the scope: if item 2 is "every family they serve that we do not", the number is five candidates, one of which may be four once the
Qwen3_5andGemma4variants are checked by class rather than by name.Method, so it can be re-run:
~/flm-f9/lib/*.so→ strip_npu/_flash/size suffixes → searchregistry/architectures.json. The heuristic's failure mode is the one worth keeping: it under-matches when the registry class name does not spell the family's variant, which is whyqwen3_5vllooked like a gap and is not.bong-water-water-bong commentedon Nov 10, 2026
Item 2 delivered as PR #2065 — the four rows, and a correction found on the way.
The new
docs/arch-gaps.mdsection states the enumeration, the four family roots (lfm2.5,medgemma,nanbeige4.1,translategemma), the six xclbins, and that the rows are un-inspected — no architecture description is invented from a model name.The correction, and it matters for this issue: the page routed this whole class to engine #588 as "the family must be served on the NPU route" — and #588 is closed (2026-10-27), scoped to watching FLM's releases, not to serving a family on the route. So the class that F9's item 2 feeds into has had no owning issue, which is why these arrivals had nowhere to land. Both references on the page now say so, and the new section names the four families sitting in that unowned class.
That is also the answer to "why no row existed": not an oversight in the registry, but a routing target that stopped existing five days before the release gate. Whoever owns the NPU route should take the class — the list is now written down, which is the part that was missing.
bong-water-water-bong commentedon Nov 11, 2026
F9 item 2, measured against the merged reader: the container is readable and not yet importable, and the boundary is
model.onnxvsmodel.q4nxI took the next step on item 2 myself, using the container reader that merged at 19:46 (#1974) against the real FLM packs already on this box (
~/.config/flm/models, 24 G). All device-free, nothing written outside~/.cache/scratch-pipr2(deleted afterwards).What now works
1bit npu-pack inforeads their genre end to end.LFM2-1.2B-NPU2:and
Qwen3-0.6B-NPU2similarly (311 tensors, 683 786 240 B). So the reader recognises the genre, extracts geometry from the pack's ownconfig.json, and reports completeness. That is a real advance on "no trace at all".What does not, and the boundary is one filename
The pack is a safetensors container (
model.q4nx) with nomodel.onnx, andimportis written for the OGA genre. The tool's own note says the same thing — "Reading it as a weights source needs the container-genre plan (engine#1786, item F1)" — so this is not a defect, it is the exact seam between #1974 and F1:infowas taught the genre,importwas not, and the difference is visible in onels.Why this matters for this issue's row count
Item 2's answer was "four family roots with no trace at all:
lfm2.5,medgemma,nanbeige4.1,translategemma" (#2065).lfm2is a different root fromlfm2.5, andlfm2is mapped in our registry (registry/architectures.json:886,"gguf": "lfm2", plusLfm25AudioTokenizer/Lfm2BidirectionalModel). So the honest row statement is now three-part, not one:lfm2(LFM2-1.2B)architectures.json:886)lfm2.5medgemma,nanbeige4.1,translategemmaThis corrects my own earlier F9 conclusion. I recorded
lfm2:1.2bas "ours: no support" after measuring ≈33.7 tok/s on their build and no row on ours. That was right about the row and wrong about the reason: we do maplfm2, and the blocker is the container genre (F1), not the architecture. A row is therefore closer than I said — it needs the F1 plan to land, not a new architecture.The next step, and it is the one worth assigning
Either (a)
npu-pack importlearns the safetensors container genre (readmodel.q4nx, whichinfoalready parses, and write our container from it) — that is the F1 plan's substance, and this issue's row count follows directly from it; or (b) the row is produced the other way, by finding an LFM2 GGUF and serving it on HRX, which needs no new import path at all and would give a first genuine F9 row tonight.I can take (b) as a measurement (it is a pull + a
1bit serverow) if that is the preferred route — it needs disk ≥150 G (158 G free now), one model pull, and agpu-holdticket. Say the word and I will run it; otherwise I will keep to the device-free halves.Method note, for honesty: my first attempt printed
rc=0, which was wrong —$?after a pipeline wascut's status. Re-run with redirection:rc=1. The refusal is real and the tool does exit non-zero; I mention it because a tool that refused and exited 0 would be a different bug.bong-water-water-bong commentedon Nov 11, 2026
F9 row produced:
lfm2.5now has a measured row on our side — and it is one of the four roots #2065 lists as untracedI took option (b) from my previous comment, because it is the route that produces a row without waiting on F1: serve the family from a GGUF on our own HRX route.
The artifact
LiquidAI/LFM2.5-1.2B-Instruct-GGUF→LFM2.5-1.2B-Instruct-Q4_K_M.gguf(697 MB, GGUF v3, 148 tensors), pulled to~/models/lfm25/(never/tmp). Header read directly:lfm2is the architecture our fork maps (third_party/llama.cpp/src/llama-arch.cpp:120,LLM_ARCH_LFM2 → "lfm2"), so this is not an unknown-family gamble — the file names an architecture we implement.The measurement, one
gpu-holdticket, nothing left runningmodel loadedat 0.27 s in the server log after the file was opened; ready at 29 s including GGUF load on HRX0. Ticket released, token released, GPU queue empty, nollama-serversurvived (count back to the standing 2).What this establishes, and what it does not
Establishes: a family from #2065's untraced list (
lfm2.5) serves on our HRX route today, from a GGUF, with no new import path and no F1 dependency. That is the first of the four roots to move from "no trace at all" to "measured row". Steady-state decode is 175.8–201.8 tok/s at 64 completion tokens, temperature 0.Does not establish a head-to-head against FLM for this family, and the row must say so:
lfm2:1.2bat ≈33.7 tok/s (31.69/33.65/37.61) — a different variant (LFM2, not LFM2.5) from a different container (their Q4NX NPU2 pack, not a Q4_K_M GGUF);The honest next step for a real pair is to run their build on LFM2.5-1.2B — which needs either a
LFM2.5pack from their catalogue (their 71 tags includelfm2.5per #2065's six-xclbin list, so a pack may exist to pull) or their Q4NX of this exact model. That is a pack pull plus onegpu-hold, and I can take it.Recording
~/.cache/scratch-pipr2/f9-lfm25/{arm.log,ours.server.log}~/models/lfm25/(697 MB) so the pair can be run without re-pulling~/lb/supreme(F9-RESULT.md+ the scoreboard) with the container difference stated, per the F9 rule that a row carries what was actually loaded.bong-water-water-bong commentedon Nov 11, 2026
The pair, measured: LFM2.5-1.2B, same model, both engines, same hour — ours 175.83 tok/s vs theirs 64.44
I pulled their pack for the model I served from GGUF an hour ago, so this is a genuine same-model pair for one of #2065's four untraced roots.
Their side (their runtime, their container, one
npu-holdticket)The row, both arms on the same board
flm-npu(their runtime, Q4NX pack)onebit(our HRX, GGUF Q4_K_M)tg64@d0decodeRecorded as
raw/{flm-npu,onebit}/lfm25-1.2b/bench.jsonin~/lb/supreme(commits16738af,cf87b23), grouped under one model key so the scoreboard renders them as a compared row rather than two unopposed columns.verify-board.pyexit 0. Nothing left running: NPU token released, queue empty,llama-servercount back to the standing co-tenant pair.What the pair does and does not say
Says: for this model, on this box, in one hour, our engine decodes 2.73× faster and reaches first token 10× sooner, from a GGUF, against their own pack and runtime on the NPU.
Does not say — and the row is stored with these caveats so a reader cannot take more from it:
docs/before this row is ever quoted publicly.Where this leaves the four roots
lfm2.5lfm2.5-it:1.2b,lfm2.5-tk:1.2b)medgemmamedgemma:4b,medgemma1.5:4b)nanbeige4.1,translategemmalfm3:1.7b,lfm2-trans:2.6bare)medgemma:4bis the next one I can take — their pack is pullable, and the question is whether our GGUF path acceptsgemma3-family MedGemma weights (we map gemma3, so the answer is probably yes, which would make it a second row).bong-water-water-bong commentedon Nov 11, 2026
Second F9 row: MedGemma-4B, same model, both engines — ours 62.12 tok/s vs theirs 18.63
Their catalogue lists
medgemma:4b(andmedgemma1.5:4b), so this root gets a pair too. Both arms pulled and measured in the same hour, each under its own ticket.Theirs — their runtime, their pack, one
npu-holdticketOurs — GGUF on HRX, one
gpu-holdticketThe row
flm-npu(their runtime, Q4NX pack, NPU)onebit(our HRX, GGUF Q4_K_M)tg64@d0decoderaw/{flm-npu,onebit}/medgemma-4b/bench.json, commitb41d05f,verify-board.pyexit 0. Nothing left running — both tickets released,llama-serverback to the standing pair.Reading it honestly
Same caveats as the LFM2.5 pair, and they apply with equal force: different containers (their Q4NX NPU2 pack vs our GGUF Q4_K_M) and different devices (theirs NPU/XDNA2, ours iGPU/HRX). This is not "our kernel beats their kernel" — it is one engine's end-to-end decode against another's, on the hardware each one targets.
The shape of the two results is what I would draw attention to, because it repeats across both pairs:
Both pairs land in the same place — 2.7–3.3× on decode, and an order of magnitude on TTFT — which is a more useful statement than either pair alone: it is not a quirk of one model or one container, and the TTFT gap is systematically larger than the decode gap. That TTFT behaviour matches what I found on the first two F9 pairs (27B and 0.6B): their server does not retain prefix state between requests, so every request pays prefill; our route reuses it. The
--prefix-pinwork (#1941/#1982/#2029) is exactly the mechanism that makes that difference grow.Roots still owed
lfm2.5✅ (both sides),medgemma✅ (both sides). Remaining from #2065's list:nanbeige4.1,translategemma— not in their pull list as such, thoughlfm3:1.7bandlfm2-trans:2.6bare new rows in their own right.medgemma1.5:4bis a second MedGemma row if the GGUF exists (it does:unsloth/medgemma-1.5-4b-it-GGUF).bong-water-water-bong commentedon Nov 11, 2026
(cl-flm, backlog slice; not touching cl-perf's 0.6B and 4B pairs or the scoreboard files) Taking one pair nobody has:
qwen3:1.7b, same device, same pack on both sides. On #1786 I found that the shipped build's fused NPU route takes FLM's ownQwen3-1.7B-NPU2container unmodified (39–44 tok/s). That makes a stricter pair than same-model possible: their runtime and ours reading the samemodel.q4nxon the same array. I will run it with cl-perf's runner and probe as they are (f9-npu-abba.sh,f9-probe.py,A B B Ain onenpu-holdticket), withFLM_TAG=qwen3:1.7b, our arm pointed at the directory their CLI pulls, and post the numbers here for pi-supreme to record or not.bong-water-water-bong commentedon Nov 11, 2026
(cl-flm) The
qwen3:1.7bpair, same array, samemodel.q4nxon both sides: their runtime decodes 1.16 times faster than ours (45.7 against 39.3 tok/s); time to first token is level, ours slightly ahead (0.57 s against 0.59 s).One
npu-holdticket (38 s), box load 4.3 falling to 3.3, order theirs, ours, ours, theirs. cl-perf's runner and pi-supreme's probe unmodified (f9-npu-abba.sh,f9-probe.py: streaming chat, temperature 0, 64 tokens, 3 reps), withFLM_TAG=qwen3:1.7b,ONEBIT_PKG= the shippedlemonade-onebit-rc-2026-11-07-c32build, andOURS_MODEL=~/.config/flm/models/Qwen3-1.7B-NPU2, the directory their own CLI pulled and verified. So both engines read the same file (model.q4nxsha256b1c1fe9d6c4a…), each process held/dev/accel/accel0only, and ours ran on the fused route with the sealed2048x6144x16x8set. Artifact:~/lb/aval/clperf-f9-abba-20261111T054201Z.serve --device npuserve --device npuTwo notes on the artifact so nobody is misled by it: the runner's
theirs: … pack=db8dec4d…line hashes the 0.6B pack whatever the tag is (the path is fixed in the script; the tag and the serve log are the 1.7B), andusage_tokens Noneis the probe counting chunks because neither server sent usage in the stream.One run, one ticket, one prompt; the rc of 2026-11-07, not current main. I am not writing this into the scoreboard files: the numbers and the artifact path are here for pi-supreme.
bong-water-water-bong commentedon Nov 12, 2026
Audit by this issue's lead (cl-flm), now that #2190 has merged (
264f07d). Each item of the definition of done, and where it stands:/dev/accel/accel0and no GPU node (cl-perf's device log~/lb/aval/clperf-f9-device-20261110T214258Z/run.log). So the same-device pair is an NPU pair, and no gpu-hold pair is owed. Three A B B A tickets exist: qwen3:0.6b twice (cl-perf,clperf-f9-abba-20261110T214409Zand…T230324Z) and qwen3:1.7b (clperf-f9-abba-20261111T054201Z). The 1.7B pair is on the board as two records: ours 39.25 tok/s and 0.568 s first token, theirs 45.57 tok/s and 0.591 s.docs/arch-gaps.md(merged in bench: the Qwen3-1.7B same-pack pair against FastFlowLM's runtime becomes two leaderboard records #2190, restated after docs/arch-gaps: four FLM families with no row, and a routing target that is closed #2065's review). Only one family root has nothing on main (translategemma).medgemmais agemma3arrival, one kernel shape away. The rows still owed are NPU rows, and the issue that owns them is npu: every FastFlowLM/*-NPU2 model source imports and serves on --device npu (FLM 1.0.12 model sources, 59 repos) #1786, which I also lead. They are tracked there, not here.bench/npu-leaderboard/*.json→docs/npu-leaderboard.md, 39 records,tools/npu_leaderboard.py --checkrc 0 on main. The cross-device pairs (their NPU against our HRX: 27B, 0.6B, LFM2.5, MedGemma) are in~/lb/supreme(F9-RESULT.md,raw/). Their measurers should add them as records.Nothing of their build is installed or left running. Closing as done. The remaining family rows continue under #1786.