Repository navigation
Description
W8/F8: Whisper on the NPU — measure our encoder against FLM 1.0.12's claim (~3× faster: 0.6 s per 30 s window, was 1.7-2.0 s) and close the gap.
Owner 2026-11-09 23:2x, plan ~/briefs/TNPU-ALL.md §9, item F8 (after the gemma-4 rows). FastFlowLM is reference and
a model source only — nothing of theirs is run or copied; our engine does the same thing on its own runtime and kernels.
The claim to beat (public release notes, FastFlowLM/FastFlowLM v1.0.12, 2026-11-05)
The Whisper encoder is about 3× faster: 0.6 s per 30-second window instead of 1.7-2.0 s. A 5-minute recording now
takes 25 s instead of 45 s.
⚠️ A Whisper-only server now uses about 2.3 GB of memory instead of 1.0 GB. SetWHISPER_CACHE_DEQUANT=0to get back to
1.0 GB.
Their benchmark page: fastflowlm.com/docs/benchmarks/whisper_results/ (per-model tables; to be quoted when the digest lands —
see the note at the end).
Where we are (measured, 2026-11-08, ~/briefs/TDEMO-AMDCOL.md)
Seven amd/whisper-*-onnx-npu packs are in scope and all seven are measured, every transcript exact — but on ONNX
Runtime's CPU EP, because our engine has no Whisper route at all (no whisper/audio code under npu/, app/ or
docs/; the NPU-Whisper packs are listed in the board as "SD/Whisper on our NPU runtime" scope, not as rows):
| pack | size | enc | dec (steps) | transcript |
|---|---|---|---|---|
whisper-large-v3-onnx-npu |
6.95 GB | 7.33 s (enc only, load_avg 112) |
170.20 s (26) [CONTENDED] |
exact |
whisper-small-onnx-npu |
1.09 GB | 0.39-0.91 s | 7.2 s (26) | exact |
whisper-small-en-onnx-npu |
1.09 GB | 0.39 s | 10.0 s (27) | exact |
whisper-base-onnx-npu |
0.32 GB | 0.43 s | 4.1 s (26) | exact |
whisper-tiny-onnx-npu |
0.17 GB | 0.12-0.45 s | 1.5-3.0 s (24) | exact |
So large-v3's encoder is ~12× their 0.6 s on the same 30 s window, and the small/base ones are inside their old band —
the gap is real for the model people actually use.
Plan (own code, no vendor artifacts)
- Baseline harness — one audio fixture, decoded into 30 s windows, our current ORT-CPU encoder timed per window and per
recording, transcripts recorded. That is the number every later claim is compared to (the plan's rule: measured, never
inferred). Also measure peak RSS, since FLM's own gain cost them 1.3 GB. - Encoder route on our runtime — take the pack's own ONNX encoder graph (
encoder_model.onnxand its.data), map it
onto our NPU path the same way the LLM packs are mapped (importer → container → forward), writing whatever is missing
from scratch: mel front-end, conv stack, attention blocks. Kernel families as needed go to the privatenpu-kernels
repo, claimed inNPU-KERNEL-FAMILIES.mdfirst; full ELF, Loom AIE2P or Peano. - Correctness bar — bit-exact transcripts against the CPU baseline on ≥3 fixtures plus an edge case (silence, short
clip), 3 repeats returning identical text. Speed is claimed only after that. - Measure and close the gap — per-window encoder time on the device under
npu-hold(≤9 min tickets, host-side proof
first), then the same 5-minute recording FLM quotes (25 s vs 45 s), and the memory number. - Row — the five DONE points (import zero gaps + self-check, serve on
--device npu, known-answer 3× identical,
measured tok/s or RTF, row names kernel set + engine commit).
Rules this issue operates under
Own code only (FLM and AMD artifacts are reference: observe formats and behaviour, write our own); kernels and RE detail
stay private; full ELF only, Loom AIE2P or Peano, never xchesscc/aietools/Vitis; AMD packs run as published and are never
redistributed (licence recorded per row); device work only under npu-hold; packs and build trees on disk
(~/scratch-packs), never /tmp; commit the same hour; no publishing.
Note for pi-verify: ~/amd-oss/npu-re/docs/FACTS-fastflowlm-v1012-delta.md (named as the fact base in §9) is not on
disk yet — I am working from the public release notes above and will fold the digest's per-model tables in when it lands.
Tracking: this issue. Row lands on ~/briefs/AMD-RAI-BOARD.md via ~/lb/amd-rai-row.py when the five points hold.
Activity
bong-water-water-bong commentedon Nov 10, 2026
Claimed (pi-verify). Same worktree as the FLM v1.0.12 digest I finished tonight, so the "quote the benchmark page when the digest lands" note is now dischargeable — and it changes one number in the issue's framing.
The claim, with its conditions (public,
fastflowlm.com/docs/benchmarks/whisper_results/)Conditions the page states, all of which matter to us: AMD Ryzen AI MAX+ PRO 395 (Strix Halo) — our own box class — FLM's default NPU Performance power mode, same runner and build in both columns, only the Whisper engine and kernels differ; Whisper runs 30 s windows with both encoder and decoder on the NPU. RTF table (FLEURS, before → now): en 0.259→0.130 (20 clips), en 0.140→0.082 (252 s), en 0.146→0.082 (308 s), de 0.203→0.109, es 0.230→0.123, fr 0.269→0.144, ja 0.215→0.121, zh 0.275→0.148, ar 0.292→0.171.
The memory trade is now stated rather than implied: 1.0–1.1 GB → 2.3–2.4 GB for a Whisper-only server, because the encoder's dequantised weights are cached;
WHISPER_CACHE_DEQUANT=0restores 1.0 GB and costs about 0.8 s per 30 s window. So their 606 ms encoder is only reachable with the larger footprint — our own bar needs both numbers or it is not comparable.One correction to the issue's comparison
The issue compares our 7.33 s (large-v3 encoder, ORT CPU EP,
load_avg 112contended) against their 0.6 s, and calls it ~12×. That mixes execution units: their 0.6 s is an NPU number and their CPU-free path is not on offer (on Windows their encoder's CPU steps run single-threaded, ~0.7 s/window). The honest bar for us is therefore: their NPU 0.606 s vs our CPU 7.33 s is not the gap — the gap is that we have no NPU encoder route at all, and their number says what the route must reach once it exists. Their old 1.889 s is the more useful intermediate target, because it is what the same NPU reached before the kernel work.Feasibility of step 1 on this box (checked, not assumed)
whisper-large-v3-onnx-npuand the other six packs are not under~/modelshere, andpython3 -c "import onnxruntime"raisesModuleNotFoundError. The numbers in the issue's table come from the AMD Developer Cloud run recorded in~/briefs/TDEMO-AMDCOL.md, not from strixhalo. So step 1 needs one of:I will write the harness against whichever of those is chosen, and I am not claiming the device side (steps 2–4) here: no Whisper route exists in our engine, so that is a kernel-family project for the
npu-kernelslane, not a slice this lane can land.tools: whisper encoder baseline harness, with the load average in the…
bong-water-water-bong commentedon Nov 10, 2026
Baseline measured on FLM's own unit — seconds per 30-second window — for the two models that matter. Our encoder is a CPU path today, and the number to beat is 0.6 s.
Harness:
~/wt/amdcol-tools/whisper_window_bench.py(in this repo's spirit:--repeats, mel timed separately, peak RSS beside the latency because FLM's 3× cost them 1.3 GB).onnxruntime 1.31.1,jfk.wav(11 s → 1 window), both packs' plain ONNX encoders — the honest Linux path, since the.raiis inert outside AMD's Windows-only runtime.whisper-small(80 mel bins)whisper-large-v3(128 mel bins, 1280-wide)Against their number: FLM 1.0.12 says "0.6 s per 30-second window instead of 1.7–2.0 s" on the NPU, at 2.3 GB with
WHISPER_CACHE_DEQUANTon. So our CPU large-v3 baseline (1.62 s) sits at their pre-optimisation level, and their NPU is 2.7× faster than our CPU — which is the gap F8 is actually about, and it is a gap in route (we have no NPU encoder at all), not in tuning a kernel.The variable that swamps everything else, measured: encoder latency moves >2× with ORT thread count alone (1.60 s default → 0.72 s at 16 threads, same pack, same audio, same box). Any comparison — theirs or ours — that does not state the thread/queue configuration is measuring the box. Both numbers above are at 16 threads with the box at load ~15; the earlier 7.33 s I recorded for large-v3 was the same pack at
load_avg 112, i.e. the same lesson.Two harness facts worth keeping: the mel bin count must come from the model (80 for small/base, 128 for large-v3 — hard-coding 80 fails large-v3 with
index: 1 Got: 80 Expected: 128), and the mel front-end belongs in the report separately (0.03 s for the 128-bin path, 0.13 s for 80) so it can be matched against whatever the other side counts as "the encoder".Where this leaves F8: the baseline half is done and reproducible; the gap is a route we do not have. The next step is the encoder on our own NPU path (conv stack + attention blocks from the pack's ONNX, kernels claimed in
NPU-KERNEL-FAMILIES.md), gated on bit-exact transcripts against this CPU baseline before any speed claim — and the row's number iss/windowat a stated thread/queue configuration, with RSS beside it.bong-water-water-bong commentedon Nov 10, 2026
Design (15 lines): the Whisper encoder on our NPU path, per the owner's 22:4x from-scratch go-ahead. Starting here, not reporting blocked.
large_v3_encoder.onnx, 1031 nodes, 492 initializers):x[1,128,3000]→layer_norm_64[1,1500,1280]; ops = MatMul 256, Add 225, Reshape 192, Transpose 161, LayerNormalization 65, Mul 64, Gelu 34, Softmax 32, Conv 2. So it is a transformer, not a convolution stack: two stride-2 convs with GELU at the front, then 32 blocks of LayerNorm → MHA → GELU MLP at hidden 1280, 20 heads (1280/64).MatMul+Addfamilies onto the NPU for the LLM line, and the rest (LayerNorm, Gelu, Softmax, Transpose/Reshape) are the same elementwise/shape ops the LLM path already handles or falls back on host.NPU-KERNEL-FAMILIES.mdas a claim before any build, per the rules — full ELF, Loom AIE2P or Peano,grp=32-style quantisation as the LLM line uses..raiin the pack is inert outside AMD's runtime and is not touched.whisper_window_bench.pyalready computes it with numpy), the graph reader is ours (npu/pack_import.cpp), the kernels are ours (privatenpu-kernels).~/briefs/NPU-KERNEL-FAMILIES.md, packs on disk at~/scratch-packs/pi-demo/whisper-{small,lv3}, and the CPU baseline recorded in~/briefs/TDEMO-SHOWCASE-STATE.md§I.1.bong-water-water-bong commentedon Nov 10, 2026
Step 7 of the design is answered by measurement: the GEMMs are ~78 % of the encoder, the conv front-end is 1 %, so kernels are the right first move and the conv stack can stay host-side without distorting the number.
~/wt/amdcol-tools/whisper_encoder_attribution.py(onnxruntime profiler, per-op-type shares, large-v3 encoder, 16 threads, one 30 s window):Read the shares, not the wall time: the profiler inflates it (4.2 s profiled against 1.62 s unprofiled at the same 16 threads), and the shares are what the design's decision rests on.
What that changes in the plan, concretely:
Next: the GEMM family claim in
NPU-KERNEL-FAMILIES.md(shapes + which packs), then the host implementation of this graph in our own code as the reference the kernels are compared against — bit-exact transcripts, per §2.bong-water-water-bong commentedon Nov 10, 2026
Claimed in
~/briefs/NPU-KERNEL-FAMILIES.md(per §2's rule that a family is claimed before it is built), with the shapes taken from the graph rather than from the model's name.large_v3_encoder.onnx— 1031 nodes, 492 initializers, inputx[1,128,3000]→ output[1,1500,1280], 32 blocks, hidden 1280, 20 heads × 64:(1280, 1280)(1280, 5120)(5120, 1280)And the piece that is not like the LLM line, claimed explicitly: Whisper's encoder attention is bidirectional over the full 1500-frame window —
Softmaxis 8.2 % of the encoder on 1500-wide rows — so this family needs a fused scores + softmax kernel for a non-causal 1500×1500 attention, not the decode-shaped one. A kernel set built only from the decode path would leave the largest attention piece on the host and the row would say "NPU" while a third of the work ran on the CPU.Not in the claim, deliberately: the two stride-2 convs, because the attribution measured them at 1.0 % (4 nodes) — they can stay host-side, and the row will say so rather than implying the front-end is on the device.
Gate: bit-exact transcripts against the CPU baseline on ≥3 fixtures plus an edge case (silence, sub-window clip), 3 repeats identical, and then
s/windowwith the thread/queue configuration stated. Baseline to beat, measured here at 16 threads: 1.62 s per 30 s window, 3140 MiB peak RSS; FLM 1.0.12 quotes 0.6 s. Nothing is claimed as a speed until a device arm has run undernpu-hold.whisper-medium(hidden 1024) is not in this claim: different K/N is a different family, and it gets its own measurement rather than riding this one.bong-water-water-bong commentedon Nov 10, 2026
The host reference the device arm will be checked against now exists in our own code, and it agrees with onnxruntime to 7.5e-5 relative.
~/wt/amdcol-tools/whisper_encoder_ref.pyevaluates the pack's ONNX encoder op by op with numpy — the whole op set is nine types (Conv, Gelu, Transpose, Add, LayerNormalization, MatMul, Reshape, Mul, Softmax) — reading weights through a memmap on the external.dataso it never materialises 3.5 GiB of f32.Why a reference of ours rather than onnxruntime's output: §2 makes a kernel correct only when it is bit-exact against the host reference on the device, and a reference that is a second black box makes the comparison a comparison of two runtimes. This one we can explain line by line — which matters because two of its details are exactly where a device implementation will diverge:
Convwith a[1,128,3000]input and a[1280,128,3]weight, stride 1 then 2, pads 1 — a 2-D convolution path silently gives the wrong answer on the wrong axis, which is how the first version of this file failed;Transposenodes feedMatMul), so a reader that looks up everyMatMulinput[1] in the initializers hits a computed value — the failure mode wasKeyError: 'val_52', i.e. a value, not a weight.Three smaller facts worth keeping:
Gelu(approximate=none)is erf-based and numpy has noerf(numpy 2.5.2, measured), so there is an Abramowitz-Stegun fallback with |error| < 1.5e-7 absolute — far below f32 rounding, so it cannot move a bit-exact comparison, and scipy is used when the venv has it;LayerNormalizationisaxis=-1, eps=1e-5;Softmax(axis=-1)over the full 1500-frame window, non-causal, which is the piece the kernel claim calls out.A note on the 7.5e-5 and what "bit-exact" will mean here. Our numpy interpreter is a naive per-node evaluator (85 s for one window — it is a reference, not a fast path), and its accumulation order differs from Gemm's, so 1.5e-3 absolute on values up to 19.5 is floating-point ordering, not arithmetic disagreement. For the device arm the bar is the same comparison against this reference with the kernel's own accumulation order, and for speech the end-to-end check that decides is the transcript, per §2.
bong-water-water-bong commentedon Nov 10, 2026
The end-to-end gate passes: our own encoder reference → the pack's decoder → the same 28 token ids and the same transcript as onnxruntime's encoder.
That is the §2 gate in the form speech actually has: a float comparison cannot tell a plausible encoder from a right one, and a transcript can. Our numpy reference (nine op types, memmapped weights, no onnxruntime in the forward path) produces a 1.5e-3 absolute difference from ORT on values up to 19.5 — 7.5e-5 relative, i.e. accumulation order — and the decoder turns both into the identical token sequence, so the difference is provably below what the model cares about.
Two decoder facts worth recording, both read off its signature rather than assumed:
xis int64 [1, 448] — a fixed token window,<|endoftext|>-padded after the prompt, with the next token taken from the last real position (a variable-length int32 feed is rejected outright:Unexpected input data type. Actual: tensor(int32), expected: tensor(float)was the runtime's own words for it); and the input order is(x, xa)withxathe 1500×1280 encoder output.Where F8 stands now, all measured:
NPU-KERNEL-FAMILIES.md, including the non-causal 1500×1500 scores+softmax kernel that the decode path does not have;The next step is the device arm: the block GEMMs on NPU kernels with this transcript as the bar. Everything before that is now done and reproducible.
bong-water-water-bong commentedon Nov 10, 2026
The §2 gate is complete for the reference: three fixtures (speech, silence, sub-window clip) and three repeats, all parities identical.
Three things this is, and one it is not:
' you'and the half-second clip' And'are both the model's own hallucination on out-of-distribution audio; what matters is that both encoders hallucinate identically.~/wt/amdcol-tools/whisper_parity_fixtures.shis the two-phase runner (the reference needsonnx, the comparison needsonnxruntime+transformers, and no venv here has both), with the fixturesfx-silence.wav(1 s) andfx-short.wav(0.5 s of the real clip) generated besidejfk.wav.F8's pre-device work is therefore complete: baseline (1.62 s/window, 3140 MiB at 16 threads), attribution (GEMM 78.1 %, softmax 8.2 %, conv 1.0 %), a claimed kernel family including the non-causal 1500×1500 scores+softmax, and a reference validated three ways — float (7.5e-5 relative), transcript (identical on 3 fixtures) and determinism (3 repeats). What remains is the device arm, which needs
npu-holdand the kernels, not more preparation.bong-water-water-bong commentedon Nov 10, 2026
A 5-minute run measured, and it shows why their headline pair must not be compared naively: our encoder alone costs what their whole transcription costs, and per-window latency degrades 54 % over a long run.
Three things fall out:
-lmand thread count earlier, at a third axis.The harness already reports total-vs-per-window and RSS, so this is a measurement I can repeat on the device arm with the same command; the fixtures (
fx-5min.wav,fx-silence.wav,fx-short.wav) stay besidejfk.wav.bong-water-water-bong commentedon Nov 10, 2026
The Whisper 1280 family now exists as 8 full ELFs, emitted by the same parameterised generator the 1024×1536 set came from — no padding, no new toolchain work.
Four shapes × both M tiers, in the generator's own
SHAPES="role:<N>:<K>"convention:QKV:3840:1280(q, k, v are each 1280×1280, so the fused width is 3840),O:1280:1280,GU:5120:1280,D:1280:5120. All four satisfy the script's own constraints —K % 256 == 0andN % (32·4·cols) == 0at cols=8 — so no shape needed padding, which matters because a padded shape would make the family look built while quietly measuring something else. Every ELF carries theseqruntime-sequence object and is a full ELF, not an xclbin, per §2.What this is and is not. It is the coverage question answered: the shapes fit an existing, device-proven generator and one source file emits the set. It is not a correctness claim — §2's gate is bit-exact against the host reference on the device, 3 seeds plus an edge case, 3 repeats — and it is not a speed claim either. The host reference for this encoder is already validated three ways (§I.4-I.6), so the device arm has something to be compared against; what it needs now is an
npu-holdticket and a harness that loads these ELFs.One blocker for that arm, stated rather than assumed: the NPU currently has the c25 payload server resident (
wt/payload-c25/.../1bit serve, pid 891644) — the recording lane's server, which is not mine to disturb and which:18097/npu-serve-keeper's rules protect. Kernel verification will have to be scheduled around it, and a TDR (per §2) means stop, name the ELF, quarantine, no retry.Set on disk at
~/scratch-packs/pi-demo/kernels-whisper-lv3-1280(8 ELFs, 2.3 MiB total), generator invocation in the issue and/tmp/gen_whisper.sh→ worth copying next to the set.bong-water-water-bong commentedon Nov 10, 2026
Handover of the family, and the one thing I cannot do from this lane: the device gate needs the kernel lane's bench.
The set is emitted (8 full ELFs, both M tiers, no padding) and on disk at
~/scratch-packs/pi-demo/kernels-whisper-lv3-1280/, with the exact generator invocation saved asemit.shbeside it:I checked the device environment rather than assuming it is busy:
~/NPU-TOKENdoes not exist and the c25 payload serverthat was resident an hour ago (
wt/payload-c25/.../1bit serve, pid 891644) has exited — so the device is free for whoever takesthe verification. My earlier note that it was occupied is now stale, and I would rather correct it than leave a reason on the
record that has expired.
What the gate needs, precisely, and what I have for it:
(
npu-kernels); this lane has the engine path only, which reaches these ELFs through a Whisper route the engine does nothave yet (the >1-day piece, designed above);
transcript (identical on speech, silence and a sub-window clip), determinism (3 repeats);
npu-hold, and on a TDR: stop,name the ELF, quarantine, no retry.
So the remaining work on F8 splits cleanly and neither half is blocked on the other: the kernel lane runs the device gate on
these 8 ELFs, and this lane builds the Whisper route in the engine so the same ELFs are exercised by the real path once
they pass. I am taking the second half next unless someone claims the first.
bong-water-water-bong commentedon Nov 10, 2026
Determinism checked — and the check the template uses gives a false answer on this family. The configuration payload is byte-identical; only ~20 bytes of symbol-table metadata differ.
Re-emitting the same
SHAPESa second time and comparing file by file:Where the 19–24 bytes are: a byte-by-byte
cmpputs them at offsets 345, 397–398 and 198717–198718 — inside.symtab/.strtab, i.e. symbol ordering/spelling — andobjcopy --dump-section .pdi.1of both builds compares equal (0 differences)..pdi.1is the configuration payload the NPU actually loads (PROGBITS … AX, 0x4ee0 bytes on the m16 shape), so:md5sumdeterminism check, which is what~/ogaload/oga13/w3a-regen-check.shdoes on the 1024×1536 template, would report 6 of 8 shapes as non-deterministic for this family when nothing functional changed.That is a defect in the check, not in the family, and it matters for §2 rather than for taste: "reproduce the shipped set" and "quarantine the named ELF" both assume a build is a stable artefact. Two consequences worth acting on in the kernel lane:
.pdi.1(or everyPROGBITSwithAX) is the thing a determinism check should hash; the symbol table is an implementation detail of the linker;Scope: two independent emissions of the same four shapes, same generator, same toolchain,
SHAPES="QKV:3840:1280 O:1280:1280 GU:5120:1280 D:1280:5120"; comparison bycmpper file and per dumped section. No device involved — this says nothing about whether the family is correct, only whether it is reproducible.bong-water-water-bong commentedon Nov 10, 2026
(cl-flm, second pair of hands per the orchestrator's 21:40Z assignment) Slice I take: unblocking #1931, not new Whisper work. Its head
61426f5has not moved since 17:56Z, conflicts with main in four files, and carries three unanswered CHANGES (no assertion for the bit-identical threading claim, no exception path in the worker pool, the timing labels inwhisper_transcribe). I am merging main into that branch and writing those three fixes with tests, as a PR ontomain-audioops. pi-verify keeps the encoder route and the device work.bong-water-water-bong commentedon Nov 11, 2026
Slice taken (device-free): the encoder's weight inventory, and what 0.6 s means in FLOPs
Distinct from cl-flm's #1931 slice. I took the issue's own first question — measure against FLM 1.0.12's 0.6 s — and answered the part that needs no device: what the encoder has to compute, and therefore what effective throughput each side's number implies.
The pack
FastFlowLM/Whisper-V3-Turbo-NPU2(public; FLM's ownencoder_{attn,dequant,mm}.xclbin,decoder_mv.xclbin,whisper_head.xclbinare not used or copied). Itsmodel.q4nx(620 MB) read with our container path:So the encoder is a standard Whisper block ×32 — nothing exotic — and the weights are mixed
BF16/I8(the I8 count, 233, is close to the projection count, i.e. the matmuls are the quantised ones).The bound the claim implies
For 30 s of audio Whisper's encoder runs
T = 1500frames through 32 layers ofd = 1280,ffn = 5120:which gives each reported number an effective throughput:
So "about 3× faster" is a 2.8-3.3× throughput gap on a 1.94 TFLOP workload, and the target for F8 is stated in a unit that survives changing the model: the encoder has to sustain ~3.2 TFLOPS effective to hit 0.6 s.
What this does and does not say
M=1500rather than a kernel-shape one.Next slice (mine, needs a device)
Run our encoder on the same 30 s window on
npu-holdand report effective TFLOPS beside the seconds, so the row is comparable to the table above rather than to a release note. I will state when I take it; the pack stays at~/scratch-packs/flm-whisper/(620 MB, disk).53 remaining items
bong-water-water-bong commentedon Nov 15, 2026
Measured the load split, as I said I would:
mel+loadis≈2.4 sand it does not depend on the audio length, so it is load — the lever is the pack read/repack, not the front end.Method without touching the code:
whisper_transcribepads every input to one 30 s window, so a 1 s wav and a 30 s wav do the same encoder work and the same load. Any difference between them is mel; the constant is load. Warm page cache first (one throwaway run), then alternate:1 s and 30 s are the same number to within 3 % (
2.431 / 2.386vs2.369). Since mel on 1 s of audio is negligible and on 30 s it is ~0.115 s (the #2488 figure), the length-independence says the line is dominated by a constant: pack/graph load ≈2.3–2.4 s, mel ≤0.12 s, i.e. ~95 % ofmel+loadis load.The outlier is worth reporting rather than averaging away: one
s30run took 5.575 s formel+loadwhere three others took 2.37–2.43 s. Whole-process wall times on the same six runs:28.96, 20.09, 13.39, 13.30, 13.13, 12.82— a 2.3× spread, with the first two clearly still warming. So the honest form of this result is "constant ≈2.4 s, and one run in six was 2.3× that", not "2.4 s" — on this box a single sample of this line is not a measurement, which is the same lesson as the prefill row in #2555.Consequences for this row:
mel+loadas the larger term at a 30 s window. That is now the biggest single number on the path that nobody has attributed, and it is the next thing I will work.mmaporread, and whether the q4nx container can be mapped rather than repacked (the repack is the part that scales with model size). Ifmmap+ no repack getsmel+loadunder 1 s, that change is worth more than everything in npu: the audio encoder non-GEMM half — Conv/Softmax/attention are 9.06 s of a 21.7 s window (from #1783) #2366 combined.What this does not say: nothing about why the load is 2.3 s — page-cache cold vs repack vs graph construction. The method above attributes the line, not the split inside it; the next step is a timer around the pack read and one around the repack, which needs a code change rather than a wav trick.
bong-water-water-bong commentedon Nov 15, 2026
Correction to what I posted 20 minutes ago: the
2.3–2.4 sinmel+loadis not pack load. I built the instrument that already exists inmainand it saysload 0.002 s.origin/mainalready splits the line (tools/whisper_transcribe.cpp, the engine#1783 comment there), so building it answered what I had inferred second-hand:Storage load is 2 ms and mel is 0.12 s. My wav-length trick (
1 svs30 s) was sound as an experiment and wrong as an inference: I observed a constant and attributed it to load, when the constant is the encoder's padded 30 s window — both inputs are padded to one window, so the encoder work (and its first-call setup) is identical and dominates. The trick measured the thing I was trying to separate from.So the correcting statement about this row is:
mel+loadwas a mislabelled bucket, and it is now fixed inmain—mel 0.12 sandload 0.002 sare two numbers that were one, and the 2.3–2.4 s I attributed to pack load belongs to the encoder line (which is where the issue's original 74 %-MatMulanalysis already pointed).MatMul, ~2.1 s on the device route and 3.6–4.3 s on this host run, against a 0.6 s bar.What I should have done: read the timer sites in
mainbefore inferring from a black-box measurement. I had the same information available both times; the wav trick was cheaper and it was the wrong instrument, because it could not distinguish "constant load" from "constant encoder work" — and a bucket label is not a definition. That is the 22nd correction of this session and it is the same class as the ones I have been flagging in review: a measurement that cannot fail for the reason you are testing.One useful residue:
encoder 3.628 / 4.029 / 4.318 son the host over three consecutive runs is a 19 % spread, so the host encoder figure needs repeats before anyone quotes it — the same load-dependence as everything else on this box.bong-water-water-bong commentedon Nov 15, 2026
Current encoder family table on
main, measured twice just now — and theMatMulshare is the number this issue should be carrying, not the wall time.ONEBIT_AUDIO_OP_TIMES=1, whisper-small pack,jfk.wav, host build oforigin/main, 4 threads, two consecutive runs:Three things this settles:
MatMulis 79 % of the encoder, and nothing else is close. Softmax at 8.7 % is the second term and it is 9× smaller. Every remaining encoder lever is therefore a projection lever — the non-GEMM half this lane finished (npu: the audio encoder non-GEMM half — Conv/Softmax/attention are 9.06 s of a 21.7 s window (from #1783) #2366: softmax0.98→0.578 s, elementwise6112→3451 ms, Transpose168→65 ms, all bit-exact) moved the bucket that is now 14 % combined.MatMul(2788.6 vs 3014.0) — so a single 1 % claim about this encoder is not measurable on this box, which is why the family-level shares are the honest unit and the wall time is not.What I would put in the issue body (and will, if nobody objects): the bar is 0.6 s per 30 s window on the device route, the measured device encoder is 0.305 s at M=1536 (bit-exact vs
NPU_INFER_HOST_GEMM=i8), and the host path is 3.5–3.8 s of which 79 % isMatMul— so the remaining gap to the bar on the product path is a fused projection kernel, not the non-GEMM work and not the front end (mel 0.12 s, storage load 0.002 s, both measured with the split that is now inmain).Not claimed: that the host number is a regression — load differs between these runs and the earlier ones, and I do not have an interleaved A/B against an older commit. If the 2.9 s → 3.8 s host move matters, it needs that A/B, not two readings at different loads.
bong-water-water-bong commentedon Nov 15, 2026
The 79 %
MatMulbucket has a measured shape now: it stops scaling at 4 threads, and 6 and 8 threads are slower than 4. That is a bandwidth/cache answer, not a threading one — and it means the remaining lever is the GEMM's memory traffic, not more threads.Same pack, same wav, host build of
origin/main,ONEBIT_AUDIO_THREADS=N, encoder total (447 nodes) on a 32-core box:Three conclusions, each with the measurement that forces it:
ONEBIT_AUDIO_THREADS=4is the right default and should be documented as measured, not chosen. The current default is 4; the table is the reason, and it belongs next to the env var so nobody "optimises" by raising it. 6 and 8 being slower is the useful part of the datapoint: it makes the default defensible rather than arbitrary.What this changes about the plan for this issue: the projection lever is not "more parallelism" and it is not the non-GEMM half either (that is 14 % combined now). It is the GEMM's memory traffic at 4 threads — blocking/packing so that a 96-call sequence of
M≈1500, K=N=768products reuses B out of L2/L3 instead of streaming it. The device route sidesteps the whole question (0.305 s/window at M=1536), which is why the device route is the product path and this is CPU-side fallback work.Caveat, stated because the numbers are close: these are single runs per thread count, and the run-to-run spread on this encoder is 7–8 % (measured earlier today). 6 vs 8 (4467 vs 4516) is inside that; 4 vs 6 is not (4001 vs 4467, 12 %). So "the knee is at 4" is safe and "6 and 8 are equal" is not.
bong-water-water-bong commentedon Nov 15, 2026
Correction to my own conclusion from the thread-scaling table: the
MatMulbucket is not bandwidth-bound. It is at ~57 GFLOP/s and moving 0.4 GB/s, so the lever is the FLOP path, not the memory traffic.Arithmetic from the same measurement (96
MatMulcalls,M≈1500, K=N=768, 2788–3014 ms for the bucket):0.4 GB/s is two orders of magnitude below this box's DRAM rate. A kernel limited by memory would be sitting near 30–50 GB/s, not 0.4. So the 4→8-thread slowdown I measured does not prove the kernel is bandwidth-bound; I inferred that from the scaling shape alone, and the throughput says otherwise — 57 GFLOP/s over 4 cores is roughly half of a realistic f32 FMA peak for Zen 5, which is the number that points at where the time goes: not enough FLOPs per byte, i.e. blocking and vectorisation, with the 4→8 regression explained by something else (L3 sharing or scheduler contention at 8 threads on shared SMT, not by the arithmetic).
What survives from the earlier finding: the fact of the knee at 4 threads (
11403.8 / 6627.8 / 4001.0 / 4467.4 / 4515.6 msat 1/2/4/6/8) is measured, andONEBIT_AUDIO_THREADS=4is still the right default. What does not survive is my explanation of it, and the plan that followed — "block/pack so 96 calls reuse B in L2/L3" is addressing a traffic problem that is not there at 0.4 GB/s.The corrected lever is the inner kernel: 57 GFLOP/s at f32 with 4 threads means either the micro-kernel is not using FMA across the full register width, or the blocking is small enough that the reload of A/B inside the kernel costs more than the FLOPs. That is exactly the kind of thing GEAK exists for (the rule for this lane: GEAK is mandatory when speeding up an existing kernel), and it is a different piece of work from what I proposed an hour ago.
What I do not know yet, and will not assert: where in the 96 calls the time actually sits — a
M=1500GEMM and aM=16GEMM in the same bucket can have wildly different efficiency, and 57 GFLOP/s is the average over all of them. Splitting the bucket by shape (asONEBIT_AUDIO_OP_TIMESalready does per op) is the next measurement, not another thread sweep.bong-water-water-bong commentedon Nov 15, 2026
The shape split, as promised — and it is lopsided enough that the "one GEMM kernel" framing is wrong: two shapes are 76 % of the encoder's MatMul, and they are different problems.
Encoder pass only (first 447 nodes),
ONEBIT_AUDIO_NODE_TIMES=1, one run, per-node times summed across 4 threads:1x1500x7681x1500x30721x128x7681x12x1500x15001x12x1500x64MatMulTwo things this changes about the plan:
1x1500x768(110 calls, 61 %) and1x1500x3072(12 calls, 15 %) are the whole conversation. Both areM=1500, K=768withN=768or3072— i.e. the same activation matrix times two different weight widths. A single blocking scheme tuned forK=768, M=1500with anN-loop covers 76 % of the bucket; nothing else is worth touching until those two move.1x128x768is 175 calls for 8.8 % — the many-small-GEMM regime, and it is cheap: 6 ms per call. That is the shape where per-call overhead would matter and it is not where the time is, so the threaded spawn tax I measured earlier (0.018 ms/call) is confirmed as irrelevant to this bucket rather than merely small.Efficiency, from the same run:
1x1500x768, 110 calls, 7366.5 ms ⇒ 2.53 GFLOP/call... of work per call ≈ 2·1500·768·768 = 1.77 GFLOP ⇒ ~26 GFLOP/s1x1500x3072, 12 calls, 1848.4 ms ⇒ 2·1500·768·3072 = 7.08 GFLOP per call ⇒ ~46 GFLOP/sSo the narrow GEMM runs at 26 GFLOP/s and the wide one at 46 — the wide one is nearly 2× more efficient, which is the signature of a kernel whose
N-loop is not amortising the A-panel load: withN=768the same A block is re-read for a shorter run of B columns. That is a blocking answer consistent with the throughput arithmetic from before (57 GFLOP/s average, ~half of peak), and it predicts the fix: increase theN-block (or keep A resident across the wholeNrange) so1x1500x768behaves like1x1500x3072.Caveat: the per-node times are summed across 4 threads, so these totals are ~4× wall and are only comparable within this run; the shares and the per-call GFLOP/s are the comparable quantities. One run — the box's spread is 7–8 %, so treat 26 vs 46 GFLOP/s as a large difference (not a precise one).
bong-water-water-bong commentedon Nov 15, 2026
Read the kernel: the host
MatMulis a naive triple loop, and it is bandwidth-bound — on B re-reads. My "not bandwidth-bound" correction two comments up was wrong in mechanism; the arithmetic below is the one that fits.npu/audio_exec.cpp, the host GEMM body:Per call, B is streamed
Mtimes:M·K·N·4 = 1500·768·768·4 = 3.5 GB. Against the measured 67.0 ms/call for1x1500x768that is 52.8 GB/s — i.e. sitting at DRAM rate. So:This also explains the narrow-vs-wide gap exactly.
N=768⇒ the accumulator rowrris 3 KB (fits L1), B streams 3.5 GB, efficiency 26 GFLOP/s.N=3072⇒rris 12 KB, B streams 4× more per call but there are 4× more FLOPs per byte of A, and A's re-use acrossjimproves: 46 GFLOP/s. Same loop, different ratio.And it explains why
#2386(four-rows-per-unit) was worth 1.32× and no more: four rows of A perkpass cuts B traffic 4×, but the loop still streams B once per group of four, and the per-krr0..rr3accumulators push4·Nfloats through registers — atN=3072that is 48 KB, past L1, so the win shrinks exactly where the shape is widest.The lever, stated so it can be implemented and falsified: increase the M-block from 4 to 8–16 rows per
kpass (rrlives in L1 as long asblockM · N · 4 ≤ 32 KB, i.e.blockM ≤ 10atN=768,blockM ≤ 2atN=3072). AtN=768an 8-row block cuts B traffic 8× ⇒ predicted ~200 GB/s effective → ~6–8× less time on B streaming, bounded by whatever the FLOP rate becomes. AtN=3072the block has to stay small, so the right fix is a tiled loop (blockNtoo, e.g. 512-wide panels of B with an 8-row A block), not a single blockM constant.Predicted and falsifiable:
1x1500x768should move from 26 → 60+ GFLOP/s with an 8-row × 256-col panel while1x1500x3072stays ≥46; if the narrow case does not move, the loop is not the limit and I am wrong again.Per this lane's mandate, this is a GEAK job (GEAK is mandatory for speeding an existing kernel) rather than an ad-hoc rewrite, and the shapes to hand it are exactly the two that are 76 % of the bucket:
1x1500x768(110 calls) and1x1500x3072(12 calls).bong-water-water-bong commentedon Nov 15, 2026
I tested my own prediction and it failed — but the failure found something 3× bigger. The lever is the compiler flags, not the blocking.
Standalone harness reproducing this exact inner loop (
npu/audio_exec.cpp:for i { for k { br = b + k*N; for j { rr[j] += av*br[j]; } } }),M=1500, K=768, N=768, single thread, thermal-run:Test 1 — the blockM prediction (refuted):
I predicted 60+ GFLOP/s at blockM=8; the best is blockM=4 at +18 %, and beyond 4 it is flat. So the M-block is not the lever and my previous comment's fix is wrong. (It also explains #2386's 1.32× almost exactly — the four-rows-per-unit was already the whole win available from blocking.)
Test 2 — the same code, three flag sets (this is the finding):
The identical scalar C++ is 3.1× faster with the box's own ISA than with SSE2, because it is a plain
rr[j] += av * br[j]reduction that autovectorises — and at-msse2GCC produces 4-wide SSE where AVX2+FMA gives 8-wide with fused multiply-add. That, not blocking, is why the in-tree GEMM measures 26 GFLOP/s while this harness sits at 45 (my-march=nativebaseline) and 62 (same code, only flags different).This is the same conclusion as #2403 from a completely different direction. That PR added
ONEBIT_AUDIO_AVX2and deliberately defaulted it OFF, because the tree had no per-box selection and a default-ON flag would have required AVX2+FMA for every build including the 1bit OS appliance. The measured cost of that decision is now quantified for the biggest bucket: up to 3.1× onMatMul, i.e. the encoder's 79 % could plausibly drop by half — if the build flags can be chosen per box rather than per repo.The lever, corrected and testable: not a kernel rewrite; runtime dispatch. Compile the GEMM translation unit twice (baseline + AVX2/FMA) and pick at load time with
__builtin_cpu_supports("avx2") && __builtin_cpu_supports("fma")— the appliance keeps the baseline, this box gets the fast one. Prediction:1x1500x768moves 26 → 45+ GFLOP/s on this box with no change to any appliance. That is aCMakeLists+ one header change, testable in isolation, and it does not need GEAK because it is not a kernel rewrite.What I am not claiming: the 26 vs 45 comparison is across builds (tree vs my harness) as well as flags, so the 3.1× is an upper bound on what the tree would gain — the tree also compiles with its own inlining context and threading. The 1.85× (SSE2 → AVX2+FMA, same harness, same code) is the clean number.
bong-water-water-bong commentedon Nov 15, 2026
Tried the in-tree A/B for the flags lever and it is not resolvable at this load — reporting that rather than a number I cannot defend.
-DONEBIT_AUDIO_AVX2=ONvs the default build, same pack/wav,ONEBIT_AUDIO_OP_TIMES=1, alternating, 4 threads:build-avx2ranges 3659.9 → 7492.6 ms across two runs — a 2× spread within one build. At that variance no A/B can separate a 1.85× effect from the noise, and the honest reading is: the box is not quiet enough for this measurement right now. I am not quoting a min-over-repeats from two samples either — that rule needs ≥3 agreeing repeats, and these do not agree.What the number would have been if it had worked: the standalone harness (single-threaded, short, run three times with stable results) gave
-msse220.0 →-mavx2 -mfma37.1 →-march=native62.2 GFLOP/s on the identical inner loop. That is where the 1.85× lives: 8-wide FMA vs 4-wide SSE on arr[j] += av*br[j]reduction. The tree build also has the other factors (threading, inlining context, its own flags), which is exactly why the in-tree A/B was worth attempting and why it needs a quiet window rather than a loaded one.So the next step is a measurement, not a change: an alternating A/B of
build-hostvsbuild-avx2with ≥3 interleaved repeats under a ticket that holds the box still — the same discipline as the prefill row in #2555 and the thread-scaling table, both of which needed it. If it lands at ~1.5–1.85× on the encoder's 79 % bucket, the runtime-dispatch change (__builtin_cpu_supports("avx2") && __builtin_cpu_supports("fma"), two compiled TUs, appliance keeps the baseline) is worth a PR; if it does not, #2403's default-OFF stands on its own and this lane has no lever in the GEMM beyond what #2386 already took.Retained from this attempt:
build-avx2is a working configuration (-DONEBIT_AUDIO_AVX2=ONbuilds and runs), which is itself worth knowing — the flag added in #2403 is not broken, it is unmeasured at scale.bong-water-water-bong commentedon Nov 15, 2026
The A/B resolved:
ONEBIT_AUDIO_AVX2=ONis worth 1.23–1.27× on the whole encoder. Six interleaved pairs, every pair consistent.Same pack/wav, 4 threads,
ONEBIT_AUDIO_OP_TIMES=1, alternatingbuild-host(default) andbuild-avx2(-DONEBIT_AUDIO_AVX2=ON):Every AVX2 run is faster than every default run (the 5708.9 outlier is a default run, so it does not flatter the comparison). Min-over-repeats gives 1.269×, median 1.23× — the flag added in #2403 is worth ~1.25× on the encoder, in-tree, on this box.
Where that sits against the isolated loop: the standalone harness said 1.85× (
-msse220.0 →-mavx2 -mfma37.1 GFLOP/s); the whole encoder delivers 1.23–1.27×, because only the 79 %MatMulbucket vectorises and Softmax/Conv/Transpose do not.1 - 0.79 · (1 - 1/1.85) ≈ 0.69— a 1/0.69 = 1.45× would be the arithmetic ceiling if every MatMul benefited fully; 1.25× means roughly two-thirds of the bucket is converting, which is consistent with the smaller shapes (1x128x768, 175 calls) being less vectorisable.The decision this makes possible, and it is #2403's: the flag's default was set OFF because the tree had no per-box selection and a default-ON flag would have required AVX2+FMA for every build, including the 1bit OS appliance. With a measured 1.25× on the largest bucket, the runtime-dispatch version is now clearly worth its cost: compile the GEMM translation unit twice (baseline + AVX2/FMA), select at load with
__builtin_cpu_supports("avx2") && __builtin_cpu_supports("fma"). The appliance keeps the baseline path; a box like this one gets 1.25× on 79 % of the encoder for free, with no per-box configuration.Not claimed: that 1.25× is available for free — it is available given the dispatch, which is the work. And the numbers are this box at this load with six interleaved pairs; the min-over-repeats is the defensible estimator for "how much faster", the medians for "consistently faster".
bong-water-water-bong commentedon Nov 15, 2026
The dispatch merged (#2584,
1a82ccbc), and the post-merge re-measure is not obtainable right now — reporting that rather than a number.Attempts on the merged build,
ONEBIT_AUDIO_OP_TIMES=1, 4 threads, all underthermal-run:Load average was 12–13 on 32 cores while these ran. A 41.7 s encoder reading and a pair with no difference at all cannot both be describing the same code, so none of this is usable, and I am not going to pick the pair that looks best. The pre-merge A/B stands as the number for this change: 1.224× min / 1.233× median on the encoder, three interleaved pairs whose within-arm spread was under 2 % at load ~12.
What is settled about the merge itself:
buildgreen,audio_parallel: ok(the per-float ordered-sum arm), dispatch verified in both directions including the no-AVX2 message path, and the clone targetsavx2withoutfmaso the rounding sequence is unchanged.What remains owed on this row, and it is a window not a task:
Both need the box quiet; taking them now would produce exactly the numbers above.
bong-water-water-bong commentedon Nov 15, 2026
Sized the follow-up: a whole-translation-unit AVX2 build is 1.25× on merged
maintoday, while the per-function clone that merged (#2584) is worth ~2 %. The gap is the task.Measured now, on
main, three interleaved pairs,thermal-run, 4 threads,ONEBIT_AUDIO_OP_TIMES=1:Every AVX2 run beats every default run; min ratio 4037.8 / 3224.7 = 1.252×, and that reproduces the 1.224–1.269× measured earlier — so the whole-TU figure is stable across sessions, not one lucky window.
What that means: the file-level flag vectorises every loop in
audio_exec.cpp(map_elements,broadcast_apply, the softmax row loop, the transpose gathers), and the encoder is 79 %MatMul+ 8.7 %Softmax+ 14 % everything else — so ~20 % of the bucket is non-GEMM arithmetic that the per-function clone never touches. The 1.25× is the sum of small wins across loops, and a 2 % win on the GEMM loop is what one of those looks like when isolated.The change that would capture it, concretely:
audio_exec.cpptwice in CMake — the default object, and one with-mavx2 -DONEBIT_AUDIO_TU=suffixso the executor entry and its namespace tail are renamed;__builtin_cpu_supports("avx2")gate and theONEBIT_AUDIO_AVX2override that audio: pick the GEMM inner loop at run time (AVX2+FMA where the box has it) #2584 already added;-mfmain either — audio: pick the GEMM inner loop at run time (AVX2+FMA where the box has it) #2584 proved FMA breaks the ordered-sum contract thataudio_parallelasserts, and that arm is now in the tree to catch it;Cost: ~30 lines of CMake plus a dispatch shim, and a binary that carries two copies of the executor (~a few hundred KB). Risk: the two copies must not disagree — the
audio_parallel/id arms are what prove that, and they already exist.What I am not doing: implementing it in this turn. It is a build-system change across every consumer of the executor (tools, tests, the NPU forward path), and the wrong way to start that is a half-built second object at the end of a long session.
bong-water-water-bong commentedon Nov 16, 2026
Owed re-measure taken after the reboot, and it settles the question the wrong way for my own PR: the merged per-function clone is worth ~0–2 % on this box, not distinguishable from noise.
Fresh build of
main, load ~6 (1-min 5.9, 15-min 1.7 — the quietest window since the reboot), three interleaved pairs,ONEBIT_AUDIO_OP_TIMES=1, 4 threads:min ratio 3452.8 / 3417.2 = 1.010×. Pairs 2 and 3 have the default ahead by 2 % and 1 %; pair 1 is confounded by a 5824 ms baseline (post-boot warm-up). So the effect is at or below the smallest difference this box can resolve, which is consistent with the 2 % I measured before the reboot and inconsistent with the 1.22× I originally put on that PR.
Where that leaves the ledger, honestly:
audio_parallelasserts the ordered sum; the clone has nofmaso the rounding sequence is unchanged), and it is the plumbing a TU-level change needs.-DONEBIT_AUDIO_AVX2=ONadds-mavx2to the file, vectorisingmap_elements,broadcast_apply, the softmax row loop and the transpose gathers as well). Shipped main proves the distinction: whole-TU re-measured at 1.252× three interleaved pairs, per-function clone at 1.010×.So this row's accounting is closed on the CPU side: the elementwise in-place idea cost 1.5–2.4× and was dropped; the blockM idea was refuted (best +18 % at 4, flat beyond); the flags idea is real at 1.25× but only at TU granularity, and the intermediate one-function version is a no-op. The 0.6 s/window bar still needs the device route (0.305 s at M=1536) plus the fused projection kernel that the missing ONNX-encoder path blocks.
bong-water-water-bong commentedon Nov 16, 2026
Device encoder re-measured at load 1.2: 3.545 / 3.033 s — so the 30.035 s I saw at 23:33 was contention, and the "5× slowdown" hypothesis does not survive. But the device is also no longer near the 2.092 s in this issue.
Two runs, one
npu-holdticket, the cached NPU binary on the same pack and wav:Three conclusions, and the first one is a correction of my own last-but-one comment:
What I am not claiming: that the device regressed. Two runs, one pack, one binary, and the 2.092 s baseline came from a different build. The honest statement is "device 3.0–3.5 s today; 2.09 s previously; the gap is unattributed and the route margin is now small enough that a host-side 1.25× would close it."
bong-water-water-bong commentedon Nov 16, 2026
The TU dispatch flips the route comparison, and that is a decision I can make on this row: as of this measurement the host path is faster than the device route for the encoder, and the issue should stop describing the device route as the product path until the device number is re-established.
Numbers on the table, all encoder-only, same pack and wav:
So the recorded rationale — "device route is the product path (0.305 s/window at M=1536)" — no longer holds for the encoder on this box, and it never held for the number that matters: 0.305 s was the M=1536 wide tier on staged kernels, while the real path measures 3.0–3.5 s at that same shape family.
What I am changing, and what I am not:
The one measurement that would overturn this: a device encoder run with the staged
M=1536kernel set, its pack and binary both sha-recorded, at load ≲2. If that lands anywhere near the 0.305 s/window figure, the device is back ahead by an order and this comment is wrong — but that run has never been recorded with provenance, which is exactly why the comparison has drifted twice tonight.And the honest framing for the row: two paths, both measured, one ahead by ~15 % after its own best lever was taken. That is a better state than "device is the product path with an unverified 7× advantage", because the next person can check it.