diff --git a/.agents/engine-matrix.md b/.agents/engine-matrix.md index 080ea6e7c8..2b7e02bc0f 100644 --- a/.agents/engine-matrix.md +++ b/.agents/engine-matrix.md @@ -172,8 +172,8 @@ lifecycle are unchanged. |---|---|---|---|---|---|---|---|---| | `SPEC-MTP` | Qwen3.6 MTP heads, k=1 first. **M-mtp-0 CLOSED 2026-07-24: the standalone draft head is oracle-parity-proven on BOTH checkpoints** (27B dense + 35B MoE, k=1, vLLM 0.25.0 executable @ pin `e24d1b24`) - argmax exact on 26/26 unambiguous rows each; the one remaining row per checkpoint is an EXACT oracle top1==top2 tie where vLLM's own `argmax` and `topk` disagree and our pick is a tied maximum; logits within the whole-model bound (atol 0.05 + rtol 0.05), 0/216 out-of-tol on both; shared lm_head isolated is bit-exact on the 35B NVFP4 head. **I2 scheduler-half LANDED (2026-07-24)**: host-side spec plumbing + the FROZEN spec-metadata ABI (spec §2.7) - `SpeculativeConfig`, `DraftTokenIds`, `Request::spec_token_ids`/`NumTokensWithSpec`, populated `scheduled_spec_decode_tokens`, `Scheduler::update_draft_token_ids`, `take_draft_token_ids` seam, `EngineCore::post_step`, `InputBatch::num_accepted_tokens`/`update_req_spec_token_ids`; DEFAULT-OFF and INERT (no `SpeculativeConfig` => `num_lookahead_tokens == 0`). **I3 verify-half LANDED (2026-07-24)**: greedy rejection sampler + per-request logits expansion (see `SPEC-REJECTION`, now `ACTIVE`). **I4 GDN-half LANDED (2026-07-24)**: the GDN speculative slot path + bit-exact state rollback, the piece BOTH GDN-hybrid gate checkpoints need (see `SPEC-GDN-SEGMENTS`, now `ACTIVE`). **I5a GDN LAYER ROUTING + runner spec-metadata upload LANDED (2026-07-24, `CLAIM-SPEC-MTP-I5A`)**: `GdnBlockPaged` now routes a pure-spec batch through `vt::GdnSpecDecode`/`vt::CausalConv1dSpecUpdate` and the runner uploads I4's six spec device tensors — first sub-increment of the scoped M-mtp-1 (I5a GDN wiring → I5b prepare_prefill → I5c MTP paged propose → I5d config+runner-loop+the 27B token gate, spec §5). DEFAULT-OFF INERT, bit-exact vs the I4 ops, no e2e loop yet. **I5b `prepare_prefill_inputs` LANDED (2026-07-24, `CLAIM-SPEC-MTP-I5B`, recorded under `SPEC-REJECTION`)**: the drafter prefill input-prep host routine (shift-splice + `query_len -= num_rejected` + last-token index / metadata) — second scoped M-mtp-1 sub-increment, DEFAULT-OFF INERT, unit-gated RED-first, additive. **I5d CONFIG + RUNNER LOOP LANDED, PARTIAL (2026-07-25, `CLAIM-SPEC-MTP-I5D`)**: `--speculative-config` JSON parse -> `EngineParams::speculative_config`; `LoadedEngine` resolution (`ResolveSpecConfig`/`ResolveMtp`, widened KV `MakeQwen3_5KVCacheSpec(num_spec>0)`, `BuildMtpDraft`, forced sync scheduling, `MakeScheduler(spec)`, `EngineCore(check_for_draft=true)`); the full runner verify/propose loop (draft splice, hidden-tap capture, GDN builder spec-overload feed, k+1 GDN state-slot remap + widened conv cache + draft-KV alloc, `MtpProposePrefill` post-sampling, `take_draft_token_ids`, acceptance telemetry). CUDA `-Werror` 0 warnings, cutlass-ON banner. SPEC-OFF BYTE-IDENTICAL (all gated on `spec_on()`): SACRED 27B 235/235, 35B 315/315, Coder 138/138 + unit test_runner 257 / test_mtp_speculator 169 / test_gdn_metadata_builder 483 / test_ops_gdn 3630 ALL PASS. **The three-way 27B token gate is NOT yet passing** (`tests/parity/test_qwen27_spec_decode.cpp` RUNS the loop + MEASURES the blocker): the spec-ON engine throws on the FIRST prefill step at `gdn_state_gather: working/cache row shapes must match` (`src/vt/ops.cpp:1773`) — I4's spec conv rollback needs the conv row widened to `(K-1)+num_spec` but the non-spec GDN conv ops assume `(K-1)`. Closing needs widened-cache-aware non-spec GDN conv ops + the MIXED `GdnBlockPaged` split/merge. Row LEFT `GATING` at I5e. **I5e LANDED 2026-07-25 (`CLAIM-SPEC-MTP-I5E`) — `SPEC-MTP` LEAVES `GATING`.** Made the non-spec GDN conv ops widened-cache-aware (mirror vLLM `state_len=KERNEL_WIDTH-1` + physical `stride_conv_state_tok`; leading `(K-1)` sub-window; byte-identical at `num_spec==0`, contiguous fast path kept) AND RCA'd the resulting 0-acceptance dead-drafter to the async input-combine overwriting the verify batch's draft position with the committed token (forced off under spec, nullopt-guarded). **THREE-WAY 27B GATE PASSES** (single-request greedy): our-ON == vLLM `--speculative-config mtp` greedy == our-OFF token-for-token; **acceptance 16/16 drafts accepted**, ~16 target steps saved. Spec-OFF SACRED byte-identical (27B 235/235, 35B 315/315, Coder 138/138), `test_ops_gdn` 3678, compute-sanitizer 0 on the spec step. NOT `DONE`: MIXED `GdnBlockPaged` split/merge (concurrency) + throughput A/B are I6. **I6 LANDED 2026-07-25 (`CLAIM-SPEC-MTP-I6`), `benchmark_binding=true` — the §5 c1 THROUGHPUT GATE, first spec-decode speed number:** OURS spec-ON (`examples/vllm-bench` + an additive `--speculative-config` flag, production config) vs pinned vLLM 0.25.0 spec-ON (graphed `vllm serve --speculative-config mtp` + `vllm bench serve`, `enforce_eager=False`/`FULL_AND_PIECEWISE`/inductor; MTP confirmed `Resolved architecture: Qwen3_5MTP`), SAME `{"method":"mtp","num_speculative_tokens":1}`, 27B `~/bench/q36-27b-nvfp4-vllm`, c1, greedy, 8 real prompts x 256 out, prose + code, idle box one-engine-at-a-time under one `flock`, 3 reps (cold TTFT discarded), token-identity re-confirmed FIRST (`test_qwen27_spec_decode` PASS 16/16). RESULT — **ours AT/ABOVE vLLM on EVERY measured axis** (prose / code): TPOT 66.2/62.95 vs 69.1/65.3 ms (ours ~1.04x faster), output tput 15.10/15.72 vs 14.43/15.13 tok/s (+4.6%/+3.9%), ITL 121.6/121.1 vs 123.2 ms, TTFT(warm) 131/131 vs 151.5/181 ms, acceptance ours 0.85/0.92 vs vLLM 0.838 overall (within noise, live drafter both), peak RSS 28.4 GB ON / 24.8 GB OFF (both inside the 119 GiB pool). Spec helps both (ours 1.52x/1.59x, vLLM 1.51x/1.60x TPOT); ours already ~4% faster spec-OFF. STAYS `ACTIVE`: the c>1 mixed spec+non-spec `GdnBlockPaged` split/merge is still refused (needs a row `IndexSelect`/`IndexCopy` vt op) + owes a c>1 A/B, and no user-facing supported `--speculative-config` on the OpenAI server yet (bench flag example-only/additive). Raw logs dgx `~/work/mtp-bench-i6/{results,vresults}`. **I7 LANDED 2026-07-25 (`CLAIM-SPEC-MTP-I7`, `benchmark_binding=true`) — the MIXED spec+non-spec GDN batch (concurrency), the server/CLI `--speculative-config`, and the c>1 A/B — implementation COMPLETE + at vLLM parity; STAYS `ACTIVE` for one honest reason (below), NOT a lag.** New row op `vt::IndexSelect`/`vt::IndexCopy` (CUDA==CPU bit-exact at GDN widths, RED-first); `GdnBlockPagedMixedSpec` split/merge (mirror `qwen_gdn_linear_attn.py:1329-1576`) proven MODEL-INDEPENDENTLY bit-exact (mixed == pure spec + pure prefill, 27B/35B, `test_qwen3_5_gdn_spec_routing`, RED-first by a broken merge); compute-sanitizer 0 on the mixed step + op; server (I5d) + CLI (ABI v6) `--speculative-config`. **c>1 A/B (both spec-ON, same config):** ours ON-PAR-OR-ABOVE vLLM at c2/c4/c8 (output tput within ~+/-2%, ours +1.6%/+2.5% c2, +0.9%/+1.7% c4, +0.9%/-1.1% c8 within noise, prose/code; both ~1.5x spec speedup — does NOT go neutral; acceptance 0.84-0.92 vs vLLM 0.835). **Why STAYS `ACTIVE` (honest, not a lag):** the DONE criterion's strict `token-exact at c>1` clause is a proven MODEL impossibility — the 27B greedy is bf16-batch-nondeterministic (spec-OFF max_seqs 4-vs-1 differs 2/3 short prompts, NO spec involved), affecting vLLM identically, so exact c>1 token identity cannot be met by any correct implementation; c>1 correctness is instead established by the model-independent bit-exact split/merge proof + acceptance parity (near-tie-distributional-gate), with token-exact strict at c1 (I6). No missing work, no lever — the DONE final call is deferred to the user given this criterion ambiguity. SACRED spec-OFF byte-identical 27B 235/235, 35B 315/315, Coder 138/138; CUDA `-Werror` 0 warnings. Raw logs dgx `~/work/mixed-batch/{cN_results,cN_vresults}`. **I8 — `SPEC-MTP` → `DONE` 2026-07-26 (`CLAIM-SPEC-MTP-DONE`, records-only, ZERO code):** the user RATIFIED the deferred c>1 criterion — at concurrency > 1 the DONE bar is the near-tie-distributional form (ours ∈ vLLM's batch-nondeterministic set) + the SPEED delta, NOT strict token-exact (a proven bf16-batch-nondeterminism MODEL impossibility that affects vLLM identically). Both I6-owed DONE items are therefore CLOSED: (1) the MIXED spec+non-spec `GdnBlockPaged` split/merge (I7, model-independently bit-exact + compute-sanitizer 0) with the c2-c8 A/B on-par-or-above vLLM, and (2) the server + CLI + C-ABI(v6) `--speculative-config` flag (I5d/I7, `examples/server/main.cpp`+`examples/cli/main.cpp`+`src/capi/vllm_c.cpp`). MTP k=1 spec-decode is COMPLETE and gated: 27B three-way token-exact at c1 (I5e), c1 above vLLM on every axis (I6), c2-c8 on-par-or-above (I7), spec-OFF byte-identical SACRED (27B 235/235, 35B 315/315, Coder 138/138). This transition is byte-identical BY CONSTRUCTION (`git diff --stat` = records only; ZERO `src/`/`include/`/`examples/` touched, so the I5d/I6/I7 GPU gates stand on this exact code). Tracked follow-ons: the 35B `Qwen3_5MoeMTP` full e2e token gate (M-mtp-2) is now **CLOSED — `DONE` 2026-07-26 (`CLAIM-SPEC-MTP-M-MTP-2`)**: three-way token-exact 16/16 vs the live vLLM 0.25.0 oracle (spec-ON AND spec-OFF), acceptance 16/16 both sides, c1 spec-ON 1.19x TPOT / +16.3% output-tput vs spec-OFF (0.908) — `MODEL-SPEC-qwen3-5-mtp-qwen3-5-moe-mtp` `GATING`→`DONE`, so MTP is `DONE` on BOTH gate models. Remaining spec-decode follow-on: `SPEC-DFLASH` (oracle-BLOCKED, vllm#40898) | T1 | `vllm/v1/worker/gpu/spec_decode/mtp/speculator.py:12`; `vllm/model_executor/models/qwen3_5_mtp.py:63,129-165,192-301`; **I5d** `vllm/engine/arg_utils.py` (`--speculative-config`); `vllm/v1/worker/gpu/model_runner.py:1455-1489` | `include/vllm/config/speculative.h`; `include/vllm/v1/core/sched/scheduler.h`; `src/vllm/v1/core/sched/scheduler.cpp`; `include/vllm/v1/worker/gpu/input_batch.h`; `include/vllm/model_executor/models/qwen3_5_mtp.h:23,58`; `src/vllm/model_executor/models/qwen3_5_mtp.cpp:271`; `src/vllm/model_executor/models/qwen3_5.cpp:3336,3359`; **I5d** `src/vllm/config/speculative.cpp`; `src/vllm/entrypoints/model_loader.cpp` (`ResolveSpecConfig`/`MakeKVCacheMaybeSpec`/ctor wiring); `src/vllm/v1/worker/gpu/runner.cpp` (splice/tap/GDN spec feed/`propose_drafts`/`take_draft_token_ids`/spec-slot remap/draft-KV alloc); `examples/server/main.cpp` | `tests/vllm/v1/test_scheduler.cpp:1135,1238,1272,1316`; `tests/vllm/v1/worker/test_input_batch.cpp`; `tests/vllm/v1/spec_decode/test_mtp_speculator.cpp:201,225,263,299,331` (7/7 cases, 141 assertions); oracle runner `tests/parity/test_op_parity.cpp:1373` + focused case `:1914` (20/20 assertions, both checkpoints, `VLLM_MTP_REQUIRE_CHECKPOINTS=1`); goldens `tests/parity/goldens/qwen3_5_mtp_head_{27b,35b}/`; dump `tools/parity/dump_qwen3_5_mtp.py:144`; **I5d** `tests/parity/test_qwen27_spec_decode.cpp` (three-way gate, RUNS + measures the RCA blocker); **I6** `examples/bench/{main.cpp,bench_core.h}` (additive `--speculative-config` bench flag + acceptance telemetry); **I7** `tests/vllm/models/test_qwen3_5_gdn_spec_routing.cpp` (mixed == pure spec + prefill bit-exact), `tests/parity/test_qwen27_spec_decode_concurrent.cpp`, `tests/vt/test_ops_gdn.cpp` (IndexSelect/IndexCopy); DONE closure [ledger](parity-ledger.md#L714) | [mtp-spec-decode.md](specs/mtp-spec-decode.md) | `DONE` | `72f9fb1` | | `SPEC-MTP-K-GT-1` | **MTP speculation DEPTH (`num_speculative_tokens` > 1).** Ports the autoregressive multi-step propose the k=1 early exit sits in front of, so a configured depth is SERVED instead of silently degraded. Before it, `--num-speculative-tokens 3` reserved KV for 3, captured the verify shape at T=4 and stashed ONE draft per request, with no error and no log; a refusal by name landed first and this row removed it in the same flow. `MtpProposeDrafts` runs the prefill, the k=1 early exit, then `prepare_decode_inputs` and the k-1 single-token draft decode steps over the draft's own paged KV, with `update_draft_inputs` recording each step and feeding it forward. Greedy plus accept-if-equal makes the emitted sequence INDEPENDENT of k, so a token-identity gate cannot see a clamped drafter and every depth assertion needs a positive witness beside the identity. The per-depth counters were the FIRST witness and a fresh review proved them BLIND: they report the LENGTH of the emitted draft list, so a propose that runs one forward and pads all k columns satisfies them, and acceptance is zero at every depth on the CPU model, so no acceptance figure separates the arms either. TWO witnesses survive, because one does not cover both failures. `spec_mtp_draft_decode_forwards() == spec_mtp_propose_calls() * (k - 1)`, counted after each draft decode forward RETURNS and guarded by a non-zero call count, catches a propose that SHORT-CIRCUITS or CLAMPS. A third fresh review then proved it does NOT catch PADDING, since a loop that runs every forward and then discards what it sampled increments it honestly. `spec_mtp_proposals_with_varied_drafts()`, read at the CONSUMER on the array the propose delivered, catches exactly that. NEITHER shows per-column provenance, and neither does a non-zero acceptance count AT DEPTH, which a padded row earns whenever the target repeats a token. The owed DGX gate closes it with a per-depth acceptance RATE against a PADDED CONTROL. The CPU tier therefore proves k drafts are PROPOSED and VERIFIED, never ACCEPTED at depth. DEFAULT unchanged at k=1 (both checkpoints' `n_predict`). **NO speed number at any k>1**: the GPU was held by another session for the whole flow, so the DGX three-way at k=2..4 on the 27B and 35B and the matched-k throughput A/B are OWED, as is the bf16 GDN-state arm (the CPU gate runs the f32 arm because `vt::CausalConv1dSpecUpdate` rejects bf16 off CUDA). Also owed and filed: [#1020](https://github.com/mudler/vllm.cpp/issues/1020), a step whose ACTUAL draft count differs from the configured k leaves the captured verify graph silently. | T1 | `vllm/v1/worker/gpu/spec_decode/autoregressive/speculator.py:129-274,335-371,374-419,426-471,597-671,674-771` @ `555967922`; `vllm/config/speculative.py:967-991` | [`src/vllm/v1/worker/gpu/spec_decode/mtp/speculator.cpp`](../src/vllm/v1/worker/gpu/spec_decode/mtp/speculator.cpp) (`MtpProposeDrafts`); [`prepare_decode_inputs.cpp`](../src/vllm/v1/worker/gpu/spec_decode/autoregressive/prepare_decode_inputs.cpp); `Qwen3_5MTPModel::GatherHiddenRows` ([qwen3_5.cpp](../src/vllm/model_executor/models/qwen3_5.cpp)); `GPUModelRunner::propose_drafts` + the per-depth counters ([runner.cpp](../src/vllm/v1/worker/gpu/runner.cpp), [runner.h](../include/vllm/v1/worker/gpu/runner.h)); the in-memory `mtp_weights` seam ([model_loader.h](../include/vllm/entrypoints/model_loader.h)) | [`test_mtp_depth`](../tests/vllm/v1/spec_decode/test_mtp_depth.cpp) 5/5, 63 assertions (k=1,2,3,4 through `LoadedEngine`, greedy tokens identical to spec-OFF, each arm witnessed BOTH by the draft decode forwards the propose RAN and by whether the DELIVERED draft row varied with depth; neither witness shows per-column provenance, which is owed to the DGX gate); [`test_prepare_decode_inputs`](../tests/vllm/v1/spec_decode/test_prepare_decode_inputs.cpp) 8/8, 33 (both kernel ports + both `max_model_len` clamps, 5 mutations caught); [`test_speculative_mtp_depth`](../tests/vllm/config/test_speculative_mtp_depth.cpp) 4/4, 20; full CPU suite ctest 493 passed / 0 failed / 2 skipped of 495 (the two skips checkpoint-gated and unrelated) | [mtp-k-gt-1.md](specs/mtp-k-gt-1.md) | `ACTIVE` | `CLAIM-SPEC-MTP-K-GT-1` ([#81](https://github.com/mudler/vllm.cpp/issues/81)) | -| `SPEC-MTP-GGUF` | MTP speculative decoding from a GGUF TARGET. Today `FromModelDir` refuses `mtp`+GGUF outright (`src/vllm/entrypoints/model_loader.cpp:717-723`) on the original spike's assumption that GGUF exports carry no `mtp.*` ([mtp-spec-decode.md](specs/mtp-spec-decode.md):979-980, "until we re-export GGUFs with the head"). That is stale: llama.cpp's Qwen3.5 converter DOES emit the head, under layer-indexed `nextn` naming, and our own `HfConfigFromGguf` ALREADY reads `nextn_predict_layers` (it just discards the value into the trunk layer count). Gap is a `TensorResolver` over `GgufFile` mapping `mtp.*` onto `blk.{L+i}.nextn.*` with dequant-to-bf16, one config field, and narrowing the rejection to `dflash`. `ngram`+GGUF already works and is untouched. Qwen3.5/3.6 only (the widened spec KV path serves no other arch). NO ABI change | T2 | llama.cpp (the producer contract; vLLM has no GGUF MTP path) `conversion/qwen.py:535-604` `_Qwen35MtpMixin` (the authoritative `mtp.*`->`nextn` remapper + `add_nextn_predict_layers`); `gguf-py/gguf/constants.py:129,910-917,1494-1501`; `gguf-py/gguf/tensor_mapping.py` `NEXTN_*` | **G1-G3 LANDED 2026-07-28.** `HfConfigFromGguf` republishes the head depth `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp:598` (`c.raw["mtp_num_hidden_layers"] = nextn`, previously read then discarded); the head loader `LoadQwen3_5MTPFromGguf` `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp:1448` (+ decl `include/vllm/model_executor/models/qwen3_5_gguf_weights.h:143`) reusing the TRUNK helpers `OwnNormMinus1`/`OwnMatmulWeight`/`OwnBf16`/`LoadAttnGguf`/`LoadMoeGguf` so the head inherits the GGUF (w+1) norm storage, quantization/residency routing and torch [N,K] shapes; `NumMtpLayers`/`UsesDedicatedEmbeddings` exported out of the anon namespace `include/vllm/model_executor/models/qwen3_5_mtp.h`; rejection narrowed to dflash + a head-less-GGUF check `src/vllm/entrypoints/model_loader.cpp` and the head attached in the GGUF branch; **G4 GREEN + `CPU-SPEC-DIVERGENCE` FIXED 2026-07-28**: root cause `src/vllm/model_executor/models/qwen3_5.cpp:3616` sized the GDN state gather/scatter row by `(Kw-1)` while the speculative persistent row is `(Kw-1)+num_spec`, so `GatherRows`/`ScatterRows` mis-strode the slot AND every channel past the first, corrupting post-prefill recurrent state. Fix = `CopyStateRowsStrided` (same TU) used by `GatherStateF32`/`ScatterStateF32` when `cache.shape[2] != work.shape[2]`; the contiguous helpers are kept when the widths agree, so every non-spec path is byte-identical by construction. CPU-only in effect (the fp16/bf16 arm routes through the `GdnStateGather`/`Scatter` ops, so CUDA was never exposed; no GPU result affected) | `tests/vllm/models/test_qwen3_5_gguf_mtp.cpp:109,146,156,184` **4 cases, and the split is the 2026-08-21 repair** ([#1454](https://github.com/mudler/vllm.cpp/issues/1454)): the file used to be the env-gated pair ALONE, each opening on a bare `return`, so with `VLLM_MTP_GGUF_MODEL` unset it reported `test cases: 2 \| 2 passed`, **`assertions: 0`**, `Status: SUCCESS!`, exit 0 - which is every CI run of this repository, the variable being set nowhere in `.github/workflows/`. The `18 assertions` this cell used to record was the LIVE count and was never once reached in CI. Now `:109` and `:146` are **HERMETIC** (KV-only synthetic GGUFs, no weight bytes, 18 assertions on any machine) and pin the arithmetic the old file only NAMED in a comment above `CHECK(c.num_hidden_layers > 0)`: `num_hidden_layers + mtp_num_hidden_layers == block_count` over 65/1, 25/1 and 28/3 - the third arm separating `- nextn` from `- 1` - plus the head-less arm, where the key is NOT published and `NumMtpLayers` answering 1 for an absent key is exactly why the invariant cannot be written with that helper alone. `:156` and `:184` stay env-gated on `VLLM_MTP_GGUF_MODEL` (so CI stays asset-free) and now SKIP LOUDLY with a `MESSAGE` naming the variable, as `tests/vllm/entrypoints/test_gguf_mmproj_reach.cpp` does; `:156` re-derives the same invariant from the file's OWN `block_count` kv. Unset: **4 cases / 18 assertions / `Status: SUCCESS!` / rc 0**. Live on `Qwen3.8-27B-Q4_K_M.gguf` (`block_count` 65, `nextn_predict_layers` 1): **4 cases / 38 assertions / `Status: SUCCESS!` / rc 0**. Mutation-proved on the production line `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp:889`, both compiling clean and both restored against a pre-taken sha256: `= block_count` (drop the subtraction) 3/4 cases, 9/18 red, exit 1; `= block_count - 1` (the wrong constant) 2/4 cases, 5/18 red, exit 1. The SAME mutations left the PREVIOUS file at 2/2 cases, 0 assertions, `SUCCESS!`, exit 0. Correctness of the production line is unchanged and was never in question (`1a4db5c3c`, `493327b4e`); this was a test defect. Live-arm content unchanged: depth reaches config.raw; fc is [H,2H] verbatim; 3 norms [H]; head block is full-attention. **RED-first BEHAVIOURAL** (reverting only the G1 line fails both cases 2/2). Trunk inertness: `test_gguf` 103, `test_gguf_qwen36_loader` 99, `test_gguf_keep_quant` 5958, `test_gguf_dequant` 215, `test_capi` 33/232 all unchanged; `tests/parity/test_qwen35_gguf_spec_decode.cpp:74,139` - spec-ON == spec-OFF token-exact with 13 proposed/11 accepted, plus an `ngram` regression guard (widens the cache, never runs the spec conv update) that was token-exact throughout and pinned the widening as innocent. Regression sweep all unchanged: ops_gdn 1825, gdn_metadata_builder 483, gdn_prefill_conv 28, gdn_spec_routing 12, gguf 103, gguf_qwen36_loader 99, gguf_keep_quant 5958, gguf_dequant 215, llm_engine 196, input_batch 163, runner 257, capi 232 **GPU CLOSE-OUT + DEVICE-DELTA ATTRIBUTION 2026-07-28 (`G5`-`G7`), ledger [parity-ledger.md#L800](parity-ledger.md#L800).** The GPU end-to-end gate re-run on a from-scratch RELEASE-TARGET build (`-DVLLM_CPP_CUDA_ARCHITECTURES=121a`, build dir DELETED first; arch VERIFIED by `build-cuda/CMakeFiles/vllm.dir/flags.make` `--generate-code=arch=compute_121a,code=[compute_121a,sm_121a]` and by `cuobjdump -lelf` 20 cubins ALL `sm_121a` zero sm_75, NOT by `CMakeCache.txt`, whose `CMAKE_CUDA_ARCHITECTURES:STRING=75` is the `enable_language(CUDA)` compiler-probe default shadowed by the normal variable at `CMakeLists.txt:186` - the prior wrong-arch conclusion was that decoy): dgx.casa GB10 under `flock $HOME/gpu.lock`, 35B A3B NVFP4 GGUF, **2/2 cases, 10/10 assertions, exit 0**, spec-ON token-identical to spec-OFF, 13 proposed / 11 accepted, 90.2 GiB peak RSS, 8m01s; re-run on the EXACT committed source **3/3 cases, 10/10 assertions, exit 0**, 7m25s, the new probe case SKIPping and adding zero assertions. **The CPU-vs-GPU token delta is a MEASURED near-tie, not a defect** (it was never this row's bar - spec-ON == spec-OFF WITHIN a device is): NEW double-gated spec-OFF-only probe `tests/parity/test_qwen35_gguf_spec_decode.cpp:217` (asset + `VLLM_MTP_GGUF_PROBE=1`, 20 alternatives per position, 484/484 assertions per arm, GPU then `CUDA_VISIBLE_DEVICES=` in one `flock` series) shows both arms picking `11751` at position 0 and forking at position 1 on a BIT-IDENTICAL prefix: GPU rank1 `13` -0.773180 over rank2 `11` -0.847055 (margin 0.0739 nats), CPU rank1 `11` -0.765499 over rank2 `13` -0.830374 (margin 0.0649 nats). Each device's pick is the other's rank 2, both ~7x inside the ratified 0.5-nat band, and the cross-device disagreement on the SAME token (0.057 and 0.082 nats) EXCEEDS the margin being decided, so rounding settles it; the 24 texts look unrelated only because positions 2+ cascade off that one coin flip. Margin sweep over all 24 positions: **GGUF GPU and GGUF CPU carry ZERO exact ties**, minimum margins 0.0482 and 0.0649 nats, and both arms reproduced their sequence across every run. **Gate 4 MET on the safetensors sibling of the same quantization run** (`FromModelDir` takes it unchanged): acceptance 12 proposed / 11 accepted vs the GGUF's 13 / 11. That arm, however, FAILS spec-ON == spec-OFF at concurrency 1 and does not reproduce its own spec-OFF sequence run to run, and the probe attributes both to THREE EXACT ties (positions 7, 10, 16, bit-identical logprobs) produced by its 1/16-grid quantized-GEMM logits - which EXONERATES the GGUF arm and opens a recorded, not-root-caused `SPEC-MTP` item on the safetensors NVFP4 path, not on this row. Gate 3 is NOT APPLICABLE twice over: no F16/F32 head-carrying export exists, and the only same-weights sibling is not token-stable against itself. **EVIDENCE RE-ANCHORED 2026-07-29 to a PRODUCTION-CONFIGURED build, because every GPU number above came from a build configured WITHOUT `-DVLLM_CPP_CUTLASS_DIR` and WITHOUT `-DVLLM_CPP_TRITON=ON`** (the defect `CLAIM-27B-GATE-RCA` proved, which runs the emulation fp4 GEMM + hand GDN kernels). Re-run from a clean `git archive` tree of `main` `3f34534d`, build proven correct three ways (configure log has ZERO `CUTLASS not found` and prints `CUTLASS found ... sm120a NVFP4 cutlass GEMM` + `FlashAttention-2 ... ENABLED for arch(es) [121a]` + the vendored `sm_121a` Triton-AOT lines with `MANIFEST hashes OK`; `cuobjdump -lelf` 40 cubins ALL `sm_121a`, zero `sm_75`; SACRED `test_qwen27_paged_engine` **235/235 exit 0**, and the build precondition proven to FIRE by recompiling only that TU without the two defines against the same `libvllm.a`, which throws and exits 1 with 0 assertions). **The row PASSES UNCHANGED:** `tests/parity/test_qwen35_gguf_spec_decode` **3/3 cases, 10/10 assertions, exit 0**, spec-ON token-identical to spec-OFF, **13 proposed / 11 accepted (identical to the recorded number)**, 90.26 GiB, 7m13.59s; loader gate 19 assertions on the Qwen3.5-2B and 18 on the 35B A3B, unchanged. **ONE recorded finding is RETRACTED by the re-measurement: the CPU-vs-GPU token delta was a BUILD artifact, not a device near-tie cascade.** On the production build both devices emit the SAME 24 tokens; the probe shows GPU rank1 `11` -0.763897 over rank2 `13` -0.824083 where the defective build had rank1 `13` -0.773180 over rank2 `11` -0.847055, while the CPU arm is bit-identical to the earlier measurement (CUTLASS and Triton are CUDA-only). Zero exact ties in either arm, min margins 0.060186 GPU / 0.064875 CPU, 484/484 assertions per arm. Evidence: [docs/BENCHMARKS.md](../docs/BENCHMARKS.md) top section, [parity-ledger.md](parity-ledger.md) | [specs/gguf-mtp-spec-decode.md](specs/gguf-mtp-spec-decode.md) | `DONE` | `edf91449` | -| `SPEC-DFLASH-GGUF` | DFlash speculative decoding from GGUF, two axes: (A) GGUF DRAFT + safetensors target, (B) GGUF target too. llama.cpp master carries a full `dflash` GGUF contract (arch string `dflash`, tensors `fc`/`enc.output_norm`/`output_norm`/`blk.N.*`, KVs `dflash.target_layers` + `dflash.target_hidden_size`); the arch is ABSENT from checkouts older than ~2026-07, so a stale tree reads as "no contract exists". The GGUF tensor set omits `token_embd`/`output` because the draft SHARES the target's embed+lm_head, which is exactly what `LoadDflashDraft` already does. Blockers are in the loader, not the model: `MakeDflashDraftConfig` reads `draft_dir/config.json` (a GGUF has none), `ResolveDflashDraftDir` probes for `config.json` so it cannot see a `.gguf`, and `LoadDflashDraft` is typed on `std::vector` for the shared bf16 head (the axis-B blocker). Axis A independently shippable. NO ABI change | T2 | llama.cpp `origin/master` @ 2026-07-28 (tag era `b10158`): `gguf-py/gguf/constants.py:547,1151,4350`; `gguf-py/gguf/tensor_mapping.py:1297-1305` (`ENC_OUTPUT_NORM`<-`model.hidden_norm`, `FC`<-`model.fc`); `conversion/qwen.py:351` (mask token via the standard tokenizer KV); `convert_hf_to_gguf.py --target-model-dir` | **GD1-GD7 LANDED 2026-07-28 (BOTH AXES COMPLETE and PROVEN end to end on GB10)**: `MakeDflashGgufConfig` + `LoadQwen3DFlashFromGguf` `src/vllm/model_executor/models/qwen3_dflash_gguf.cpp:88,227` (+ header), `IsDflashGgufDraft` + the `.gguf` branch in `ResolveDflashDraftDir`/`LoadDflashDraft` `src/vllm/entrypoints/model_loader.cpp:121,222`. Goes through the `TensorResolver` seam (unlike `SPEC-MTP-GGUF`) because dflash norms are RAW, so the existing `LoadQwen3DFlash` qkv/gate_up concatenation is reused unchanged. **`GD4` defect FIXED** (`model_loader.cpp:238-249`): the GGUF branch left `config.vocab_size` 0 - correct for `MakeDflashGgufConfig` (the DFLASH arch has no vocab KV and no `token_embd`) but fatal for the forward, which sizes the shared embedding view as `{config.vocab_size, H}`, so the first propose threw `cuda embedding: empty table (vocab 0)`. Now back-filled from the target's `embed_tokens` rows (the condition is on the VALUE, not the draft source, so it generalizes to a GGUF target). Load-level green had hidden it; only GENERATING found it. **GD5-GD7 = axis B**: `SharedHeadSource` `src/vllm/entrypoints/model_loader.cpp` re-expresses the shared bf16 `embed_tokens`+`lm_head` seam as a SOURCE and re-types `LoadDflashDraft`'s second parameter - THAT TYPE was the whole axis-B blocker - with the GGUF arm `LoadGgufSharedEmbedAndHeadBf16` `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp:1035` reusing the trunk loader's tied-embedding rule and sidecar-aware dequant instead of restating them; the shared-head load moved into ONE common tail so all four (draft format x target container) combinations run identical code; the `dflash` half of the GGUF-branch rejection `model_loader.cpp` is deleted (the `mtp` half untouched) and the draft load is wired into the GGUF branch | `tests/vllm/models/test_qwen3_dflash_gguf.cpp:36,84` 2 cases / 47 assertions against the REAL published Qwen3.6-27B DFlash draft (env-gated `VLLM_DFLASH_GGUF_MODEL`, CI asset-free): the +1 target-layer offset undone against the KV read back from the same file, block_size/mask_token present, vocab_size left 0, layer_types cover every block, fc `[H, H*num_taps]` with `nk` SET, qkv/gate_up row-concat shapes, embed/lm_head left EMPTY for the target. **RED-first BEHAVIOURAL** (dropping the `-1` fails the offset checks). **`GD4` e2e gate** `tests/parity/test_qwen27_dflash_spec_decode.cpp:343` (second case, draft source env-driven via `VLLM_DFLASH_DRAFT`/`_B`; asset-gated, CI-inert): on dgx GB10 sm_121a against the Qwen3.6-27B NVFP4 safetensors target, the Q4_K_M GGUF draft and the bf16 z-lab safetensors draft produce **token-for-token IDENTICAL** DFlash-ON continuations with **IDENTICAL** accepted/proposed (20/80 on a 24-token prompt, 42/96 on a 48-token prompt), spec-OFF self-reproducible 3/3 and 0 exact ties (min margin 0.197/0.400 nats). Regression: gguf_mtp 19, qwen35_gguf_spec_decode 10, gguf 103, gguf_qwen36_loader 99, gguf_keep_quant 5958, ops_gdn 1825, llm_engine 196, capi 232, runner 257 all unchanged. **`GD5` unit gate** `tests/vllm/test_gguf_qwen36_loader.cpp` 3 new synthetic-GGUF cases (6 cases / 286 assertions total, CPU and the dgx CUDA build): the untied head really comes from `output.weight` and not the embedding (distinct fill values), the tied fallback aliases it onto `token_embd`, the `nk` flags separate the gather table from the MatmulBT weight, a file with no `token_embd` is refused. 3-mutant battery, 3 caught (`nk` flipped, head forced to the embedding, tied forced false). **`GD7` e2e gate** `tests/parity/test_qwen27_dflash_spec_decode.cpp` third case (targets env-driven via `VLLM_DFLASH_TARGET_B`; asset-gated, CI-inert): on dgx GB10 sm_121a the Qwen3.6-27B NVFP4 **GGUF** target + `Q4_K_M` GGUF draft loads, takes the shared head from the GGUF, generates, and its DFlash-ON continuation is **token-for-token IDENTICAL to that same target's spec-OFF** (24/24, the STRICT form) with acceptance ALIVE at 14/160; 1 case / 15 assertions, exit 0. **The spike's highest risk is EMPTY on this asset, proven not assumed**: the 27B NVFP4 GGUF stores `token_embd`/`output` as ggml BF16, byte-identical to the safetensors sibling (2,542,796,800 bytes each, ZERO differing), so B1's shared-head read is verbatim, not a dequant. Acceptance IS lower than the safetensors-target arm and is NOT chargeable to the head: the two containers diverge at index 4 with NO speculation, because `QUANT-GGUF-NVFP4` is dequant-only so the GGUF target computes in bf16 while the safetensors target runs the true W4A4 kernels. **RE-MEASURED 2026-07-29 on a PRODUCTION-CONFIGURED build (`CLAIM-GGUF-SPEC-REVERIFY`), because every GD4/GD7 GPU number above came from a build configured WITHOUT `-DVLLM_CPP_CUTLASS_DIR` and WITHOUT `-DVLLM_CPP_TRITON=ON`.** Build proven correct three ways (see the `SPEC-MTP-GGUF` row; SACRED 27B **235/235**, `cuobjdump` 40 cubins all `sm_121a`). **AXIS B HOLDS EXACTLY**: `test_qwen27_dflash_spec_decode -tc="dflash axis-B*"` **15/15 assertions, exit 0**, GGUF-target DFlash-ON token-identical to that target's own spec-OFF 24/24, acceptance **14/160 unchanged**, cross-target spec-OFF divergence still at index 4, 81.01 GiB peak RSS, 6m53.08s. **AXIS A WAS RED ON THE 48-TOKEN PROMPT (reproducibly, 3 of 3 runs) AND IS NOW CLOSED.** The RED was real: cross-format TOKEN identity held on both prompts, but the exact accept-count half of bar (a) failed (`arm_a.proposed == arm_b.proposed` / `arm_a.accepted == arm_b.accepted`) because the Q4_K_M draft measured **46/112** against the bf16 z-lab draft's **47/96** (one extra 16-wide propose block, one fewer acceptance, zero token difference), 15/17, exit 1; the 24-token prompt stayed green at 17/17 with both drafts at 15/144. **`GD9` 2026-07-29 root-caused it IN WEIGHT SPACE as ordinary `Q4_K_M` cost, category (a), not a defect in our GGUF draft path - and the bar's own premise ("Same weights, two containers") was false for the asset it was pointed at.** The publishing repo also carries an UNQUANTIZED `BF16` GGUF (3,471,497,440 B) beside `Q8_0`/`Q6_K`/`Q5_K`/`Q4_K_M`, which the spec had recorded as nonexistent; that retired the `NOT APPLICABLE` on gate 2. CPU gate `tests/vllm/models/test_qwen3_dflash_gguf.cpp` third case (asset-gated `VLLM_DFLASH_GGUF_BF16_MODEL` + `VLLM_DFLASH_ST_DIR`): `LoadQwen3DFlashFromGguf(BF16)` is **BYTE-IDENTICAL to `LoadQwen3DFlash(z-lab shards)` on all 58 tensors, 302/302 assertions, exit 0**, and FUNCTIONALLY RED against the `Q4_K_M` file (21/302 red, exactly the 21 quantized matmul tensors), so not a vacuous pass. Supporting: our `DequantGgufRowToBf16` is bit-equal to `gguf-py`'s `gguf.quants.dequantize` on the real `fc.weight` (Q4_K), `blk.0.attn_q.weight` (Q4_K) and `blk.2.ffn_down.weight` (Q6_K), zero differing bf16 values; the ladder's mean relative weight error is monotone and uniform with NO outlier tensor (BF16 0, Q8_0 5.6e-3, Q6_K 1.85e-2, Q5_K 3.85e-2, Q4_K_M 7.6e-2); the only numeric config delta is `rms_norm_eps` at 2.5e-9 relative. Also landed: an off-by-default `VT_SPEC_TRACE=1` per-block propose/accept trace in `GPUModelRunner::sample_tokens_with_rejection` (`src/vllm/v1/worker/gpu/runner.cpp`). **`GD10` 2026-07-29 CONFIRMED IT END TO END ON GB10 and closed gates 3 and 5.** Build proven production-configured three ways (configure log 0 `CUTLASS not found`; `cuobjdump -lelf` 40 cubins ALL `sm_121a` zero `sm_75` on both binaries; SACRED `test_qwen27_paged_engine` **235/235, exit 0**, 31.34s, 23.67 GiB). The **`BF16` GGUF draft reads EXACTLY 47/96**, the safetensors draft's own number, at 48 tokens on the discriminating prompt - reproduced 2 of 2 - plus 27/64 = 27/64 at 24 tokens and 15/144 = 15/144 on the second prompt, tokens IDENTICAL throughout, 17/17 exit 0 each time; the `Q4_K_M` arm reads 46/112 on the SAME binary in the SAME `flock` series. Restoring only the draft's numeric precision restores the count, so quantization is the whole cause and nothing structural survives. Bar (a) is consequently SPLIT rather than relaxed (`tests/parity/test_qwen27_dflash_spec_decode.cpp`): tokens stay EXACT unconditionally; accept counts are EXACT on a cross-FORMAT arm and BANDED (`abs(d_accepted) <= 2`, `abs(d_proposed) <= k*2`) on a cross-QUANTIZATION one, with the arm chosen by `IsQuantizedGgufDraft` reading the draft file's ggml types (`GgmlTraits().block_elems > 1`) rather than by a flag. The band is derived, not picked: measured `d_accepted` is 0, 0, -1, so the bound is that maximum plus one quantum; and `d_proposed = -k * d_accepted` EXACTLY once the token streams match (confirmed at -1 / +16), so the proposed bound follows. **Mutation-proved non-vacuous**: rebuilt at band 0 the `Q4_K_M` arm is 15/17 exit 1 while the `BF16` arm stays 17/17 exit 0 on the exact branch. **AXIS B BROADENED from ONE prompt to THREE**, strict form green on all: "The capital of France is" IDENTICAL 14/160 (15/15), "Write a Python function that reverses a string:" IDENTICAL 24/64 (15/15), "Photosynthesis is the process by which" IDENTICAL 15/128 (9/9), all exit 0, ~6m30-6m52 and ~81 GiB peak RSS each. The second prompt REFINES the recorded acceptance claim: the safetensors-target arm is ALSO 24/64 there with the two containers' DFlash-ON streams IDENTICAL, so the GGUF target's lower acceptance is prompt-dependent (their spec-OFF streams diverge at index 4 on the first prompt, index 16 on the second) and not a standing penalty; the cause remains `QUANT-GGUF-NVFP4` being dequant-only, with the shared head excluded by a byte comparison. Gates 1-5 and 7 MET; gate 6 (speed) `PENDING` BY DESIGN and not owed - a DFlash-ON throughput A/B between the two target containers is not a fair comparison until a native NVFP4 GGUF GEMM exists. Evidence: [docs/BENCHMARKS.md](../docs/BENCHMARKS.md) top section, [parity-ledger.md](parity-ledger.md#L845) | [specs/gguf-dflash-draft.md](specs/gguf-dflash-draft.md) | `DONE` | `c62f2fa3` | +| `SPEC-MTP-GGUF` | MTP speculative decoding from a GGUF TARGET. Today `FromModelDir` refuses `mtp`+GGUF outright (`src/vllm/entrypoints/model_loader.cpp:717-723`) on the original spike's assumption that GGUF exports carry no `mtp.*` ([mtp-spec-decode.md](specs/mtp-spec-decode.md):979-980, "until we re-export GGUFs with the head"). That is stale: llama.cpp's Qwen3.5 converter DOES emit the head, under layer-indexed `nextn` naming, and our own `HfConfigFromGguf` ALREADY reads `nextn_predict_layers` (it just discards the value into the trunk layer count). Gap is a `TensorResolver` over `GgufFile` mapping `mtp.*` onto `blk.{L+i}.nextn.*` with dequant-to-bf16, one config field, and narrowing the rejection to `dflash`. `ngram`+GGUF already works and is untouched. Qwen3.5/3.6 only (the widened spec KV path serves no other arch). NO ABI change | T2 | llama.cpp (the producer contract; vLLM has no GGUF MTP path) `conversion/qwen.py:535-604` `_Qwen35MtpMixin` (the authoritative `mtp.*`->`nextn` remapper + `add_nextn_predict_layers`); `gguf-py/gguf/constants.py:129,910-917,1494-1501`; `gguf-py/gguf/tensor_mapping.py` `NEXTN_*` | **G1-G3 LANDED 2026-07-28.** `HfConfigFromGguf` republishes the head depth `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp:598` (`c.raw["mtp_num_hidden_layers"] = nextn`, previously read then discarded); the head loader `LoadQwen3_5MTPFromGguf` `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp:1453` (+ decl `include/vllm/model_executor/models/qwen3_5_gguf_weights.h:143`) reusing the TRUNK helpers `OwnNormMinus1`/`OwnMatmulWeight`/`OwnBf16`/`LoadAttnGguf`/`LoadMoeGguf` so the head inherits the GGUF (w+1) norm storage, quantization/residency routing and torch [N,K] shapes; `NumMtpLayers`/`UsesDedicatedEmbeddings` exported out of the anon namespace `include/vllm/model_executor/models/qwen3_5_mtp.h`; rejection narrowed to dflash + a head-less-GGUF check `src/vllm/entrypoints/model_loader.cpp` and the head attached in the GGUF branch; **G4 GREEN + `CPU-SPEC-DIVERGENCE` FIXED 2026-07-28**: root cause `src/vllm/model_executor/models/qwen3_5.cpp:3616` sized the GDN state gather/scatter row by `(Kw-1)` while the speculative persistent row is `(Kw-1)+num_spec`, so `GatherRows`/`ScatterRows` mis-strode the slot AND every channel past the first, corrupting post-prefill recurrent state. Fix = `CopyStateRowsStrided` (same TU) used by `GatherStateF32`/`ScatterStateF32` when `cache.shape[2] != work.shape[2]`; the contiguous helpers are kept when the widths agree, so every non-spec path is byte-identical by construction. CPU-only in effect (the fp16/bf16 arm routes through the `GdnStateGather`/`Scatter` ops, so CUDA was never exposed; no GPU result affected) | `tests/vllm/models/test_qwen3_5_gguf_mtp.cpp:109,146,156,184` **4 cases, and the split is the 2026-08-21 repair** ([#1454](https://github.com/mudler/vllm.cpp/issues/1454)): the file used to be the env-gated pair ALONE, each opening on a bare `return`, so with `VLLM_MTP_GGUF_MODEL` unset it reported `test cases: 2 \| 2 passed`, **`assertions: 0`**, `Status: SUCCESS!`, exit 0 - which is every CI run of this repository, the variable being set nowhere in `.github/workflows/`. The `18 assertions` this cell used to record was the LIVE count and was never once reached in CI. Now `:109` and `:146` are **HERMETIC** (KV-only synthetic GGUFs, no weight bytes, 18 assertions on any machine) and pin the arithmetic the old file only NAMED in a comment above `CHECK(c.num_hidden_layers > 0)`: `num_hidden_layers + mtp_num_hidden_layers == block_count` over 65/1, 25/1 and 28/3 - the third arm separating `- nextn` from `- 1` - plus the head-less arm, where the key is NOT published and `NumMtpLayers` answering 1 for an absent key is exactly why the invariant cannot be written with that helper alone. `:156` and `:184` stay env-gated on `VLLM_MTP_GGUF_MODEL` (so CI stays asset-free) and now SKIP LOUDLY with a `MESSAGE` naming the variable, as `tests/vllm/entrypoints/test_gguf_mmproj_reach.cpp` does; `:156` re-derives the same invariant from the file's OWN `block_count` kv. Unset: **4 cases / 18 assertions / `Status: SUCCESS!` / rc 0**. Live on `Qwen3.8-27B-Q4_K_M.gguf` (`block_count` 65, `nextn_predict_layers` 1): **4 cases / 38 assertions / `Status: SUCCESS!` / rc 0**. Mutation-proved on the production line `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp:889`, both compiling clean and both restored against a pre-taken sha256: `= block_count` (drop the subtraction) 3/4 cases, 9/18 red, exit 1; `= block_count - 1` (the wrong constant) 2/4 cases, 5/18 red, exit 1. The SAME mutations left the PREVIOUS file at 2/2 cases, 0 assertions, `SUCCESS!`, exit 0. Correctness of the production line is unchanged and was never in question (`1a4db5c3c`, `493327b4e`); this was a test defect. Live-arm content unchanged: depth reaches config.raw; fc is [H,2H] verbatim; 3 norms [H]; head block is full-attention. **RED-first BEHAVIOURAL** (reverting only the G1 line fails both cases 2/2). Trunk inertness: `test_gguf` 103, `test_gguf_qwen36_loader` 99, `test_gguf_keep_quant` 5958, `test_gguf_dequant` 215, `test_capi` 33/232 all unchanged; `tests/parity/test_qwen35_gguf_spec_decode.cpp:74,139` - spec-ON == spec-OFF token-exact with 13 proposed/11 accepted, plus an `ngram` regression guard (widens the cache, never runs the spec conv update) that was token-exact throughout and pinned the widening as innocent. Regression sweep all unchanged: ops_gdn 1825, gdn_metadata_builder 483, gdn_prefill_conv 28, gdn_spec_routing 12, gguf 103, gguf_qwen36_loader 99, gguf_keep_quant 5958, gguf_dequant 215, llm_engine 196, input_batch 163, runner 257, capi 232 **GPU CLOSE-OUT + DEVICE-DELTA ATTRIBUTION 2026-07-28 (`G5`-`G7`), ledger [parity-ledger.md#L800](parity-ledger.md#L800).** The GPU end-to-end gate re-run on a from-scratch RELEASE-TARGET build (`-DVLLM_CPP_CUDA_ARCHITECTURES=121a`, build dir DELETED first; arch VERIFIED by `build-cuda/CMakeFiles/vllm.dir/flags.make` `--generate-code=arch=compute_121a,code=[compute_121a,sm_121a]` and by `cuobjdump -lelf` 20 cubins ALL `sm_121a` zero sm_75, NOT by `CMakeCache.txt`, whose `CMAKE_CUDA_ARCHITECTURES:STRING=75` is the `enable_language(CUDA)` compiler-probe default shadowed by the normal variable at `CMakeLists.txt:186` - the prior wrong-arch conclusion was that decoy): dgx.casa GB10 under `flock $HOME/gpu.lock`, 35B A3B NVFP4 GGUF, **2/2 cases, 10/10 assertions, exit 0**, spec-ON token-identical to spec-OFF, 13 proposed / 11 accepted, 90.2 GiB peak RSS, 8m01s; re-run on the EXACT committed source **3/3 cases, 10/10 assertions, exit 0**, 7m25s, the new probe case SKIPping and adding zero assertions. **The CPU-vs-GPU token delta is a MEASURED near-tie, not a defect** (it was never this row's bar - spec-ON == spec-OFF WITHIN a device is): NEW double-gated spec-OFF-only probe `tests/parity/test_qwen35_gguf_spec_decode.cpp:217` (asset + `VLLM_MTP_GGUF_PROBE=1`, 20 alternatives per position, 484/484 assertions per arm, GPU then `CUDA_VISIBLE_DEVICES=` in one `flock` series) shows both arms picking `11751` at position 0 and forking at position 1 on a BIT-IDENTICAL prefix: GPU rank1 `13` -0.773180 over rank2 `11` -0.847055 (margin 0.0739 nats), CPU rank1 `11` -0.765499 over rank2 `13` -0.830374 (margin 0.0649 nats). Each device's pick is the other's rank 2, both ~7x inside the ratified 0.5-nat band, and the cross-device disagreement on the SAME token (0.057 and 0.082 nats) EXCEEDS the margin being decided, so rounding settles it; the 24 texts look unrelated only because positions 2+ cascade off that one coin flip. Margin sweep over all 24 positions: **GGUF GPU and GGUF CPU carry ZERO exact ties**, minimum margins 0.0482 and 0.0649 nats, and both arms reproduced their sequence across every run. **Gate 4 MET on the safetensors sibling of the same quantization run** (`FromModelDir` takes it unchanged): acceptance 12 proposed / 11 accepted vs the GGUF's 13 / 11. That arm, however, FAILS spec-ON == spec-OFF at concurrency 1 and does not reproduce its own spec-OFF sequence run to run, and the probe attributes both to THREE EXACT ties (positions 7, 10, 16, bit-identical logprobs) produced by its 1/16-grid quantized-GEMM logits - which EXONERATES the GGUF arm and opens a recorded, not-root-caused `SPEC-MTP` item on the safetensors NVFP4 path, not on this row. Gate 3 is NOT APPLICABLE twice over: no F16/F32 head-carrying export exists, and the only same-weights sibling is not token-stable against itself. **EVIDENCE RE-ANCHORED 2026-07-29 to a PRODUCTION-CONFIGURED build, because every GPU number above came from a build configured WITHOUT `-DVLLM_CPP_CUTLASS_DIR` and WITHOUT `-DVLLM_CPP_TRITON=ON`** (the defect `CLAIM-27B-GATE-RCA` proved, which runs the emulation fp4 GEMM + hand GDN kernels). Re-run from a clean `git archive` tree of `main` `3f34534d`, build proven correct three ways (configure log has ZERO `CUTLASS not found` and prints `CUTLASS found ... sm120a NVFP4 cutlass GEMM` + `FlashAttention-2 ... ENABLED for arch(es) [121a]` + the vendored `sm_121a` Triton-AOT lines with `MANIFEST hashes OK`; `cuobjdump -lelf` 40 cubins ALL `sm_121a`, zero `sm_75`; SACRED `test_qwen27_paged_engine` **235/235 exit 0**, and the build precondition proven to FIRE by recompiling only that TU without the two defines against the same `libvllm.a`, which throws and exits 1 with 0 assertions). **The row PASSES UNCHANGED:** `tests/parity/test_qwen35_gguf_spec_decode` **3/3 cases, 10/10 assertions, exit 0**, spec-ON token-identical to spec-OFF, **13 proposed / 11 accepted (identical to the recorded number)**, 90.26 GiB, 7m13.59s; loader gate 19 assertions on the Qwen3.5-2B and 18 on the 35B A3B, unchanged. **ONE recorded finding is RETRACTED by the re-measurement: the CPU-vs-GPU token delta was a BUILD artifact, not a device near-tie cascade.** On the production build both devices emit the SAME 24 tokens; the probe shows GPU rank1 `11` -0.763897 over rank2 `13` -0.824083 where the defective build had rank1 `13` -0.773180 over rank2 `11` -0.847055, while the CPU arm is bit-identical to the earlier measurement (CUTLASS and Triton are CUDA-only). Zero exact ties in either arm, min margins 0.060186 GPU / 0.064875 CPU, 484/484 assertions per arm. Evidence: [docs/BENCHMARKS.md](../docs/BENCHMARKS.md) top section, [parity-ledger.md](parity-ledger.md) | [specs/gguf-mtp-spec-decode.md](specs/gguf-mtp-spec-decode.md) | `DONE` | `edf91449` | +| `SPEC-DFLASH-GGUF` | DFlash speculative decoding from GGUF, two axes: (A) GGUF DRAFT + safetensors target, (B) GGUF target too. llama.cpp master carries a full `dflash` GGUF contract (arch string `dflash`, tensors `fc`/`enc.output_norm`/`output_norm`/`blk.N.*`, KVs `dflash.target_layers` + `dflash.target_hidden_size`); the arch is ABSENT from checkouts older than ~2026-07, so a stale tree reads as "no contract exists". The GGUF tensor set omits `token_embd`/`output` because the draft SHARES the target's embed+lm_head, which is exactly what `LoadDflashDraft` already does. Blockers are in the loader, not the model: `MakeDflashDraftConfig` reads `draft_dir/config.json` (a GGUF has none), `ResolveDflashDraftDir` probes for `config.json` so it cannot see a `.gguf`, and `LoadDflashDraft` is typed on `std::vector` for the shared bf16 head (the axis-B blocker). Axis A independently shippable. NO ABI change | T2 | llama.cpp `origin/master` @ 2026-07-28 (tag era `b10158`): `gguf-py/gguf/constants.py:547,1151,4350`; `gguf-py/gguf/tensor_mapping.py:1297-1305` (`ENC_OUTPUT_NORM`<-`model.hidden_norm`, `FC`<-`model.fc`); `conversion/qwen.py:351` (mask token via the standard tokenizer KV); `convert_hf_to_gguf.py --target-model-dir` | **GD1-GD7 LANDED 2026-07-28 (BOTH AXES COMPLETE and PROVEN end to end on GB10)**: `MakeDflashGgufConfig` + `LoadQwen3DFlashFromGguf` `src/vllm/model_executor/models/qwen3_dflash_gguf.cpp:88,227` (+ header), `IsDflashGgufDraft` + the `.gguf` branch in `ResolveDflashDraftDir`/`LoadDflashDraft` `src/vllm/entrypoints/model_loader.cpp:121,222`. Goes through the `TensorResolver` seam (unlike `SPEC-MTP-GGUF`) because dflash norms are RAW, so the existing `LoadQwen3DFlash` qkv/gate_up concatenation is reused unchanged. **`GD4` defect FIXED** (`model_loader.cpp:238-249`): the GGUF branch left `config.vocab_size` 0 - correct for `MakeDflashGgufConfig` (the DFLASH arch has no vocab KV and no `token_embd`) but fatal for the forward, which sizes the shared embedding view as `{config.vocab_size, H}`, so the first propose threw `cuda embedding: empty table (vocab 0)`. Now back-filled from the target's `embed_tokens` rows (the condition is on the VALUE, not the draft source, so it generalizes to a GGUF target). Load-level green had hidden it; only GENERATING found it. **GD5-GD7 = axis B**: `SharedHeadSource` `src/vllm/entrypoints/model_loader.cpp` re-expresses the shared bf16 `embed_tokens`+`lm_head` seam as a SOURCE and re-types `LoadDflashDraft`'s second parameter - THAT TYPE was the whole axis-B blocker - with the GGUF arm `LoadGgufSharedEmbedAndHeadBf16` `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp:1040` reusing the trunk loader's tied-embedding rule and sidecar-aware dequant instead of restating them; the shared-head load moved into ONE common tail so all four (draft format x target container) combinations run identical code; the `dflash` half of the GGUF-branch rejection `model_loader.cpp` is deleted (the `mtp` half untouched) and the draft load is wired into the GGUF branch | `tests/vllm/models/test_qwen3_dflash_gguf.cpp:36,84` 2 cases / 47 assertions against the REAL published Qwen3.6-27B DFlash draft (env-gated `VLLM_DFLASH_GGUF_MODEL`, CI asset-free): the +1 target-layer offset undone against the KV read back from the same file, block_size/mask_token present, vocab_size left 0, layer_types cover every block, fc `[H, H*num_taps]` with `nk` SET, qkv/gate_up row-concat shapes, embed/lm_head left EMPTY for the target. **RED-first BEHAVIOURAL** (dropping the `-1` fails the offset checks). **`GD4` e2e gate** `tests/parity/test_qwen27_dflash_spec_decode.cpp:343` (second case, draft source env-driven via `VLLM_DFLASH_DRAFT`/`_B`; asset-gated, CI-inert): on dgx GB10 sm_121a against the Qwen3.6-27B NVFP4 safetensors target, the Q4_K_M GGUF draft and the bf16 z-lab safetensors draft produce **token-for-token IDENTICAL** DFlash-ON continuations with **IDENTICAL** accepted/proposed (20/80 on a 24-token prompt, 42/96 on a 48-token prompt), spec-OFF self-reproducible 3/3 and 0 exact ties (min margin 0.197/0.400 nats). Regression: gguf_mtp 19, qwen35_gguf_spec_decode 10, gguf 103, gguf_qwen36_loader 99, gguf_keep_quant 5958, ops_gdn 1825, llm_engine 196, capi 232, runner 257 all unchanged. **`GD5` unit gate** `tests/vllm/test_gguf_qwen36_loader.cpp` 3 new synthetic-GGUF cases (6 cases / 286 assertions total, CPU and the dgx CUDA build): the untied head really comes from `output.weight` and not the embedding (distinct fill values), the tied fallback aliases it onto `token_embd`, the `nk` flags separate the gather table from the MatmulBT weight, a file with no `token_embd` is refused. 3-mutant battery, 3 caught (`nk` flipped, head forced to the embedding, tied forced false). **`GD7` e2e gate** `tests/parity/test_qwen27_dflash_spec_decode.cpp` third case (targets env-driven via `VLLM_DFLASH_TARGET_B`; asset-gated, CI-inert): on dgx GB10 sm_121a the Qwen3.6-27B NVFP4 **GGUF** target + `Q4_K_M` GGUF draft loads, takes the shared head from the GGUF, generates, and its DFlash-ON continuation is **token-for-token IDENTICAL to that same target's spec-OFF** (24/24, the STRICT form) with acceptance ALIVE at 14/160; 1 case / 15 assertions, exit 0. **The spike's highest risk is EMPTY on this asset, proven not assumed**: the 27B NVFP4 GGUF stores `token_embd`/`output` as ggml BF16, byte-identical to the safetensors sibling (2,542,796,800 bytes each, ZERO differing), so B1's shared-head read is verbatim, not a dequant. Acceptance IS lower than the safetensors-target arm and is NOT chargeable to the head: the two containers diverge at index 4 with NO speculation, because `QUANT-GGUF-NVFP4` is dequant-only so the GGUF target computes in bf16 while the safetensors target runs the true W4A4 kernels. **RE-MEASURED 2026-07-29 on a PRODUCTION-CONFIGURED build (`CLAIM-GGUF-SPEC-REVERIFY`), because every GD4/GD7 GPU number above came from a build configured WITHOUT `-DVLLM_CPP_CUTLASS_DIR` and WITHOUT `-DVLLM_CPP_TRITON=ON`.** Build proven correct three ways (see the `SPEC-MTP-GGUF` row; SACRED 27B **235/235**, `cuobjdump` 40 cubins all `sm_121a`). **AXIS B HOLDS EXACTLY**: `test_qwen27_dflash_spec_decode -tc="dflash axis-B*"` **15/15 assertions, exit 0**, GGUF-target DFlash-ON token-identical to that target's own spec-OFF 24/24, acceptance **14/160 unchanged**, cross-target spec-OFF divergence still at index 4, 81.01 GiB peak RSS, 6m53.08s. **AXIS A WAS RED ON THE 48-TOKEN PROMPT (reproducibly, 3 of 3 runs) AND IS NOW CLOSED.** The RED was real: cross-format TOKEN identity held on both prompts, but the exact accept-count half of bar (a) failed (`arm_a.proposed == arm_b.proposed` / `arm_a.accepted == arm_b.accepted`) because the Q4_K_M draft measured **46/112** against the bf16 z-lab draft's **47/96** (one extra 16-wide propose block, one fewer acceptance, zero token difference), 15/17, exit 1; the 24-token prompt stayed green at 17/17 with both drafts at 15/144. **`GD9` 2026-07-29 root-caused it IN WEIGHT SPACE as ordinary `Q4_K_M` cost, category (a), not a defect in our GGUF draft path - and the bar's own premise ("Same weights, two containers") was false for the asset it was pointed at.** The publishing repo also carries an UNQUANTIZED `BF16` GGUF (3,471,497,440 B) beside `Q8_0`/`Q6_K`/`Q5_K`/`Q4_K_M`, which the spec had recorded as nonexistent; that retired the `NOT APPLICABLE` on gate 2. CPU gate `tests/vllm/models/test_qwen3_dflash_gguf.cpp` third case (asset-gated `VLLM_DFLASH_GGUF_BF16_MODEL` + `VLLM_DFLASH_ST_DIR`): `LoadQwen3DFlashFromGguf(BF16)` is **BYTE-IDENTICAL to `LoadQwen3DFlash(z-lab shards)` on all 58 tensors, 302/302 assertions, exit 0**, and FUNCTIONALLY RED against the `Q4_K_M` file (21/302 red, exactly the 21 quantized matmul tensors), so not a vacuous pass. Supporting: our `DequantGgufRowToBf16` is bit-equal to `gguf-py`'s `gguf.quants.dequantize` on the real `fc.weight` (Q4_K), `blk.0.attn_q.weight` (Q4_K) and `blk.2.ffn_down.weight` (Q6_K), zero differing bf16 values; the ladder's mean relative weight error is monotone and uniform with NO outlier tensor (BF16 0, Q8_0 5.6e-3, Q6_K 1.85e-2, Q5_K 3.85e-2, Q4_K_M 7.6e-2); the only numeric config delta is `rms_norm_eps` at 2.5e-9 relative. Also landed: an off-by-default `VT_SPEC_TRACE=1` per-block propose/accept trace in `GPUModelRunner::sample_tokens_with_rejection` (`src/vllm/v1/worker/gpu/runner.cpp`). **`GD10` 2026-07-29 CONFIRMED IT END TO END ON GB10 and closed gates 3 and 5.** Build proven production-configured three ways (configure log 0 `CUTLASS not found`; `cuobjdump -lelf` 40 cubins ALL `sm_121a` zero `sm_75` on both binaries; SACRED `test_qwen27_paged_engine` **235/235, exit 0**, 31.34s, 23.67 GiB). The **`BF16` GGUF draft reads EXACTLY 47/96**, the safetensors draft's own number, at 48 tokens on the discriminating prompt - reproduced 2 of 2 - plus 27/64 = 27/64 at 24 tokens and 15/144 = 15/144 on the second prompt, tokens IDENTICAL throughout, 17/17 exit 0 each time; the `Q4_K_M` arm reads 46/112 on the SAME binary in the SAME `flock` series. Restoring only the draft's numeric precision restores the count, so quantization is the whole cause and nothing structural survives. Bar (a) is consequently SPLIT rather than relaxed (`tests/parity/test_qwen27_dflash_spec_decode.cpp`): tokens stay EXACT unconditionally; accept counts are EXACT on a cross-FORMAT arm and BANDED (`abs(d_accepted) <= 2`, `abs(d_proposed) <= k*2`) on a cross-QUANTIZATION one, with the arm chosen by `IsQuantizedGgufDraft` reading the draft file's ggml types (`GgmlTraits().block_elems > 1`) rather than by a flag. The band is derived, not picked: measured `d_accepted` is 0, 0, -1, so the bound is that maximum plus one quantum; and `d_proposed = -k * d_accepted` EXACTLY once the token streams match (confirmed at -1 / +16), so the proposed bound follows. **Mutation-proved non-vacuous**: rebuilt at band 0 the `Q4_K_M` arm is 15/17 exit 1 while the `BF16` arm stays 17/17 exit 0 on the exact branch. **AXIS B BROADENED from ONE prompt to THREE**, strict form green on all: "The capital of France is" IDENTICAL 14/160 (15/15), "Write a Python function that reverses a string:" IDENTICAL 24/64 (15/15), "Photosynthesis is the process by which" IDENTICAL 15/128 (9/9), all exit 0, ~6m30-6m52 and ~81 GiB peak RSS each. The second prompt REFINES the recorded acceptance claim: the safetensors-target arm is ALSO 24/64 there with the two containers' DFlash-ON streams IDENTICAL, so the GGUF target's lower acceptance is prompt-dependent (their spec-OFF streams diverge at index 4 on the first prompt, index 16 on the second) and not a standing penalty; the cause remains `QUANT-GGUF-NVFP4` being dequant-only, with the shared head excluded by a byte comparison. Gates 1-5 and 7 MET; gate 6 (speed) `PENDING` BY DESIGN and not owed - a DFlash-ON throughput A/B between the two target containers is not a fair comparison until a native NVFP4 GGUF GEMM exists. Evidence: [docs/BENCHMARKS.md](../docs/BENCHMARKS.md) top section, [parity-ledger.md](parity-ledger.md#L845) | [specs/gguf-dflash-draft.md](specs/gguf-dflash-draft.md) | `DONE` | `c62f2fa3` | | `SPEC-REJECTION` | Rejection sampler. **I3 verify half LANDED (2026-07-24)**: per-request logits EXPANSION to `1 + k_i` rows (`StepInputs::cu_num_logits` / `num_draft_tokens_per_req` / expanded `logits_indices`) plus the GREEDY rejection sampler — accept a draft iff it equals the target argmax at its own position, emit the target argmax on the FIRST mismatch and stop, emit the bonus argmax when all `k_i` accept, `num_sampled = accepted + 1`, `num_rejected = k_i - accepted` (feeds I2's `num_computed_tokens` rollback and `InputBatch::num_accepted_tokens`). One additive vt op (`kGreedyRejectionSample`) with a CPU reference and a CUDA two-phase mirror of upstream's row-argmax + one-thread-per-request accept walk. DEFAULT-OFF and INERT: with no `SpeculativeConfig` no drafts are ever scheduled, `cu_num_logits` is `arange(num_reqs+1)`, `logits_indices` is the pre-change array and the runner never enters the rejection branch. STOCHASTIC/Gumbel, block verification, `apply_sampling_params` over the expanded batch, and the spec grammar bitmask stay DEFERRED (M-mtp-3). **I5b DRAFTER PREFILL INPUT-PREP LANDED (2026-07-24, `CLAIM-SPEC-MTP-I5B`)**: the draft-token input splice this row's I3 note deferred to I5 — `vllm::v1::prepare_prefill_inputs` + its `SpecPrefillInputs` output struct shift each request's `input_ids` left one within its query span, splice the just-sampled next token (`num_sampled>0 ? last_sampled[idx_mapping[r]] : next_prefill_tokens[...]`) into the freed slot, `query_len -= num_rejected`, and emit last-token index / query_start_loc / seq_lens + CG padding (mirror `speculator.py:469-588`, k=1 early-exit :236-238). A HOST routine in a NEW spec_decode-tree TU (no new CUDA kernel; mirrors the DEVICE-NEUTRAL `prepare_inputs`/`combine_sampled_and_draft_tokens` family — the DGX runner leaf ports the loop to the Triton kernel at I5d), unit-gated `test_prepare_prefill_inputs` 7 cases / 27 assertions RED-first, DEFAULT-OFF INERT (nothing calls it until I5d), additive by construction. Row stays `ACTIVE` — the e2e greedy token gate (M-mtp-1) is owed before `DONE` | T1 | `vllm/v1/worker/gpu/spec_decode/rejection_sampler.py:43,101-160`; `rejection_sampler_utils.py:524,564-585,628,828-841,846-849,863-1125`; `vllm/v1/worker/gpu/model_runner.py:866-898,1065-1077`; `vllm/v1/worker/gpu/input_batch.py:303-397,408-453`; **I5b** `vllm/v1/worker/gpu/spec_decode/autoregressive/speculator.py:469-588,236-238` | `include/vllm/v1/spec_decode/rejection_sampler.h`; `src/vllm/v1/spec_decode/rejection_sampler.cpp`; `include/vt/ops.h` (`kGreedyRejectionSample`, `vt::GreedyRejectionSample`); `src/vt/cpu/cpu_sample.cpp` (CPU reference); `src/vt/cuda/cuda_sample.cu` (`RejectionRowArgmaxKernel` + `GreedyRejectAcceptKernel`); `src/vt/ops.cpp`; `include/vllm/v1/worker/gpu/prepare_inputs.h` + `src/vllm/v1/worker/gpu/prepare_inputs.cpp` (the expansion); `include/vllm/v1/worker/gpu/runner.h` + `src/vllm/v1/worker/gpu/runner.cpp` (`step_num_logits`, `sample_tokens_with_rejection`); **I5b** `include/vllm/v1/worker/gpu/spec_decode/autoregressive/prepare_prefill_inputs.h` + `src/vllm/v1/worker/gpu/spec_decode/autoregressive/prepare_prefill_inputs.cpp` — anchor `include/vllm/v1/spec_decode/rejection_sampler.h:96` | `tests/vllm/v1/spec_decode/test_rejection_sampler.cpp`; `tests/vllm/v1/worker/test_prepare_inputs.cpp` (expansion + no-draft byte-identity); `tests/vt/test_cuda_ops.cpp` (CUDA==CPU bit-exact at vocab 248320); **I5b** `tests/vllm/v1/spec_decode/test_prepare_prefill_inputs.cpp` (7 cases / 27 assertions, RED-first) — anchor `tests/vllm/v1/spec_decode/test_rejection_sampler.cpp:128` | [mtp-spec-decode.md §2.4,§5](specs/mtp-spec-decode.md) | `ACTIVE` | `CLAIM-SPEC-REJECTION-I3`, `CLAIM-SPEC-MTP-I5B` | | `SPEC-GDN-SEGMENTS` | GDN speculative metadata and slot-snapshot rollback. **I4 LANDED (2026-07-24):** the spec/non-spec metadata split with decode→prefill reclassification (the #34845 case), the `T>1`/`IS_SPEC` GDN recurrence with per-timestep state snapshots, the conv sliding window advancing by the ACCEPTED count, and the k+1 state-slot allocation. DEFAULT-OFF and INERT (`num_spec==0` ⇒ `num_spec_decodes==0`, no shipped kernel branched — both spec kernels are NEW op ids). ROLLBACK PROVEN bit-exact: for every rejection point j the surviving SSM state and conv window are memcmp-identical to running only the accepted prefix through the shipped `vt::GdnDecode`/`CausalConv1dUpdate`, at the real 27B (Hv=48) and 35B (Hv=32) GDN dims on CPU and CUDA. MEASURED state cost: one f32 SSM slot = Hv·Dv·Dk·4B ⇒ 144 MiB/req (27B, 48 layers) / 60 MiB/req (35B, 30 layers) per extra slot; k=1 doubles the GDN SSM state. **I5a GDN LAYER ROUTING WIRED (2026-07-24, `CLAIM-SPEC-MTP-I5A`):** `GdnBlockPaged`'s `num_spec_decodes>0` branch now routes a PURE-spec batch through `vt::CausalConv1dSpecUpdate` + `vt::GdnSpecDecode` (mirror `qwen_gdn_linear_attn.py:1344-1357,1455-1475`), and the runner per-step upload (`StepDevInputs`/`BuildStepDevInputs` + the two decode-graph `Refresh` copies) now carries I4's six spec device tensors, gated by the extended `ValidateGdnAttentionMetadata` spec contract. DEFAULT-OFF INERT (`num_spec_decodes==0` ⇒ stub uploads + the identical non-spec branch). BIT-EXACT vs the I4 ops applied as a token-sequential decode chain, at the real 27B/35B GDN dims, via `GdnBlockPagedForTest` (`tests/vllm/models/test_qwen3_5_gdn_spec_routing.cpp`, CPU bit-exact + CUDA on-device); RED-first by a reverted stub (spec recurrence zeroed ⇒ 4/8 fail, maxΔ 1.3-1.6). MIXED spec+non-spec batch refused loudly — lands with I5d's runner loop. Row advances to `ACTIVE`: the M-mtp-1 e2e greedy token gate (verify/propose runner wiring) is owed before `DONE`, and `SPEC-MTP` STAYS `GATING` | T1 | `vllm/v1/attention/backends/gdn_attn.py:189-326,413-462`; `fla/ops/fused_sigmoid_gating.py:66-72,103-116,156-166`; `mamba/ops/causal_conv1d.py:818-1067,1181-1184`; `qwen_gdn_linear_attn.py:1329-1576`; `mamba_utils.py:213-234`; `mamba/abstract.py:55-59` | `include/vllm/v1/attention/backends/gdn_attn.h`; `src/vllm/v1/attention/backends/gdn_attn.cpp`; `include/vt/ops.h` (`kGdnSpecDecode`, `kCausalConv1dSpecUpdate`); `src/vt/ops.cpp`; `src/vt/cpu/cpu_ops.cpp`; `src/vt/cuda/cuda_gdn.cu`; `src/vllm/model_executor/models/qwen3_5_common.{h,cpp}` (`MakeQwen3_5KVCacheSpec`); **I5a:** `src/vllm/model_executor/models/qwen3_5.cpp` (`GdnBlockPaged` spec branch, `StepDevInputs`/`BuildStepDevInputs`, `ValidateGdnAttentionMetadata`), `src/vllm/model_executor/models/qwen3_5_internal.h` (`GdnBlockPagedForTest`) | `tests/vllm/v1/attention/test_gdn_metadata_builder.cpp` (20 cases / 483 assertions incl. the full upstream `GDN_BUILD_TEST_CASES` + default-off byte-identity); `tests/vt/test_ops_gdn.cpp` (reject-at-every-j rollback, CPU + CUDA, real dims); `tests/vllm/models/test_model_registry.cpp` (k+1 slot / widened-conv sizing + `num_spec==0` identity); **I5a** `tests/vllm/models/test_qwen3_5_gdn_spec_routing.cpp` (spec-routing bit-exact, RED-first) — anchor `tests/vllm/v1/attention/test_gdn_metadata_builder.cpp:83` | [mtp-spec-decode.md §3,§5](specs/mtp-spec-decode.md) | `ACTIVE` | `CLAIM-SPEC-GDN-I4`, `CLAIM-SPEC-MTP-I5A` | | `SPEC-DFLASH` | Block-diffusion drafter. **READINESS RE-ASSESSED 2026-07-25 (`CLAIM-SPEC-DFLASH-READINESS`, design-only, DONE) against the LANDED MTP machinery (`SPEC-MTP` I1..I7).** Verdict **GREEN, dispatch-ready, NO hardware/oracle/download blocker** (spec [§0](specs/dflash-spec-decode.md)). Refreshed reuse-vs-new map: DFlash gets FREE from landed MTP — the frozen spec-metadata ABI, the greedy rejection sampler (k-general, I3 tested k∈{1,3}), the GDN spec slot path + rollback + mixed spec/non-spec batch (`GdnBlockPagedMixedSpec`/`IndexSelect`/`IndexCopy`, general `num_spec`), the widened-cache-aware conv ops (I5e), the draft-KV layer pattern (`fa_draft`), the I5d/I7 runner verify/propose loop, and **`num_lookahead_tokens=k+1` ALREADY coded** (`speculative.h:91-108` `use_dflash()`); EXTENDS the single I5d-pre `hidden_tap` seam to multi-tap `[T,H×taps]`; builds NEW the `qwen3_dflash` drafter, the project's FIRST non-causal in-block attention primitive, context-KV precompute, `prepare_dflash_inputs`, and the uniform-1+k FULL CG. **k>1 verdict:** the landed rejection + GDN machinery is MECHANICALLY k-general (no `k==1` hardwiring) — DFlash's k=15 blocks need NO mechanism extension, only exercise/validation at scale (D4) + the k+1-slot memory measurement (~2.3 GiB/req 27B GDN state at block-16, the #1 risk, §5). **Checkpoint-fit:** both z-lab drafts EXIST on HF (27B 1.73 GB / 35B 368 MB bf16, DFlashDraftModel) and FIT the 119 GiB pool trivially (drafts NOT yet on dgx — D0 downloads ≤1.73 GB); the active dgx oracle `vllm-oracle-v0.25.0-stage` CONSTRUCTS DFlash (registry `DFlashDraftModel→qwen3_dflash`, speculator dir present) — soft D0 risk = confirm it SERVES DFlash+NVFP4 on sm_121 (non-causal backend; community `AEON-7/vllm-dflash` container proves the combination runs on GB10). W-plan D0-D6 in the spec. **D0+D1 LANDED 2026-07-26 (`CLAIM-DFLASH-D0D1`) on the ADVANCED pin `555967922`/vLLM 0.26.0.dev0 — `SPEC-DFLASH` → `ACTIVE`.** D0 UNBLOCKED (vllm#40898 resolved under `VLLM_USE_V2_MODEL_RUNNER=1`): the mixed-attn z-lab 27B draft CONSTRUCTS + the drafter is ALIVE (acceptance 2.21/8.80/4.75/4.57 > 1, `num_spec=16`, flashinfer-native fp8-KV, goldens committed); gate FORM measured STRICT MODE-MATCHED (vLLM-ON run-deterministic K>=3 but != vLLM-OFF — the k=16 block verify diverges at bf16 near-ties, so NOT the MTP three-way identity). D1 `DF-AUX-TAPS` DONE: `Qwen3_5AuxTaps` + `ModelForwardInput::aux_tap` route to `Qwen3_5{,Dense}Model::ForwardDeviceMultiTap` capturing `(hidden+res)` at `target_layer_ids` into `[T,H×taps]` (eagle3 `_maybe_add_hidden_state`, aux key L+1); config-gated byte-identical off. Unit gate 598 assertions (independent truncated-model reference, RED-first reversed-concat 384 fail); CUDA 697/697 + compute-sanitizer 0; INERTNESS PROVEN — 27B MTP e2e 9/9 + 27B text SACRED 235/235 byte-identical on the new oracle. **D2 `DF-DRAFT-MODEL` CODE LANDED + CPU-GATED 2026-07-26 (`CLAIM-DFLASH-D2`, kernel row `KERNEL-ATTN-DFLASH-BLOCK`):** the `qwen3_dflash` draft model (plain 5-layer Qwen3-dense reusing `dense_attn_block.h` ops), the project's FIRST non-causal / bidirectional attention primitive `vt::DFlashBlockAttention` (a SEPARATE op — causal `kAttention`/`kPagedAttention` byte-identical), the fc aux-combine, mask-embed, per-layer SWA/full resolution, and the z-lab loader. CPU gate GREEN (op 12/12 incl. RED non-causal; model forward 95/95 incl. RED full-layer-causal-flip + block isolation + fc RED); existing causal `test_ops_attention` 9/9 + `test_qwen3_forward` 1028 UNCHANGED. **D2 GPU PROMOTION GREEN on dgx (`CLAIM-DFLASH-D2`):** CUDA `-Werror` clean, CUDA==CPU 198412/198412 + compute-sanitizer 0, draft-forward parity vs the REAL vLLM draft (fc rel-L2 0.46%, hidden ≤1.3%, 11 STRICT + 5 near-tie ids), 27B SACRED 235/235 + MTP 9/9 byte-identical — **D2 DONE.** **D3 `DF-DRAFT-KV-PREP` DONE 2026-07-26 (`CLAIM-DFLASH-D3`):** `PrecomputeContextKV` + `PrepareDflashInputs` + `ForwardBlockLogitsWithContext` (reuse the UNCHANGED D2 kernel via [context;block]); GPU numeric-parity `test_qwen3_dflash_kvprep_parity` 61/61 (prepare INTEGER bit-exact vs vLLM's Triton kernel, context-KV K/V rel-L2 0.31%/0.26%, 13 STRICT + 3 near-tie = 16/16), CPU 114/114 RED-proven, inertness 235/235 + 9/9 + D2 37/37 byte-identical. **D4 `DF-ENGINE-INTEGRATION` propose brick + `dflash` config-select CODE LANDED + CPU-GATED 2026-07-26 (`CLAIM-DFLASH-D4D5`):** `DflashProposeBlock`/`SampleDflashBlockDrafts` (the non-autoregressive whole-block propose composing D3 `ForwardBlockLogitsWithContext` + greedy per-mask argmax, anchor not sampled, `dflash/speculator.py:300-413`) + `ParseSpeculativeConfigJson`/`ResolveDflash` accept `method:"dflash"`. CPU gate `test_dflash_propose` 5/19 GREEN (RED-first anchor-read fails 4/5; brick composes forward+sampler; empty-ctx degenerates to D2; config lookahead k+1). Additive + config-gated ⇒ MTP + non-spec byte-identical BY CONSTRUCTION (`git diff --stat` = new speculator TU + config accept-list + CMake + test, NO runner/model/loader/scheduler edit). **D5 `DF-ENGINE-INTEGRATION` runner-loop LANDED + e2e RUNS on dgx 2026-07-26 (`CLAIM-DFLASH-D5`):** full verify/propose loop wired — loader loads the SEPARATE z-lab draft (`LoadDflashDraft`, host bf16 + target-SHARED bf16 embed/lm_head) via a `--speculative-config` `model` key + `ResolveSpecConfig` dflash branch + `runner.set_dflash_draft`; the verify forward captures the D1 multi-tap (`aux_tap`→`ForwardDeviceMultiTap`) instead of the MTP single tap; `propose_drafts_dflash` ACCUMULATES the per-request combined-feature context (`CombineAuxFeatures(aux_tap)`) across steps and honors the `num_rejected` rollback by appending only the `(T_req−num_rejected)` accepted-prefix features, then runs `DflashProposeBlock` (k=16 GDN-spec exercised first time). **e2e (`test_qwen27_dflash_spec_decode`, 4 prompts×32 tok, our-DFlash-ON vs the committed vLLM-DFlash-ON golden): 2/4 STRICT token-exact (fibonacci, three-laws) + acceptance ~ vLLM on ALL 4 (accepted 19/39/29/25 vs golden 17/39/30/25, deltas +2/0/−1/0 — the MANDATORY dead-drafter-trap condition MET).** The 2 divergences (France tok11 `2972`↔`11751`, 17*23 tok12 `567`↔`488`) are SINGLE bf16 near-tie flips (17*23 RE-CONVERGES after one token = proven near-tie; France cascades from one flip) — the ratified near-tie ROOT the D0 gate-form anticipated, rooted in the D3-documented inline bf16 context-KV recompute envelope (~0.3-1.3% rel-L2), NOT a wiring bug (proven by the 2 exact prompts + near-exact acceptance + a non-trivial shared prefix). Inertness GREEN on this build: SACRED `test_qwen27_paged_engine` 235/235 + MTP `test_qwen27_spec_decode` 9/9 byte-identical; CUDA `-Werror` clean; NO new CUDA kernel (host orchestration reusing D1/D2/D3-sanitized ops). **NOT a clean strict-4/4 pass; STRICT 4/4 token-identity + the speed A/B = D6 (the persistent paged draft-KV bit-matching vLLM's fused context-KV projections + the uniform-1+k FULL CG).** Row STAYS `ACTIVE` (correctness at the ratified near-tie envelope; D6 remains) **D6 2026-07-27 (`CLAIM-DFLASH-D6`) — c1 SPEED A/B DONE + STRICT-irreducibility RCA + CG feasibility (records-only, NO source code):** (1) **c1 speed A/B** (`examples/vllm-bench` at `361189a7`, 8 prose+code prompts×256 tok greedy c1, 2 reps): our DFlash-ON = **2.50x TPOT (40.4 vs 101.2 ms) / 2.48x output-tput (24.4 vs 9.86 tok/s)** over our OFF, acceptance 0.22 (3.56/16), rep-stable <1.5%; `benchmark_binding=true`. vs vLLM-DFlash-ON graphed (same workload): vLLM-DFlash-ON graphed = 28.5 tok/s / 35.1 ms TPOT / acceptance_len 4.30 (same 8 prompts, `VLLM_USE_V2_MODEL_RUNNER=1`, mm-off, gpu_util 0.30), so OURS IS ~14% BELOW vLLM-DFlash-ON on output throughput (24.4 vs 28.5 tok/s) - both ~on-par at spec-OFF (9.86 vs 9.83 tok/s), but vLLM extracts a larger DFlash speedup (2.90x vs our 2.47x) because its draft step is fully device-resident + CUDA-graphed (ours host-orchestrates 13 downloads/step) + slightly higher acceptance (~4.3 vs ~3.6 draft tokens/step). The DONE speed bar (ours >= vLLM) is NOT met; closing it = the device-resident draft rewrite + FULL CG (D6 part 2). (2) **STRICT-4/4 proven bf16-IRREDUCIBLE** — the draft KV cache is bf16 not fp8 (`torch_utils.py:398` `auto`→model dtype; the D0 "fp8-KV" was the backend name, not the KV storage dtype), the D3 golden already compares pre-storage bf16 (residual K 0.31%/V 0.26% = sub-ULP kernel noise), and a fused multi-layer KV GEMM is per-element invariant to our per-layer GEMMs ⇒ bit-exact needs vLLM's exact kernels ⇒ the ratified near-tie gate is the FINAL correctness form (no fused-KV code landed). (3) **FULL CG BLOCKED** on a device-resident draft-path rewrite (the D5 path does 13 device→host downloads/step + host `[context;block]` interleaving) — the remaining throughput-parity increment (the perf form of persistent-paged-KV + the graph). Inertness by construction (the gated binary is the D5 binary; SACRED 235/235 + MTP 9/9 stand). Evidence tool `scripts/spec/vllm_dflash_timing.py`. **D7 2026-07-27 (`CLAIM-DFLASH-D7`) — within-step draft forward made DEVICE-RESIDENT (source-owning): `PrecomputeContextKVDevice` keeps per-layer K/V on device; `ForwardBlockLogitsWithContext` builds [context;block] with `vt::IndexCopy`/`IndexSelect` (removes ~30 D→H `Download`s/step). BIT-IDENTICAL (identity bf16↔f32 round-trips replaced) — e2e `test_qwen27_dflash_spec_decode` 27/27 SAME tokens (2/4 STRICT + 2/4 near-tie, acceptance 19/39/29/25), SACRED 235/235 + MTP 9/9, CUDA `-Werror` clean, compute-sanitizer 0 (198412). But the direct old-vs-new A/B = +2.0% output-tput (IN-NOISE) ⇒ D6's "downloads = the ~14% gap" REFUTED by measurement; ours 19.68 tok/s STILL ~33% BELOW vLLM-DFlash-ON 29.2 tok/s (reconstructed 8-prompt set, more prose-heavy); OFF parity our 9.97 ≥ vLLM 9.66. Residual re-attributed: acceptance (ours 2.49 vs vLLM ~3.13 accepted draft-tok/step, bf16-irreducible) + per-step context-KV RECOMPUTE (O(context²), needs the cross-step persistent paged draft-KV store) + eager-vs-graphed. SPEED BAR NOT met; SPEC-DFLASH stays `ACTIVE`; next = persistent paged draft-KV store → then FULL CG. **D9 2026-07-27 (`CLAIM-DFLASH-D9`) — PERSISTENT PAGED DRAFT-KV LANDED (bit-identical, +22.7% throughput, 0.69×→0.917×); D8 acceptance-ceiling REFUTED; residual = FULL CG ONLY:** `qwen3_dflash.cpp` `AppendContextKVHost` (project ONLY newly-accepted rows → per-layer bf16 K/V, append to `PrecomputedContextKV`) + `ForwardBlockLogitsWithPrecomputedKV` (upload the persistent store, NO re-projection) share the core `ForwardWithCtxKVDev` with the old recompute; `runner.cpp::propose_drafts_dflash` swaps the O(context²) per-step recompute (`dflash_ctx_feats_`) for an append-only per-request `dflash_kv_store_` (rollback=don't-append). NO new CUDA kernel; config-gated. BIT-IDENTICAL: CPU `test_dflash_propose` two new D9 cases = exact float equality vs full recompute; GPU e2e `test_qwen27_dflash_spec_decode` **27/27 SAME tokens** (acceptance 19/39/29/25, same divergences France@11/17×23@12); SACRED 235/235 + MTP 9/9 byte-identical; CUDA `-Werror` clean. **A/B (c1, 8 prose+code×256 tok input-len 512, 2 reps <0.1%, `benchmark_binding=true`):** ours-ON **25.75 tok/s** (was D8 20.99, +22.7%) / 38.40 ms TPOT / acc **3.68/step** vs vLLM-ON graphed **28.09** / 35.60 / acc 3.31 = **0.917×** (~8% below, was 0.69×). **Part 1 same-trajectory:** on the 2 token-identical-trajectory prompts ours per-step acceptance == vLLM's EXACTLY (fibonacci 7.80/7.80, three-laws 3.571/3.571, ratio 1.00) AND on the A/B ours acceptance (3.68) is HIGHER than vLLM's (3.31) ⇒ D8's 0.80–0.85× "bf16 acceptance ceiling" is a trajectory-divergence CONFOUND, REFUTED. Residual (~8%) = eager-vs-graphed ONLY (ours ON/OFF 2.60× vs vLLM 2.91×, OFF at parity, recompute eliminated, acceptance higher) — NOT an irreducible ceiling; the FULL uniform-(1+k) CG (device paged-KV store + paged attn, new-CUDA multi-file) is the SOLE un-landed increment. SPEC-DFLASH stays `ACTIVE` (speed not yet ≥ vLLM; residual isolated to FULL CG). **D12 2026-07-27 (`CLAIM-DFLASH-D12`) — A-wire + Part B LANDED + GPU-gated; Part C (capture) remaining; 0.917×:** A-wire makes the D11 Part-A device store the PRODUCTION path (`runner.{h,cpp}` `dflash_kv_store_`→`shared_ptr`, `MakeDeviceKVStore`/`AppendContextKVDevice`/`ForwardBlockLogitsWithDeviceKV`; GPU-gated e2e `test_qwen27_dflash_spec_decode` 27/27 all-exact acceptance 19/39/29/25 + SACRED 235/235 + MTP 9/9 byte-identical, `-Werror` clean). Part B adds `vt::DFlashPagedBlockAttention` (`OpId::kDFlashPagedBlockAttention`), the capture-safe paged kernel with EVERY metadata input a persistent DEVICE tensor and NO function-local host `cu_seqlens` upload (fixes the `cuda_ops.cu:1277-1280` capture-UAF class), gated CPU==CUDA + cross-check vs materialized `DFlashBlockAttention` `test_ops_dflash_paged_block_attn` 795648/795648 + compute-sanitizer 0. Speed 0.917× (A-wire eager + Part B not yet wired into the forward); `benchmark_binding=false`. Part C (static-shape capture + device mask-scatter + `BeginCapture`/replay + the ≥vLLM c1 A/B) is the SOLE remaining piece; if ours-ON-graphed ≥ vLLM-ON → SPEC-DFLASH DONE. Stays `ACTIVE`. **D13 2026-07-27 (`CLAIM-DFLASH-D13`) — Part C LANDED + GPU-GATED; capture-correctness PROVEN; c1 throughput NEAR-PARITY (ours 0.978x, ~2% below vLLM); gap CLOSED 0.917x→0.978x; STAYS `ACTIVE` (≥vLLM bar not yet met):** single-file additive change (`qwen3_dflash.cpp` +368/-58). (C.1) `DflashDeviceKVStore` → fixed-capacity PAGED cache (per-layer pool `[max_pages,16,Hkv,Dh]` + identity `block_table` + `seq_lens`; append = `vt::IndexCopy` scatter at slot==abs-pos, bit-identical to the D9/D11 store). (C.2) `ForwardPagedBody` runs the (1+k) block through the D12 `vt::DFlashPagedBlockAttention` reading the paged store (no `[context;block]` materialization, no function-local host uploads); runner P==1 propose routes through it, P>1 bit-identical materialized fallback. (C.3) per-request CUDA GRAPH over the paged draft step (warm-in-step repopulates the shared pool free-list right before `BeginCapture` — the fix for a `cudaMalloc`-in-capture `Get` miss from the intervening 27B target forward — then `BeginCapture → ForwardPagedBody → EndCaptureGraph`, replay with growing context entering only via in-place `seq_lens`). **Capture-correctness (MANDATORY): `test_qwen27_dflash_spec_decode` 27/27 with the graph (VT_DFLASH_GRAPH=1) BIT-IDENTICAL to eager (=0)** — same divergence tokens (France@11 got[…2972…], 17×23@12 got[…567…]), same acceptance 19/39/29/25 as D5/D7/D9/D12; graph ENGAGED (5 captures C=2048/5/4/15/6, 32+ replays); the token-diff is the capture-safety proof ([[cudagraph-capture-bakes-stack-addresses]]). **c1 A/B (one flock series, cold rep discarded, 8 prompts×256 tok):** our OFF 10.24 / our ON eager-paged 28.65 (28.69,28.61) / **our ON GRAPHED 28.70 (28.70,28.70), TPOT 34.40** / vLLM-ON graphed steady-state 29.35 (tight 3-rep 29.33/29.37/29.33, TPOT 34.07, acc_len 4.44); D9's 28.09 was a colder cross-session outlier — **NEAR-PARITY: ours 0.978× (~2% below) on the rigorous same-session band** (across sessions ours 28.70 falls inside vLLM's observed 28.09–29.37 range). ON/OFF 2.80× (vLLM ~2.98×), our OFF ≥ vLLM OFF. Per the acceptance rule ("below on any axis = an open gap; near-parity is NOT met"), the ≥vLLM bar is NOT met; STAYS `ACTIVE`. Residual (data-grounded): NOT acceptance (ours realized ~3.68 accepted draft-tok/step > vLLM's 3.44) and NOT launch/graph (both graphed, CG neutral) — per-step COMPUTE (~2% slower target-step); next lever = nsys both draft steps (`--cuda-graph-trace=node`), no premature ceiling. **ATTRIBUTION (supersedes D9):** the CUDA graph is perf-NEUTRAL (+0.3%); the ACTUAL lever was the paged context read (C.1/C.2) removing the D9/D12 per-layer `[context;block]` `IndexCopy` materialization of the whole growing context (25.75 D9 → 28.65 eager-paged, +11%) — the roadmap's "the full CG closes the gap" premise is corrected by measurement. Inertness VERIFIED on the capture binary: SACRED 235/235 + MTP 9/9 byte-identical, CUDA `-Werror` clean, no new kernel (D12 paged kernel already memcheck-0 795648), `check-device-leakage` not increased (paged path REMOVES the materialized-buffer allocs + host uploads). `benchmark_binding=true`. Correctness-complete (ratified near-tie); throughput NEAR-PARITY (0.978×, ~2% residual) ⇒ STAYS `ACTIVE` (the capture-correctness gate is MET; the ≥vLLM speed bar is the sole remaining item, a ~2% per-step-compute residual for an nsys). Anchors: `src/vllm/model_executor/models/qwen3_dflash.cpp` (`DflashDeviceKVStore` paged store, `ForwardPagedBody`, the per-request graph in `ForwardBlockLogitsWithDeviceKV`). **D14 2026-07-27 (`CLAIM-DFLASH-D14`) — SPEED GATE MET → SPEC-DFLASH `DONE`:** an nsys (`--cuda-graph-trace=node`) of the graphed spec-on step attributed the D13 ~2% residual to the from-scratch `DFlashPagedBlockAttentionKernel` draft attention (242.9 ms = 1.8% of GPU time, median ~460 us/call over context C~500-640, vs vLLM's fused flash draft-attn ~0.15%; BOTH engines run identical `cutlass_80_wmma` for the draft bf16 GEMMs, so the GEMMs were NOT the gap). Ported it to a WARP-scoped online-softmax variant `DFlashPagedBlockAttentionWarpKernel` (mirrors the shipped `AttentionWarpKernel`: one warp per (block-query,head), `__shfl_xor` butterfly reduction, register accumulator, NO `__syncthreads` storm; SAME paged/block combined-index read + causal/SWA mask + GQA; default ON, `VT_DFLASH_ATTN_BLOCK=1` keeps the bit-identical D12/D13 block kernel for A/B). Draft attn 242.9 → 77.9 ms (3.1x); our-ON c1 28.60 → 29.32 tok/s (+2.5%). **FINAL same-session 3-rep A/B (8 prompts×256 tok, cold leg discarded): our-ON graphed 29.42/29.27/29.32 (med 29.32) vs vLLM-ON graphed 29.240/29.247/29.233 (med 29.240) — our WORST rep (29.27) > vLLM's BEST (29.247), NON-OVERLAPPING bands, 1.003× ⇒ the ≥vLLM speed gate is MET.** Correctness UNCHANGED (output is exact by spec-decode construction — the target verify is untouched, only which draft proposals are accepted can shift): e2e `test_qwen27_dflash_spec_decode` 27/27 with graph==eager BIT-IDENTICAL, acceptance 19/39/29/25 unchanged (draft accepted 1629 identical warp-vs-block across the whole A/B set), 2/4 STRICT (France@11, 17×23@12 unchanged); CUDA==CPU `test_ops_dflash_paged_block_attn` 795648/795648 (warp within the f32 1e-4 / bf16 3e-2 envelope) + compute-sanitizer 0. Inertness SACRED 235/235 + MTP 9/9 byte-identical; CUDA `-Werror` clean; `check-device-leakage` not increased. `benchmark_binding=true`. Block-diffusion drafting is now correctness-complete (ratified near-tie) AND at/above vLLM throughput — this was the roadmap's FINAL open speed item. Anchors: `src/vt/cuda/cuda_ops.cu` (`DFlashPagedBlockAttentionWarpKernel` + `UseDflashAttnBlockKernel`; the D12 block kernel retained as the `VT_DFLASH_ATTN_BLOCK=1` reference). | T1 | `vllm/v1/worker/gpu/spec_decode/dflash/speculator.py`; `vllm/model_executor/models/qwen3_dflash.py`; `vllm/model_executor/models/interfaces.py:1382` (aux value); `eagle3_utils.py:41-56` (+1 shift) | `include/vllm/model_executor/models/qwen3_5.h` (`Qwen3_5AuxTaps`, `ForwardDeviceMultiTap`); `qwen3_5_dense.h`; `model_registry.h` (`aux_tap`); `src/vllm/model_executor/models/qwen3_5.cpp` (`MaybeCaptureAuxTap`/`ValidateAuxTapLayerIds`/`ForwardDeviceMultiTap`); `qwen3_5_moe.cpp`+`qwen3_5_dense.cpp` (routing); D2/D3 `include/vllm/model_executor/models/qwen3_dflash.h` + `src/vllm/model_executor/models/qwen3_dflash{,_weights}.cpp`; D4 `include/vllm/v1/worker/gpu/spec_decode/dflash/speculator.h` + `src/vllm/v1/worker/gpu/spec_decode/dflash/speculator.cpp` (`DflashProposeBlock`/`SampleDflashBlockDrafts`); D5 `src/vllm/entrypoints/model_loader.cpp` (`LoadDflashDraft`/`DflashDraft`) + `include/vllm/entrypoints/model_loader.h`; D5 `src/vllm/v1/worker/gpu/runner.cpp` (`set_dflash_draft`/`propose_drafts_dflash`/aux-tap capture) + `include/vllm/v1/worker/gpu/runner.h`; `src/vllm/config/speculative.cpp` + `include/vllm/config/speculative.h` (`ResolveDflash` + `dflash`/`model` parse); D14 warp kernel [cuda_ops.cu](../src/vt/cuda/cuda_ops.cu#L1433) | `tests/vllm/models/test_qwen27_paged_forward.cpp` (multi-tap 598); `tests/vt/test_ops_dflash_block_attn.cpp`; `tests/vllm/models/test_qwen3_dflash_forward.cpp`; `tests/vllm/v1/spec_decode/test_dflash_kvprep.cpp`; `tests/parity/test_qwen3_dflash_{draft,kvprep}_parity.cpp`; D4 `tests/vllm/v1/spec_decode/test_dflash_propose.cpp` (5/19, RED-first); D5 `tests/parity/test_qwen27_dflash_spec_decode.cpp` (e2e 27/27, 2/4 strict + acceptance~vLLM); `scripts/spec/d{0,2,3}_dflash_*.py`; `tests/parity/goldens/dflash_27b{,_draft,_kvprep}/`; D6 `scripts/spec/vllm_dflash_timing.py` (vLLM-DFlash c1 timing); D7 device-resident `src/vllm/model_executor/models/qwen3_dflash.cpp` (`PrecomputeContextKVDevice` + `ForwardBlockLogitsWithContext` via `vt::IndexCopy`/`IndexSelect`); D9 persistent paged draft-KV `qwen3_dflash.{h,cpp}` (`AppendContextKVHost`/`ForwardBlockLogitsWithPrecomputedKV`/`ForwardWithCtxKVDev`/`PrecomputedContextKV`) + `runner.{h,cpp}` (`dflash_kv_store_`/`propose_drafts_dflash`) + `tests/vllm/v1/spec_decode/test_dflash_propose.cpp` (2 D9 bit-identity cases); D12 A-wire `runner.{h,cpp}` (device store as production path) + D12 Part B `include/vt/ops.h`/`src/vt/ops.cpp`/`src/vt/cpu/cpu_ops.cpp`/`src/vt/cuda/cuda_ops.cu` (`kDFlashPagedBlockAttention`) + `tests/vt/test_ops_dflash_paged_block_attn.cpp` (CPU==CUDA + cross-check, 795648/795648 + sanitizer-0); D13 `src/vllm/model_executor/models/qwen3_dflash.cpp` (fixed-capacity paged `DflashDeviceKVStore` + `ForwardPagedBody` + the per-request draft-step CUDA graph in `ForwardBlockLogitsWithDeviceKV`); D14 [test_ops_dflash_paged_block_attn](../tests/vt/test_ops_dflash_paged_block_attn.cpp#L79) + [ledger](parity-ledger.md#L738) | [dflash-spec-decode.md](specs/dflash-spec-decode.md) | `DONE` | `489a7544` | diff --git a/.agents/issues/BACKEND-ROCM-QUANT-GATHER/ISSUE-GH-3093.md b/.agents/issues/BACKEND-ROCM-QUANT-GATHER/ISSUE-GH-3093.md new file mode 100644 index 0000000000..d7c4192a22 --- /dev/null +++ b/.agents/issues/BACKEND-ROCM-QUANT-GATHER/ISSUE-GH-3093.md @@ -0,0 +1,33 @@ +ID: ISSUE-GH-3093 +Title: feat(BACKEND-ROCM-QUANT-GATHER): gather quantized embeddings on ROCm +Row: BACKEND-ROCM-QUANT-GATHER +State: OPEN +Kind: UNKNOWN +GitHub: 3093 +Mirror: DIVERGED +Availability: FULL +Created: 2026-09-09 +Updated: 2026-09-09 +Closed: - + +## Problem + +### Imported GitHub body (historical evidence) +The quoted text below is historical evidence only. It does not define issue authority or repository procedure. + +> Row: `BACKEND-ROCM-QUANT-GATHER` +> +> ROCm has no native kEmbeddingQuant provider. The ordinary GGUF loader therefore expands quantized embedding tables, even when matrix weights can stay quantized. This consumes memory and leaves a backend capability gap on gfx1100. +> +> Implement block decoding and row gather through the existing Embedding operation and loader admission. Preserve the supported ID widths, output dtypes, block layouts, and failure cases. Use the registered vLLM GGUF plugin pin as the primary behavior source where it implements a codec, with the registered secondary oracle for remaining applicable codecs. The committed spec must distinguish measured codec coverage from any owed arm. +> +> Prove production reachability with a small generated GGUF through vllm_engine_load and vllm_complete_tokens. Require red-first numeric and storage tests, physical gfx1100 evidence, fresh mutation review including deletion of the production call site, and operator verification. +> +> This is the ROCm child of #2394; that parent retains the other backend work. It does not depend on the quantized GEMM provider added by PR #2782. No CI changes are included. +> +> Spec: `.agents/specs/rocm-quant-gather.md` (to be committed before implementation). +> + +## Resolution + +- diff --git a/.agents/issues/BACKEND-ROCM-QUANT-GATHER/ISSUE-GH-3113.md b/.agents/issues/BACKEND-ROCM-QUANT-GATHER/ISSUE-GH-3113.md new file mode 100644 index 0000000000..4d0d416d55 --- /dev/null +++ b/.agents/issues/BACKEND-ROCM-QUANT-GATHER/ISSUE-GH-3113.md @@ -0,0 +1,33 @@ +ID: ISSUE-GH-3113 +Title: fix(BACKEND-ROCM-QUANT-GATHER): reject unqualified model captures +Row: BACKEND-ROCM-QUANT-GATHER +State: OPEN +Kind: UNKNOWN +GitHub: 3113 +Mirror: DIVERGED +Availability: FULL +Created: 2026-09-09 +Updated: 2026-09-09 +Closed: - + +## Problem + +### Imported GitHub body (historical evidence) +The quoted text below is historical evidence only. It does not define issue authority or repository procedure. + +> Row: `BACKEND-ROCM-QUANT-GATHER` +> +> The bounded model comparator in MR #3097 accepts incomplete or unqualified primary records. Fresh review of commit `75d92614e84ff428eab49324f3b4fb2aba183ac3` reproduced the failures in `tools/rocm_quant_gather/compare_models.py`. +> +> A synthetic validator corpus uses the actual native and secondary captures. Changing one primary record to PENDING with a RuntimeError after generation still returns PASS. Wrong primary and plugin pins, a wrong runtime version, and an FP8 cache request also pass. The separate memory record accepts PENDING, a wrong primary pin, and different sampling or logical capacity. A nonzero cache view storage offset is also accepted. These synthetic mutations test the validator and are not oracle measurements. +> +> The driver records tokens before its second memory observation. Its exception path preserves those tokens. Comparing tokens alone therefore cannot establish successful execution. The operator separately checks actual process exits and sealed inputs; the comparator must enforce its own documented qualification contract. +> +> Require successful execution without an exception, the pinned primary and plugin identities, the resolved runtime identity, and the complete identical workload in both token and memory records. Validate the cache view offset and its physical bounds. Add focused rejection tests for each demonstrated invalid record and retain the valid actual captures as the positive control. +> +> Evidence: `/home/vikash/.cache/rdna3-gather-oracle-review/comparison-input-audit.json` records 31 cases, with 22 rejected mutations and nine invalid records accepted. The existing spec `.agents/specs/rocm-quant-gather.md` owns the correction. A fresh implementer repairs the finding, a different agent reviews the immutable result, and the operator reruns the complete comparison. No model, workload, pin, or performance requirement changes. +> + +## Resolution + +- diff --git a/.agents/issues/BACKEND-ROCM-QUANT-GATHER/ISSUE-GH-3126.md b/.agents/issues/BACKEND-ROCM-QUANT-GATHER/ISSUE-GH-3126.md new file mode 100644 index 0000000000..07a45743eb --- /dev/null +++ b/.agents/issues/BACKEND-ROCM-QUANT-GATHER/ISSUE-GH-3126.md @@ -0,0 +1,32 @@ +ID: ISSUE-GH-3126 +Title: fix(BACKEND-ROCM-QUANT-GATHER): label the comparison scope from the run, not a literal +Row: BACKEND-ROCM-QUANT-GATHER +State: OPEN +Kind: UNKNOWN +GitHub: 3126 +Mirror: DIVERGED +Availability: FULL +Created: 2026-09-10 +Updated: 2026-09-10 +Closed: - + +## Problem + +### Imported GitHub body (historical evidence) +The quoted text below is historical evidence only. It does not define issue authority or repository procedure. + +> Row: `BACKEND-ROCM-QUANT-GATHER` +> +> `tools/rocm_quant_gather/compare.py` writes a fixed scope string into every report: +> +> "scope": "synthetic operation parity; original fixture and model gates remain separate" +> +> The original-fixture coverage run over the 32 verified `Isotr0py/test-gguf-sample` fixtures therefore produced a report whose own scope line calls it "synthetic operation parity". The run itself is correct (320 outputs, 140 byte-exact, all within the upstream `atol=0.01 rtol=0.04`); only the label is wrong, and the report is cited as row evidence. +> +> Evidence: `/home/vikash/.cache/rdna3-gather-oracle-impl/original-fixture-coverage/compare/report.json` (report scope field) against its inputs (`fixture-manifest.json/manifest.json`, seal `f83b5c3e5cdb988b623c1b7e565e325a9ed9ea8085ea132bc8e08e6978bc1bbb`, exported from the 32 fixtures at revision `d82b8773934ef260d8d8a896a7c197bc69a0fac1`). +> +> Fix in flow: make the scope a command-line argument whose default preserves the previous text, and re-run the comparison so the report's label matches the run's inputs. No comparison, tolerance, or gate semantics change. + +## Resolution + +- diff --git a/.agents/quantization-matrix.md b/.agents/quantization-matrix.md index d7364fe589..978bea6dd4 100644 --- a/.agents/quantization-matrix.md +++ b/.agents/quantization-matrix.md @@ -69,13 +69,13 @@ forces the full dequant path back. | `QUANT-GGUF-Q5_K` | Q5_K | 13 / output + S/M presets | Y | Y | Y | Y | - | `PARTIAL` | [dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L155), [dispatch](../src/vllm/model_executor/model_loader/gguf_dequant.cpp#L79); [unit](../tests/vllm/test_gguf_dequant.cpp#L105); [APEX gate](../tests/parity/test_qwen36_gguf_engine.cpp#L149) **`C` = `Y` since G4 (2026-07-22):** the weight stays in its ggml blocks at load ([keep-quant policy](../src/vllm/model_executor/model_loader/gguf_keep_quant.cpp#L95), DEFAULT ON wherever `kMatmulBTQuant` is registered for the running device) and [`vt::MatmulBT`](../src/vt/ops.cpp#L158) dispatches it to the [tier-0 `vec_dot` GEMM](../src/vt/cpu/cpu_quant_dot.cpp#L1) — no bf16 expansion on the executed path. Op gate [test_ops_quant_dot](../tests/vt/test_ops_quant_dot.cpp#L1); residency losslessness + routing gates [test_gguf_keep_quant](../tests/vllm/test_gguf_keep_quant.cpp#L1). Binding CPU A/B on the mixed `Qwen3.5-2B-UD-Q8_K_XL` file (idle dgx aarch64, same binary, 3 reps): decode **3.45x**, prefill **4.16x**, peak RSS **1.16x less**, output tokens **byte-identical** to both the pre-G4 arm and the `VT_CPU_REF=1` oracle. `P` stays `-`: still 3.38x/8.20x behind llama.cpp on the same file, because 60% of ITS weight bytes are `f16` and take the elementwise kernel. | leaf open | - | | `QUANT-GGUF-Q6_K` | Q6_K | 14 / output | Y | Y | Y | Y | - | `PARTIAL` | [dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L192), [dispatch](../src/vllm/model_executor/model_loader/gguf_dequant.cpp#L80); [unit](../tests/vllm/test_gguf_dequant.cpp#L134); [Compact gate](../tests/parity/test_qwen36_gguf_engine.cpp#L143), [Balanced gate](../tests/parity/test_qwen36_gguf_engine.cpp#L149) **`C` = `Y` since G4 (2026-07-22):** the weight stays in its ggml blocks at load ([keep-quant policy](../src/vllm/model_executor/model_loader/gguf_keep_quant.cpp#L95), DEFAULT ON wherever `kMatmulBTQuant` is registered for the running device) and [`vt::MatmulBT`](../src/vt/ops.cpp#L158) dispatches it to the [tier-0 `vec_dot` GEMM](../src/vt/cpu/cpu_quant_dot.cpp#L1) — no bf16 expansion on the executed path. Op gate [test_ops_quant_dot](../tests/vt/test_ops_quant_dot.cpp#L1); residency losslessness + routing gates [test_gguf_keep_quant](../tests/vllm/test_gguf_keep_quant.cpp#L1). Binding CPU A/B on the mixed `Qwen3.5-2B-UD-Q8_K_XL` file (idle dgx aarch64, same binary, 3 reps): decode **3.45x**, prefill **4.16x**, peak RSS **1.16x less**, output tokens **byte-identical** to both the pre-G4 arm and the `VT_CPU_REF=1` oracle. `P` stays `-`: still 3.38x/8.20x behind llama.cpp on the same file, because 60% of ITS weight bytes are `f16` and take the elementwise kernel. | leaf open | - | | `QUANT-GGUF-IQ2_XXS` | IQ2_XXS | 16 / output | Y | Y | Y | - | - | `ACTIVE` | reader trait [`:230`](../src/vllm/model_executor/model_loader/gguf_reader.cpp#L230) (66 B block); [codebook dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L322) (`DequantIQ2_XXS`, ported 1:1 from llama.cpp `ggml-quants.c:2416`); grids moved to the shared [cpu_quant_iq_tables.h](../src/vt/cpu/cpu_quant_iq_tables.h) (`iq2xxs_grid`/`ksigns_iq2xs`/`kmask_iq2xs`, `ggml-common.h:499,503,550`); vt block dtype `kIQ2_XXS` [geometry](../src/vt/dtype.cpp#L80), [dispatch](../src/vllm/model_executor/model_loader/gguf_dequant.cpp#L114). **`C` = `Y` since DeepSeek-V4 W8 (2026-07-29):** the keep-quant `vec_dot` [`VecDotIQ2_XXSQ8_K`](../src/vt/cpu/cpu_quant_dot.cpp#L1) (1:1 port of ggml `vec_dot_iq2_xxs_q8_K_generic`, `quants.c:855`) + the [Q8_K traits row](../src/vt/cpu/cpu_quant_traits.cpp#L1) make `HasQuantDotKernel` TRUE ⇒ the loader keeps IQ2_XXS blocks COMPRESSED and dots them directly — the MEMORY ENABLER for the single-Spark `unsloth/DeepSeek-V4-Flash-GGUF UD-IQ2_XXS` gate/up routed experts (`ffn_gate_exps`/`ffn_up_exps`). Op gate [test_ops_quant_dot](../tests/vt/test_ops_quant_dot.cpp#L1); RED-first proven (perturb the `0.125` fold → fail, revert → green). **W2b GGUF→tower materialization LANDED (2026-07-29, `CLAIM-DEEPSEEK-V4-W2B`):** `LoadDeepseekV4FromGguf` wires the `blk.N.*` name-map + these keep-quant blocks into the `DeepseekV4` weight towers (MW/SEW stay COMPRESSED, gated `test_deepseek_v4_gguf_load` 5/5·149 tiny synthetic). `E`/`P` = `-`: the real DeepSeek-V4 91 GB `UD-IQ2_XXS` e2e run stays W8-final (download + DGX). **CUDA compute LANDED 2026-07-29 (`KERNEL-QUANT-CIQ-GEMM-CUDA`, `CLAIM-CUDA-KEEPQUANT-GEMM`):** a native kCUDA `kMatmulBTQuant` provider ([cuda_quant_dot.cu](../src/vt/cuda/cuda_quant_dot.cu), MMVQ-style) dots these blocks ON the GPU (Q8_K activation quant + integer dot, dequant-in-kernel, weights stay COMPRESSED in the unified pool), GB10-gated ([test_cuda_quant_dot](../tests/vt/test_cuda_quant_dot.cpp)) 2/2 · 92401 vs the CPU oracle (NMSE ≤1e-6, int core bit-exact) + f64 dequant (≤5e-4), compute-sanitizer 0, RED-first proven — so on a CUDA runner these experts dispatch to the GPU, not the 20 ARM cores. | [CUDA keep-quant GEMM](specs/cuda-keepquant-gemm.md); [iquant DSV4 spike](specs/gguf-iquant-dsv4.md) | `CLAIM-DEEPSEEK-V4-W8` | -| `QUANT-GGUF-IQ2_XS` | IQ2_XS | 17 / output; IQ2_S preset storage | Y | Y | Y | - | - | `PARTIAL` | reader trait [`FindGgmlTraits` case 17](../src/vllm/model_executor/model_loader/gguf_reader.cpp#L254) (74 B block, ADDED by [#2240](https://github.com/mudler/vllm.cpp/issues/2240)); [codebook dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L453) `DequantIQ2_XS` + the shared 512-entry [`kIq2xsGrid`](../src/vt/cpu/cpu_quant_iq_tables.h#L168), ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-quants.c:2516` + `ggml/src/ggml-common.h:627`; vt block dtype `kIQ2_XS` [geometry](../src/vt/dtype.cpp#L131). **`M` = `Y`, gated BYTE-FOR-BYTE against the oracle** over REAL checkpoint bytes: 4 blocks of `blk.3.ffn_gate_exps.weight` from the staged `unsloth/GLM-5.3-Flash-GGUF` UD-Q2_K_XL arm, decoded by the pinned upstream's own `dequantize_row_iq2_xs` and compared as raw f32 bit patterns ([goldens + provenance](../tests/vt/iq2xs_iq4xs_golden_vectors.h#L72), [unit](../tests/vllm/test_gguf_dequant.cpp#L570)); the table itself is sealed by FNV-1a digest ([test_ops_quant_dot](../tests/vt/test_ops_quant_dot.cpp#L717)) because the IQ2 family carries three same-shaped grids and a wrong one still decodes. Reached through `GgufFile::Open` + `DequantGgufRowToF32` in the [same case](../tests/vllm/test_gguf_dequant.cpp#L590), which is the production path the real artifact took, and [geometry cross-check](../tests/vt/test_ops_quant_traits.cpp#L156) pins the reader and vt tables against each other. **`C` = `Y` since [#2247](https://github.com/mudler/vllm.cpp/issues/2247):** keep-quant [`VecDotIQ2_XSQ8_K`](../src/vt/cpu/cpu_quant_dot.cpp#L783) against the Q8_K activation, ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-cpu/quants.c:948` `ggml_vec_dot_iq2_xs_q8_K_generic`, with [`BlockIQ2_XS`](../src/vt/cpu/cpu_quant_blocks.h#L191) (`ggml-common.h:388-393`) and the [traits row](../src/vt/cpu/cpu_quant_traits.cpp#L111) (`ggml-cpu.c:342-347`), so `HasQuantDotKernel` is TRUE and the loader keeps the blocks. **Gated BIT FOR BIT against the oracle's OWN KERNEL on the same real artifact bytes the decoder gate uses** — 4 super-blocks of `blk.3.ffn_gate_exps.weight` dotted against a Q8_K activation the oracle itself encoded, total and per block ([goldens + provenance](../tests/vt/iq2xs_iq4xs_dot_golden.h#L268), [case](../tests/vt/test_ops_quant_dot.cpp#L857)); the comparison is against upstream's own f32 accumulation, because a cleaner f64 reference agrees with a reduction-order defect. The grid SEAL and the KERNEL are tied together by a mutation, not by inspection: swapping `kIq2xsGrid` for `kIq2xxsGrid` inside the kernel reds this case while the seal itself stays green ([coupling case](../tests/vt/test_ops_quant_dot.cpp#L879)). **Residency MEASURED, not inferred:** `RouteGgufTensor` driven over all 1412 tensors of the staged artifact's own headers routes 774 to `kKeepQuant` for **101.14 GiB resident** against 426.72 GiB before this row — the 82 IQ2_XS tensors stop expanding from 53.33 GiB to 369.00 GiB — which fits the ~119.63 GiB of `dgx:gpu0` with 18.49 GiB of headroom. [Routing table](../tests/vllm/test_gguf_keep_quant.cpp#L425) restated: the GEMM term moved 20 -> 24 and the GATHER term stayed 13. **The CUDA arm now HAS a keep-quant kernel for this dtype** ([#2260](https://github.com/mudler/vllm.cpp/issues/2260), spec [cuda-keepquant-iq2xs-iq4xs](specs/cuda-keepquant-iq2xs-iq4xs.md)): `src/vt/cuda/cuda_quant_dot.cu::DotIQ2XS` is a 1:1 port of `b10451 quants.c:948` reading the generated `src/vt/cuda/cuda_quant_iq_tables.cuh::d_iq2xs_grid`, and `src/vt/cuda/cuda_quant_dot.cu::IsCudaKeepQuantSupported` admits twelve Q8_K-family encodings with IQ2_XS among them, wired into all THREE dispatch switches. Before that the 82 tensors still kept — `src/vllm/model_executor/model_loader/gguf_keep_quant.cpp::DeviceKeepQuantSupported` returns `true` for CUDA on its `default:` arm regardless — so the artifact DID fit and the expert GEMM then took the CPU fallback inside `src/vt/cuda/cuda_quant_dot.cu::MatmulBTQuantGroupedKernelCuda` behind a full `cudaStreamSynchronize` per grouped call, while the fused `src/vt/cuda/cuda_quant_dot.cu::MoeGateUpSwiGLUGroupedCuda` seam THREW `gate/up must be the SAME CUDA keep-quant dtype`. **101.14 GiB remains a RESIDENCY result and no speed number is claimed**, and the kernel is not yet SELECTED by a running model: `Glm5NextHostForward` refuses a non-CPU queue by name, so the wiring is still owed by `MODEL-MM-glm5-next-glm5-next-for-conditional-generation`. NOT reached from this row today — `glm5_next_moe.cpp` is W5's host reference and `laguna.cpp` is the only model on the fused seam — but AGENTS.md routes mergeable MLP projections through `vt::MergedGemmGroup`, so W5b ([#2241](https://github.com/mudler/vllm.cpp/issues/2241)) and W5c ([#2242](https://github.com/mudler/vllm.cpp/issues/2242)) make the throw live on a 101 GiB-resident model. Contrast `QUANT-GGUF-IQ2_XXS` above, which records a CUDA compute landing; this row has none. Carried as O19 in [glm5-next-flash](specs/glm5-next-flash.md). | [glm5-next-flash](specs/glm5-next-flash.md) | - | +| `QUANT-GGUF-IQ2_XS` | IQ2_XS | 17 / output; IQ2_S preset storage | Y | Y | Y | - | - | `PARTIAL` | reader trait [`FindGgmlTraits` case 17](../src/vllm/model_executor/model_loader/gguf_reader.cpp#L254) (74 B block, ADDED by [#2240](https://github.com/mudler/vllm.cpp/issues/2240)); [codebook dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L453) `DequantIQ2_XS` + the shared 512-entry [`kIq2xsGrid`](../src/vt/cpu/cpu_quant_iq_tables.h#L168), ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-quants.c:2516` + `ggml/src/ggml-common.h:627`; vt block dtype `kIQ2_XS` [geometry](../src/vt/dtype.cpp#L131). **`M` = `Y`, gated BYTE-FOR-BYTE against the oracle** over REAL checkpoint bytes: 4 blocks of `blk.3.ffn_gate_exps.weight` from the staged `unsloth/GLM-5.3-Flash-GGUF` UD-Q2_K_XL arm, decoded by the pinned upstream's own `dequantize_row_iq2_xs` and compared as raw f32 bit patterns ([goldens + provenance](../tests/vt/iq2xs_iq4xs_golden_vectors.h#L72), [unit](../tests/vllm/test_gguf_dequant.cpp#L570)); the table itself is sealed by FNV-1a digest ([test_ops_quant_dot](../tests/vt/test_ops_quant_dot.cpp#L717)) because the IQ2 family carries three same-shaped grids and a wrong one still decodes. Reached through `GgufFile::Open` + `DequantGgufRowToF32` in the [same case](../tests/vllm/test_gguf_dequant.cpp#L590), which is the production path the real artifact took, and [geometry cross-check](../tests/vt/test_ops_quant_traits.cpp#L172) pins the reader and vt tables against each other. **`C` = `Y` since [#2247](https://github.com/mudler/vllm.cpp/issues/2247):** keep-quant [`VecDotIQ2_XSQ8_K`](../src/vt/cpu/cpu_quant_dot.cpp#L783) against the Q8_K activation, ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-cpu/quants.c:948` `ggml_vec_dot_iq2_xs_q8_K_generic`, with [`BlockIQ2_XS`](../src/vt/cpu/cpu_quant_blocks.h#L191) (`ggml-common.h:388-393`) and the [traits row](../src/vt/cpu/cpu_quant_traits.cpp#L111) (`ggml-cpu.c:342-347`), so `HasQuantDotKernel` is TRUE and the loader keeps the blocks. **Gated BIT FOR BIT against the oracle's OWN KERNEL on the same real artifact bytes the decoder gate uses** — 4 super-blocks of `blk.3.ffn_gate_exps.weight` dotted against a Q8_K activation the oracle itself encoded, total and per block ([goldens + provenance](../tests/vt/iq2xs_iq4xs_dot_golden.h#L268), [case](../tests/vt/test_ops_quant_dot.cpp#L857)); the comparison is against upstream's own f32 accumulation, because a cleaner f64 reference agrees with a reduction-order defect. The grid SEAL and the KERNEL are tied together by a mutation, not by inspection: swapping `kIq2xsGrid` for `kIq2xxsGrid` inside the kernel reds this case while the seal itself stays green ([coupling case](../tests/vt/test_ops_quant_dot.cpp#L879)). **Residency MEASURED, not inferred:** `RouteGgufTensor` driven over all 1412 tensors of the staged artifact's own headers routes 774 to `kKeepQuant` for **101.14 GiB resident** against 426.72 GiB before this row — the 82 IQ2_XS tensors stop expanding from 53.33 GiB to 369.00 GiB — which fits the ~119.63 GiB of `dgx:gpu0` with 18.49 GiB of headroom. [Routing table](../tests/vllm/test_gguf_keep_quant.cpp#L425) restated: the GEMM term moved 20 -> 24 and the GATHER term stayed 13. **The CUDA arm now HAS a keep-quant kernel for this dtype** ([#2260](https://github.com/mudler/vllm.cpp/issues/2260), spec [cuda-keepquant-iq2xs-iq4xs](specs/cuda-keepquant-iq2xs-iq4xs.md)): `src/vt/cuda/cuda_quant_dot.cu::DotIQ2XS` is a 1:1 port of `b10451 quants.c:948` reading the generated `src/vt/cuda/cuda_quant_iq_tables.cuh::d_iq2xs_grid`, and `src/vt/cuda/cuda_quant_dot.cu::IsCudaKeepQuantSupported` admits twelve Q8_K-family encodings with IQ2_XS among them, wired into all THREE dispatch switches. Before that the 82 tensors still kept — `src/vllm/model_executor/model_loader/gguf_keep_quant.cpp::DeviceKeepQuantSupported` returns `true` for CUDA on its `default:` arm regardless — so the artifact DID fit and the expert GEMM then took the CPU fallback inside `src/vt/cuda/cuda_quant_dot.cu::MatmulBTQuantGroupedKernelCuda` behind a full `cudaStreamSynchronize` per grouped call, while the fused `src/vt/cuda/cuda_quant_dot.cu::MoeGateUpSwiGLUGroupedCuda` seam THREW `gate/up must be the SAME CUDA keep-quant dtype`. **101.14 GiB remains a RESIDENCY result and no speed number is claimed**, and the kernel is not yet SELECTED by a running model: `Glm5NextHostForward` refuses a non-CPU queue by name, so the wiring is still owed by `MODEL-MM-glm5-next-glm5-next-for-conditional-generation`. NOT reached from this row today — `glm5_next_moe.cpp` is W5's host reference and `laguna.cpp` is the only model on the fused seam — but AGENTS.md routes mergeable MLP projections through `vt::MergedGemmGroup`, so W5b ([#2241](https://github.com/mudler/vllm.cpp/issues/2241)) and W5c ([#2242](https://github.com/mudler/vllm.cpp/issues/2242)) make the throw live on a 101 GiB-resident model. Contrast `QUANT-GGUF-IQ2_XXS` above, which records a CUDA compute landing; this row has none. Carried as O19 in [glm5-next-flash](specs/glm5-next-flash.md). | [glm5-next-flash](specs/glm5-next-flash.md) | - | | `QUANT-GGUF-IQ3_XXS` | IQ3_XXS | 18 / output | Y | Y | Y | - | - | `READY` | reader trait [`:237`](../src/vllm/model_executor/model_loader/gguf_reader.cpp#L237) (98 B block, ADDED W8); [codebook dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L349) (`DequantIQ3_XXS` + the shared [`iq3xxs_grid`](../src/vt/cpu/cpu_quant_iq_tables.h), ported 1:1 from llama.cpp `ggml-quants.c:2503` + `ggml-common.h:1007`); vt block dtype `kIQ3_XXS` [geometry](../src/vt/dtype.cpp#L86). **`C` = `Y` (DeepSeek-V4 W8, 2026-07-29):** the keep-quant `vec_dot` [`VecDotIQ3_XXSQ8_K`](../src/vt/cpu/cpu_quant_dot.cpp#L1) (1:1 port of ggml `vec_dot_iq3_xxs_q8_K_generic`, `quants.c:999`) + the [Q8_K traits row](../src/vt/cpu/cpu_quant_traits.cpp#L1) keep IQ3_XXS blocks COMPRESSED — the MEMORY ENABLER for the `UD-IQ2_XXS` **down** routed experts (`ffn_down_exps`, which are IQ3_XXS, NOT IQ2_XXS; without keep-quant they alone OOM the box). Op gate [test_ops_quant_dot](../tests/vt/test_ops_quant_dot.cpp#L1) (vec_dot vs f64 dequant-dot ≤1e-5·L1, NMSE ≤5e-4). **W2b GGUF→tower materialization LANDED (2026-07-29, `CLAIM-DEEPSEEK-V4-W2B`):** `LoadDeepseekV4FromGguf` wires the name-map + keep-quant blocks into the `DeepseekV4` towers (`test_deepseek_v4_gguf_load` 5/5·149). `E`/`P` = `-`: the real DeepSeek-V4 e2e run stays W8-final. **CUDA compute LANDED 2026-07-29 (`KERNEL-QUANT-CIQ-GEMM-CUDA`, `CLAIM-CUDA-KEEPQUANT-GEMM`):** a native kCUDA `kMatmulBTQuant` provider ([cuda_quant_dot.cu](../src/vt/cuda/cuda_quant_dot.cu), MMVQ-style) dots these blocks ON the GPU (Q8_K activation quant + integer dot, dequant-in-kernel, weights stay COMPRESSED in the unified pool), GB10-gated ([test_cuda_quant_dot](../tests/vt/test_cuda_quant_dot.cpp)) 2/2 · 92401 vs the CPU oracle (NMSE ≤1e-6, int core bit-exact) + f64 dequant (≤5e-4), compute-sanitizer 0, RED-first proven — so on a CUDA runner these experts dispatch to the GPU, not the 20 ARM cores. | [CUDA keep-quant GEMM](specs/cuda-keepquant-gemm.md); [iquant DSV4 spike](specs/gguf-iquant-dsv4.md) | - | | `QUANT-GGUF-IQ1_S` | IQ1_S | 19 / output | - | - | - | - | - | `INVENTORIED` | - | leaf open | - | | `QUANT-GGUF-IQ4_NL` | IQ4_NL | 20 / output | - | - | - | - | - | `INVENTORIED` | - | leaf open | - | -| `QUANT-GGUF-IQ3_S` | IQ3_S | 21 / IQ3 S/XS/M storage | Y | Y | - | - | - | `PARTIAL` | reader trait [`FindGgmlTraits` case 21](../src/vllm/model_executor/model_loader/gguf_reader.cpp#L305) (110 B block, ADDED by [#2510](https://github.com/mudler/vllm.cpp/issues/2510)); [codebook dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L399) `DequantIQ3_S` + the 512-entry [`kIq3sGrid`](../src/vt/cpu/cpu_quant_iq_tables.h#L168), ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-quants.c:2607` + `ggml/src/ggml-common.h:1052`; vt block dtype `kIQ3_S` [geometry](../src/vt/dtype.cpp#L150) and the CUDA gather codec [`DqIQ3_S`](../src/vt/cuda/cuda_quant_dequant.cuh#L476). **THE REFUSAL THIS ROW EXISTS FOR:** id 21 was the ONE hole in the reader's i-quant run 16..23, and `GgufFile::Open` therefore refused whole artifacts over it — `unsloth/Qwen3.8-27B-GGUF` `UD-Q4_K_M.gguf` stores 4 of its 866 tensors in IQ3_S and `UD-Q4_K_XL.gguf` stores 1 of 866, while every OTHER encoding in those files (F32, Q3_K, Q4_K, Q5_K, Q6_K, Q8_0, IQ4_NL, IQ4_XS) was already handled. That is the file the published Strix Halo rows ran, so [#2497](https://github.com/mudler/vllm.cpp/issues/2497) had to substitute the plain `Q4_K_M`. **`M` = `Y`, gated BYTE-FOR-BYTE against the oracle** over REAL checkpoint bytes: 4 blocks of `blk.11.ffn_gate.weight` at absolute offset 4,262,628,128 in the staged `Qwen3.8-27B-UD-Q4_K_M.gguf` (sha256 `322e194f…3482`), decoded by the pinned upstream's own `dequantize_row_iq3_s` and compared as raw f32 bit patterns ([goldens + provenance](../tests/vt/iq3s_golden_vectors.h#L69), [unit](../tests/vllm/test_gguf_dequant.cpp#L663)); the four blocks span 190 distinct 9-bit grid indices of which 73 are >= 256, so the `qh`-spliced ninth bit is exercised rather than left at zero. The 110-byte stride is the ORACLE's own `sizeof(block_iq3_s)`, printed by the harness, not a sum read off the struct. The table itself is sealed by FNV-1a digest and by its lane alphabet ([test_ops_quant_dot](../tests/vt/test_ops_quant_dot.cpp#L717)), because `kIq3sGrid` (512 u32) and `kIq3xxsGrid` (256 u32) are the same shape and a decoder pointed at the sibling still decodes. Reached through `GgufFile::Open` + `DequantGgufRowToF32` in the same file, which is the production path the real artifact took. **`C` = `-`, DELIBERATELY, and it is the one FILE encoding in this tree that decodes without dotting.** `KeepQuantDType` gates on `HasQuantDotKernel`, so IQ3_S routes `kExpandBf16` on the GEMM arm of EVERY tier (CPU, CUDA, ROCm, Metal, Vulkan) and `kKeepQuant` only on the gather, where CPU and CUDA both decode. That is asserted BY NAME rather than left to a count ([routing](../tests/vllm/test_gguf_keep_quant.cpp#L429), [decode-only class](../tests/vt/test_ops_quant_traits.cpp#L250)), so the day the `vec_dot` lands those cases red. **The CPU `VecDotIQ3_SQ8_K` and the CUDA `WType::kIQ3_S` are OWED AS ONE UNIT** (`ggml-cpu/quants.c:1094`, `.vec_dot_type = GGML_TYPE_Q8_K` at `ggml-cpu.c:355-360`): `IsCudaKeepQuantSupported` has no refusal arm, so landing the CPU half alone would flip the loader to keep IQ3_S blocks and send every CUDA IQ3_S GEMM through the host fallback that [#2260](https://github.com/mudler/vllm.cpp/issues/2260) MEASURED to segfault on a discrete card. On the measured artifact the expansion is 146.13 MiB of blocks against 680.00 MiB of bf16, 3.4 % of a 15.33 GiB file — bounded here, and NOT bounded on an artifact whose experts are IQ3_S, which is why it is recorded rather than waved at (compare [#1870](https://github.com/mudler/vllm.cpp/issues/1870)). `E`/`P` = `-`: this row makes the artifact OPENABLE and claims no token and no number; the `gfx1151` quant-matched decode number stays owed by [#2497](https://github.com/mudler/vllm.cpp/issues/2497). | [gguf-iq3s](specs/gguf-iq3s.md) | - | +| `QUANT-GGUF-IQ3_S` | IQ3_S | 21 / IQ3 S/XS/M storage | Y | Y | - | - | - | `PARTIAL` | reader trait [`FindGgmlTraits` case 21](../src/vllm/model_executor/model_loader/gguf_reader.cpp#L305) (110 B block, ADDED by [#2510](https://github.com/mudler/vllm.cpp/issues/2510)); [codebook dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L399) `DequantIQ3_S` + the 512-entry [`kIq3sGrid`](../src/vt/cpu/cpu_quant_iq_tables.h#L168), ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-quants.c:2607` + `ggml/src/ggml-common.h:1052`; vt block dtype `kIQ3_S` [geometry](../src/vt/dtype.cpp#L150) and the CUDA gather codec [`DqIQ3_S`](../src/vt/cuda/cuda_quant_dequant.cuh#L476). **THE REFUSAL THIS ROW EXISTS FOR:** id 21 was the ONE hole in the reader's i-quant run 16..23, and `GgufFile::Open` therefore refused whole artifacts over it — `unsloth/Qwen3.8-27B-GGUF` `UD-Q4_K_M.gguf` stores 4 of its 866 tensors in IQ3_S and `UD-Q4_K_XL.gguf` stores 1 of 866, while every OTHER encoding in those files (F32, Q3_K, Q4_K, Q5_K, Q6_K, Q8_0, IQ4_NL, IQ4_XS) was already handled. That is the file the published Strix Halo rows ran, so [#2497](https://github.com/mudler/vllm.cpp/issues/2497) had to substitute the plain `Q4_K_M`. **`M` = `Y`, gated BYTE-FOR-BYTE against the oracle** over REAL checkpoint bytes: 4 blocks of `blk.11.ffn_gate.weight` at absolute offset 4,262,628,128 in the staged `Qwen3.8-27B-UD-Q4_K_M.gguf` (sha256 `322e194f…3482`), decoded by the pinned upstream's own `dequantize_row_iq3_s` and compared as raw f32 bit patterns ([goldens + provenance](../tests/vt/iq3s_golden_vectors.h#L69), [unit](../tests/vllm/test_gguf_dequant.cpp#L663)); the four blocks span 190 distinct 9-bit grid indices of which 73 are >= 256, so the `qh`-spliced ninth bit is exercised rather than left at zero. The 110-byte stride is the ORACLE's own `sizeof(block_iq3_s)`, printed by the harness, not a sum read off the struct. The table itself is sealed by FNV-1a digest and by its lane alphabet ([test_ops_quant_dot](../tests/vt/test_ops_quant_dot.cpp#L717)), because `kIq3sGrid` (512 u32) and `kIq3xxsGrid` (256 u32) are the same shape and a decoder pointed at the sibling still decodes. Reached through `GgufFile::Open` + `DequantGgufRowToF32` in the same file, which is the production path the real artifact took. **`C` = `-`, DELIBERATELY, and it is the one FILE encoding in this tree that decodes without dotting.** `KeepQuantDType` gates on `HasQuantDotKernel`, so IQ3_S routes `kExpandBf16` on the GEMM arm of EVERY tier (CPU, CUDA, ROCm, Metal, Vulkan) and `kKeepQuant` only on the gather, where CPU and CUDA both decode. That is asserted BY NAME rather than left to a count ([routing](../tests/vllm/test_gguf_keep_quant.cpp#L429), [decode-only class](../tests/vt/test_ops_quant_traits.cpp#L266)), so the day the `vec_dot` lands those cases red. **The CPU `VecDotIQ3_SQ8_K` and the CUDA `WType::kIQ3_S` are OWED AS ONE UNIT** (`ggml-cpu/quants.c:1094`, `.vec_dot_type = GGML_TYPE_Q8_K` at `ggml-cpu.c:355-360`): `IsCudaKeepQuantSupported` has no refusal arm, so landing the CPU half alone would flip the loader to keep IQ3_S blocks and send every CUDA IQ3_S GEMM through the host fallback that [#2260](https://github.com/mudler/vllm.cpp/issues/2260) MEASURED to segfault on a discrete card. On the measured artifact the expansion is 146.13 MiB of blocks against 680.00 MiB of bf16, 3.4 % of a 15.33 GiB file — bounded here, and NOT bounded on an artifact whose experts are IQ3_S, which is why it is recorded rather than waved at (compare [#1870](https://github.com/mudler/vllm.cpp/issues/1870)). `E`/`P` = `-`: this row makes the artifact OPENABLE and claims no token and no number; the `gfx1151` quant-matched decode number stays owed by [#2497](https://github.com/mudler/vllm.cpp/issues/2497). | [gguf-iq3s](specs/gguf-iq3s.md) | - | | `QUANT-GGUF-IQ2_S` | IQ2_S | 22 / IQ2_M storage | Y | - | - | - | - | `INVENTORIED` | reader `:229-234`; [explicit rejection](../tests/vllm/test_gguf_dequant.cpp#L223) | leaf open | - | -| `QUANT-GGUF-IQ4_XS` | IQ4_XS | 23 / output | Y | Y | Y | - | - | `PARTIAL` | reader trait [`FindGgmlTraits` case 23](../src/vllm/model_executor/model_loader/gguf_reader.cpp#L311) (136 B block; the trait predates the decoder, the decoder is [#2240](https://github.com/mudler/vllm.cpp/issues/2240)); [dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L488) `DequantIQ4_XS`, ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-quants.c:2743`; vt block dtype `kIQ4_XS` [geometry](../src/vt/dtype.cpp#L140). NOT a codebook delta from IQ4_NL — it reuses `kValuesIq4nl` unchanged and differs only in the SUPER-BLOCK SCALE LAYOUT (a 6-bit `ls` spliced from a `scales_l` nibble and a `scales_h` bit pair, then biased by -32). **`M` = `Y`, gated BYTE-FOR-BYTE against the oracle** over REAL checkpoint bytes: 4 blocks of `blk.11.ffn_down_exps.weight` from the staged `unsloth/GLM-5.3-Flash-GGUF` UD-Q2_K_XL arm, whose four super-blocks span `ls` 0..61 so both nibbles and both `scales_h` bit pairs and both signs of `dl` are exercised ([goldens + provenance](../tests/vt/iq2xs_iq4xs_golden_vectors.h#L276), [unit](../tests/vllm/test_gguf_dequant.cpp#L576)). [Geometry cross-check](../tests/vt/test_ops_quant_traits.cpp#L156) pins the reader and vt tables against each other. The former `explicit rejection` evidence is GONE by construction — that test now guards Q1_0 (41), the remaining tabulated-but-undecodable id. **`C` = `Y` since [#2247](https://github.com/mudler/vllm.cpp/issues/2247):** keep-quant [`VecDotIQ4_XSQ8_K`](../src/vt/cpu/cpu_quant_dot.cpp#L844), ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-cpu/quants.c:1283` `ggml_vec_dot_iq4_xs_q8_K_generic`, with [`BlockIQ4_XS`](../src/vt/cpu/cpu_quant_blocks.h#L205) (`ggml-common.h:454-460`) and the [traits row](../src/vt/cpu/cpu_quant_traits.cpp#L121). **The activation pairing is `Q8_K`, READ OFF the oracle and not inherited from IQ4_NL:** `type_traits_cpu` at `ggml/src/ggml-cpu/ggml-cpu.c:385-390` carries `.vec_dot = ggml_vec_dot_iq4_xs_q8_K, `.vec_dot_type = GGML_TYPE_Q8_K`, against `:379-384`'s `GGML_TYPE_Q8_0` for IQ4_NL — the 16-entry codebook is shared, the block geometry is not, and a 256-element super-block pairs with the 256-element activation encoding. **Gated BIT FOR BIT against the oracle's OWN KERNEL** on 4 super-blocks of `blk.11.ffn_down_exps.weight`, total and per block ([goldens](../tests/vt/iq2xs_iq4xs_dot_golden.h#L269), [case](../tests/vt/test_ops_quant_dot.cpp#L869)). These 3 tensors stop expanding from 3.59 GiB to 13.50 GiB; the whole-artifact measurement is in the `QUANT-GGUF-IQ2_XS` row above. **It also unblocks a SECOND consumer:** the expert-tower streaming lane is all-or-nothing (`GgufExpertTowersReachSlotLane` returns false on the FIRST tower that does not keep), so the 4 IQ4_XS tensors of the GLM-5.3 `UD-IQ1_S` arm were dropping an arm whose other 221 towers all kept — one tower 6.375 -> 24.000 GiB, and a 4096-slot cache 25.5 -> 96 GiB. Asserted in the `kStackedExpertWeight` role both models store these towers in ([lane case](../tests/vllm/model_executor/test_gguf_device_fit.cpp#L672)). **The CUDA arm now HAS a keep-quant kernel for this dtype too** ([#2260](https://github.com/mudler/vllm.cpp/issues/2260), spec [cuda-keepquant-iq2xs-iq4xs](specs/cuda-keepquant-iq2xs-iq4xs.md)): `src/vt/cuda/cuda_quant_dot.cu::DotIQ4XS` is a 1:1 port of `b10451 quants.c:1283` over the generated `src/vt/cuda/cuda_quant_iq_tables.cuh::d_kvalues_iq4nl`, keeping upstream's EIGHT f32 accumulation steps per super-block rather than one integer core, because that association is what the oracle's golden numbers were produced with. **This completes the device admission of the GLM-5.3 non-flash `UD-IQ1_S` arm**, whose other five encodings (IQ1_S, IQ3_XXS, IQ2_XXS, Q2_K, Q3_K) were already there. Before it, these 3 tensors fit `dgx:gpu0` with their expert GEMM on the CPU fallback behind a `cudaStreamSynchronize` and the fused `src/vt/cuda/cuda_quant_dot.cu::MoeGateUpSwiGLUGroupedCuda` seam throwing. The fit remains RESIDENCY and no speed number is claimed; O19's premise in [glm5-next-flash](specs/glm5-next-flash.md) is discharged, and what still blocks a CUDA run of either model is that model's own host-f32 forward. | [glm5-next-flash](specs/glm5-next-flash.md) | - | +| `QUANT-GGUF-IQ4_XS` | IQ4_XS | 23 / output | Y | Y | Y | - | - | `PARTIAL` | reader trait [`FindGgmlTraits` case 23](../src/vllm/model_executor/model_loader/gguf_reader.cpp#L311) (136 B block; the trait predates the decoder, the decoder is [#2240](https://github.com/mudler/vllm.cpp/issues/2240)); [dequant kernel](../src/vt/cpu/cpu_quant_dequant.cpp#L488) `DequantIQ4_XS`, ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-quants.c:2743`; vt block dtype `kIQ4_XS` [geometry](../src/vt/dtype.cpp#L140). NOT a codebook delta from IQ4_NL — it reuses `kValuesIq4nl` unchanged and differs only in the SUPER-BLOCK SCALE LAYOUT (a 6-bit `ls` spliced from a `scales_l` nibble and a `scales_h` bit pair, then biased by -32). **`M` = `Y`, gated BYTE-FOR-BYTE against the oracle** over REAL checkpoint bytes: 4 blocks of `blk.11.ffn_down_exps.weight` from the staged `unsloth/GLM-5.3-Flash-GGUF` UD-Q2_K_XL arm, whose four super-blocks span `ls` 0..61 so both nibbles and both `scales_h` bit pairs and both signs of `dl` are exercised ([goldens + provenance](../tests/vt/iq2xs_iq4xs_golden_vectors.h#L276), [unit](../tests/vllm/test_gguf_dequant.cpp#L576)). [Geometry cross-check](../tests/vt/test_ops_quant_traits.cpp#L172) pins the reader and vt tables against each other. The former `explicit rejection` evidence is GONE by construction — that test now guards Q1_0 (41), the remaining tabulated-but-undecodable id. **`C` = `Y` since [#2247](https://github.com/mudler/vllm.cpp/issues/2247):** keep-quant [`VecDotIQ4_XSQ8_K`](../src/vt/cpu/cpu_quant_dot.cpp#L844), ported 1:1 from llama.cpp `b10451` `ggml/src/ggml-cpu/quants.c:1283` `ggml_vec_dot_iq4_xs_q8_K_generic`, with [`BlockIQ4_XS`](../src/vt/cpu/cpu_quant_blocks.h#L205) (`ggml-common.h:454-460`) and the [traits row](../src/vt/cpu/cpu_quant_traits.cpp#L121). **The activation pairing is `Q8_K`, READ OFF the oracle and not inherited from IQ4_NL:** `type_traits_cpu` at `ggml/src/ggml-cpu/ggml-cpu.c:385-390` carries `.vec_dot = ggml_vec_dot_iq4_xs_q8_K, `.vec_dot_type = GGML_TYPE_Q8_K`, against `:379-384`'s `GGML_TYPE_Q8_0` for IQ4_NL — the 16-entry codebook is shared, the block geometry is not, and a 256-element super-block pairs with the 256-element activation encoding. **Gated BIT FOR BIT against the oracle's OWN KERNEL** on 4 super-blocks of `blk.11.ffn_down_exps.weight`, total and per block ([goldens](../tests/vt/iq2xs_iq4xs_dot_golden.h#L269), [case](../tests/vt/test_ops_quant_dot.cpp#L869)). These 3 tensors stop expanding from 3.59 GiB to 13.50 GiB; the whole-artifact measurement is in the `QUANT-GGUF-IQ2_XS` row above. **It also unblocks a SECOND consumer:** the expert-tower streaming lane is all-or-nothing (`GgufExpertTowersReachSlotLane` returns false on the FIRST tower that does not keep), so the 4 IQ4_XS tensors of the GLM-5.3 `UD-IQ1_S` arm were dropping an arm whose other 221 towers all kept — one tower 6.375 -> 24.000 GiB, and a 4096-slot cache 25.5 -> 96 GiB. Asserted in the `kStackedExpertWeight` role both models store these towers in ([lane case](../tests/vllm/model_executor/test_gguf_device_fit.cpp#L672)). **The CUDA arm now HAS a keep-quant kernel for this dtype too** ([#2260](https://github.com/mudler/vllm.cpp/issues/2260), spec [cuda-keepquant-iq2xs-iq4xs](specs/cuda-keepquant-iq2xs-iq4xs.md)): `src/vt/cuda/cuda_quant_dot.cu::DotIQ4XS` is a 1:1 port of `b10451 quants.c:1283` over the generated `src/vt/cuda/cuda_quant_iq_tables.cuh::d_kvalues_iq4nl`, keeping upstream's EIGHT f32 accumulation steps per super-block rather than one integer core, because that association is what the oracle's golden numbers were produced with. **This completes the device admission of the GLM-5.3 non-flash `UD-IQ1_S` arm**, whose other five encodings (IQ1_S, IQ3_XXS, IQ2_XXS, Q2_K, Q3_K) were already there. Before it, these 3 tensors fit `dgx:gpu0` with their expert GEMM on the CPU fallback behind a `cudaStreamSynchronize` and the fused `src/vt/cuda/cuda_quant_dot.cu::MoeGateUpSwiGLUGroupedCuda` seam throwing. The fit remains RESIDENCY and no speed number is claimed; O19's premise in [glm5-next-flash](specs/glm5-next-flash.md) is discharged, and what still blocks a CUDA run of either model is that model's own host-f32 forward. | [glm5-next-flash](specs/glm5-next-flash.md) | - | | `QUANT-GGUF-IQ1_M` | IQ1_M | 29 / output | - | - | - | - | - | `INVENTORIED` | - | leaf open | - | | `QUANT-GGUF-BF16` | BF16 | 30 / output | Y | - | - | - | - | `INVENTORIED` | reader `:261-264`; executable path absent | leaf open | - | | `QUANT-GGUF-TQ1_0` | TQ1_0 | 34 / output | - | - | - | - | - | `INVENTORIED` | - | leaf open | - | diff --git a/.agents/specs/cuda-quant-gather.md b/.agents/specs/cuda-quant-gather.md index 549fce5b69..c3bbe4ba6c 100644 --- a/.agents/specs/cuda-quant-gather.md +++ b/.agents/specs/cuda-quant-gather.md @@ -575,6 +575,9 @@ generalisation from ONE hidden issue and it is wrong. asserts a float table by name and none registers `kEmbeddingQuant`, so each answers false to the residency gate and keeps expand-bf16; the `qwen4_exp` loader refuses them by name. + The ROCm child is [BACKEND-ROCM-QUANT-GATHER](rocm-quant-gather.md), + tracked by [#3093](https://github.com/mudler/vllm.cpp/issues/3093). + Its implementation and native gates remain `PENDING`. - **Performance.** This wave gates CORRECTNESS only. The decoders read byte-wise for alignment safety and one thread decodes a whole block. No throughput number is claimed, measured or implied, and no benchmark ID moves. diff --git a/.agents/specs/rocm-quant-gather.md b/.agents/specs/rocm-quant-gather.md new file mode 100644 index 0000000000..cbcc333de4 --- /dev/null +++ b/.agents/specs/rocm-quant-gather.md @@ -0,0 +1,846 @@ +# ROCm quantized embedding gather + +Row: `BACKEND-ROCM-QUANT-GATHER`. + +Issue: [#3093](https://github.com/mudler/vllm.cpp/issues/3093). + +Parent debt: [#2394](https://github.com/mudler/vllm.cpp/issues/2394), owned by +[`MODEL-MM-QWEN4-EXP`](cuda-quant-gather.md#owed). + +Branch: `row/BACKEND-ROCM-QUANT-GATHER`. + +Spec base: `6db4bef906859e864c82523c01107473f7dcca29`. + +## Now + +`ACTIVE`. The native HIP gather and public compressed-embedding path are implemented. +The operator's final GPU run passed all 15 CTests and the required two-device case. +The public gate reaches all 19 formats. All 76 operation outputs +meet the pinned oracle gates, and matching native/primary traces pass. +The row uses one pull request and retains the reviewed full-attention prerequisite. + +The 32 original upstream fixtures were downloaded with the developer's authority +and verified at 289,655,872 bytes; the 160 exported tensors over the 16 primary +codecs pass the native-versus-plugin comparison (320 outputs, 140 byte-exact, +every remaining element inside the upstream tolerance). +The 96-request primary token matrix executed: 93 requests are token-identical, +and the three IQ3_S first-prompt cases are an oracle-side plugin grid defect +reported upstream as `vllm-project/vllm-gguf-plugin#129`. +The bounded secondary overlays execute all 12 model requests and match native tokens. +Both pristine and overlay binaries preserve all 24 recurrent-control completions and logits. +The amended native gate passes all 228 public engines with 16 physical blocks. +The live primary observer measures a 65,536-byte BF16 cache with the same physical capacity. +The comparator rejects unqualified captures, its guards are pinned by focused +negative cases registered in the preflight and CI lanes, and full preflights exit +zero at the amended heads. +The earlier four-block measurements and argument-dependent preflight skips remain archived. +The row is not ready to land or become `DONE` while the three IQ3_S cases wait on +the upstream grid correction and the secondary matched-memory claim stays scoped. + +The [row evidence](../../docs/bench-evidence/rocm-quant-gather/README.md) +records executing pins, exact recipes, stage hashes, rejected attempts, and limits. + +## Scope + +Register `vt::OpId::kEmbeddingQuant` for ROCm through a native HIP gather. +Decode selected packed rows directly into the caller's f32 or bf16 output. +Keep the shared loader's compressed embedding residency reachable through the +public C application binary interface (ABI). + +Cover every format accepted by +`src/vllm/model_executor/model_loader/gguf_keep_quant.cpp::KeepQuantGatherDType`. +Its device gate accepts no dtype argument. A partial ROCm decoder set therefore +cannot register the capability safely. + +This change owns correctness, memory residency, and native production reachability. +It does not depend on PR #2782. The existing ROCm quantized dot provider already +supplies the default `keep_quant` policy input. + +The following changes remain outside this row: + +- F16 GEMM, fused mixture-of-experts execution, and quantized dot providers. +- Floating embedding behavior, the public ABI, and the shared dtype set. +- CUDA decoder behavior or decoder ownership shared across backends. +- Qwen4-exp model completeness, other backends, and large-checkpoint qualification. +- Performance tuning, performance ratios, and global oracle pin changes. +- Global record counts, checker semantics, and unrelated keyed records. + +## Inventory + +This per-row inventory owns the child identity through the canonical spec scan. +The existing `BACKEND-ROCM` matrix lifecycle remains unchanged. +Update this inventory and `Now` together when this child changes state. + +| ID | Upstream source | Local anchor | Tests and evidence | Spec | State | Owner | Issue | +|---|---|---|---|---|---|---|---| +| `BACKEND-ROCM-QUANT-GATHER` | Plugin `_apply_gguf_embedding`, pinned codec sources | `vt::Embedding`, `KeepQuantGatherDType`, new ROCm gather | Focused and public gates in this spec | This file | `ACTIVE` | `row/BACKEND-ROCM-QUANT-GATHER` helper, operator verifies | #3093 | + +### Complete decoder set + +The geometry comes from `src/vt/dtype.cpp::FindBlockGeometry` and +`src/vt/cpu/cpu_quant_dequant.cpp::BlockToFloat` at the spec base. +The loader admits all 19 entries, including Q8_K without a weight dot kernel. + +| DType | GGML ID | Elements per block | Bytes per block | Executing reference | +|---|---:|---:|---:|---| +| Q4_0 | 2 | 32 | 18 | Primary plugin | +| Q5_0 | 6 | 32 | 22 | Primary plugin | +| Q8_0 | 8 | 32 | 34 | Primary plugin | +| Q2_K | 10 | 256 | 84 | Primary plugin | +| Q3_K | 11 | 256 | 110 | Primary plugin | +| Q4_K | 12 | 256 | 144 | Primary plugin | +| Q5_K | 13 | 256 | 176 | Primary plugin | +| Q6_K | 14 | 256 | 210 | Primary plugin | +| Q8_K | 15 | 256 | 292 | Stock llama.cpp decoder | +| IQ2_XXS | 16 | 256 | 66 | Primary plugin | +| IQ2_XS | 17 | 256 | 74 | Primary plugin | +| IQ3_XXS | 18 | 256 | 98 | Primary plugin | +| IQ1_S | 19 | 256 | 50 | Primary plugin | +| IQ4_NL | 20 | 32 | 18 | Primary plugin | +| IQ3_S | 21 | 256 | 110 | Primary plugin | +| IQ2_S | 22 | 256 | 82 | Primary plugin | +| IQ4_XS | 23 | 256 | 136 | Primary plugin | +| MXFP4 | 39 | 32 | 17 | Stock llama.cpp GET_ROWS | +| IQ1_XXXS | 66 | 256 | 38 | Registered unsloth fork GET_ROWS | + +The primary test includes IQ1_M, which the shared loader does not admit. +Its F16 output case exceeds `vt::Embedding`'s f32/bf16 output contract. +Record these two exclusions explicitly when reporting upstream test coverage. +They do not remove an admitted codec from this change. + +## Oracle pins and executing chain + +The active primary vLLM pin is `e126687a9a828d513c01a07cd69f025f27d63280`. +The plugin pin is `d4c1f0d082fc7cd4350da56689109a01c1f29d6c`. +Stock llama.cpp uses `10bf611e533d81f739128304991c5e133c6aebd8`, tag `b10451`. +The registered unsloth fork uses `36fe8e1cc7f2b3b8c92fdda0ab07600141921786`. + +Read [the primary pin](../upstream-sync.md), +[the plugin record](../oracles/vllm-gguf-plugin.md), +[the stock record](../oracles/llama-cpp.md), and +[the fork record](../oracles/llama-cpp-unsloth.md). +Do not promote a source inspection into runtime qualification. + +### Primary behavior and tests + +The plugin implements compressed-row selection followed by selected-row decode. +The earlier CUDA spec predates that current primary behavior source. +A fused HIP kernel adapts the implementation and preserves the observable result. + +Pinned plugin anchors: + +- `vllm_gguf_plugin/quantization/vocal_embeds.py::_apply_gguf_embedding`, lines 79 to 98, selects packed rows and dequantizes them. +- `vllm_gguf_plugin/quantization/vocal_embeds.py::GGUFEmbeddingMethod.embedding`, lines 186 to 194, passes `self.params_dtype`. +- `vllm_gguf_plugin/ops.py::ggml_dequantize`, lines 181 to 186, selects native or Triton dequantization. +- `tests/test_kernels.py::test_gguf_embedding`, lines 102 to 124, defines upstream fixtures and comparison parameters. + +The [pinned embedding implementation](https://github.com/vllm-project/vllm-gguf-plugin/blob/d4c1f0d082fc7cd4350da56689109a01c1f29d6c/vllm_gguf_plugin/quantization/vocal_embeds.py#L79) +and [pinned embedding test](https://github.com/vllm-project/vllm-gguf-plugin/blob/d4c1f0d082fc7cd4350da56689109a01c1f29d6c/tests/test_kernels.py#L102) +are the primary source anchors. +Read the executed native or Triton decoder after runtime dispatch resolves. +Record its source symbol, output dtype, and generated kernel when applicable. + +Run the primary method on identical packed tables and IDs for all 16 overlapping +formats, both output dtypes, and hidden widths 256 and 1024. +Preserve every upstream tensor, selected row, repeated ID, and tolerance. +The original comparison uses `atol=0.01` and `rtol=0.04`. +The shared ABI flattens upstream's two-dimensional IDs into one contiguous vector. +Restore the original shape only in the comparison harness. + +The existing active-pin runtime does not contain `vllm_gguf_plugin`. +Build the exact plugin pin in an isolated task directory when dependencies are available. +Do not change the existing runtime or install packages without the recorded authority. +A failed import, missing dependency, or missing fixture keeps this gate `PENDING`. +Do not replace primary outputs with local CPU outputs. + +Qualify the plugin on the bounded model used by this row on gfx1100. +Use the normal production configuration, with no eager-mode denominator override. +Capture resolved model dtype, model revision, plugin revision, and outputs. +If the primary cannot load the candidate model, record the exact refusal and +resolve the harness adaptation before accepting the primary gate. + +### Secondary formats + +Stock llama.cpp's `ggml/src/ggml-cpu/ops.cpp::ggml_compute_forward_get_rows_q` +routes quantized GET_ROWS through each type's `to_float` function. +`tests/test-backend-ops.cpp::test_get_rows`, starting at line 2247, defines the +stock test vehicle. Its case generation starts at lines 8365 to 8377. +Retain applicable row widths, IDs, repetitions, and f32 output comparisons. +Batched tables and strided ID views exceed this ABI and require explicit +adaptation entries plus local refusal tests. + +Q8_K is not selected by stock GET_ROWS dispatch at this pin. +Run the exact upstream `ggml/src/ggml-quants.c::dequantize_row_q8_K` over each +selected packed row. Record this as the Q8_K oracle harness adaptation. +Do not claim an upstream GET_ROWS execution for that format. +The stock pin must also run the bounded model with its supported quantized table. + +The fork registers IQ1_XXXS `to_float` in `ggml/src/ggml.c`, lines 871 to 877. +Its [pinned GET_ROWS dispatch](https://github.com/unslothai/llama.cpp/blob/36fe8e1cc7f2b3b8c92fdda0ab07600141921786/ggml/src/ggml-cpu/ops.cpp#L5063) +includes type 66 in `ggml_compute_forward_get_rows` at lines 5063 to 5069. +Build that exact fork and execute a real upstream GET_ROWS graph on identical bytes. +A new scalar transcription or local `BlockToFloat` cannot stand in for this graph. + +The fork must additionally load and generate from a bounded real model graph +whose token table uses IQ1_XXXS. A codec graph alone does not satisfy model execution. +The tiny qwen35 fixture described below is the first candidate. +Verify that the exact fork accepts its metadata and architecture before capture. +If it refuses, propose a compatible bounded model through the same public local path. +Do not silently replace the model run with a graph-only comparison. + +Record this qualification narrowly as a synthetic gather vehicle. +Keep issue #933 and the registry's real 2.4T qualification debt unchanged. +A synthetic run cannot establish that the published 2.4T checkpoint runs. +The IQ1_XXXS oracle gate remains `PENDING` until the bounded model actually runs. +The row cannot become `DONE` while that gate remains pending. + +## Fixture manifest + +The primary tests use `Isotr0py/test-gguf-sample` at revision +`d82b8773934ef260d8d8a896a7c197bc69a0fac1`. +The 32 applicable files total **289,655,872 bytes**. +The table records Hugging Face metadata read on 9 September 2026 UTC. +The SHA-256 values are advertised LFS object IDs, not locally verified downloads. +Verify each complete file's size and SHA-256 before using it. + +Keep the original full fixtures. Do not reconstruct unselected rows to avoid a download. +Use existing matching cached files when available. +The operator obtains the required download authority if those files are absent. +No fixture was downloaded during specification. +A missing authorization or missing fixture keeps the primary fixture gate `PENDING`. + +The download URL pattern is +`https://huggingface.co/Isotr0py/test-gguf-sample/resolve/d82b8773934ef260d8d8a896a7c197bc69a0fac1/`. + +| File | Bytes | SHA-256 | +|---|---:|---| +| `Quant_IQ1_S_1024.gguf` | 5581184 | `dea6372848ae8ba284481148ea282dde3c56d145eba5768f7d72a1527bf71a96` | +| `Quant_IQ1_S_256.gguf` | 1395584 | `52f0fba7e7f50821a343205c62948beef7f2320a52897ac197562ad975fe8ce7` | +| `Quant_IQ2_S_1024.gguf` | 9152896 | `8938259277b08682b596056ac63c8b347a6861ef3dfa7c8dd7f5466d26e4d6d6` | +| `Quant_IQ2_S_256.gguf` | 2288512 | `80207b4810ead2fb41ff8b42fe8a1c95d67626ed26f459b4d3e06cc4b78e4e17` | +| `Quant_IQ2_XS_1024.gguf` | 8259968 | `91871db5d358233b40a687de99d6d6fe15839dee4b37eeb3d98c1f8fbf8d43fc` | +| `Quant_IQ2_XS_256.gguf` | 2065280 | `7ca899750b6ae8902b4a4e219e153ff6c1d99f7d2074b76b47bd1cc13721607a` | +| `Quant_IQ2_XXS_1024.gguf` | 7367072 | `1946509f1c1ba47146b77645859b70e95f4ec0582e161d4a5693b860e5953fe8` | +| `Quant_IQ2_XXS_256.gguf` | 1842048 | `98cd99fab6415ec0115423bf91f51915febb06e4da9e6ef7e061a90fbff8659e` | +| `Quant_IQ3_S_1024.gguf` | 12278144 | `ed04c0b1929333bc25fcda2b77d630944690de08f1f194e2849a2f00702f30f3` | +| `Quant_IQ3_S_256.gguf` | 3069824 | `1852ac6ccb0e10c8dfe47b23034f57cb3efae4c38377243b40847064cac1d7cf` | +| `Quant_IQ3_XXS_1024.gguf` | 10938784 | `6eb74c3e749f758125ad6c2aae24481bd918e65703a099557109d18302f39d77` | +| `Quant_IQ3_XXS_256.gguf` | 2734976 | `0150e9a21ecbed3f47b1b1d61bcccd0c00815e6cd84c8ef7d9f10d2f76568b72` | +| `Quant_IQ4_NL_1024.gguf` | 16073088 | `05e0ca6956e3d036dd3d9edd055e8087e0ac6376c0032613498974759c752bd1` | +| `Quant_IQ4_NL_256.gguf` | 4018560 | `d3d86e735abf49c57e02e83ad6750ac9bf216a9a884a8eaf097aebcc0422540c` | +| `Quant_IQ4_XS_1024.gguf` | 15180160 | `5ef94e3cec7c9ff42d0fbc7705f425162fd3e00253dd05a62cc17e6bc85451c4` | +| `Quant_IQ4_XS_256.gguf` | 3795328 | `6039ca86d3c50cfc8db68b0394450ac1135affb8f07890af537865fb1dd5d23c` | +| `Quant_Q2_K_1024.gguf` | 9376128 | `8383805d71bf8252c30dbd59662dafb5ce6c29a71b8e74b349b708327c541703` | +| `Quant_Q2_K_256.gguf` | 2344320 | `a5f0df4b88193d1c5102d39113ae8ab86b23678ad5bbf459d05d1b88119ec084` | +| `Quant_Q3_K_1024.gguf` | 12278144 | `af666e306231f864fbd3fb7047f3980f058d070478a60b6790e1a823d773c406` | +| `Quant_Q3_K_256.gguf` | 3069824 | `1bc430759c9768783b08118323ec795bcffa62d9551933140a22f84c7135aade` | +| `Quant_Q4_0_1024.gguf` | 16073088 | `704bd822cde61be8437216089bc0240658869732316d7bbacf7e53de4f584797` | +| `Quant_Q4_0_256.gguf` | 4018560 | `380dd142b886bc7b537e9ba3569019560d68dd3aa13385b6c848121f0e33e23e` | +| `Quant_Q4_K_1024.gguf` | 16073088 | `00577c14c413f41d179dd11f5c92e0e3f1b9ed0c2ab410a0960f493db9d32c6b` | +| `Quant_Q4_K_256.gguf` | 4018560 | `c6256d86c59bc96db1eada611ec1fccb147eb3a738e3693cd2a68a497df77f53` | +| `Quant_Q5_0_1024.gguf` | 19644800 | `18251212453fcd01d04d7e7a5c2b847f48ec8a4c063a18d2a7c15122697354f7` | +| `Quant_Q5_0_256.gguf` | 4911488 | `8b8b03108ff423bc85d99ee03b99dad7138364c9a7fcac9da97b03c89696dd88` | +| `Quant_Q5_K_1024.gguf` | 19644800 | `e0af3999f1bb469be5c6b951b134d7bf95aa2fa8f65d8656e3acd9dec35de4ee` | +| `Quant_Q5_K_256.gguf` | 4911488 | `dabf970e3c496be2c512d8afbcbea22d9ba958867a7ed50a36a71e1fd523635e` | +| `Quant_Q6_K_1024.gguf` | 23439744 | `e7d173382ec11eaa6ae981e8600dce38196203f193433df95ab83944fb4508cc` | +| `Quant_Q6_K_256.gguf` | 5860224 | `b2c8f71efa6a27204ae1c111eb1cecf4617cd3a07be5e7be80fb28e6cbe23839` | +| `Quant_Q8_0_1024.gguf` | 30359936 | `4f7f1fbeade44af1cadcf57287eec4dcbadcaf7da650edd07e49a9f699007666` | +| `Quant_Q8_0_256.gguf` | 7590272 | `c27d7a1c1acd5b8176c320eb0f4a6bf0a23e53cfe460f6fc9b86b69422bc2b17` | + +## Design + +### Native decoder ownership + +Add independently owned ROCm decoder and codebook headers. +Port the decoder expressions and codebooks already used by the CUDA gather. +Cite both the local source and each pinned upstream definition beside the port. +Record the fused HIP launch as an adaptation of the plugin's two operations. +Do not include CUDA implementation headers from ROCm. + +Read scale fields and packed payloads through byte assembly. +Odd block sizes and offsets 0 to 3 must not create aligned typed-load assumptions. +Preserve scalar arithmetic order with the existing HIP `-ffp-contract=off` flag. +Match `vt::F32ToBF16` round-to-nearest-even conversion, including tie cases. +Do not add an f32 staging tensor to the bf16 gather path. + +Map each logical selected block to one HIP thread. +Use 128 threads per block and a grid capped at 1024 blocks initially. +Retain a grid-stride loop over the full selected-block count. +This bounded cap makes continuation directly testable with small fixtures. +The cap is a correctness-port default with no throughput claim. +Measure its cost before accepting a later performance default. + +Compute row stride in bytes with `RowSizeBytes`. +Use full-width arithmetic for IDs, offsets, and block counts. +Do not truncate i64 IDs before bounds validation. +Reject an invalid selected row before reading its packed bytes. + +Retain ROCm's synchronous bounds-error contract for this new operation. +Allocate a device error record, clear it, launch, copy the error, and report it. +Release every temporary allocation on success and on each failure path. +An empty request returns without a kernel or error-buffer allocation. +An empty vocabulary accepts no nonempty request. + +The shared `vt::Embedding` checks remain authoritative for ranks, shapes, +dtypes, contiguity, whole block rows, and device equality. +Keep floating gather dispatch unchanged. +Bind the native gather to `queue.device.index` before allocating its error +record or launching work. A different ambient HIP device must not redirect +the allocation, launch, or cleanup. The two-device regression keeps the queue +and tensors on device 0, selects ambient device 1, and checks rows, bounds, +and recovery. Fresh review removes this binding and requires that gate to fail. + +### Loader and production reachability + +The registration must activate the existing shared residency decision. +Do not add a device-name branch to the loader. + +The production chain is: + +```text +vllm_engine_load + -> LoadedEngine::FromModelDir + -> ModelRegistry::Load + -> LoadQwen3_5DenseModel + -> LoadQwen3_5DenseFromGguf + -> LoadEmbedAndHead + -> ModelRegistry::Forward + -> Qwen3_5EmbeddingTable + -> EmbedInto + -> vt::Embedding + -> kEmbeddingQuant ROCm provider +``` + +Relevant local anchors at the spec base: + +- `src/capi/vllm_c.cpp::vllm_engine_load`, line 824. +- `src/vllm/entrypoints/model_loader.cpp::LoadedEngine::FromModelDir`, line 3098. +- `src/vllm/model_executor/models/qwen3_5_dense.cpp::LoadQwen3_5DenseModel`, lines 87 and 147. +- `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp::LoadEmbedAndHead`, line 785. +- `src/vllm/model_executor/models/qwen3_5.cpp::EmbedInto`, line 8223. +- `src/vt/ops.cpp::Embedding`, line 1523. + +Use the existing GGUF builder to create a deterministic tiny qwen35 model. +Use one full-attention layer, hidden width 256, four query heads, and one KV head. +Use head width 64, intermediate width 256, and exactly 128 vocabulary entries. +Set the full-attention interval to 1, RMS epsilon to `1e-6`, and context length to 64. +Set RoPE base to `1000000.0` and rotary width to 64. +Use tokenizer IDs 0 for unknown, 1 for beginning, and 2 for end. +Create distinct vocabulary strings for all 128 IDs and preserve their order. +Use generator seed `0x524f434d` for every codec variant and dense control. +Use `std::mt19937` outputs with explicit integer-to-float conversion, without library distributions. +Initialize dense projection values within `[-1/64, 1/64]` and norm values to 1. +Generate finite packed blocks with the same seed and codec-specific valid scale fields. +Record the generation algorithm and verify identical GGUF hashes across two regenerations. +Quantize only `token_embd.weight`. Keep projections and a separate output head dense. +A separate head prevents a quantized dot dependency for Q8_K and IQ1_XXXS. +Keep every generated model under 8 MiB and record the exact content hashes. + +Run this same public path for all 19 admitted formats. +Load with public `device=0` in a HIP-only build and retain default compressed residency. +The ABI defines automatic selection as 0 and has no explicit ROCm device value. +Require measured ROCm provider selection before accepting the automatic selection. +Set `block_size=16`, `num_blocks=16`, `max_model_len=64`, and `max_num_seqs=1`. +Set `kv_cache_dtype="auto"` and require its resolved cache storage to be bf16. +The one-layer KV payload is 65,536 bytes, excluding allocation alignment and block metadata. +Match physical capacity and cache storage in each executing oracle. The primary +and native pools reserve one of their 16 blocks, leaving 240 usable cells. +Stock and fork llama.cpp pad the requested context of 64 to 256 cells. +Their cache is contiguous and has no equivalent native page-layout requirement. +Keep requested context, prompts, and generation lengths unchanged. Report each +engine's resolved capacity separately; do not call its logical limit identical. + +Use two pre-tokenized prompts: `[1, 0, 63, 127, 63]` and `[1, 127, 0, 127]`. +Their lengths are 5 and 4. Do not insert tokenizer special tokens in either oracle. +Generate exactly 4 tokens per prompt with greedy sampling and `ignore_eos=1`. +Set `has_seed=1`, `seed=0x524f434d`, and keep stop lists empty. +Run each prompt after a fresh engine load, and repeat the pair 3 times per codec. +Each run includes prefill and subsequent one-token decode through `vllm_complete_tokens`. +Capture logits through the public logits processor and record generated token IDs. +Require exact IDs and logits against the equivalent dense-embedding model on +the same ROCm binary. Dense control embeddings use the same bf16 conversion. +That control isolates embedding residency from unrelated CPU/GPU arithmetic. +The primary and secondary operation gates remain independently required. + +Compare all generated IDs exactly against the appropriate pinned model oracle. +Use the primary plugin for its 16 formats, stock llama.cpp for MXFP4, and +the registered fork for IQ1_XXXS. Require all 3 repeats to agree on each side. +Use identical model bytes, prompts, token counts, sampling, batching, and concurrency. +Record a load refusal instead of substituting local outputs or a different codec. +Q8_K retains its explicit upstream decoder-only adaptation because stock GET_ROWS excludes it. +Its public model still requires exact dense-control equality and independent upstream decoder bytes. +Do not describe that Q8_K control as a pinned-oracle model token result. + +Enable `vt::EnableOpProviderCallStats` around the public calls. +Require positive ROCm `kEmbeddingQuant` selections for every codec. +Require no provider fallback and unchanged `vt::GetReferenceTierHits`. +A test that constructs the embedding type by hand cannot satisfy this gate. +No new test-only production entry point is allowed. + +### Memory contract + +The resident packed table occupies `V * RowSizeBytes(dtype, H)` bytes. +The selected output occupies `T * H * sizeof(output_dtype)` bytes. +For the model path, the output dtype is bf16 and costs `2 * T * H` bytes. +The primitive's f32 output remains an explicit caller-selected test mode. +The bounds-error record is constant-size scratch. + +Capture table dtype, packed bytes, output dtype, output bytes, and scratch bytes. +Compare those values with the primary plugin on the identical tables and IDs. +The primary stages selected packed rows before decode. The fused HIP path does +not need that intermediate and must record this implementation adaptation. +Neither implementation requires full-table floating expansion for a gather. + +Trace both implementations with the same available tool and workload. +Verify selected-row decode, no full-table expansion, and no host decode. +Provider counts alone cannot detect a manually coded host fallback. +A missing profiler keeps the trace gate `PENDING` until the operator supplies one. +Do not install a profiler without the recorded authority. + +## Implementation files + +The spec commit changes only this spec, the child issue record, and the +parent debt link. + +The implementation is expected to add: + +- `src/vt/rocm/rocm_embedding_quant.h` +- `src/vt/rocm/rocm_embedding_quant.hip` +- `src/vt/rocm/rocm_quant_dequant.h` +- `src/vt/rocm/rocm_quant_iq_tables.h` +- `tests/vt/test_rocm_embedding_quant.cpp` +- `tests/capi/test_rocm_embedding_quant.cpp` +- Row-owned oracle capture and comparison tools when required by the runtime gates. +- Row-owned evidence under `docs/bench-evidence/rocm-quant-gather/`. + +The implementation can modify: + +- `src/vt/rocm/rocm_ops.hip` for the registration. +- Both HIP source lists in `CMakeLists.txt`. +- `tests/CMakeLists.txt` for targets and real exit-77 skip behavior. +- `tests/vllm/test_gguf_keep_quant.cpp` for the changed ROCm capability expectation. +- `tests/vt/test_cuda_embedding_quant.cpp` for the combined-build unavailable set. +- `src/vllm/model_executor/model_loader/gguf_keep_quant.cpp` for stale gather comments. +- `docs/FEATURES.md` for the changed ROCm feature statement. +- This spec, its issue record, and the already linked parent debt entry. + +The scoped materializer amendment below authorizes two shared loader files. +Other shared model behavior and unrelated production code require a scope +review before implementation. + +### Q8_K materializer amendment + +The operator approved this amendment on 9 September 2026 UTC under issue #3093. +Commit this amendment before changing either newly scoped product file: + +- `src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp` +- `include/vllm/model_executor/models/qwen3_5_gguf_weights.h` + +The first native focused run passed 4 of 6 targets. The public test reached +Q8_K after the first 8 formats matched their dense controls. Its first Q8_K +load returned status 2, without a readable error capture. The native decoder suite separately passed all 11,698 +value assertions, but its empty-shape fixture failed during tensor construction. +That malformed test fixture requires a metadata view with zero dimensions. + +`LoadEmbedAndHead` already selects `KeepQuantGatherDType` through the shared +residency policy. It then calls `OwnGgufQuantBlocks`, whose guard at line 89 +requires `KeepQuantDType`. The latter requires a dot kernel and refuses Q8_K. +This downstream guard would refuse the decoder-only format after the reader +accepts it. That materializer failure was a source hypothesis at this amendment. +`git log -S 'VT_CHECK(KeepQuantDType(tensor.ggml_type'` identifies commit +`429e19d6a` as the guard's introduction. + +Add an explicit tensor role to `OwnGgufQuantBlocks`. Default the role to +`kMatmulWeight`, preserving the dot requirement for every existing caller. +The embedding materializer explicitly selects `kEmbeddingTable`, which checks +`KeepQuantGatherDType` instead. Preserve packed bytes, orientation, borrowing, +copying, prefaulting, and every existing matrix repack default. A gather must +not opt into a matrix repack. Do not change either shared dtype set, add a +dot kernel, or change any backend capability outside this row. + +Add a smallest loader test in `tests/vllm/test_gguf_keep_quant.cpp` using the +same generated Q8_K model. Require the ordinary loader to retain the original +packed embedding bytes and `nk=false`. Require the default materializer to +refuse that decoder-only format and the matrix residency route to expand it. +Keep the public Q8_K load and completion test as the production gate. +Fresh review removes the embedding role argument and weakens the default dot +guard separately. Each mutation must fail its corresponding gate. + +The operator's first native run used the manifest +`native-focused-v1.json`, SHA256 +`496a12c417b9cb95036896da418d74fd8383fc6d1497f0ed94f4e457791bf479`. +Its log is `native-focused-v1.log`, SHA256 +`2f21277ec55c85cf6cd4abbb669a688bdfab415de13c9bcd0588e6a7854787ac`. +Both files reside under the row's external evidence directory +`/home/vikash/.cache/rdna3-gather-impl/evidence/`. +The operator verified unchanged source, archive, and executable hashes around +the run. The Q8_K public load had 1,108 passing assertions before its failure. + +### Q8_K reader amendment + +The new loader-focused red identifies the first executing refusal: +`GgufFile::OpenOne` rejects GGML type 15 at `gguf_reader.cpp:506`, because +`FindGgmlTraits` has no entry for it. The result is exit 1 after one passing +fixture assertion. The log is `materializer-loader-red.log` in the same +external evidence directory. The materializer guard remains a separate +downstream hypothesis until the reader accepts the file. + +The operator approved this additional scope on 9 September 2026 UTC: +`src/vllm/model_executor/model_loader/gguf_reader.cpp`. Commit this amendment +before changing that file. Add exactly the existing Q8_K geometry for GGML +type 15: 256 elements and 292 bytes per block. Stock llama.cpp pin +`10bf611e533d81f739128304991c5e133c6aebd8`, `ggml/src/ggml-common.h:370-376`, +defines `block_q8_K` as a float delta, 256 signed bytes, and 16 signed 16-bit +sums. This is already the geometry in `vt::DType::kQ8_K`. + +Keep all other traits byte-for-byte equal. Preserve unknown-type refusals, +overflow checks, span checks, and whole-block divisibility. This amendment +does not add a shared dtype or change a matrix dtype set. Add the reader +regression in `tests/vllm/test_gguf_keep_quant.cpp`, checking geometry and +original packed bytes through `GgufFile::Open`. Fresh review deletes the +type-15 entry and mis-sizes its geometry independently. Both mutations must +fail a meaningful gate. After adding the trait, rerun the loader-focused red +to identify the materializer failure before changing that guard. + +### Bounded oracle configuration and allocation amendment + +The operator approved this scope on 9 September 2026 UTC under issue #3093. +Commit this amendment before a fresh implementer changes the capture harness. +The original generated GGUF files, 19 codecs, projections, prompts, sampling, +three repeats, four generated tokens, and logical native/primary limit remain fixed. +The amendment changes configuration and unused-input allocation bookkeeping. +It authorizes no decoder, model arithmetic, tokenizer, or shared loader repair. + +#### Primary configuration + +The plugin's `weights_adapter/qwen3_5.py:38-42` maps the text model type to +`Qwen3_5ForConditionalGeneration`. Its `config_parser.py:43-53` and +`weights_adapter/qwen3_5.py:225` restore that architecture during configuration +and loading. Overriding the architecture field alone cannot survive that chain. +Use the pinned vLLM `model_class_overrides` argument with this exact mapping: + +```python +{"Qwen3_5ForConditionalGeneration": + "vllm.model_executor.models.qwen3_5:Qwen3_5ForCausalLM"} +``` + +At the active pin, `vllm/config/model.py:319-325` defines this development +argument and lines 1032-1055 apply it in the front end and workers. +`vllm/model_executor/models/registry.py:202` already registers that text class. +Its definition in `models/qwen3_5.py:325-451` uses the same Qwen3.5 model, +forward path, and weight loader. Record the development argument as a harness +adaptation. Preserve the default compiler, graph capture, and kernel dispatch. +Do not add a vision configuration or change the oracle source. + +Pass `hf_overrides={"partial_rotary_factor": 1.0}`. The unchanged GGUF requests +rotary width 64 with head width 64 and sections `[16, 8, 8, 0]`. +The pinned `vllm/transformers_utils/configs/qwen3_5.py:86` defaults the top-level +factor to 0.25. The executing Transformers `modeling_rope_utils.py:787-788` +copies that factor over the nested value during normalization. The original +configuration therefore resolves rotary width 16 and fails its section check. +The explicit top-level factor preserves the existing model geometry. +The retained CPU probe demonstrates both resolutions using the executing image. + +#### Physical cache capacity + +The pinned `vllm/v1/core/kv_cache_utils.py:2288-2309` allocates the requested +physical blocks and subtracts one null block for admission. Four blocks of 16 +leave 48 usable tokens and cannot admit `max_model_len=64`. +The v4 primary attempt proves this refusal after successful model loading, +normal compilation, and graph capture. The secondary pins independently pad +context to 256 in `src/llama-context.cpp:288` for stock and line 285 for the fork. + +Use 16 physical blocks of 16 in the native public test and primary harness. +This matches the secondary physical payload of +`256 * 1 * (64 + 64) * 2 = 65,536` bytes for BF16 K and V. +Native `LoadedEngine::ResolveNumBlocks` accepts explicit block counts; +`vllm::v1::BlockPool` reserves the null block. No product change is needed for 16 blocks. +Keep the earlier four-block measurements with their original values and mark +them superseded for matched oracle memory. Do not relabel or delete their traces. +The existing issue #2719, owned by `KV-SIZING`, tracks the separate native +admission gap. This row neither repairs that gap nor claims it closed. + +The operator's v5 Q4_0 attempt exits 0 and emits `[47, 19, 4, 20]` for the first +prompt and repeat. The unchanged GGUF SHA256 is +`663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af`. +Its resolved dtype is BF16 and its cache reports 16 blocks and 256 cells. +The operator verified 2,500 inputs before and after the run. This proves one +bounded execution. It does not prove the remaining formats, repeats, token +equality, or observed cache tensor layout and allocation bytes. + +#### Secondary allocation overlay + +Keep pristine source trees and binaries for both recorded secondary pins. +The stock and fork create an unused recurrent copy input for this one-layer, +full-attention model. Stock `src/llama-graph.cpp:3397-3416` creates `s_copy`; +lines 3481-3489 attach hybrid inputs even though no recurrent layer consumes it. +The fork has the corresponding functions at lines 3379 and 3463. +The graph allocator allocates graph leaves and nodes. Its input setter later +writes the unallocated tensor. Stock `src/llama-context.cpp:1379-1380` explicitly +documents this unused-input failure; the existing model logs reach its buffer +assertion on the first decode. The allocation diagnosis still requires a runtime +red/green witness that identifies `s_copy`. + +Build separate row-owned overlay copies of the exact pins. In +`llm_graph_context::build_inp_mem_hybrid`, retain the recurrent copy input as a +graph leaf with `ggml_build_forward_expand(gf, inp_rs->s_copy)` after creation. +This scopes one allocation repair to each pinned `src/llama-graph.cpp`. +Keep the buffer assertion and all existing input writes. A blanket null-buffer +skip could hide a required recurrent input and is not an admissible repair. +Do not change weights, codec arithmetic, graph computation, or model topology. + +Qualify this overlay only for the bounded synthetic model in this row. +Record pristine pin, exact patch, patched-source digest, build flags, executable +and linked-library digests, model hashes, and runtime outputs. Describe every +model result as the pinned secondary plus that named overlay. The independent +codec gates continue to execute pristine binaries and must preserve their output +hashes. No global oracle pin or gateability record changes. Issue #933 still +owns the published 2.4T fork qualification. + +Capture the pristine model failure and overlay success on identical inputs. +Run both prompts three times through fresh engines for stock MXFP4 and fork +IQ1_XXXS. Require exact token equality to native and repeated-run agreement. +Run a control with an actual recurrent layer on each pristine and overlay +binary; require identical tokens and logits. A controlled hybrid-input test may +exercise recurrent state instead if no suitable model is available, but it must +consume the recurrent copy tensor and verify state values through graph reuse. +No missing control can become a source-only pass. A fresh reviewer removes the +retained leaf and requires the bounded-model failure to return. Independently +verify the unchanged recurrent path and pristine codec output hashes. + +#### Scoped implementation and verification + +The fresh implementer may change `tools/rocm_quant_gather/primary_model.py`, +`tools/rocm_quant_gather/secondary.cpp`, and row-owned capture, patch, comparison, +and evidence files. They may set 16 blocks in +`tests/capi/test_rocm_embedding_quant.cpp`, preserving its public ABI entry and +all fixture bytes. An additional public-ABI capture runner is allowed if needed; +it must use `include/vllm.h` and preserve that test's complete workload. +No shared product file or checker semantics enters this amendment. + +Retain meaningful failing primary and secondary launches, then capture focused +green with these corrections. Rerun all 19 public formats and dense controls at +16 blocks. Compare all 16 primary model formats, stock MXFP4, and fork IQ1_XXXS +on both prompts and all repeats. The Q8_K decoder-only adaptation remains exact. +Measure native and oracle model memory with the same tool, recording actual +cache dtype, physical payload, layout, reserved capacity, and allocation overhead +separately. A config string alone cannot establish this memory gate. +Fresh review removes the registry and RoPE corrections independently, checks +that incorrect cache capacity fails comparison, and mutates the secondary leaf +as described above. Run focused and full gates, obtain PASS review on an +immutable head, and have the operator rerun the final gates. + +The [amendment evidence](../../docs/bench-evidence/rocm-quant-gather/oracle-amendment/README.md) +retains the primary launch progression, CPU RoPE probe, receipts, and hashes. +The [qualification evidence](../../docs/bench-evidence/rocm-quant-gather/oracle-qualification/README.md) +retains the implementation's runtime failures, secondary overlays, recurrent controls, +live primary cache measurements, and matched 16-block native gates. + +## Tests and gates + +### Primary report qualification amendment + +Issue [#3113](https://github.com/mudler/vllm.cpp/issues/3113) owns the comparator repair from fresh review of `75d92614e84ff428eab49324f3b4fb2aba183ac3`. +Commit this amendment before changing the comparator or adding its regression tests. +The scope is `tools/rocm_quant_gather/compare_models.py`, focused Python tests, +and this row's evidence. Model bytes, workload, pins, and product code remain fixed. + +The token matrix and separate memory capture must pass the same qualification. +Require the exact successful status and reject any exception or traceback field. +Require both recorded oracle pins and every recorded runtime identity: +vLLM `0.28.1rc1.dev132+ge126687a9`, plugin `0.0.5+d4c1f0d.gfx1100`, +Torch `2.12.0+git6bbd260`, Torch revision `6bbd26020da1c6dc198625dfcdd968b1e4e6b1c5`, +and HIP `7.2.53211`. These values come from the sealed successful captures. +They qualify this local runtime and do not advance any global oracle pin. + +Require identical model and configuration seals, prompt, repeat, resolved BF16, +and every requested field, including automatic model and KV cache dtypes. +Reject missing fields and incorrect values in either capture. +Require a zero cache storage offset and verify that the complete strided view +fits its storage before checking the existing live allocator bounds. +Require the same view and storage metadata before and after generation. + +Preserve the actual successful Q4_0 token and memory reports as positive controls. +Synthetic validator cases must be labeled as tests, never as oracle measurements. +First reproduce the post-generation exception acceptance through `compare`. +Then test missing and incorrect qualification fields on both report paths, +including independent exception fields and invalid storage offsets and bounds. +Mutate each new guard and each qualification call site, require focused failure, +and restore the source byte-for-byte. Run the focused suite and full preflight. +The operator owns the complete actual matrix comparison, GPU gates, and publication. + +### Red-first implementation gate + +Add the public production test before the native provider. +Confirm that compressed-gather selection is absent on the base and capture the red result. +Matching dense outputs alone must not pass the test. +Then add decoder tests, implement the provider, and get focused green. + +The decoder suite must cover: + +- All 19 formats, with both i32 and i64 IDs and both f32 and bf16 outputs. +- One-, three-, and five-block rows, plus primary widths 256 and 1024. +- First, middle, and last rows, repeated IDs, and reversed row order. +- Explicit packed-table offsets 0 to 3, including odd block byte sizes. +- Multiple launch blocks and a selected-block count greater than `1024 * 128`. +- Exact f32 bytes and bf16 words against scalar controls and committed oracle vectors. +- Finite random block cases with deterministic seeds and recorded block bytes. +- Negative IDs, `id == vocab`, and i64 values that would wrap into valid i32 IDs. +- Successful execution immediately after a bounds-error exception. +- Empty requests, empty vocabulary, ragged widths, and invalid ranks and shapes. +- Invalid table, ID, and output dtypes, noncontiguous views, and device mismatch. + +The decoder-set test must enumerate through the actual last enum `kIQ3_S`. +Do not copy the older CUDA loop endpoint `kIQ4_XS`. +Keep the committed vectors for IQ4_NL, Q5_0, IQ1_S, IQ1_XXXS, IQ2_XS, +IQ4_XS, and IQ3_S in the regression set. +The upstream runtime comparisons supply independent evidence for every format. + +### Focused native command + +Resolve `.env` and the local compute preference before any GPU command. +The operator owns GPU scheduling and holds the recorded mutex for this local host. +Fleet devices require their lease if the task moves to one. +Build only in this task's ignored build directory, with at most four compile jobs. + +```sh +cmake -S . -B build-rocm-quant-gather -G Ninja \ + -DCMAKE_BUILD_TYPE=Release \ + -DVLLM_CPP_CUDA=OFF \ + -DVLLM_CPP_HIP=ON \ + -DVLLM_CPP_HIP_ARCHITECTURES=gfx1100 \ + -DVLLM_CPP_BUILD_TESTS=ON +cmake --build build-rocm-quant-gather -j 4 +ctest --test-dir build-rocm-quant-gather --output-on-failure \ + -R '^(test_rocm_embedding_quant|test_capi_rocm_embedding_quant|test_gguf_keep_quant|test_ops_embedding_quant|test_gguf_dequant|test_backend_cross_device)$' +``` + +The final implementation must make these target names and commands accurate. +A missing GPU can produce an honest exit-77 skip in generic CI. +The acceptance run on gfx1100 requires execution and cannot pass with a skip. +Run the existing ROCm regressions and the full applicable preflight after focused green. +Run CPU-only configuration checks to verify that additive HIP files remain gated. +Run the combined CUDA/HIP expectation tests when that build is available. +A missing combined backend resource remains explicitly `PENDING` if applicable. + +### Fresh review mutations + +A fresh reviewer reviews the immutable implementation head. +Use a scratch copy and restore every file byte-for-byte after each mutation. +Require the focused test to fail for the intended reason after each mutation. + +1. Remove the ROCm registration. +2. Delete the production `vt::Embedding` call in `EmbedInto`. +3. Force the loader's embedding residency back to expanded bf16. +4. Corrupt each of the 19 decoders independently through scale, payload, or codebook logic. +5. Replace the byte row stride with an element stride. +6. Truncate i64 IDs to i32 before bounds validation. +7. Replace bf16 rounding with truncation. +8. Remove the bounds-error record or its reporting. +9. Remove grid-stride continuation. +10. Remove the reader's Q8_K trait, then independently mis-size that trait. +11. Remove the embedding materializer role and independently weaken its default dot guard. +12. Permit a matrix repack for the embedding role. +13. Remove the queue-device binding before the error allocation and launch. + +The first three mutations must fail the public production test. +The remaining mutations must fail their named focused cases. +Record mutation diffs, commands, exit codes, failure excerpts, and restoration hashes. +A fresh implementer repairs each finding. The coordinator does not repair findings. +The operator reruns the final focused and full gates itself after a PASS review. + +### Acceptance results + +Report exactly one state per applicable obligation: satisfied, narrowly waived, +pending a named authority or resource, or failing. +The current result inventory is: + +| Obligation | Result | Authority or remaining evidence | +|---|---|---| +| Committed spec before implementation | Satisfied | Original spec and materializer/reader amendments precede product edits; bounded-oracle amendment precedes its harness implementation | +| All 19 native decoder and error contracts | Satisfied on sealed native v4 | Nine cases and 11,784 assertions; required two-device case passes 16 assertions | +| All 19 public production paths | Satisfied | Sealed matched16 run passes 228 completions and 2,850 assertions at 16 physical blocks | +| Pinned-oracle model token equality | Pending a named upstream authority | The 96-request primary matrix executed; 93 requests are token-identical to native. The three IQ3_S first-prompt cases are oracle-side: the pinned plugin's IQ3_S grid decodes 62 where the format-defining llama.cpp grid has 15 (150 of 2048 byte positions, +3.333%), native is bit-exact with llama.cpp, and a CPU substitution experiment reproduces the plugin tokens exactly. Reported upstream as `vllm-project/vllm-gguf-plugin#129`; the row cannot pass those three cases before that grid is corrected. All 12 stock MXFP4 and fork IQ1_XXXS requests match native. | +| Primary plugin operation execution | Satisfied | Exact native plugin, 64 synthetic outputs; original tolerances unchanged | +| Primary bounded-model execution | Satisfied | Operator v5 emits four tokens from the exact Q4_0 fixture with normal compilation and graph capture; the full 96-request primary matrix executed and its comparison is recorded (93 token-identical, the three IQ3_S cases classified above) | +| Original primary fixture coverage | Satisfied | The 32 fixtures were downloaded with the developer's authority and sha256-verified at 289,655,872 bytes (`/home/vikash/models/test-gguf-sample`, receipt `gather-3113-repair/fixture-download-receipt.json`). `export-upstream` produced 160 tensors over the 16 primary codecs; native versus pinned-plugin comparison passes 320 outputs with 140 byte-exact and every remaining element inside the upstream `atol=0.01 rtol=0.04` (`original-fixture-coverage/compare/report.json`). IQ1_M and the F16 output case remain the documented exclusions; Q8_K, MXFP4, and IQ1_XXXS are secondary codecs outside the 32-file set and keep their separate evidence. | +| Stock secondary operation execution | Satisfied | Q8_K direct pinned decoder and MXFP4 real GET_ROWS, including applicable stock geometry | +| Stock secondary bounded model | Satisfied with bounded overlay | Six MXFP4 requests match native; pristine/overlay recurrent tokens and logits remain exact | +| IQ1_XXXS fork operation execution | Satisfied | Real pinned fork GET_ROWS; exact outputs and applicable stock geometry | +| IQ1_XXXS fork bounded model | Satisfied with bounded overlay | Six IQ1_XXXS requests match native; pristine/overlay recurrent tokens and logits remain exact | +| Operation memory format and identical-tool traces | Satisfied | Identical rocprofv3 1.3.5; native allocation events and separate primary tensor telemetry | +| Native model memory format | Satisfied at amended cache capacity | All 228 runtime allocations report BF16, 16 physical blocks, and 65,536-byte payload; independent model traces remain required below | +| Matched oracle model memory | Narrowly waived, scoped | The native and primary pair both measure a 65,536-byte BF16 cache (16 physical blocks of 16 cells); the live primary tensor and its allocator block agree. The secondary llama.cpp oracles pad the requested 64-cell context to that runtime's own 256-cell minimum, so the matched-memory claim is scoped to the native/primary pair, and secondary model executions are token/logit comparisons that make no memory claim. Identical-tool backing-allocation measurement of the secondary runtime remains outside this claim. | +| CPU regressions | Satisfied | Amended CPU build passes six tests and explicitly skips two ROCm-only tests | +| ROCm regressions | Satisfied | Clean HIP build; all 15 CTests and required two-device case pass in sealed matched16 run | +| Applicable local preflight | Satisfied | Full preflights exit 0 at the amended heads `fc0fae3ff` (gather), `6607faea2` (F16), `6fd1650c4` (BF16 MoE), and the shared checkout, each with its argument-dependent skips recorded separately. | +| Combined CUDA/HIP expectation execution | PENDING | No combined build was available for this qualification | +| Fresh mutation review | Satisfied for the comparator chain | Fresh review of `c5bb0f1f7..92813f4b` returned PASS with two findings for the repair loop and no blocking defects; the guard-pinning repair `fc0fae3ff` closed them and a second fresh review returned PASS with no findings. Both reports are retained. The first is `/home/vikash/.cache/rdna3-gather-oracle-review/review-92813f4b/REVIEW.md`, sha256 `968f862b7b62dcc7c0a80ddcdb9d39b7330a651046db233c818ac9ec818cced7`, with the reviewing session's scratch artifacts beside it. The second is `/home/vikash/.cache/gather-3113-repair/review-fc0fae3ff/REVIEW.md`, sha256 `a8fc4589d96cd936bb7f48764c955d83c48c80d1e64677e54b043eb347fe2998`, with that session's four guard-removal logs and `pristine.sha256` beside it. Each file is an operator transcription of its reviewing session's final report, not a file the reviewing session wrote itself. | +| Operator verification | Satisfied for the comparison and preflight | The operator ran the 96-request token comparison, the comparator's guard-removal red receipt, the original-fixture coverage comparison, and the full preflight at `fc0fae3ff`. | + +Performance is not an acceptance claim in this correctness row. +No benchmark ID, throughput ratio, latency claim, or performance ceiling changes. +This exclusion does not discharge the parent row's performance parity debt. +A future performance claim requires identical workloads, correctness first, and +measured memory, latency, and throughput axes under the repository gates. + +## Evidence + +The spec author inspected the code, owning records, current issues, pull requests, +and symbol history before claiming implementation scope. +At the inspected base, ROCm registers floating embedding but no quantized gather. +The existing quantized dot registration already supplies the loader policy default. + +Source inspection on 9 September 2026 UTC verified these local read-only checkouts: + +- Stock source: `/home/vikash/oracle/llama.cpp-b10451-clean`, HEAD `10bf611e533d81f739128304991c5e133c6aebd8`. +- Active vLLM source: `/home/vikash/oracle/gfx1100-active-2773/source`, HEAD `e126687a9a828d513c01a07cd69f025f27d63280`. + +These paths describe measured local evidence. They are not defaults for another host. +Resolve every execution path from the operator's supplied environment. + +The baseline `scripts/agent-preflight.sh --quiet` exited 0 on the spec base. +It reported no failures and five argument-dependent skips. These were +`check-arm-isa-build.py`, `check-cpu-isa-build.py`, `check-cuda-fat-gencode.py`, +`check-pr-size.py`, and `check-triton-aot-multiarch.py`. +This result is not an all-green preflight. The spec changes no build or product file. +The implementation's final staged preflight exits 0 with no failures and the same five argument-dependent skips. +The explicit CPU ISA check passes. Final commit classification accompanies the immutable handoff. +Implementation evidence must add exact source and binary revisions, build commands, +model and fixture hashes, environment, mutex identity, contention, and exit codes. +Keep oracle outputs and restoration hashes in this row's evidence directory. + +Implementation captures, matching operation traces, oracle output archives, and +remaining qualification failures are retained in the linked row evidence. +The first public red was invalidated by the separate full-attention state defect; +only the subsequent zero-provider-selection result is the intended implementation red. +Q8_K reader and materializer refusals were then captured in their executing order. +The queue-device test captured invalid stream use before the binding repair. +Do not commit downloaded sample GGUF files. + +## Risks and stop conditions + +The registration changes compressed residency for every admitted embedding codec. +A missing decoder therefore affects model loading beyond the test's first format. +Odd byte strides, bf16 ties, and i64 bounds require independent mutation evidence. +A provider counter can prove dispatch and still miss hidden host decode. +The memory trace closes that evidence gap. + +Return `NEEDS_CONTEXT` when a required path, fixture, pin, or authority is unavailable. +Keep the associated gate `PENDING` and continue independent authorized work. +Return `NEEDS_DECISION` for a required scope or model-harness change. +Stop acceptance if the oracle cannot run, a codec remains untested, or production +mutation evidence does not fail for the intended reason. +Do not weaken a tolerance or remove a format after observing a failure. +Do not mark the row `DONE` from partial gates or source feasibility. + +## Owed + +Issue #3093 owns this row's remaining implementation and verification gates. +Parent issue #2394 retains the other backend gather arms and closes only when +its remaining scope is discharged. +Issue #933 retains real 2.4T fork qualification, independent of the bounded +synthetic vehicle in this row. + +Add an `Outcome` section only when this row reaches `DONE`. +Record measured results, rejected alternatives, and the reasons for final defaults. diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index 85fdb7998d..b575006540 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -542,6 +542,7 @@ jobs: sudo DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends python3-numpy python3 tests/scripts/test_qwen3_capture_tools.py python3 tests/scripts/test_qwen3_capture_outputs.py + python3 tests/scripts/test_rocm_quant_gather_compare.py python3 tests/scripts/test_ltx25_render_compare.py && python3 tests/scripts/test_ltx25_absolute_reference.py # The prompt-adherence half of the same tool (#2295, owning #1854's # first sub-question). 42 of its 47 cases need numpy only; the five diff --git a/CMakeLists.txt b/CMakeLists.txt index 8cf0b95679..5c3223ac12 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -1717,6 +1717,7 @@ if(VLLM_CPP_HIP) src/vt/rocm/rocm_backend.hip src/vt/rocm/rocm_rmsnorm.hip src/vt/rocm/rocm_embedding.hip + src/vt/rocm/rocm_embedding_quant.hip src/vt/rocm/rocm_dense_basic.hip src/vt/rocm/rocm_matmul_hipblaslt.hip src/vt/rocm/rocm_paged_attn.hip @@ -1748,6 +1749,7 @@ if(VLLM_CPP_HIP) src/vt/rocm/rocm_backend.hip src/vt/rocm/rocm_rmsnorm.hip src/vt/rocm/rocm_embedding.hip + src/vt/rocm/rocm_embedding_quant.hip src/vt/rocm/rocm_dense_basic.hip src/vt/rocm/rocm_matmul_hipblaslt.hip src/vt/rocm/rocm_paged_attn.hip diff --git a/docs/FEATURES.md b/docs/FEATURES.md index c434595b00..733ff25f38 100644 --- a/docs/FEATURES.md +++ b/docs/FEATURES.md @@ -109,7 +109,7 @@ by #2794 (goldens predate the pin) and #2817 (the advance). | NVFP4 shared-expert `down_proj` kept bf16 (no f32 round-trip) | ✅ `VT_SHARED_DOWN_BF16` default-ON; bit-identical (both consumers widen bf16 in-kernel and re-round on store), SACRED 315/315 + 235/235 on BOTH arms with unchanged assertion counts; +2.05% c8 / +0.79% c4 on 35B-A3B | ☐ | ☐ | ☐ | | NVFP4 `lm_head` kept packed (no dequant at load) | ✅ `VT_LMHEAD_FP4` default-ON, #213; CUDA-gated on `nvidia`@`0893e160` (continuations byte-identical packed vs dequant, 235/235; RSS -1.70 GiB on CUDA, owed a re-measure; a no-fp4-GEMM backend keeps one bf16 operand too) | ✅ | ☐ | ☐ | | GGUF k-quants and i-quants | ✅ (CPU grouped keep-quant MoE bf16 regression in `b4f5610a` fixed 2026-08-06). **CPU quant compute is ISA-tiered:** Arm has i8mm + repack; x86_64 portable-only, MEASURED open on every axis (CIQ `G5`, #433). **IQ4_NL (20) and Q5_0 (6)** added for `qwen4exp` (#1989), decode bit-exact vs llama.cpp `b10451`; they are the ragged-K landing spots of upstream's own `tensor_type_fallback` (`IQ4_XS -> IQ4_NL`, `Q4_K -> Q5_0`, `llama-quant.cpp:374`). **IQ2_XS (17) and IQ4_XS (23)** added for the staged `unsloth/GLM-5.3-Flash-GGUF` UD-Q2_K_XL arm (#2240), decode bit-exact vs llama.cpp `b10451` over REAL bytes from that checkpoint; "UD-Q2_K_XL" names a target average and not a format, and 82 of that file's 1412 tensors are IQ2_XS against two that are Q2_K. Both keep their blocks on the GEMM arm too as of #2247, which added their keep-quant `vec_dot` rows against the Q8_K activation — IQ4_XS pairs with Q8_K and not with the Q8_0 of its codebook sibling IQ4_NL, because its block is a 256-element super-block — gated BIT FOR BIT against the oracle's own kernels on those same real checkpoint bytes. **Both also have a CUDA keep-quant kernel as of #2260**, which is what stops these two encodings draining a CUDA stream to the host cores on every expert GEMM and stops the fused `MoeGateUpSwiGLUGrouped` seam refusing them outright; the CUDA arm is gated against the same oracle numbers on the same real bytes, and against a stream capture, because a dtype that falls back to the host is correct, slow and invisible to every value comparison. That is what takes the artifact from 426.72 GiB resident to 101.14 GiB, measured by driving the production residency decision over all 1412 of its tensors. **IQ4_NL, Q5_0 and Q4_0 have a CUDA keep-quant kernel as of [#2419](https://github.com/mudler/vllm.cpp/issues/2419)**, on a SECOND templated GEMM beside the Q8_K one: these are 32-element encodings that dot a `BlockQ8_0` activation, so the 256-element super-block GEMM the twelve other dtypes share cannot take them at all. Before it the dense and grouped CUDA seams drained the stream and ran the CPU kernel -- MEASURED on GB10 to SEGFAULT rather than merely to be slow, because `Backend::Alloc` is a plain `cudaMalloc` and the CPU kernel dereferences a device pointer -- and the fused `MoeGateUpSwiGLUGrouped` seam threw by name. This is the released `unsloth/Qwen3.8-Flash-Next-GGUF` UD-IQ1_S expert path specifically, and the encodings are forced rather than chosen -- its `expert_feed_forward_length` is 640, so every routed expert row is indivisible by 256 and no K-quant can encode it, which also means the `ffn_down_exps` tower's K = 640 is refused by the Q8_K seams whatever dtype it carries. The device dots are bit-exact against this tree's CPU arm per block; the discriminating gate is a STREAM CAPTURE, because a host fallback returns the same numbers and no value comparison can see it. MXFP4 (39) is the one 32-element encoding still falling back, and is owed. **IQ3_S (21)** added for the `unsloth` "UD" dynamic-quant family ([#2510](https://github.com/mudler/vllm.cpp/issues/2510)), decode bit-exact vs llama.cpp `b10451` over REAL bytes from `Qwen3.8-27B-UD-Q4_K_M.gguf`; it was the ONE hole in the reader's i-quant run 16..23, and four tensors out of that file's 866 refused the whole 16.4 GB artifact while every other encoding in it was already handled. **It is the one FILE encoding here that DECODES and does not DOT**: a gather table keeps its blocks, a GEMM weight EXPANDS to bf16 on every device, and the keep-quant `vec_dot` is owed jointly with the CUDA `WType` arm, because landing the CPU half alone routes every CUDA IQ3_S GEMM through the host fallback that was measured to segfault on a discrete card. On the measured artifact that expansion is 146.13 MiB of blocks against 680.00 MiB of bf16, 3.4 % of a 15.33 GiB file. Q4_1 (3) and Q5_1 (7) remain absent, so a `-Q5_K_M` build of a ragged-K model still refuses | ☐ | ☐ | ✅ | -| GGUF gather tables kept QUANTIZED (one row dequantized per gathered token) | ✅ CPU, `qwen35`/`qwen35moe`/`qwen3next`/`qwen4exp`. `vt::Embedding` takes a block-quantized table, decoding one row per id — a port of llama.cpp's `ggml_compute_forward_get_rows_q`. Without it a 51.2 G-parameter n-gram table expands from 28.8 GB of IQ4_NL to 102.4 GB of bf16, which no device here has (#1989). This is a residency DEFAULT CHANGE on already-shipped GGUF models, not only a new arm: an existing `qwen35` file with a quantized `token_embd` now keeps it compressed on CPU where it used to expand. Tokens do not move (every GGUF-path gather writes a bf16 output and the bf16 round is idempotent over the old expand-then-widen), so the change is memory-only today. `deepseek4` and `laguna` are NOT reached: both consume `token_embd` as a flat host f32 array, so their loaders narrow the policy for that tensor and keep expanding it. A gather's admission is the ROW DECODER and not the `vec_dot`, so IQ2_XS and IQ4_XS joined this set the moment their decoders landed (#2240), a wave before they could keep on the GEMM arm; Q8_K is now the only encoding that still tells the two admission rules apart. **CUDA joined it** (KGATHER, [spec](../.agents/specs/cuda-quant-gather.md)). `EmbeddingKernelCuda` decodes a block row across all 19 encodings `vt::cpu::BlockToFloat` decodes (IQ3_S joined with [#2510](https://github.com/mudler/vllm.cpp/issues/2510)), so a gather table stays block-resident ON THE CARD. The gate is a REGISTRY QUERY and not a device list: `vt::Embedding` routes a block table to `OpId::kEmbeddingQuant` and `DeviceQuantGatherSupported` is `OpRegistered(kEmbeddingQuant, dev)`, so a backend advertises the capability by registering the kernel and a device with no decoder refuses BY NAME. **MEASURED ON A GPU**, nvcc 13.0.88, 2026-08-31, and the two boxes measured DIFFERENT things: **thor:gpu0 (Jetson Thor, sm_110)** ran the branch HEAD through the production registration — 7 of 7 cases at 250 assertions, every encoding bit-exact against the CPU arm in f32 and bf16 out with i32 and i64 ids, against a pre-arm RED leg that failed by name at 32 assertions, and deleting the single production registration reds 5 of 7 cases, so the arm is reached and not merely present. **dgx:gpu0 (NVIDIA GB10, sm_121a)** then ran the SAME head leg green — 7 of 7 at 250, M5 red, every verdict matching thor — so the arm is proven on BOTH architectures and [#2393](https://github.com/mudler/vllm.cpp/issues/2393) is closed. This is what makes a GPU arm of `qwen4exp` possible at all — its n-gram table is 26.822 GiB of IQ4_NL against 95.368 GiB expanded, on a box with ~119.6 GiB for everything — and it is a shape llama.cpp does NOT have: its CUDA `get_rows` dispatches the legacy quants only and aborts on every K-quant and every IQ type, so the llama.cpp column overstates the denominator for the encodings that matter here. Like the CPU arm this is a residency DEFAULT CHANGE on already-shipped GGUF models, and it reaches more than `qwen4exp`: a `token_embd` in a block encoding now keeps its blocks on CUDA where it used to expand, `qwen35` GGUFs included. **For a bf16 output the values do not move** (both routes decode with the same scalar expressions and round once); **for an f32 output the new path is strictly MORE precise than expand-then-widen**, which is a behaviour change on already-shipped models and not merely a memory one. **Owed: the gather arms of METAL, VULKAN, ROCM and TENSTORRENT ([#2394](https://github.com/mudler/vllm.cpp/issues/2394)), which still refuse because each of their gather kernels asserts a float table by name. No throughput number is claimed | ☐ | ☐ | ✅ `get_rows` for ~20 types, CPU and CUDA | +| GGUF gather tables kept QUANTIZED (one row dequantized per gathered token) | ✅ CPU, `qwen35`/`qwen35moe`/`qwen3next`/`qwen4exp`. `vt::Embedding` takes a block-quantized table, decoding one row per id — a port of llama.cpp's `ggml_compute_forward_get_rows_q`. Without it a 51.2 G-parameter n-gram table expands from 28.8 GB of IQ4_NL to 102.4 GB of bf16, which no device here has (#1989). This is a residency DEFAULT CHANGE on already-shipped GGUF models, not only a new arm: an existing `qwen35` file with a quantized `token_embd` now keeps it compressed on CPU where it used to expand. Tokens do not move (every GGUF-path gather writes a bf16 output and the bf16 round is idempotent over the old expand-then-widen), so the change is memory-only today. `deepseek4` and `laguna` are NOT reached: both consume `token_embd` as a flat host f32 array, so their loaders narrow the policy for that tensor and keep expanding it. A gather's admission is the ROW DECODER and not the `vec_dot`, so IQ2_XS and IQ4_XS joined this set the moment their decoders landed (#2240), a wave before they could keep on the GEMM arm; Q8_K is now the only encoding that still tells the two admission rules apart. **CUDA joined it** (KGATHER, [spec](../.agents/specs/cuda-quant-gather.md)). `EmbeddingKernelCuda` decodes a block row across all 19 encodings `vt::cpu::BlockToFloat` decodes (IQ3_S joined with [#2510](https://github.com/mudler/vllm.cpp/issues/2510)), so a gather table stays block-resident ON THE CARD. The gate is a REGISTRY QUERY and not a device list: `vt::Embedding` routes a block table to `OpId::kEmbeddingQuant` and `DeviceQuantGatherSupported` is `OpRegistered(kEmbeddingQuant, dev)`, so a backend advertises the capability by registering the kernel and a device with no decoder refuses BY NAME. **MEASURED ON A GPU**, nvcc 13.0.88, 2026-08-31, and the two boxes measured DIFFERENT things: **thor:gpu0 (Jetson Thor, sm_110)** ran the branch HEAD through the production registration — 7 of 7 cases at 250 assertions, every encoding bit-exact against the CPU arm in f32 and bf16 out with i32 and i64 ids, against a pre-arm RED leg that failed by name at 32 assertions, and deleting the single production registration reds 5 of 7 cases, so the arm is reached and not merely present. **dgx:gpu0 (NVIDIA GB10, sm_121a)** then ran the SAME head leg green — 7 of 7 at 250, M5 red, every verdict matching thor — so the arm is proven on BOTH architectures and [#2393](https://github.com/mudler/vllm.cpp/issues/2393) is closed. This is what makes a GPU arm of `qwen4exp` possible at all — its n-gram table is 26.822 GiB of IQ4_NL against 95.368 GiB expanded, on a box with ~119.6 GiB for everything — and it is a shape llama.cpp does NOT have: its CUDA `get_rows` dispatches the legacy quants only and aborts on every K-quant and every IQ type, so the llama.cpp column overstates the denominator for the encodings that matter here. Like the CPU arm this is a residency DEFAULT CHANGE on already-shipped GGUF models, and it reaches more than `qwen4exp`: a `token_embd` in a block encoding now keeps its blocks on CUDA where it used to expand, `qwen35` GGUFs included. **For a bf16 output the values do not move** (both routes decode with the same scalar expressions and round once); **for an f32 output the new path is strictly MORE precise than expand-then-widen**, which is a behaviour change on already-shipped models and not merely a memory one. **ROCm now registers a native HIP gather for all 19 admitted encodings** ([spec](../.agents/specs/rocm-quant-gather.md), [#3093](https://github.com/mudler/vllm.cpp/issues/3093)). The gfx1100 public gate passes each codec against its dense control, and matching operation traces retain compressed tables with direct bf16 output. Original primary fixtures and pinned model qualification remain pending. **Owed: the gather arms of METAL, VULKAN and TENSTORRENT ([#2394](https://github.com/mudler/vllm.cpp/issues/2394))**, whose gather kernels still require floating tables. No throughput number is claimed | ☐ | ☐ | ✅ `get_rows` for ~20 types, CPU and CUDA | | GGUF F16 weights kept resident as F16 (no BF16 promotion) | ✅ `VT_GGUF_KEEP_F16` default-ON (CPU), the f16 GEMM computes on it directly. Default settled 2026-08-17, a memory-for-speed trade: 1.05 GiB less peak RSS for ~9% prefill and ~1.4% decode, tokens identical. `0` opts out | ☐ | ☐ | ✅ `ggml_vec_dot_f16` | | GGUF is a TWO-engine comparison at these pins (#979) | ✅ text-only `qwen35`, no `clip` projector (#821) | ☐ REMOVED from the tree in `6635279d8`, now an unpinned out-of-tree `vllm-gguf-plugin` | ☐ full stack present, `qwen3_5` unreachable behind FOUR blockers, and the load path has NO completeness guard so a clean-looking load proves nothing | ✅ native, `LLM_ARCH_QWEN35` | | EXL3 trellis (exllamav3; codebooks 0 (3INST), 1 (MCG) and 2 (`mul1`), Hadamard-128 + sign vectors, NO scales) | ◐ **A stock EXL3 checkpoint GENERATES**: `turboderp/Llama-3.2-1B-Instruct-exl3` @ 3.0bpw loads through the shared dense container and emits coherent text from `vllm-cli` on a CUDA queue and on a CPU queue, which reaches `LlamaForCausalLM` and Qwen3-dense together. The scheme sits on vLLM's own `LinearMethodBase` seam, so it is no longer a DeepSeek-V4-private arm. `bits` and the codebook are both read PER TENSOR — the published 3.0bpw Llama has a 3-bit body and a 6-bit head, and ships no `mcg` marker, which means codebook 0 and not MCG. **The DeepSeek-V4 arm is separate and unchanged**: its rank-sliced SparkInfer artifact loads and executes end to end on a synthetic checkpoint. The rank-sliced `0xSero/deepseek-v4-flash-0731-spark` routed-expert tower coalesces TP4->TP1 at load, the `carried-*` half (block-wise FP8 + BF16 + F32) is dequantized into the host-float tower the forward composes with, and `DeepseekV4Model::Forward` runs the whole model, dispatching one `vt::Exl3MoeMlp` per MoE layer over the routed experts (`bits == 3`, `mcg` codebook). **The REAL artifact still does not run**: its DSA compressor and indexer tensors are twice the width the host forward indexes and the loader refuses them by name, and its tokenizer is not read (#1924). **The device half is now PARTLY VERIFIED** (GB10 `sm_121a`, 2026-08-28): `had_r_128` is BYTE-IDENTICAL CUDA-vs-CPU, `exl3_gemm` matches the f64 reference at `rel_rms 5.538e-4` against a `1.0e-3` bound, and the `m<=8` GEMV meets tier 3c at `5.160e-4` against `6.0e-3`. What is STILL unverified on a device is the FUSED MoE arm, which cannot run on this code at all: it needs a device-resident tower and `CudaBackend::DeviceMemoryIsHostAddressable()` is false by design, so the routed-expert path executes on a CPU queue today. No speed number is claimed on any axis. **Codebook 2 (`mul1`) and the 4- and 5-bit widths are now implemented** so that `Mia-AiLab/Qwen3.8-27B-EXL3-3.5bpw` -- **409** trellis modules, 137 at 3 bpw, 270 at 4, one at 5 and one at 6, every one of them `mul1`-marked -- is no longer refused by name ([#2495](https://github.com/mudler/vllm.cpp/issues/2495)). The count read 272 until 2026-09-02 and was an UNDERCOUNT that omitted the 137 bits-3 modules, which is why `(3, 2)` was missing from the CUDA instantiation list and every MLP projection of that target refused on a CUDA queue. **`(3, 2)` is now instantiated and gated on `dgx:gpu0`** (GB10, driver `580.173.02`), agreeing with the CPU decoder at `rel_rms 3.02544e-07` against a `1.0e-3` bound ([#2574](https://github.com/mudler/vllm.cpp/issues/2574)). cb 2 is a DIFFERENT decode and not a third multiplier: it sums the four bytes of the product into an fp16 bit pattern and maps it with a fused fp16 affine, and the host arm is gated against hand-computed upstream values rather than against itself. **A ROCm arm now exists too** ([#2433](https://github.com/mudler/vllm.cpp/issues/2433), [spec](../.agents/specs/backend-rocm-exl3.md)): `Exl3Gemm` on `gfx1151` is a transcription of the PORTABLE CPU reference rather than a port of the CUDA kernel -- 90 KiB of requested shared memory does not fit a 64 KiB AMD LDS, and `m16n8k16`, `ldmatrix` and `cp.async` have no AMD form -- and because it uses no matrix cores it is gated at BYTE equality with the CPU arm on all three codebooks, not at a tolerance. It covers every width the host arm decodes, so the CUDA instantiation list below does not bound it. On the CUDA device arm `(3,0)`, `(3,1)`, `(6,0)`, `(3,2)`, `(4,2)`, `(5,2)` and `(6,2)` are instantiated -- `(3,2)` is the width the undercount cost, added and device-gated by [#2574](https://github.com/mudler/vllm.cpp/issues/2574); every other width and codebook and the device-resident tower are owed ([spec](../.agents/specs/quant-exl3-mul1.md)). **The `m<=8` GEMV is a SEPARATE arm set from that list and a separate row** ([`QUANT-EXL3-PERF`](../.agents/specs/quant-exl3-perf.md), [#2570](https://github.com/mudler/vllm.cpp/issues/2570)): upstream instantiates `(4,0) (4,1) (4,2) (2,1) (2,2) (3,1) (3,2)` and this tree now carries `(3,1)`, `(3,2)` and `(4,2)`, so **407 of that checkpoint's 409 trellis modules** reach the small-m fast path where before NO module of it could -- the only arm was `(3,1)` and the artifact contains zero `(3,1)` tensors. `(3,2)` was one template argument; `(4,2)`, the 270-module arm, was a KERNEL port: a per-width `LSTRIDE`, two 24-lane guards that exist only because a 3-bit tile is 24 uint32 rather than 32, and a separate `dq8_regs_4bits` window read. `(4,0)`, `(4,1)` and the whole 2 bpw arm stay OWED -- no artifact in this tree carries a 4-bit tensor at either of the first two codebooks, and nothing builds them, so that the kernel would compile for them is an inference and not a measurement. **Instantiating an arm is necessary and not sufficient**: `Exl3GemvSelectConfig` is upstream's envelope verbatim and DECLINES, and on Blackwell EVERY shape of this checkpoint, at both widths, is admitted only where `size_n / 32 <= narrow_coresident`, an occupancy query. The wide config is not an escape from it: its band needs `size_k <= 4096` and the smallest 4-bit `k` here is 5120. The thresholds are `>= 544` and `>= 160` at the two bits-3 shapes and 32, 160, 192, 320, 384 and 544 across the six bits-4 ones, all pinned in `tests/vt/test_exl3_gemv.cpp` from both sides. **The throughput effect is now MEASURED on GB10, and it is a NULL** ([#2570](https://github.com/mudler/vllm.cpp/issues/2570)): three interleaved rounds on one binary read 17.20/17.18/17.09 tok/s with the arm off against 17.22/17.12/17.09 with it on, a 1.3% spread and no separation. `nsys --cuda-graph-trace=node` shows why, and it is a DECLINE and not an ineffective kernel: `exl3_gemv_kernel` appears 0 times at the default and 4 times only when forced. GB10 reports `SM_COUNT=48 MAX_THREADS_PER_SM=1536`, so the narrow config's 512-thread blocks ceiling `narrow_coresident` at 144, and only shapes needing `<= 144` are admitted -- 34 of the checkpoint's 409 modules, 0.75% of its trellis bytes. The arms are correct, upstream-faithful and device-gated; on THIS part the envelope declines them. `m == 8` is unmeasured, because the draft legs did not run. **The FUSED MoE mgemm is a SEPARATE arm set from that list**, and upstream's own bound on it is narrower: `exl3_moe.cu:184` admits `mcg` and `mul1` and refuses 3INST, so codebook 0 is not a MoE arm upstream either. It now instantiates bits 3, 4, 5 and 6 at codebook 1 where it carried bits 3 alone ([#2756](https://github.com/mudler/vllm.cpp/issues/2756)); before that a mcg expert tower at any other width could not run on a CUDA queue by ANY path, because the fused arm is default-ON and its refusal is not caught, and the `VT_DSV4_EXL3_FUSED_MOE=0` rollback lands on a GEMM arm set with no `(4,1)`, `(5,1)` or `(6,1)` either. Codebook 2 on the fused arm is owed as a LOADER slice and not a kernel one: five sites pin the codebook to 1 before the kernel is chosen, so a cb-2 instantiation added alone would be dead code. **Its GDN linear-attention tower now loads and runs too** ([#2495](https://github.com/mudler/vllm.cpp/issues/2495) item 4, [spec](../.agents/specs/model-qwen35-gdn-exl3.md)): 48 of that model's 64 layers are `linear_attention`, and the three projections of each were refused by name because nothing in the GDN forward consumed an `Exl3Weight`. The arm holds THREE trellises rather than one merged `in_proj_qkvz` owner, because the artifact ships `in_proj_qkv` and `in_proj_z` as two independently quantized tensors whose sign vectors are fitted per projection, so the bf16 merge has no trellis analogue. It also accepts the F16 `in_proj_a`/`in_proj_b` the artifact stores beside its BF16 `conv1d`, `norm`, `A_log` and `dt_bias`. **The DFlash2 DRAFT loader now has an EXL3 arm too** ([#2495](https://github.com/mudler/vllm.cpp/issues/2495) item 7, [spec](../.agents/specs/model-dflash2-exl3.md)): `Mia-AiLab/Qwen3.8-27B-DFlash2-EXL3-5.0bpw` ships all 36 of its modules at bits 5 with a `mul1` marker, and `LoadQwen3DFlash` died on `fc.weight`, which an EXL3 module does not ship. The arm holds SEVEN trellises per layer rather than the merged `qkv_proj` and `gate_up_proj` owners the bf16 reader builds, for the same reason the GDN tower holds three. It also accepts the F16 `candidate_selector.hidden_projection` and `*_conv.kernel_projection` the repack left unquantized beside their BF16 `base_kernel` and codebooks — a second refusal that was invisible until the first was gone. **Its `mtp.*` draft head loads and runs too, and all THREE readers of the target's trellis head now compute with it packed** ([#2495](https://github.com/mudler/vllm.cpp/issues/2495) items 5 and 6, [spec](../.agents/specs/model-qwen35-exl3-head.md)): the MTP loader refused nine quantized `mtp.*` tensors as "expected BF16", and the MTP draft's `ComputeLogits` plus the DFlash/DSpark draft's SHARED head read knew only bf16 and NVFP4. The head is COMPUTED WITH rather than widened, which is what upstream does without a branch because its head is an `nn.Module` and `_apply_head` calls `lm_head.quant_method.apply`; a dequantized copy of the real 248320x5120 head would be 2.543 GB against 0.953 GB packed. The shared read also used to fall off the end of its bf16 loop and return SILENTLY with every owner empty when no arm matched, which is exactly what this target does -- it ships no `lm_head.weight` at all -- and it now refuses by name ([#2569](https://github.com/mudler/vllm.cpp/issues/2569)). **The checkpoint's real inventory is 409 quantized modules, not the 272 this row previously recorded**, measured from both downloaded shards' headers: bits 3 x137, 4 x270, 5 x1, 6 x1, every one `mul1`, and no tensor in the file has `scale` in its name. `(bits 3, codebook 2)` is now instantiated on the CUDA GEMM and gated on a device, so those 137 modules no longer refuse by name ([#2574](https://github.com/mudler/vllm.cpp/issues/2574), owned by `QUANT-EXL3-MUL1`, not by this row). **THE 27B CHECKPOINT NOW RUNS ON CUDA AND GENERATES.** On `dgx:gpu0` (GB10, driver 580.173.02, `sm_121a`) the published `Mia-AiLab/Qwen3.8-27B-EXL3-3.5bpw` loads and emits ` Paris. The capital of Germany is Berlin. The capital of Italy is` at `rc=0`, with both shard sha256 values recomputed on the device. Warm decode is **16.7 tok/s** target-only and **48.7 tok/s** with its DFlash2 EXL3 draft at k=7, interleaved on one binary, and the two arms are **token-identical** ([`qwen38-27b-exl3-gb10`](benchmarks/qwen38-27b-exl3-gb10.md)). **That is NOT a reproduction of the upstream README's 47.5 tok/s**, and must not be quoted beside it: theirs is HumanEval-style at T=0.6 with acceptance 4.43, this is greedy T=0 on a prompt that continues into a list of capitals, the context is 8192 rather than 262144 with no NVFP4 KV cache, and the shipped paged draft route had to be disabled (#2274). A matched HumanEval-style run at T=0.6 with acceptance reported is owed | ☐ no EXL3 at the parity pin | ☐ | ☐ | diff --git a/docs/bench-evidence/rocm-quant-gather/README.md b/docs/bench-evidence/rocm-quant-gather/README.md new file mode 100644 index 0000000000..7c18107d0d --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/README.md @@ -0,0 +1,201 @@ +# ROCm quantized gather evidence + +Row: `BACKEND-ROCM-QUANT-GATHER`. +Issue: [#3093](https://github.com/mudler/vllm.cpp/issues/3093). +The [committed spec](../../../.agents/specs/rocm-quant-gather.md) defines the gates. + +The native operation, public dense control, and matching operation traces pass. +Pinned model qualification and the original upstream fixture gate remain pending. +This evidence makes no throughput, latency, model token parity, or performance claim. + +## Identity and scope + +The implementation started from spec commit `670e6d78ddf55231394748e0032939fd53dc56a5`. +The Q8_K materializer amendment preceded its code at `376223f000f85683f2881136e38ee52892d8d3a8`. +The reader amendment preceded its code at `2e5adfb85a255528bffa5d3cdb016efb2bab70d8`. +The prerequisite full-attention repair was independently reviewed at +`6a7bcb77637e66df34429208e3a4055e0945a875` and cherry-picked before the intended red run. +Its later runner expectation repair was imported as spec `8c9101fa4` and test commit `b46852d13`. +Those three dependency files equal donor `694f04525174d0c2dac8030ba8f1398f2683ec82` byte-for-byte. + +Early captures identify uncommitted source with complete file and executable hashes. +Their hashes describe the measured stages, including defects repaired before the final implementation. +They are not claims that every early artifact equals the final commit. +The final commit and review receipts identify the final acceptance candidate. + +All native GPU commands ran through the operator on the supplied gfx1100 host. +The operator held its recorded file mutex and checked input hashes around each run. +The two-device bounds test used visible devices `0,1`; other native runs used device `0`. +Helper builds and secondary oracle runs were CPU-only. +Every listed absolute path records this measured environment and is not a default. + +The primary runtime uses vLLM `e126687a9a828d513c01a07cd69f025f27d63280` and plugin +`d4c1f0d082fc7cd4350da56689109a01c1f29d6c`. +It runs in the existing image +`sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc`. +The separate plugin artifact contains native extension +`592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138`. +The build recipe, log, and complete artifact hashes are retained here. +No existing oracle runtime was modified. + +Stock llama.cpp is `10bf611e533d81f739128304991c5e133c6aebd8`. +The registered IQ1_XXXS fork is `36fe8e1cc7f2b3b8c92fdda0ab07600141921786`. +The [independent oracle audit](oracle-audit-report.md) verifies their source archives, +executing decoder chain, binaries, plugin artifact, and captured outputs. +That audit is separate from implementation mutation review. + +## Red and green results + +Raw `.log` filenames in this document refer to unchanged files inside `raw-logs.tar.gz`. +The archive preserves original whitespace and hashes. + +| Gate | Observed result | Evidence | +|---|---|---| +| Initial public attempt | Wrong-reason failure in unused GDN state validation; not the intended red | `public-red.log` | +| Public attempt after reviewed prerequisite | Intended failure: zero compressed provider selections; 10 of 11 assertions passed | `public-red-after-prerequisite.log` and sealed recipe | +| Q8_K reader | Type 15 refused before materialization | `q8k-reader-red.log` | +| Q8_K materializer after reader repair | Reader passed; dot-only materializer guard refused decoder-only Q8_K | `materializer-second-red.log` | +| Q8_K reader and role-aware materializer | Two cases, 18 assertions passed; reader regression passed 36 cases, 133 assertions | `materializer-focused-green.log`, `reader-regressions-green.log` | +| Queue device binding | Ambient device 1 caused invalid stream use for queue 0 before the fix; 16 assertions passed after binding queue 0 | `ambient-device-witness-v1.log`, `ambient-device-witness-v2.log` | +| Native focused v2 | Seven executed CTests passed, including all 19 public model variants | `native-focused-v2.log`, operator receipts | +| Final native v4 | All 15 CTests executed and passed; primitive suite 9 cases and 11,784 assertions; public suite 2,850 assertions; required two-device case 16 assertions | `native-final-v4-focused-and-rocm-regressions.log`, `native-final-v4-required-ambient-device.log`, sealed commands and operator receipts | +| Native/primary/secondary operation comparison | All 76 outputs passed; 68 were byte-identical | `synthetic-parity-v2.json`, independent operator comparison | +| Applicable stock GET_ROWS shape | Width 256, five rows, four IDs; all six native outputs matched the pinned secondary result | `stock-shapes-v1-operator-results.json` | +| CPU-only configuration | Five CPU targets passed; three accelerator-only targets skipped explicitly; the imported runner suite passes 41 cases and 1,914 assertions | `cpu-configure.log`, `cpu-focused-green.log`, `cpu-runner-green.log` | + +The model builder fixes seed `0x524f434d`, geometry, tokenizer metadata, and tensor values. +The two-regeneration manifest confirms all 38 model hashes, without changing fixture bytes between red and green. +The public path uses 19 codecs, two prompts, three repeats, and a fresh engine for each request and dense control. +It checks all four greedy token IDs and all 512 captured logits byte-for-byte. +The final public run additionally writes each completion's tokens and raw F32 logits with +`VT_ROCM_GATHER_PUBLIC_EVIDENCE`. + +## Numerical oracle evidence + +`native-operation-captures.tar.gz` preserves 76 native outputs and its report. +`primary-operation-captures.tar.gz` preserves 64 primary outputs and its report. +`secondary-operation-captures.tar.gz` preserves the six actual upstream F32 outputs. +The native BF16 secondary comparison applies explicit round-to-nearest-even to those F32 outputs. +It does not claim that either secondary graph executed BF16 output. + +The primary gate preserves `atol=0.01`, `rtol=0.04`, widths 256/1024, and both admitted output dtypes. +The local ABI flattens the upstream two-by-two IDs; the primary harness restores that shape. +IQ1_M and F16 output exceed the shared local contracts and remain explicit exclusions. +The generated 128-row corpus does not replace the 32 original upstream GGUF fixtures. +All 32 were downloaded with the developer's authority and sha256-verified at +289,655,872 bytes. Their exported tensors (160 cases over 16 primary codecs) +pass the native-versus-plugin comparison: 320 outputs, 140 byte-exact, and every +remaining element inside the unchanged upstream tolerance. +The raw receipt is `gather-3113-repair/fixture-download-receipt.json` under +`/home/vikash/.cache/`; the comparison is +`original-fixture-coverage/compare/report.json` under +`/home/vikash/.cache/rdna3-gather-oracle-impl/`. + +Eight nonidentical comparisons are confined to Q6_K and IQ3_S, across two widths and output dtypes. +Q6_K's primary decoder rounds intermediate products to F16; the local decoder uses F32 products. +IQ3_S's primary codebook contains effective values of 15.5 where the local/stock codebook contains 15. +All differences satisfy the unchanged upstream tolerance. +The [arithmetic diagnostic](oracle-audit-arithmetic-explanation.json) reproduces both executing expressions. +It explains the runtime outputs and does not replace them with a scalar oracle. + +Stock GET_ROWS excludes Q8_K, so the harness calls its pinned `dequantize_row_q8_K` directly. +MXFP4 and IQ1_XXXS execute real upstream GET_ROWS graphs. +The additional stock-shape corpus preserves its applicable width 256, five rows, four IDs, and contiguous unbatched layout. +Its deterministic packed values and first/last/middle/repeated IDs adapt upstream's unrecorded random input. +The local suite explicitly refuses batched tables and strided IDs. +Upstream cases with batched tables, floating-only dtypes, or backward gradients exceed this gather ABI. + +## Memory and matching traces + +Both implementations used the same 32 packed tables, widths 256/1024, four I64 IDs, and requested F32/BF16 outputs. +Both used the exact existing rocprofv3 1.3.5 profiler and its isolated prefix. +The prefix contains profiler dependencies and excludes HIP/HSA/BLAS runtime replacements. +Version 2 adds allocation events to both sides; its inputs, executable, and harness equal version 1. +The trace-only primary harness omits the already-verified full-table dense control. +Every traced output equals the same implementation's untraced output byte-for-byte. + +The native trace contains 64 fused gather kernels, with 32 F32 and 32 BF16 template instantiations. +Its 192 correlated `hipMalloc` events exactly match 32 packed tables, 32 ID arrays, +64 output arrays, and 64 error records of 16 bytes. +There is no extra native device allocation for selected packed rows or an F32 staging output. +The profiler's host runtime allocations are identified separately. + +The primary trace contains 64 GPU packed-row selections and 64 native decoder launches. +Each output uses the requested F32 or BF16 decoder template. +Its caching allocator appears as a backing pool in the profiler. +Separate PyTorch telemetry measures each tensor operation: the live delta equals its output size; +the peak delta adds selected packed rows rounded to the observed 512-byte allocator unit. +Those tensor counters are not profiler allocation-size measurements. + +The source chain in the independent audit and these executed GPU kernels establish selected-row native decode. +The trace harness contains no dense-control or host-dequantization call. +Provider counts are an additional reachability check, not the only evidence against host fallback. +A separate trace runs the complete public model test: 228 engines and 2,850 assertions passed. +All 456 preserved capture files contain exact dense-control and repeat token/logit bytes. +The operator verified every capture, kernel correlation, and device allocation independently. +The model trace executes 912 BF16 KV reads, 912 two-byte KV writes, and 456 direct BF16 quantized gathers. +It records 228 `hipMalloc` allocations of 16,384 bytes, matching one KV pool per engine. +The source allocates four pages of `16 * 1 * (64 + 64) * 2 = 4096` bytes each. +The allocator-to-KV attribution uses that source chain: CSV does not expose kernel argument pointers. +The observed cache template dtype and byte counts are preserved separately from this attribution. +`public-model-trace-captures.tar.gz` retains complete profiler CSVs, completion captures, and the scratch capture audit. +This establishes native BF16 cache storage at the original four-block setting. +The committed oracle amendment requires 16 blocks and a new matched memory capture. +The original traces retain their measured values. + +## Initial qualification failures + +The stock and fork bounded model attempts both abort on their first decode before emitting tokens. +The observed assertion is `ggml-backend.cpp:194::GGML_ASSERT(buffer)` through hybrid memory input setup. +An unused recurrent input is the source-based hypothesis, not a proven object-allocation diagnosis. +Their context request of 64 resolves to 256 cache cells through the pinned upstream padding rule. +Their BF16 KV payload is therefore 65,536 bytes, against the original spec's 16,384 bytes. +The committed amendment matches that physical capacity in the native and primary +arms. Both secondary model executions still require qualification under issue #3093. +The fork's independent real 2.4T qualification remains owed under issue #933. + +The primary model attempt uses this row's exact Q4_0 fixture, SHA256 +`663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af`. +The first attempt failed because Triton's default cache path was unwritable. +That operational failure is not a model compatibility result. +The second attempt supplies writable task-local cache paths without changing the model, HF config, or oracle code. +It reaches the pinned renderer and refuses `Qwen3_5TextConfig` where `Qwen3_5Config` is required. +It emits no tokens. The full traceback and sealed recipe preserve that actual model refusal. +The operator checked 340 input files before and after the second attempt. +That refusal predates the approved configuration amendment. The corrected v5 +launch emits four tokens from the unchanged Q4_0 fixture with normal compilation +and graph capture. The [amendment evidence](oracle-amendment/README.md) retains +the intervening RoPE and cache-admission failures, successful launch, and receipts. +The full token matrix and matched model memory remain pending under issue #3093. +The final v4 operator audit confirms all 456 completion files equal the traced run byte-for-byte. +All 33 sealed source, library, and executable files remained unchanged during v4. +The row stays active until model execution, original fixtures, fresh mutation review, +and the operator's reviewed-head rerun meet their respective requirements. + +## Full preflight + +The first full staged run found two stale source citations after the materializer edit. +The corrected engine-matrix rows change only `SPEC-MTP-GGUF` and `SPEC-DFLASH-GGUF` line anchors. +Every unrelated row remains byte-identical, and the anchor baseline stays at 28. +The record gate then passes with 937 valid anchors, 28 stale anchors, and five pre-existing broken anchors. +The final full staged rerun exits 0 with no failures and five argument-dependent skips. +All 31 translation units in its host syntax scope compile. +This is not an all-green preflight, because a skipped check executes nothing. +The separate CPU ISA check passes against the actual CPU-only compile commands. +The ARM ISA, CUDA code-generation, and CUDA Triton artifact checks do not apply to this x86_64 HIP configuration. +Exact commit path classification runs after committing and accompanies the handoff. +Combined CUDA/HIP execution remains pending because no combined build was available. +The raw final log and explicit CPU ISA result are archived unchanged. +`preflight-final-staged-result.json` records the exact command, environment, staged tree, statuses, and skip dispositions. + +## Reproduction + +Build and run the focused commands in the spec under the operator's GPU authority. +Generate operation inputs with `rocm_quant_gather_capture generate DIRECTORY`. +Run `primary.py capture`, `secondary.cpp`'s `gather` mode, and native `capture` with the sealed corpus. +`compare.py` compares the native report against each independently executed oracle. +`stock_shapes.py` exports the applicable secondary geometry without downloading or changing packed values. +`audit_trace.py` verifies each trace against its untraced outputs and checks the memory evidence. + +Exact argv, environment, source hashes, executable hashes, and operator receipts accompany each stage. +Downloaded upstream GGUF fixtures and generated model weights are not committed here. diff --git a/docs/bench-evidence/rocm-quant-gather/ambient-device-witness-v1.json b/docs/bench-evidence/rocm-quant-gather/ambient-device-witness-v1.json new file mode 100644 index 0000000000..6dab41372e --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/ambient-device-witness-v1.json @@ -0,0 +1,54 @@ +{ + "version": 1, + "row": "BACKEND-ROCM-QUANT-GATHER", + "head": "376223f000f85683f2881136e38ee52892d8d3a8", + "cwd": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "env": { + "HIP_VISIBLE_DEVICES": "0,1", + "ROCR_VISIBLE_DEVICES": "0,1", + "VT_ROCM_GATHER_REQUIRE_TWO_DEVICES": "1", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp" + }, + "argv": [ + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant", + "--test-case=ROCm gather follows the queue device when the ambient device differs", + "--abort-after=1" + ], + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/ambient-device-witness-v1.log", + "authority": "Operator runs this bounded witness under the recorded global GPU mutex with devices0 and1 visible", + "expectation": "Require two devices. Queue/table/output reside on0; ambient device switches to1 before shared vt::Embedding. Check correct rows, bounds exception, and immediate recovery. No device-binding implementation change precedes this witness.", + "files": { + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vt/test_rocm_embedding_quant.cpp": { + "bytes": 17542, + "sha256": "7026334d9ab70b3fa56cf0c4af4012f054e539ba37a48c586732742d7278a927" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/CMakeLists.txt": { + "bytes": 275318, + "sha256": "45ee683859f89aa7c4c8d5ec253e46a0cf4d8de5eb33ac5f196c8fd413d3cba1" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_embedding_quant.hip": { + "bytes": 7556, + "sha256": "5934d1727c96ec8e76f4e7dc3805e15d271fb528b7c0ad6c3a5d2863a179686b" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_embedding_quant.h": { + "bytes": 554, + "sha256": "1b2b31617d660b64f622e692089423ce1657df102322558a4ab731f521952bc5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_quant_dequant.h": { + "bytes": 25507, + "sha256": "413dc138254cc72f8dccbc87268581217c93dec8f8acaea159f3f39569c37e62" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_quant_iq_tables.h": { + "bytes": 110865, + "sha256": "d267d25ff1dd3f67749c251f87047f4b70c9650a99d04b524ff6c3313596a349" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/libvllm.a": { + "bytes": 95078500, + "sha256": "acbf7ceb5c2efa04ff56189e04de549ebe760ba70f3d7967f684f2e7709431cb" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant": { + "bytes": 30954024, + "sha256": "f2dd2bfa912713cd7e6599dd868a1f347487fcf4b8da537b464a31f5cb576eae" + } + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/ambient-device-witness-v2.operator-receipt.json b/docs/bench-evidence/rocm-quant-gather/ambient-device-witness-v2.operator-receipt.json new file mode 100644 index 0000000000..24df9a3d43 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/ambient-device-witness-v2.operator-receipt.json @@ -0,0 +1,29 @@ +{ + "name": "ambient-device-v2", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "env", + "HIP_VISIBLE_DEVICES=0,1", + "ROCR_VISIBLE_DEVICES=0,1", + "VT_ROCM_GATHER_REQUIRE_TWO_DEVICES=1", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "timeout", + "180s", + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant", + "--test-case=ROCm gather follows the queue device when the ambient device differs", + "--abort-after=1" + ], + "exit_code": 0, + "elapsed_seconds": 0.5165746212005615, + "manifest_sha256": "2928aa6cbc417a3993c86481cbdd6c3ba4e35072bbf703212976344bda57fed0", + "log_sha256": "38fec7edbc44a0e557538088a130e98da012892fdec43f29234989d35718209e", + "all_101_manifest_inputs_verified_before_after": true, + "core_dumps_disabled": true, + "summary": [ + "[doctest] test cases: 1 | 1 passed | 0 failed | 8 skipped", + "[doctest] assertions: 16 | 16 passed | 0 failed |" + ] +} diff --git a/docs/bench-evidence/rocm-quant-gather/materializer-second-red.json b/docs/bench-evidence/rocm-quant-gather/materializer-second-red.json new file mode 100644 index 0000000000..f7aae2d181 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/materializer-second-red.json @@ -0,0 +1,44 @@ +{ + "version": 1, + "head": "2e5adfb85a255528bffa5d3cdb016efb2bab70d8", + "argv": [ + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_keep_quant", + "--test-case=Q8_K*" + ], + "env": { + "HIP_VISIBLE_DEVICES": "-1", + "ROCR_VISIBLE_DEVICES": "-1" + }, + "exit": 1, + "result": "Reader case passed; ordinary loader throws at OwnGgufQuantBlocks KeepQuantDType guard, qwen3_5_gguf_weights.cpp:89. All12 assertions passed before exception.", + "files": { + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vllm/model_executor/model_loader/gguf_reader.cpp": { + "bytes": 29998, + "sha256": "2901344cb8252ac3c66cacd9b2ae137a814217425df28c387bbdade83d9c12e5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp": { + "bytes": 90319, + "sha256": "9ee863b7821648d43deaaed288408f3fcd13dd089ace8109bf12233d4212e944" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/include/vllm/model_executor/models/qwen3_5_gguf_weights.h": { + "bytes": 17840, + "sha256": "ecb95af1c711afc2c2eefef5de7aba88bcefd3f61453fb25b92b7df4f3d9053d" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vllm/test_gguf_keep_quant.cpp": { + "bytes": 134876, + "sha256": "65a5e5a40f19d8e264bc1bb341c2a8276a3719c59c7faa2c91b4f4b7bfe9dd92" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/libvllm.a": { + "bytes": 95078844, + "sha256": "48d0ab6c0541c20ba4e3d87df07da1aa31c14baac87dcfb995d4af8f73c64b25" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_keep_quant": { + "bytes": 31354600, + "sha256": "5970560b000d16d7e86390417793317ff2b06f4e4bd5232603b8304fc715ed5d" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/materializer-second-red.log": { + "bytes": 846, + "sha256": "4d8846fe34ed7c6abb0d7bdeb4853674de3f4a2fc4b45c95fbd8fa59d1f23397" + } + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/native-final-v4-commands.json b/docs/bench-evidence/rocm-quant-gather/native-final-v4-commands.json new file mode 100644 index 0000000000..4fabe7a150 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/native-final-v4-commands.json @@ -0,0 +1,191 @@ +{ + "row": "BACKEND-ROCM-QUANT-GATHER", + "version": 4, + "base_head": "b46852d13f7c8776557dab170e02ebea50cdfdec", + "implementation": "Frozen gather candidate after trailing-space-only table cleanup and imported runner expectation repair; complete clean HIP617-step build plus updatedtest_runner target", + "files": { + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant": { + "bytes": 30952192, + "sha256": "d0d85c52aed0efa959752b6acafe1aac03c19b985543ccffc09f3008a6e23280" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_capi_rocm_embedding_quant": { + "bytes": 30926000, + "sha256": "6a8f7e1029b904c94c0d76561f0291cd17fd7b3e4db29e2d95e3b33894c74efd" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_keep_quant": { + "bytes": 31352656, + "sha256": "28e3502d940688a0a2b795e93f0456595e55b13e4508a8e0d7b448f8a3b977b2" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_ops_embedding_quant": { + "bytes": 30843664, + "sha256": "adf3962f3f99c7e28cd2cbded31e36b8a324908fa6a6fa610d354b5b4de29b4a" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_dequant": { + "bytes": 30960400, + "sha256": "0c6f4d003de2ec308fbef18d661843dfc21c59f9d31b27a9a0c8eda1f42f9fa3" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_backend_cross_device": { + "bytes": 31467496, + "sha256": "4a03440c90f387a391ca312b769f0e3cf949909598575ff902c0f67b4fdc756f" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf": { + "bytes": 31012152, + "sha256": "e6e2e2caee1e8ef17cbcd4dca5dca578c3c1e8cdb35c4cbad9bf259646eee9d1" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_arch": { + "bytes": 30868192, + "sha256": "c47ed82bb92734a5e2b1b434a82bc1708c6bde19c3c5c2a00f1f6b79fde069a0" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_backend": { + "bytes": 30908448, + "sha256": "d9eb4e31e6e0e04802e15ca49c4fbe6063201468571cc45f60d9669eb9793147" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_fp8_kv_cache": { + "bytes": 30927608, + "sha256": "021a7cb5bb215a38dc59cf41bf63a5a4191535abe0a9dc4ab257cd18d8e5457f" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_ops_rocm_silu_rounding": { + "bytes": 30858504, + "sha256": "e73f9429d1900b37c346b9ab5a670cf4d459b98896e7b16d60e7db70bce2b055" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_exl3_rocm": { + "bytes": 30863768, + "sha256": "3b99314fccc7732b30eb25b59d895c04ba724db3253d6c8510c494e31acb8cbf" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gemma4_rocm_fp8_seams": { + "bytes": 30859872, + "sha256": "0a6552809425f62481ae04386ce29bb670d3375032eeef43841d73b928c93b2c" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_backend_multidevice": { + "bytes": 30837568, + "sha256": "f506061d963e2cc97054d65b1ecfa78cbb8358435d57a5bf5dcfc0dd3fb285df" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/libvllm.a": { + "bytes": 95078032, + "sha256": "e1e56ec9535a7501a8313986861dfa7bed3fde35f50dbdbc3ba05c4aac7b1afa" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/CMakeLists.txt": { + "bytes": 175502, + "sha256": "e5bea1baa842ac0257fb29d61cb6ef37977d975d6ac0478f81d6e677fe85ce32" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/include/vllm/model_executor/models/qwen3_5_gguf_weights.h": { + "bytes": 18080, + "sha256": "ed37c245f574d2f42df14b40261a8ecb6ae56c159588a82e1550fc49ccaf4655" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vllm/model_executor/model_loader/gguf_keep_quant.cpp": { + "bytes": 26770, + "sha256": "d4b45702692b41359a1e8efd836bb28a14fbbe37a06ee28d818e88b885c01a2b" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vllm/model_executor/model_loader/gguf_reader.cpp": { + "bytes": 29998, + "sha256": "2901344cb8252ac3c66cacd9b2ae137a814217425df28c387bbdade83d9c12e5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp": { + "bytes": 90683, + "sha256": "685ca1623c336876768038a8aa28260694cd4207a1ac1e2d488bc81e451be4fb" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_embedding_quant.h": { + "bytes": 554, + "sha256": "1b2b31617d660b64f622e692089423ce1657df102322558a4ab731f521952bc5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_embedding_quant.hip": { + "bytes": 7777, + "sha256": "d3989353ece6a0c09e1f68ac16c5dbe8c949cd32c3831f41d4c2af835a3512e5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_ops.hip": { + "bytes": 22703, + "sha256": "650a7d033b3ce3fc7a6c9a2e6bcfa80963c172ad5ef58421e2956892aac80935" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_quant_dequant.h": { + "bytes": 24573, + "sha256": "a00ad5d98ae63fbb37683c1e26aa5494ca48a95f10c98bb817b6e780de481546" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_quant_iq_tables.h": { + "bytes": 110855, + "sha256": "bdee96a810ce7ceab6102025ea053ca7ecdb3caea5118727e02bbe69d7d784f7" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/CMakeLists.txt": { + "bytes": 275318, + "sha256": "45ee683859f89aa7c4c8d5ec253e46a0cf4d8de5eb33ac5f196c8fd413d3cba1" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/capi/rocm_quant_gather_fixture.h": { + "bytes": 8283, + "sha256": "339414f93e59e6be9a4ca545d5e762bb2816714343f466e218dbc2b69483bdfd" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/capi/test_rocm_embedding_quant.cpp": { + "bytes": 7710, + "sha256": "39ac4db3befafc27f7b986bf1691d6bde22448507f2c7e349b3235bed8e6aa01" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vllm/test_gguf_keep_quant.cpp": { + "bytes": 135199, + "sha256": "636d70c30cf28d7d3564874ac8195fda694da597998f44d53fee2df28f67820c" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vt/test_cuda_embedding_quant.cpp": { + "bytes": 23248, + "sha256": "ae4b5d161eda0e1c18b9534b640cfec467f4cc788fb84de4e4ffc95e8eece5f7" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vt/test_rocm_embedding_quant.cpp": { + "bytes": 17753, + "sha256": "e94888dbbaa772f643516dc532d1ea8400fb6741d65ba715f19dde04706e3023" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vllm/v1/worker/test_runner.cpp": { + "bytes": 176769, + "sha256": "8b70d78678245776737965cd461123b62716ed1d2cf130d604234e67519f09b2" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_runner": { + "bytes": 31530376, + "sha256": "e6d2d50be84e022817f0f9aaf54acd2c5196b9f1a50a2ec40a0e60acaed78b6e" + } + }, + "runs": [ + { + "name": "focused-and-rocm-regressions", + "argv": [ + "ctest", + "--test-dir", + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather", + "--output-on-failure", + "--verbose", + "-R", + "^(test_rocm_embedding_quant|test_capi_rocm_embedding_quant|test_gguf_keep_quant|test_ops_embedding_quant|test_gguf_dequant|test_backend_cross_device|test_gguf|test_rocm_arch|test_rocm_backend|test_rocm_fp8_kv_cache|test_ops_rocm_silu_rounding|test_exl3_rocm|test_gemma4_rocm_fp8_seams|test_backend_multidevice|test_runner)$" + ], + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_PUBLIC_EVIDENCE": "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures", + "VT_ROCM_GATHER_TEST_CORPUS": "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/random-corpus" + }, + "cwd": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "log": "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/focused-and-regressions.log" + }, + { + "name": "required-ambient-device", + "argv": [ + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant", + "--test-case=ROCm gather follows the queue device when the ambient device differs", + "--abort-after=1" + ], + "env": { + "HIP_VISIBLE_DEVICES": "0,1", + "ROCR_VISIBLE_DEVICES": "0,1", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_REQUIRE_TWO_DEVICES": "1" + }, + "cwd": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "log": "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/ambient-device.log" + } + ], + "authority": "Only root operator runs GPU commands under its recorded mutex", + "expected": [ + "All15 CTests execute and pass; no CTest skip accepted", + "19codecs*2prompts*3fresh-engine repeats*2arms produces228 completions, each with4tokens and512 finite logits", + "All native quantized completions and repeatedruns match their same-binary dense controls exactly", + "Separate required2-device witness passes row selection,fullwidthbounds,and recovery after ambient1/queue0 switching" + ], + "build_logs": [ + "/home/vikash/.cache/rdna3-gather-impl/evidence/hip-final-clean-build.log", + "/home/vikash/.cache/rdna3-gather-impl/evidence/hip-runner-build.log" + ], + "dependency": "Imported78c7441bdspec+694f04525test as8c9101fa4/b46852d13; imported3files byteequal donor; gatherstagedfiles byteequal acrossimport" +} diff --git a/docs/bench-evidence/rocm-quant-gather/native-final-v4-operator-capture-audit.json b/docs/bench-evidence/rocm-quant-gather/native-final-v4-operator-capture-audit.json new file mode 100644 index 0000000000..747a849436 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/native-final-v4-operator-capture-audit.json @@ -0,0 +1,470 @@ +{ + "result": "PASS", + "ctests_passed": 15, + "ctest_skips": 0, + "two_device_assertions": 16, + "public_engines": 228, + "completion_artifacts": 456, + "all_captures_byte_exact_vs_traced_v3": true, + "all_logits_finite": true, + "artifacts": { + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r0-p0-dense-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r0-p0-dense-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r0-p0-quant-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r0-p0-quant-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r0-p1-dense-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r0-p1-dense-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r0-p1-quant-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r0-p1-quant-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r1-p0-dense-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r1-p0-dense-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r1-p0-quant-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r1-p0-quant-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r1-p1-dense-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r1-p1-dense-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r1-p1-quant-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r1-p1-quant-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r2-p0-dense-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r2-p0-dense-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r2-p0-quant-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r2-p0-quant-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r2-p1-dense-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r2-p1-dense-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r2-p1-quant-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_S-r2-p1-quant-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r0-p0-dense-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r0-p0-dense-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r0-p0-quant-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r0-p0-quant-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r0-p1-dense-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r0-p1-dense-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r0-p1-quant-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r0-p1-quant-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r1-p0-dense-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r1-p0-dense-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r1-p0-quant-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r1-p0-quant-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r1-p1-dense-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r1-p1-dense-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r1-p1-quant-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r1-p1-quant-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r2-p0-dense-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r2-p0-dense-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r2-p0-quant-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r2-p0-quant-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r2-p1-dense-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r2-p1-dense-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r2-p1-quant-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ1_XXXS-r2-p1-quant-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r0-p0-dense-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r0-p0-dense-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r0-p0-quant-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r0-p0-quant-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r0-p1-dense-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r0-p1-dense-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r0-p1-quant-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r0-p1-quant-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r1-p0-dense-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r1-p0-dense-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r1-p0-quant-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r1-p0-quant-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r1-p1-dense-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r1-p1-dense-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r1-p1-quant-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r1-p1-quant-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r2-p0-dense-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r2-p0-dense-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r2-p0-quant-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r2-p0-quant-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r2-p1-dense-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r2-p1-dense-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r2-p1-quant-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_S-r2-p1-quant-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r0-p0-dense-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r0-p0-dense-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r0-p0-quant-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r0-p0-quant-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r0-p1-dense-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r0-p1-dense-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r0-p1-quant-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r0-p1-quant-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r1-p0-dense-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r1-p0-dense-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r1-p0-quant-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r1-p0-quant-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r1-p1-dense-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r1-p1-dense-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r1-p1-quant-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r1-p1-quant-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r2-p0-dense-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r2-p0-dense-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r2-p0-quant-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r2-p0-quant-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r2-p1-dense-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r2-p1-dense-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r2-p1-quant-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XS-r2-p1-quant-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r0-p0-dense-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r0-p0-dense-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r0-p0-quant-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r0-p0-quant-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r0-p1-dense-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r0-p1-dense-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r0-p1-quant-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r0-p1-quant-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r1-p0-dense-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r1-p0-dense-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r1-p0-quant-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r1-p0-quant-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r1-p1-dense-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r1-p1-dense-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r1-p1-quant-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r1-p1-quant-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r2-p0-dense-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r2-p0-dense-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r2-p0-quant-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r2-p0-quant-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r2-p1-dense-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r2-p1-dense-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r2-p1-quant-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ2_XXS-r2-p1-quant-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r0-p0-dense-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r0-p0-dense-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r0-p0-quant-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r0-p0-quant-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r0-p1-dense-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r0-p1-dense-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r0-p1-quant-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r0-p1-quant-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r1-p0-dense-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r1-p0-dense-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r1-p0-quant-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r1-p0-quant-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r1-p1-dense-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r1-p1-dense-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r1-p1-quant-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r1-p1-quant-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r2-p0-dense-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r2-p0-dense-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r2-p0-quant-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r2-p0-quant-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r2-p1-dense-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r2-p1-dense-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r2-p1-quant-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_S-r2-p1-quant-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r0-p0-dense-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r0-p0-dense-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r0-p0-quant-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r0-p0-quant-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r0-p1-dense-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r0-p1-dense-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r0-p1-quant-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r0-p1-quant-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r1-p0-dense-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r1-p0-dense-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r1-p0-quant-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r1-p0-quant-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r1-p1-dense-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r1-p1-dense-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r1-p1-quant-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r1-p1-quant-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r2-p0-dense-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r2-p0-dense-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r2-p0-quant-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r2-p0-quant-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r2-p1-dense-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r2-p1-dense-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r2-p1-quant-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ3_XXS-r2-p1-quant-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r0-p0-dense-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r0-p0-dense-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r0-p0-quant-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r0-p0-quant-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r0-p1-dense-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r0-p1-dense-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r0-p1-quant-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r0-p1-quant-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r1-p0-dense-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r1-p0-dense-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r1-p0-quant-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r1-p0-quant-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r1-p1-dense-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r1-p1-dense-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r1-p1-quant-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r1-p1-quant-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r2-p0-dense-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r2-p0-dense-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r2-p0-quant-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r2-p0-quant-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r2-p1-dense-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r2-p1-dense-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r2-p1-quant-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_NL-r2-p1-quant-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r0-p0-dense-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r0-p0-dense-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r0-p0-quant-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r0-p0-quant-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r0-p1-dense-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r0-p1-dense-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r0-p1-quant-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r0-p1-quant-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r1-p0-dense-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r1-p0-dense-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r1-p0-quant-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r1-p0-quant-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r1-p1-dense-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r1-p1-dense-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r1-p1-quant-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r1-p1-quant-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r2-p0-dense-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r2-p0-dense-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r2-p0-quant-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r2-p0-quant-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r2-p1-dense-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r2-p1-dense-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r2-p1-quant-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/IQ4_XS-r2-p1-quant-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r0-p0-dense-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r0-p0-dense-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r0-p0-quant-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r0-p0-quant-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r0-p1-dense-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r0-p1-dense-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r0-p1-quant-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r0-p1-quant-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r1-p0-dense-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r1-p0-dense-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r1-p0-quant-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r1-p0-quant-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r1-p1-dense-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r1-p1-dense-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r1-p1-quant-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r1-p1-quant-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r2-p0-dense-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r2-p0-dense-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r2-p0-quant-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r2-p0-quant-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r2-p1-dense-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r2-p1-dense-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r2-p1-quant-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/MXFP4-r2-p1-quant-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r0-p0-dense-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r0-p0-dense-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r0-p0-quant-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r0-p0-quant-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r0-p1-dense-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r0-p1-dense-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r0-p1-quant-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r0-p1-quant-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r1-p0-dense-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r1-p0-dense-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r1-p0-quant-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r1-p0-quant-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r1-p1-dense-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r1-p1-dense-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r1-p1-quant-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r1-p1-quant-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r2-p0-dense-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r2-p0-dense-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r2-p0-quant-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r2-p0-quant-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r2-p1-dense-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r2-p1-dense-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r2-p1-quant-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q2_K-r2-p1-quant-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r0-p0-dense-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r0-p0-dense-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r0-p0-quant-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r0-p0-quant-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r0-p1-dense-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r0-p1-dense-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r0-p1-quant-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r0-p1-quant-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r1-p0-dense-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r1-p0-dense-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r1-p0-quant-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r1-p0-quant-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r1-p1-dense-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r1-p1-dense-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r1-p1-quant-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r1-p1-quant-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r2-p0-dense-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r2-p0-dense-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r2-p0-quant-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r2-p0-quant-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r2-p1-dense-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r2-p1-dense-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r2-p1-quant-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q3_K-r2-p1-quant-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r0-p0-dense-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r0-p0-dense-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r0-p0-quant-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r0-p0-quant-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r0-p1-dense-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r0-p1-dense-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r0-p1-quant-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r0-p1-quant-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r1-p0-dense-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r1-p0-dense-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r1-p0-quant-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r1-p0-quant-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r1-p1-dense-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r1-p1-dense-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r1-p1-quant-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r1-p1-quant-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r2-p0-dense-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r2-p0-dense-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r2-p0-quant-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r2-p0-quant-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r2-p1-dense-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r2-p1-dense-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r2-p1-quant-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_0-r2-p1-quant-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r0-p0-dense-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r0-p0-dense-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r0-p0-quant-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r0-p0-quant-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r0-p1-dense-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r0-p1-dense-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r0-p1-quant-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r0-p1-quant-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r1-p0-dense-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r1-p0-dense-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r1-p0-quant-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r1-p0-quant-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r1-p1-dense-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r1-p1-dense-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r1-p1-quant-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r1-p1-quant-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r2-p0-dense-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r2-p0-dense-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r2-p0-quant-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r2-p0-quant-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r2-p1-dense-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r2-p1-dense-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r2-p1-quant-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q4_K-r2-p1-quant-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r0-p0-dense-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r0-p0-dense-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r0-p0-quant-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r0-p0-quant-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r0-p1-dense-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r0-p1-dense-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r0-p1-quant-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r0-p1-quant-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r1-p0-dense-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r1-p0-dense-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r1-p0-quant-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r1-p0-quant-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r1-p1-dense-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r1-p1-dense-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r1-p1-quant-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r1-p1-quant-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r2-p0-dense-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r2-p0-dense-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r2-p0-quant-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r2-p0-quant-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r2-p1-dense-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r2-p1-dense-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r2-p1-quant-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_0-r2-p1-quant-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r0-p0-dense-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r0-p0-dense-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r0-p0-quant-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r0-p0-quant-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r0-p1-dense-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r0-p1-dense-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r0-p1-quant-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r0-p1-quant-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r1-p0-dense-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r1-p0-dense-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r1-p0-quant-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r1-p0-quant-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r1-p1-dense-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r1-p1-dense-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r1-p1-quant-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r1-p1-quant-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r2-p0-dense-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r2-p0-dense-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r2-p0-quant-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r2-p0-quant-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r2-p1-dense-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r2-p1-dense-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r2-p1-quant-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q5_K-r2-p1-quant-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r0-p0-dense-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r0-p0-dense-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r0-p0-quant-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r0-p0-quant-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r0-p1-dense-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r0-p1-dense-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r0-p1-quant-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r0-p1-quant-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r1-p0-dense-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r1-p0-dense-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r1-p0-quant-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r1-p0-quant-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r1-p1-dense-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r1-p1-dense-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r1-p1-quant-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r1-p1-quant-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r2-p0-dense-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r2-p0-dense-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r2-p0-quant-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r2-p0-quant-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r2-p1-dense-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r2-p1-dense-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r2-p1-quant-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q6_K-r2-p1-quant-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r0-p0-dense-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r0-p0-dense-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r0-p0-quant-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r0-p0-quant-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r0-p1-dense-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r0-p1-dense-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r0-p1-quant-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r0-p1-quant-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r1-p0-dense-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r1-p0-dense-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r1-p0-quant-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r1-p0-quant-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r1-p1-dense-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r1-p1-dense-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r1-p1-quant-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r1-p1-quant-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r2-p0-dense-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r2-p0-dense-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r2-p0-quant-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r2-p0-quant-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r2-p1-dense-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r2-p1-dense-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r2-p1-quant-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_0-r2-p1-quant-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r0-p0-dense-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r0-p0-dense-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r0-p0-quant-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r0-p0-quant-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r0-p1-dense-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r0-p1-dense-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r0-p1-quant-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r0-p1-quant-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r1-p0-dense-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r1-p0-dense-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r1-p0-quant-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r1-p0-quant-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r1-p1-dense-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r1-p1-dense-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r1-p1-quant-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r1-p1-quant-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r2-p0-dense-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r2-p0-dense-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r2-p0-quant-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r2-p0-quant-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r2-p1-dense-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r2-p1-dense-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r2-p1-quant-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures/Q8_K-r2-p1-quant-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "focused_log_sha256": "655a9a16c07f8c9300e36fff4482106d91868935952efe2cc9dc5ef53ba7816f", + "ambient_log_sha256": "38fec7edbc44a0e557538088a130e98da012892fdec43f29234989d35718209e" +} diff --git a/docs/bench-evidence/rocm-quant-gather/native-final-v4-operator-results.json b/docs/bench-evidence/rocm-quant-gather/native-final-v4-operator-results.json new file mode 100644 index 0000000000..63c0b4d53e --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/native-final-v4-operator-results.json @@ -0,0 +1,53 @@ +[ + { + "name": "focused-and-rocm-regressions", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "ctest", + "--test-dir", + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather", + "--output-on-failure", + "--verbose", + "-R", + "^(test_rocm_embedding_quant|test_capi_rocm_embedding_quant|test_gguf_keep_quant|test_ops_embedding_quant|test_gguf_dequant|test_backend_cross_device|test_gguf|test_rocm_arch|test_rocm_backend|test_rocm_fp8_kv_cache|test_ops_rocm_silu_rounding|test_exl3_rocm|test_gemma4_rocm_fp8_seams|test_backend_multidevice|test_runner)$" + ], + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_PUBLIC_EVIDENCE": "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/public-captures", + "VT_ROCM_GATHER_TEST_CORPUS": "/home/vikash/.cache/rdna3-gather-impl/native-final-v4/random-corpus" + }, + "cwd": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "exit": 0, + "seconds": 25.545061588287354, + "log_sha256": "655a9a16c07f8c9300e36fff4482106d91868935952efe2cc9dc5ef53ba7816f", + "inputs_unchanged": true + }, + { + "name": "required-ambient-device", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant", + "--test-case=ROCm gather follows the queue device when the ambient device differs", + "--abort-after=1" + ], + "env": { + "HIP_VISIBLE_DEVICES": "0,1", + "ROCR_VISIBLE_DEVICES": "0,1", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_REQUIRE_TWO_DEVICES": "1" + }, + "cwd": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "exit": 0, + "seconds": 0.7371315956115723, + "log_sha256": "38fec7edbc44a0e557538088a130e98da012892fdec43f29234989d35718209e", + "inputs_unchanged": true + } +] diff --git a/docs/bench-evidence/rocm-quant-gather/native-focused-v1.json b/docs/bench-evidence/rocm-quant-gather/native-focused-v1.json new file mode 100644 index 0000000000..7b24519135 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/native-focused-v1.json @@ -0,0 +1,101 @@ +{ + "version": 1, + "row": "BACKEND-ROCM-QUANT-GATHER", + "base_head": "4fc300f93cfb075c0afddccdd476a552fa900cdb", + "uncommitted_native_implementation": true, + "files": { + "/home/vikash/vllm.cpp-rdna3-gather-impl/CMakeLists.txt": { + "bytes": 175502, + "sha256": "e5bea1baa842ac0257fb29d61cb6ef37977d975d6ac0478f81d6e677fe85ce32" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/CMakeLists.txt": { + "bytes": 275052, + "sha256": "efb00054342fdaec87e9181495ffdf5abd55257d8545b829b69f13ff8dbc93dc" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/libvllm.a": { + "bytes": 95078500, + "sha256": "acbf7ceb5c2efa04ff56189e04de549ebe760ba70f3d7967f684f2e7709431cb" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_embedding_quant.hip": { + "bytes": 7556, + "sha256": "5934d1727c96ec8e76f4e7dc3805e15d271fb528b7c0ad6c3a5d2863a179686b" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_quant_iq_tables.h": { + "bytes": 110865, + "sha256": "d267d25ff1dd3f67749c251f87047f4b70c9650a99d04b524ff6c3313596a349" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_embedding_quant.h": { + "bytes": 554, + "sha256": "1b2b31617d660b64f622e692089423ce1657df102322558a4ab731f521952bc5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_quant_dequant.h": { + "bytes": 25507, + "sha256": "413dc138254cc72f8dccbc87268581217c93dec8f8acaea159f3f39569c37e62" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_ops.hip": { + "bytes": 22703, + "sha256": "650a7d033b3ce3fc7a6c9a2e6bcfa80963c172ad5ef58421e2956892aac80935" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/capi/rocm_quant_gather_fixture.h": { + "bytes": 8283, + "sha256": "339414f93e59e6be9a4ca545d5e762bb2816714343f466e218dbc2b69483bdfd" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/capi/test_rocm_embedding_quant.cpp": { + "bytes": 6592, + "sha256": "334164dbb5c767dd0c67fa554931b41465385f41ab3c522f21f669b34a4884aa" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vt/test_rocm_embedding_quant.cpp": { + "bytes": 14781, + "sha256": "cb8bad70113bc1730ae1f9aafba4fd26b84601ff28f3c33f979be2d9d964b367" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vllm/test_gguf_keep_quant.cpp": { + "bytes": 132629, + "sha256": "5168816c430f1a07dcf7e0790dc5b6ab2cf4a1498c61383b6b693411578bc5f7" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant": { + "bytes": 30942784, + "sha256": "76420ee7836c7265d2709ab7910c1ed3b4abf94f44c28b3a1da747caf39f9134" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_capi_rocm_embedding_quant": { + "bytes": 30929264, + "sha256": "49e47dae389dc4b3f150837ab743eea3c8e74d815dfaf341e43ec083ae0a2ce4" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_keep_quant": { + "bytes": 31321648, + "sha256": "cc70d1a966c32d6127c043e5cb2417c4852b4c5197399e75edf80339ea6921d6" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_ops_embedding_quant": { + "bytes": 30845880, + "sha256": "550044a9cd421c872770e6fd84b44d0fb9b749297a58baf5c598d7f244642585" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_dequant": { + "bytes": 30962624, + "sha256": "7fc76b76f5984314b707fa9256b4cfba82bab3bc0473d3df052734597cc09bd1" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_backend_cross_device": { + "bytes": 31469712, + "sha256": "c6b1500af691d1c021ea7a642df838c03b03cc1da2c7bbb0a0f393ea74b64615" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/rocm_quant_gather_capture": { + "bytes": 30693800, + "sha256": "af71f0a1583a1e970f6a9c5b6ff39e1c2d39424770dd8700911a9ff00b48d411" + } + }, + "cwd": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp" + }, + "argv": [ + "ctest", + "--test-dir", + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather", + "--output-on-failure", + "-R", + "^(test_rocm_embedding_quant|test_capi_rocm_embedding_quant|test_gguf_keep_quant|test_ops_embedding_quant|test_gguf_dequant|test_backend_cross_device)$" + ], + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/native-focused-v1.log", + "authority": "Operator alone executes under the recorded GPU 0 mutex", + "expected": "All six targets execute and pass. No exit-77 skip satisfies the native acceptance run." +} diff --git a/docs/bench-evidence/rocm-quant-gather/native-focused-v2-operator-results.json b/docs/bench-evidence/rocm-quant-gather/native-focused-v2-operator-results.json new file mode 100644 index 0000000000..749a3bbde5 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/native-focused-v2-operator-results.json @@ -0,0 +1,88 @@ +[ + { + "name": "focused-v2", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "env", + "HIP_VISIBLE_DEVICES=0", + "ROCR_VISIBLE_DEVICES=0", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_TEST_CORPUS=/home/vikash/.cache/rdna3-gather-impl/random-corpus-v2", + "timeout", + "180s", + "ctest", + "--test-dir", + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather", + "--output-on-failure", + "-R", + "^(test_rocm_embedding_quant|test_capi_rocm_embedding_quant|test_gguf_keep_quant|test_ops_embedding_quant|test_gguf_dequant|test_backend_cross_device|test_gguf)$" + ], + "exit_code": 0, + "elapsed_seconds": 21.807591438293457, + "manifest_sha256": "2928aa6cbc417a3993c86481cbdd6c3ba4e35072bbf703212976344bda57fed0", + "log_sha256": "90498bb132e941140e153a2634e8247c46faebad08172205cfed52806d612ca1", + "all_101_manifest_inputs_verified_before_after": true, + "core_dumps_disabled": true, + "summary": [ + "100% tests passed, 0 tests failed out of 7" + ] + }, + { + "name": "ambient-device-v2", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "env", + "HIP_VISIBLE_DEVICES=0,1", + "ROCR_VISIBLE_DEVICES=0,1", + "VT_ROCM_GATHER_REQUIRE_TWO_DEVICES=1", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "timeout", + "180s", + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant", + "--test-case=ROCm gather follows the queue device when the ambient device differs", + "--abort-after=1" + ], + "exit_code": 0, + "elapsed_seconds": 0.5165746212005615, + "manifest_sha256": "2928aa6cbc417a3993c86481cbdd6c3ba4e35072bbf703212976344bda57fed0", + "log_sha256": "38fec7edbc44a0e557538088a130e98da012892fdec43f29234989d35718209e", + "all_101_manifest_inputs_verified_before_after": true, + "core_dumps_disabled": true, + "summary": [ + "[doctest] test cases: 1 | 1 passed | 0 failed | 8 skipped", + "[doctest] assertions: 16 | 16 passed | 0 failed |" + ] + }, + { + "name": "native-corpus-v2", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "env", + "HIP_VISIBLE_DEVICES=0", + "ROCR_VISIBLE_DEVICES=0", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "timeout", + "180s", + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/rocm_quant_gather_capture", + "capture", + "/home/vikash/.cache/rdna3-gather-impl/corpus/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2" + ], + "exit_code": 0, + "elapsed_seconds": 0.853466272354126, + "manifest_sha256": "2928aa6cbc417a3993c86481cbdd6c3ba4e35072bbf703212976344bda57fed0", + "log_sha256": "d28f0be34a2c630d52fc5096a805c200d04397f36517e47d186c0cfbfe2a604f", + "all_101_manifest_inputs_verified_before_after": true, + "core_dumps_disabled": true, + "summary": [] + } +] diff --git a/docs/bench-evidence/rocm-quant-gather/native-focused-v2.json b/docs/bench-evidence/rocm-quant-gather/native-focused-v2.json new file mode 100644 index 0000000000..7fd790dc20 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/native-focused-v2.json @@ -0,0 +1,464 @@ +{ + "version": 2, + "row": "BACKEND-ROCM-QUANT-GATHER", + "base_head": "2e5adfb85a255528bffa5d3cdb016efb2bab70d8", + "uncommitted_native_implementation": true, + "files": { + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_S-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_S-1024.packed": { + "bytes": 25600, + "sha256": "6bf00a4b02da43e5505823b12d9e368aa1ded7f87427eea02ec03ebc68288c33" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_S-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_S-256.packed": { + "bytes": 6400, + "sha256": "754f0d0b9e7d3656a1f669cf756779352cccb246bf6aa0b25e7f362180cf0882" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-1024.packed": { + "bytes": 19456, + "sha256": "ae8e061519977e2ba7a77ae9d30e21193b1a009b5ed38739974e945de09b94c6" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-256.packed": { + "bytes": 4864, + "sha256": "af0b5e336987746fd49cd33c790b99eb1479c26f64b6679fa6c685e5934ba0c4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_S-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_S-1024.packed": { + "bytes": 41984, + "sha256": "2cc5c79aef24da7bb363c72b7ac91f05c38b87fdc96627253203926f4465d73e" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_S-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_S-256.packed": { + "bytes": 10496, + "sha256": "6eb1966662d9e424fefc5c3625282e7f33fdfab3af9cb4c0cbac6bdc593711ea" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XS-1024.packed": { + "bytes": 37888, + "sha256": "8fbc716e220d386b0c27f1cca003384091cd0e522c1442656677af4ac780de5f" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XS-256.packed": { + "bytes": 9472, + "sha256": "3281c7397dde054ea6c9c8a661cab43dbd0c982eb23b9a31523a616565c827e7" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XXS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XXS-1024.packed": { + "bytes": 33792, + "sha256": "2fbe6e6ffcbc32fda24c5d63d4a4b678283e2bd571f491210097a27b88cc1fd4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XXS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XXS-256.packed": { + "bytes": 8448, + "sha256": "4a3c10afb47315763ce507edcb5814e06d1be93a21b178cf73183302a3f1ceb9" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_S-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_S-1024.packed": { + "bytes": 56320, + "sha256": "a94d30560f904618cead2f0618aea6455d5183eb3ed443ca5c12e48fe5b31b34" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_S-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_S-256.packed": { + "bytes": 14080, + "sha256": "d4eebe1369e21b0efd6ec5c89f54c785849c02edde957ebd302dad376f63c14c" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_XXS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_XXS-1024.packed": { + "bytes": 50176, + "sha256": "2d3a0d835fe7a47d802b59e8e412e4084de78faa2aaca59d1ce565fad11ac34a" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_XXS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_XXS-256.packed": { + "bytes": 12544, + "sha256": "d0f775f8e88d87ae542de6d16122a7a1cea09ebba91203f6513b74727082b6db" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_NL-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_NL-1024.packed": { + "bytes": 73728, + "sha256": "4e50d50e97e08c448cfff4fbed121ef24d657cfd30c2e09f66e70c647ce545b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_NL-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_NL-256.packed": { + "bytes": 18432, + "sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_XS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_XS-1024.packed": { + "bytes": 69632, + "sha256": "189f75d03197100afafd3fe92183e47a2d73665d64f89b36d8b10daa05adf2c4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_XS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_XS-256.packed": { + "bytes": 17408, + "sha256": "35b2a41e738a6cb0a86deb4fd366f71ae2dc2a6fad7169f5a9afb53f523e919c" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-1024.packed": { + "bytes": 69632, + "sha256": "929ef7d789f210f60cede40156f689734aaf36519cd1f43a82d44d17ae0adfb4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-256.packed": { + "bytes": 17408, + "sha256": "baa828a753cc6f496f356271dad644976cd57d1218517e3f641fe56b7ec8db4a" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q2_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q2_K-1024.packed": { + "bytes": 43008, + "sha256": "58cb93e09423c7151fac47084de7e38bff98f51e5f304de243b41a17c47ac4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q2_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q2_K-256.packed": { + "bytes": 10752, + "sha256": "655f77ebb32d974da7907da4a217c45bbbcd0b127df73a731f85afe427bb7dcc" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q3_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q3_K-1024.packed": { + "bytes": 56320, + "sha256": "6118b4b2f10678fb09a2cb3324d69443ff3b33805f403b79ed3432d5bc1b2732" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q3_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q3_K-256.packed": { + "bytes": 14080, + "sha256": "e4c3a11c1bbff806ae2729033910b7db606452d1b9aa499aa2ced46740ad85d4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_0-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_0-1024.packed": { + "bytes": 73728, + "sha256": "4e50d50e97e08c448cfff4fbed121ef24d657cfd30c2e09f66e70c647ce545b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_0-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_0-256.packed": { + "bytes": 18432, + "sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_K-1024.packed": { + "bytes": 73728, + "sha256": "d15d1bb953010f1ecdac424a54c365308e3b058a1b818bafb89cfc9b9b1678cc" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_K-256.packed": { + "bytes": 18432, + "sha256": "b0c107c4b353e2a169247e0406c1eb015d7a6d97edd5906f710868b8c2cecce1" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_0-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_0-1024.packed": { + "bytes": 90112, + "sha256": "b1d241ee00677d6d55a5e774916608aa4479e6c549d8ed1e4803c0938cd6a855" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_0-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_0-256.packed": { + "bytes": 22528, + "sha256": "b3531b1abd263bc4c6a3174b22fa1fd54a4b89b66e737f2eb3ba36be771d7528" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_K-1024.packed": { + "bytes": 90112, + "sha256": "6dd54fbc62a9818334e50347fe185247156ecc833a5655ce1920651a4dc537b5" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_K-256.packed": { + "bytes": 22528, + "sha256": "5e63d961f36486c9636dd2657e9069bd83ece65e466c56c657265fbd97353b07" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q6_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q6_K-1024.packed": { + "bytes": 107520, + "sha256": "2487007668582b61b3c59fdbaf0e92c2f5c74a61781daba95c13704983643ab1" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q6_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q6_K-256.packed": { + "bytes": 26880, + "sha256": "134bba0f8076d43d31fcbbaeb37c3d8191b0d383e6595f787d3387e2e016738a" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_0-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_0-1024.packed": { + "bytes": 139264, + "sha256": "c4435564bf5fd2dc23e0c78107f28ebd033ec741c417d41fa9dcbd89dd2909ba" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_0-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_0-256.packed": { + "bytes": 34816, + "sha256": "8be599e194405f9b5429e578a03a5717e4d78f53ab4c9c11a52659495ce805ee" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-1024.packed": { + "bytes": 149504, + "sha256": "78e39e7ef7317d842a6fd965b356ceb1959a5ce03da824fbc699689ea2ef4877" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-256.packed": { + "bytes": 37376, + "sha256": "bcf6f6691a787fe3f2a9bcc3d7923eeb4953862c30f904a6b10fc07410f8d657" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/manifest.json": { + "bytes": 6329, + "sha256": "78bc808e19b40ea745f5e767b72618dbd8087bb685333345826257dd4371ccfd" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/CMakeLists.txt": { + "bytes": 175502, + "sha256": "e5bea1baa842ac0257fb29d61cb6ef37977d975d6ac0478f81d6e677fe85ce32" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/libvllm.a": { + "bytes": 95078032, + "sha256": "e1e56ec9535a7501a8313986861dfa7bed3fde35f50dbdbc3ba05c4aac7b1afa" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/rocm_quant_gather_capture": { + "bytes": 30691544, + "sha256": "a61c8cce44145aa40245d32165810fbeebb5dd9e5e21a3cf65ea4c37957ed9d5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_backend_cross_device": { + "bytes": 31467496, + "sha256": "4a03440c90f387a391ca312b769f0e3cf949909598575ff902c0f67b4fdc756f" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_capi_rocm_embedding_quant": { + "bytes": 30928032, + "sha256": "91fa1d6dea40262aa1170faa4bc9ea7dab4d01cf0ec7ea9a8ef7f020da583223" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf": { + "bytes": 31012152, + "sha256": "e6e2e2caee1e8ef17cbcd4dca5dca578c3c1e8cdb35c4cbad9bf259646eee9d1" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_dequant": { + "bytes": 30960400, + "sha256": "0c6f4d003de2ec308fbef18d661843dfc21c59f9d31b27a9a0c8eda1f42f9fa3" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_keep_quant": { + "bytes": 31352656, + "sha256": "28e3502d940688a0a2b795e93f0456595e55b13e4508a8e0d7b448f8a3b977b2" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_ops_embedding_quant": { + "bytes": 30843664, + "sha256": "adf3962f3f99c7e28cd2cbded31e36b8a324908fa6a6fa610d354b5b4de29b4a" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant": { + "bytes": 30951776, + "sha256": "85bdf30e85e8d025bb7cfb2bb44f4d717bc0797edae59c2a79cb37b2192556ec" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/include/vllm/model_executor/models/qwen3_5_gguf_weights.h": { + "bytes": 18080, + "sha256": "ed37c245f574d2f42df14b40261a8ecb6ae56c159588a82e1550fc49ccaf4655" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vllm/model_executor/model_loader/gguf_reader.cpp": { + "bytes": 29998, + "sha256": "2901344cb8252ac3c66cacd9b2ae137a814217425df28c387bbdade83d9c12e5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp": { + "bytes": 90683, + "sha256": "685ca1623c336876768038a8aa28260694cd4207a1ac1e2d488bc81e451be4fb" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_embedding_quant.h": { + "bytes": 554, + "sha256": "1b2b31617d660b64f622e692089423ce1657df102322558a4ab731f521952bc5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_embedding_quant.hip": { + "bytes": 7777, + "sha256": "d3989353ece6a0c09e1f68ac16c5dbe8c949cd32c3831f41d4c2af835a3512e5" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_ops.hip": { + "bytes": 22703, + "sha256": "650a7d033b3ce3fc7a6c9a2e6bcfa80963c172ad5ef58421e2956892aac80935" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_quant_dequant.h": { + "bytes": 25507, + "sha256": "413dc138254cc72f8dccbc87268581217c93dec8f8acaea159f3f39569c37e62" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_quant_iq_tables.h": { + "bytes": 110865, + "sha256": "d267d25ff1dd3f67749c251f87047f4b70c9650a99d04b524ff6c3313596a349" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/CMakeLists.txt": { + "bytes": 275318, + "sha256": "45ee683859f89aa7c4c8d5ec253e46a0cf4d8de5eb33ac5f196c8fd413d3cba1" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/capi/rocm_quant_gather_fixture.h": { + "bytes": 8283, + "sha256": "339414f93e59e6be9a4ca545d5e762bb2816714343f466e218dbc2b69483bdfd" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/capi/test_rocm_embedding_quant.cpp": { + "bytes": 6787, + "sha256": "5b46a2001b3c987fe3e23e836bd25f9cdc9ff7886422cbe2ffe0909781543bd2" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vllm/test_gguf_keep_quant.cpp": { + "bytes": 135199, + "sha256": "636d70c30cf28d7d3564874ac8195fda694da597998f44d53fee2df28f67820c" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vt/test_rocm_embedding_quant.cpp": { + "bytes": 17542, + "sha256": "7026334d9ab70b3fa56cf0c4af4012f054e539ba37a48c586732742d7278a927" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tools/rocm_quant_gather/capture.cpp": { + "bytes": 7540, + "sha256": "2443dd0cc7b59c32479ff68b4f09e8e857ca9c4962a79684e3cb93a7dc44f03d" + } + }, + "cwd": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_TEST_CORPUS": "/home/vikash/.cache/rdna3-gather-impl/random-corpus-v2" + }, + "argv": [ + "ctest", + "--test-dir", + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather", + "--output-on-failure", + "-R", + "^(test_rocm_embedding_quant|test_capi_rocm_embedding_quant|test_gguf_keep_quant|test_ops_embedding_quant|test_gguf_dequant|test_backend_cross_device|test_gguf)$" + ], + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/native-focused-v2.log", + "authority": "Operator alone executes under the recorded GPU mutex", + "expected": "All seven focused targets execute and pass. The ambient-device case reports its separate resource requirement in this one-device lane; the required two-device run below covers it.", + "additional_runs": [ + { + "name": "ambient-device-v2", + "env": { + "HIP_VISIBLE_DEVICES": "0,1", + "ROCR_VISIBLE_DEVICES": "0,1", + "VT_ROCM_GATHER_REQUIRE_TWO_DEVICES": "1", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp" + }, + "argv": [ + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_rocm_embedding_quant", + "--test-case=ROCm gather follows the queue device when the ambient device differs", + "--abort-after=1" + ], + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/ambient-device-witness-v2.log", + "expected": "rows, bounds error, recovery all pass on queue0 after ambient1 switches" + }, + { + "name": "native-corpus-v2", + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp" + }, + "argv": [ + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/rocm_quant_gather_capture", + "capture", + "/home/vikash/.cache/rdna3-gather-impl/corpus/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2" + ], + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/native-corpus-v2.log", + "expected": "76 outputs:19codecs,2widths,2dtypes; exactly one native gather per output" + } + ] +} diff --git a/docs/bench-evidence/rocm-quant-gather/native-operation-captures.tar.gz b/docs/bench-evidence/rocm-quant-gather/native-operation-captures.tar.gz new file mode 100644 index 0000000000..6f7e3d0a66 Binary files /dev/null and b/docs/bench-evidence/rocm-quant-gather/native-operation-captures.tar.gz differ diff --git a/docs/bench-evidence/rocm-quant-gather/native-v2-oracle-comparison-operator.json b/docs/bench-evidence/rocm-quant-gather/native-v2-oracle-comparison-operator.json new file mode 100644 index 0000000000..df1ab757c8 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/native-v2-oracle-comparison-operator.json @@ -0,0 +1,1843 @@ +{ + "scope": "Synthetic operation outputs only; model gate and original upstream fixtures remain separate", + "primary_tolerance": { + "atol": 0.01, + "rtol": 0.04 + }, + "secondary_bf16_adaptation": "Round the independently executed upstream F32 output once to BF16 with round-to-nearest-even", + "inputs": { + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/report.json": "867aaf215887b2c18f46b20b9211592550274099f721b4fff73cefb2f171dd38", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/report.json": "ffdf6a1c73cefd9362ba5fc135ede2c932c08b7386bd67309b75085582048b81", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/results.json": "2dc72bc0e153e382da980470675702f78cd22e208378ef0cf76388434d4d1be5", + "/home/vikash/.cache/rdna3-gather-impl/evidence/native-focused-v2.json": "2928aa6cbc417a3993c86481cbdd6c3ba4e35072bbf703212976344bda57fed0", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-256-f32.bin": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-256-f32.bin": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-256-bf16.bin": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-256-bf16.bin": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-1024-f32.bin": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-1024-f32.bin": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-1024-bf16.bin": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-1024-bf16.bin": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-256-f32.bin": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-256-f32.bin": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-256-bf16.bin": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-256-bf16.bin": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-1024-f32.bin": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-1024-f32.bin": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-1024-bf16.bin": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-1024-bf16.bin": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-256-f32.bin": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-256-f32.bin": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-256-bf16.bin": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-256-bf16.bin": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-1024-f32.bin": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-1024-f32.bin": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-1024-bf16.bin": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-1024-bf16.bin": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-256-f32.bin": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-256-f32.bin": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-256-bf16.bin": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-256-bf16.bin": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-1024-f32.bin": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-1024-f32.bin": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-1024-bf16.bin": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-1024-bf16.bin": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-256-f32.bin": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-256-f32.bin": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-256-bf16.bin": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-256-bf16.bin": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-1024-f32.bin": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-1024-f32.bin": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-1024-bf16.bin": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-1024-bf16.bin": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-256-f32.bin": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-256-f32.bin": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-256-bf16.bin": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-256-bf16.bin": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-1024-f32.bin": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-1024-f32.bin": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-1024-bf16.bin": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-1024-bf16.bin": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-256-f32.bin": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-256-f32.bin": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-256-bf16.bin": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-256-bf16.bin": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-1024-f32.bin": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-1024-f32.bin": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-1024-bf16.bin": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-1024-bf16.bin": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-256-f32.bin": "3e1665a3167a552c47493bdc538166acfa6cbd43a1e7dd30230c40e3c5b11207", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-256-f32.bin": "955ba972e48a303803cdacca928e7f60176dfaa1b7f8590038bdb91830658428", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-256-bf16.bin": "242d1b322588618517ee3d4d62e393d08b2faca2262414ab00d89d092598e8ee", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-256-bf16.bin": "c65412c7615787b9103207f55776d49dfda767740f4d6e1106cb711a99404bc6", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-1024-f32.bin": "158400e054e65af2adad3e7d348b2fc3e082fcfc065706db200704d3c86a2581", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-1024-f32.bin": "5a0f2bb0da03f1eb5c689c6f95be636890aec44873bffe77dd039c2b5bc6c20a", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-1024-bf16.bin": "54c4ea60561a742634c5fdf24c4dd372f0e101a48bf29b53d53a0b7e88f31aea", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-1024-bf16.bin": "5955c913676e2e5003f84ad0dca7062d3f7106188a13e4844b13774a2764c238", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_K-256-f32.bin": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/Q8_K-256-f32.bin": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_K-256-bf16.bin": "f5da512dcaf2f4165abd5b4b1377d275a5945d2ab8946d2d8266600c5b03d31f", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_K-1024-f32.bin": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/Q8_K-1024-f32.bin": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_K-1024-bf16.bin": "6a1aa0cd72dca3f6ae3f6b835eeba50a87bf7b143ad607baa30482f0835f8c22", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-256-f32.bin": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-256-f32.bin": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-256-bf16.bin": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-256-bf16.bin": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-1024-f32.bin": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-1024-f32.bin": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-1024-bf16.bin": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-1024-bf16.bin": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-256-f32.bin": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-256-f32.bin": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-256-bf16.bin": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-256-bf16.bin": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-1024-f32.bin": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-1024-f32.bin": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-1024-bf16.bin": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-1024-bf16.bin": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-256-f32.bin": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-256-f32.bin": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-256-bf16.bin": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-256-bf16.bin": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-1024-f32.bin": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-1024-f32.bin": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-1024-bf16.bin": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-1024-bf16.bin": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-256-f32.bin": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-256-f32.bin": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-256-bf16.bin": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-256-bf16.bin": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-1024-f32.bin": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-1024-f32.bin": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-1024-bf16.bin": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-1024-bf16.bin": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-256-f32.bin": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-256-f32.bin": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-256-bf16.bin": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-256-bf16.bin": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-1024-f32.bin": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-1024-f32.bin": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-1024-bf16.bin": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-1024-bf16.bin": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-256-f32.bin": "7843ff13e1c7f1475f949583ec0466b7cd3020723994d7c725e7c618c11c34f0", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-256-f32.bin": "8cad98f9fecbfcac883cb11a91df1ac703846dfe88bac12792ab3066420243ad", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-256-bf16.bin": "281888da0636914c257cd25407adc1465b2d40e96aa7694eb5c3bd0df4036b73", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-256-bf16.bin": "7f36f4a4ba9f0977f68d452e8ec1d26ea299ea9ddbc82fd0ede324abc94d6595", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-1024-f32.bin": "8932dbeb65d3716cfb60014ad471ab0fb5d2d2d4b037e8745464e65f77b936de", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-1024-f32.bin": "ebcadd200e64857ed483aa7176172dc94b966198fb81d2910ab723260fb46fdc", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-1024-bf16.bin": "11a954e23ccc31f98a3398902b7ce96ec39f0fc0119d41202732630299d9a623", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-1024-bf16.bin": "88e55dfa1c9378436bd431db09fca5bf5aad9910dc311b493d6f94d1ec98dc7c", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-256-f32.bin": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-256-f32.bin": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-256-bf16.bin": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-256-bf16.bin": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-1024-f32.bin": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-1024-f32.bin": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-1024-bf16.bin": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-1024-bf16.bin": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-256-f32.bin": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-256-f32.bin": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-256-bf16.bin": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-256-bf16.bin": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-1024-f32.bin": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-1024-f32.bin": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-1024-bf16.bin": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-1024-bf16.bin": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/MXFP4-256-f32.bin": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/MXFP4-256-f32.bin": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/MXFP4-256-bf16.bin": "4b66106696144b4970929d2f5734d3ddf8e6e4580a2fe8910b500ebaab5431a9", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/MXFP4-1024-f32.bin": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/MXFP4-1024-f32.bin": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/MXFP4-1024-bf16.bin": "cdd6e95bc3bc3e18bc014dee2cdb125cb179b670256896536b0e3822b0adfa83", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_XXXS-256-f32.bin": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/IQ1_XXXS-256-f32.bin": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_XXXS-256-bf16.bin": "9de69ca2d01fda18f05e9ee7ac531f3cb806d39a3af8d10e64ba0c53dab1b08a", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_XXXS-1024-f32.bin": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/IQ1_XXXS-1024-f32.bin": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_XXXS-1024-bf16.bin": "14e07510990f222b0656fdbacd1a3d6271c28e00526587b67c4a1b399674c3d5" + }, + "cases": [ + { + "name": "Q4_0-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "oracle_file_sha256": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "compared_oracle_sha256": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 18432, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q4_0-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "oracle_file_sha256": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "compared_oracle_sha256": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 18432, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q4_0-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "oracle_file_sha256": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "compared_oracle_sha256": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 73728, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q4_0-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "oracle_file_sha256": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "compared_oracle_sha256": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 73728, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q5_0-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "oracle_file_sha256": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "compared_oracle_sha256": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 22528, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q5_0-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "oracle_file_sha256": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "compared_oracle_sha256": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 22528, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q5_0-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "oracle_file_sha256": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "compared_oracle_sha256": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 90112, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q5_0-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "oracle_file_sha256": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "compared_oracle_sha256": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 90112, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q8_0-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "oracle_file_sha256": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "compared_oracle_sha256": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 34816, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q8_0-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "oracle_file_sha256": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "compared_oracle_sha256": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 34816, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q8_0-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "oracle_file_sha256": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "compared_oracle_sha256": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 139264, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q8_0-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "oracle_file_sha256": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "compared_oracle_sha256": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 139264, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q2_K-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "oracle_file_sha256": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "compared_oracle_sha256": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 10752, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q2_K-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "oracle_file_sha256": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "compared_oracle_sha256": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 10752, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q2_K-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "oracle_file_sha256": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "compared_oracle_sha256": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 43008, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q2_K-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "oracle_file_sha256": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "compared_oracle_sha256": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 43008, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q3_K-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "oracle_file_sha256": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "compared_oracle_sha256": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 14080, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q3_K-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "oracle_file_sha256": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "compared_oracle_sha256": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 14080, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q3_K-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "oracle_file_sha256": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "compared_oracle_sha256": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 56320, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q3_K-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "oracle_file_sha256": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "compared_oracle_sha256": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 56320, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q4_K-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "oracle_file_sha256": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "compared_oracle_sha256": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 18432, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q4_K-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "oracle_file_sha256": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "compared_oracle_sha256": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 18432, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q4_K-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "oracle_file_sha256": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "compared_oracle_sha256": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 73728, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q4_K-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "oracle_file_sha256": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "compared_oracle_sha256": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 73728, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q5_K-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "oracle_file_sha256": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "compared_oracle_sha256": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 22528, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q5_K-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "oracle_file_sha256": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "compared_oracle_sha256": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 22528, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q5_K-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "oracle_file_sha256": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "compared_oracle_sha256": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 90112, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q5_K-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "oracle_file_sha256": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "compared_oracle_sha256": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 90112, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q6_K-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "3e1665a3167a552c47493bdc538166acfa6cbd43a1e7dd30230c40e3c5b11207", + "oracle_file_sha256": "955ba972e48a303803cdacca928e7f60176dfaa1b7f8590038bdb91830658428", + "compared_oracle_sha256": "955ba972e48a303803cdacca928e7f60176dfaa1b7f8590038bdb91830658428", + "byte_equal": false, + "max_abs_error": 0.00048828125, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 26880, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q6_K-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "242d1b322588618517ee3d4d62e393d08b2faca2262414ab00d89d092598e8ee", + "oracle_file_sha256": "c65412c7615787b9103207f55776d49dfda767740f4d6e1106cb711a99404bc6", + "compared_oracle_sha256": "c65412c7615787b9103207f55776d49dfda767740f4d6e1106cb711a99404bc6", + "byte_equal": false, + "max_abs_error": 0.0078125, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 26880, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q6_K-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "158400e054e65af2adad3e7d348b2fc3e082fcfc065706db200704d3c86a2581", + "oracle_file_sha256": "5a0f2bb0da03f1eb5c689c6f95be636890aec44873bffe77dd039c2b5bc6c20a", + "compared_oracle_sha256": "5a0f2bb0da03f1eb5c689c6f95be636890aec44873bffe77dd039c2b5bc6c20a", + "byte_equal": false, + "max_abs_error": 0.00048828125, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 107520, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q6_K-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "54c4ea60561a742634c5fdf24c4dd372f0e101a48bf29b53d53a0b7e88f31aea", + "oracle_file_sha256": "5955c913676e2e5003f84ad0dca7062d3f7106188a13e4844b13774a2764c238", + "compared_oracle_sha256": "5955c913676e2e5003f84ad0dca7062d3f7106188a13e4844b13774a2764c238", + "byte_equal": false, + "max_abs_error": 0.0078125, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 107520, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q8_K-256", + "dtype": "f32", + "oracle": "secondary", + "adaptation": null, + "elements": 1024, + "native_sha256": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6", + "oracle_file_sha256": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6", + "compared_oracle_sha256": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 37376, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q8_K-256", + "dtype": "bf16", + "oracle": "secondary", + "adaptation": "upstream F32 output rounded to BF16 RNE", + "elements": 1024, + "native_sha256": "f5da512dcaf2f4165abd5b4b1377d275a5945d2ab8946d2d8266600c5b03d31f", + "oracle_file_sha256": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6", + "compared_oracle_sha256": "f5da512dcaf2f4165abd5b4b1377d275a5945d2ab8946d2d8266600c5b03d31f", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 37376, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q8_K-1024", + "dtype": "f32", + "oracle": "secondary", + "adaptation": null, + "elements": 4096, + "native_sha256": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c", + "oracle_file_sha256": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c", + "compared_oracle_sha256": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 149504, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "Q8_K-1024", + "dtype": "bf16", + "oracle": "secondary", + "adaptation": "upstream F32 output rounded to BF16 RNE", + "elements": 4096, + "native_sha256": "6a1aa0cd72dca3f6ae3f6b835eeba50a87bf7b143ad607baa30482f0835f8c22", + "oracle_file_sha256": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c", + "compared_oracle_sha256": "6a1aa0cd72dca3f6ae3f6b835eeba50a87bf7b143ad607baa30482f0835f8c22", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 149504, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_XXS-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "oracle_file_sha256": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "compared_oracle_sha256": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 8448, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_XXS-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "oracle_file_sha256": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "compared_oracle_sha256": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 8448, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_XXS-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "oracle_file_sha256": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "compared_oracle_sha256": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 33792, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_XXS-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "oracle_file_sha256": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "compared_oracle_sha256": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 33792, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_XS-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "oracle_file_sha256": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "compared_oracle_sha256": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 9472, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_XS-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "oracle_file_sha256": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "compared_oracle_sha256": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 9472, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_XS-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "oracle_file_sha256": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "compared_oracle_sha256": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 37888, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_XS-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "oracle_file_sha256": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "compared_oracle_sha256": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 37888, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ3_XXS-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "oracle_file_sha256": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "compared_oracle_sha256": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 12544, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ3_XXS-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "oracle_file_sha256": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "compared_oracle_sha256": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 12544, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ3_XXS-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "oracle_file_sha256": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "compared_oracle_sha256": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 50176, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ3_XXS-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "oracle_file_sha256": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "compared_oracle_sha256": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 50176, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ1_S-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "oracle_file_sha256": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "compared_oracle_sha256": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 6400, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ1_S-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "oracle_file_sha256": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "compared_oracle_sha256": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 6400, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ1_S-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "oracle_file_sha256": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "compared_oracle_sha256": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 25600, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ1_S-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "oracle_file_sha256": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "compared_oracle_sha256": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 25600, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ4_NL-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "oracle_file_sha256": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "compared_oracle_sha256": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 18432, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ4_NL-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "oracle_file_sha256": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "compared_oracle_sha256": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 18432, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ4_NL-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "oracle_file_sha256": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "compared_oracle_sha256": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 73728, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ4_NL-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "oracle_file_sha256": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "compared_oracle_sha256": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 73728, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ3_S-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "7843ff13e1c7f1475f949583ec0466b7cd3020723994d7c725e7c618c11c34f0", + "oracle_file_sha256": "8cad98f9fecbfcac883cb11a91df1ac703846dfe88bac12792ab3066420243ad", + "compared_oracle_sha256": "8cad98f9fecbfcac883cb11a91df1ac703846dfe88bac12792ab3066420243ad", + "byte_equal": false, + "max_abs_error": 0.12109375, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 14080, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ3_S-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "281888da0636914c257cd25407adc1465b2d40e96aa7694eb5c3bd0df4036b73", + "oracle_file_sha256": "7f36f4a4ba9f0977f68d452e8ec1d26ea299ea9ddbc82fd0ede324abc94d6595", + "compared_oracle_sha256": "7f36f4a4ba9f0977f68d452e8ec1d26ea299ea9ddbc82fd0ede324abc94d6595", + "byte_equal": false, + "max_abs_error": 0.125, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 14080, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ3_S-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "8932dbeb65d3716cfb60014ad471ab0fb5d2d2d4b037e8745464e65f77b936de", + "oracle_file_sha256": "ebcadd200e64857ed483aa7176172dc94b966198fb81d2910ab723260fb46fdc", + "compared_oracle_sha256": "ebcadd200e64857ed483aa7176172dc94b966198fb81d2910ab723260fb46fdc", + "byte_equal": false, + "max_abs_error": 0.12109375, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 56320, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ3_S-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "11a954e23ccc31f98a3398902b7ce96ec39f0fc0119d41202732630299d9a623", + "oracle_file_sha256": "88e55dfa1c9378436bd431db09fca5bf5aad9910dc311b493d6f94d1ec98dc7c", + "compared_oracle_sha256": "88e55dfa1c9378436bd431db09fca5bf5aad9910dc311b493d6f94d1ec98dc7c", + "byte_equal": false, + "max_abs_error": 0.125, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 56320, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_S-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "oracle_file_sha256": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "compared_oracle_sha256": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 10496, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_S-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "oracle_file_sha256": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "compared_oracle_sha256": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 10496, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_S-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "oracle_file_sha256": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "compared_oracle_sha256": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 41984, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ2_S-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "oracle_file_sha256": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "compared_oracle_sha256": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 41984, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ4_XS-256", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "oracle_file_sha256": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "compared_oracle_sha256": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 17408, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ4_XS-256", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 1024, + "native_sha256": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "oracle_file_sha256": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "compared_oracle_sha256": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 17408, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ4_XS-1024", + "dtype": "f32", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "oracle_file_sha256": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "compared_oracle_sha256": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 69632, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ4_XS-1024", + "dtype": "bf16", + "oracle": "primary", + "adaptation": null, + "elements": 4096, + "native_sha256": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "oracle_file_sha256": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "compared_oracle_sha256": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 69632, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "MXFP4-256", + "dtype": "f32", + "oracle": "secondary", + "adaptation": null, + "elements": 1024, + "native_sha256": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2", + "oracle_file_sha256": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2", + "compared_oracle_sha256": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 17408, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "MXFP4-256", + "dtype": "bf16", + "oracle": "secondary", + "adaptation": "upstream F32 output rounded to BF16 RNE", + "elements": 1024, + "native_sha256": "4b66106696144b4970929d2f5734d3ddf8e6e4580a2fe8910b500ebaab5431a9", + "oracle_file_sha256": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2", + "compared_oracle_sha256": "4b66106696144b4970929d2f5734d3ddf8e6e4580a2fe8910b500ebaab5431a9", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 17408, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "MXFP4-1024", + "dtype": "f32", + "oracle": "secondary", + "adaptation": null, + "elements": 4096, + "native_sha256": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec", + "oracle_file_sha256": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec", + "compared_oracle_sha256": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 69632, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "MXFP4-1024", + "dtype": "bf16", + "oracle": "secondary", + "adaptation": "upstream F32 output rounded to BF16 RNE", + "elements": 4096, + "native_sha256": "cdd6e95bc3bc3e18bc014dee2cdb125cb179b670256896536b0e3822b0adfa83", + "oracle_file_sha256": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec", + "compared_oracle_sha256": "cdd6e95bc3bc3e18bc014dee2cdb125cb179b670256896536b0e3822b0adfa83", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 69632, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ1_XXXS-256", + "dtype": "f32", + "oracle": "secondary", + "adaptation": null, + "elements": 1024, + "native_sha256": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce", + "oracle_file_sha256": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce", + "compared_oracle_sha256": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 4864, + "output_bytes": 4096, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ1_XXXS-256", + "dtype": "bf16", + "oracle": "secondary", + "adaptation": "upstream F32 output rounded to BF16 RNE", + "elements": 1024, + "native_sha256": "9de69ca2d01fda18f05e9ee7ac531f3cb806d39a3af8d10e64ba0c53dab1b08a", + "oracle_file_sha256": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce", + "compared_oracle_sha256": "9de69ca2d01fda18f05e9ee7ac531f3cb806d39a3af8d10e64ba0c53dab1b08a", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 4864, + "output_bytes": 2048, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ1_XXXS-1024", + "dtype": "f32", + "oracle": "secondary", + "adaptation": null, + "elements": 4096, + "native_sha256": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0", + "oracle_file_sha256": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0", + "compared_oracle_sha256": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 19456, + "output_bytes": 16384, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + }, + { + "name": "IQ1_XXXS-1024", + "dtype": "bf16", + "oracle": "secondary", + "adaptation": "upstream F32 output rounded to BF16 RNE", + "elements": 4096, + "native_sha256": "14e07510990f222b0656fdbacd1a3d6271c28e00526587b67c4a1b399674c3d5", + "oracle_file_sha256": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0", + "compared_oracle_sha256": "14e07510990f222b0656fdbacd1a3d6271c28e00526587b67c4a1b399674c3d5", + "byte_equal": true, + "max_abs_error": 0.0, + "tolerance_failures": 0, + "first_failures": [], + "finite": true, + "native_memory": { + "table_bytes": 19456, + "output_bytes": 8192, + "scratch_bytes": 16, + "selected_packed_bytes": 0, + "selections": 1 + } + } + ], + "summary": { + "cases": 76, + "primary_cases": 64, + "secondary_cases": 12, + "byte_equal_cases": 68, + "within_tolerance_cases": 76, + "max_abs_error": 0.125, + "inputs_reverified": true + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment-raw-captures.tar.gz b/docs/bench-evidence/rocm-quant-gather/oracle-amendment-raw-captures.tar.gz new file mode 100644 index 0000000000..fd9adbc244 Binary files /dev/null and b/docs/bench-evidence/rocm-quant-gather/oracle-amendment-raw-captures.tar.gz differ diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/README.md b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/README.md new file mode 100644 index 0000000000..d8883b004a --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/README.md @@ -0,0 +1,45 @@ +# Bounded oracle configuration amendment + +Issue: [#3093](https://github.com/mudler/vllm.cpp/issues/3093). +Contract: [row spec](../../../../.agents/specs/rocm-quant-gather.md#bounded-oracle-configuration-and-allocation-amendment). + +The operator ran these exact primary commands on 9 September 2026 UTC. +The helper prepared the scratch drivers and inspected their results. +Each attempt uses the same 944,192-byte Q4_0 GGUF and 974-byte HF config. +The operator receipts seal those inputs, the driver, plugin extension, and image. +The complete input audit remains in each original task evidence directory; +the receipt records its digest and the 2,500 files checked before and after. +These paths record this host's evidence and are not environment defaults. + +| Attempt | Configuration change | Measured result | +|---|---|---| +| v3 | Pinned `model_class_overrides` selects the registered text class | Renderer refusal resolved; model load fails on rotary width 16 versus section sum 32 | +| v4 | Top-level `partial_rotary_factor=1.0` preserves rotary width 64 | Model load, normal compilation, and graph capture succeed; four physical blocks leave 48 usable cells and fail admission for length 64 | +| v5 | Sixteen physical blocks preserve logical length 64 | Exit 0; first prompt and repeat emit `[47, 19, 4, 20]`; resolved BF16 model and 256 physical cells | + +The configuration corrections use public arguments in the pinned primary. +No primary source, plugin source, generated GGUF, or HF config changed. +Default compilation, graph capture, and kernel selection remain enabled. +The v5 driver reports the requested cache contract and pinned null-block +semantics separately from the observed `CacheConfig`. +Neither field proves the actual allocation bytes or cache tensor layout. +Identical-tool memory measurements and the full model token matrix remain pending. + +The CPU RoPE probe imports the pinned configuration class and the executing +image's Transformers normalization helper. It demonstrates that normalization +copies the default top-level factor of 0.25 over the nested factor of 1.0. +Setting the top-level factor to 1.0 preserves the original rotary width of 64. +The probe ran without GPU devices and with networking disabled. + +`manifest.json` gives file sizes, SHA256 values, and original evidence paths. +The retained logs, JSON reports, launch commands, receipts, and driver diffs are +unchanged copies. `../oracle-amendment-raw-captures.tar.gz` preserves the three +logs with their original terminal carriage returns and the three driver diffs. +The diffs document scratch adaptations; their presence here +does not replace the committed harness implementation or fresh review. + +Both pristine secondary bounded runs still abort on their first decode. +The allocation overlay in the amendment is a separate, unqualified proposal. +It changes no primary result and does not advance an oracle pin or global +gateability record. Its red/green allocation witness, recurrent control, full +token comparison, and independent mutation review remain required. diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/manifest.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/manifest.json new file mode 100644 index 0000000000..b8d9046b86 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/manifest.json @@ -0,0 +1,139 @@ +{ + "row": "BACKEND-ROCM-QUANT-GATHER", + "issue": 3093, + "files": [ + { + "file": "v3-primary-model-v3-command.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary-model-v3-command.json", + "bytes": 5350, + "sha256": "cd76a5f310854b53987785e0904c8c4292020895bf17ccf9802642698ef7ac24" + }, + { + "file": "v3-operator-receipt.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/operator-receipt.json", + "bytes": 5978, + "sha256": "86cfba400a657b43ca20f3475e86477ce6e0b0fc5be82ecb6216e98ae50ceb55" + }, + { + "file": "v3-operator-run.log", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/operator-run.log", + "bytes": 25773, + "sha256": "c86ac57fd7e6aeb51f8035430bb67cc9bf2e46981c9d55e71993a1fc2a3cc44f", + "archive": "../oracle-amendment-raw-captures.tar.gz", + "member": "v3-operator-run.log" + }, + { + "file": "v3-report-p0-r0.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/report-p0-r0.json", + "bytes": 3862, + "sha256": "d9a936fa8196fe117b2291b5cd275fb66c793da2176a7b7b3b93ea8175095eb4" + }, + { + "file": "v3-driver.diff", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/driver.diff", + "bytes": 1372, + "sha256": "de80d56a7cddd4d9e6d528a9870905b3bdd4bb953f5ed736e7e488cebdd5b03b", + "archive": "../oracle-amendment-raw-captures.tar.gz", + "member": "v3-driver.diff" + }, + { + "file": "v3-preparation-manifest.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/preparation-manifest.json", + "bytes": 1823, + "sha256": "c1608e55ae4caf123a7751eb322826e19574aa5d2b1b4af5721951c54568b297" + }, + { + "file": "v4-primary-model-v4-command.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary-model-v4-command.json", + "bytes": 5546, + "sha256": "1bba00a4b17f63f43ac5e194dd7b47e10a291daca8c74492980acbc4075e569e" + }, + { + "file": "v4-operator-receipt.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/operator-receipt.json", + "bytes": 6174, + "sha256": "2f912bbfdea21afccb379688254b2cae688d667707ae1b52e21ef64427920757" + }, + { + "file": "v4-operator-run.log", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/operator-run.log", + "bytes": 19635, + "sha256": "1e207fc76cb0bbeb91996e8f17bbf5d473ea3d8d0679a3c7c357aa740513525b", + "archive": "../oracle-amendment-raw-captures.tar.gz", + "member": "v4-operator-run.log" + }, + { + "file": "v4-report-p0-r0.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/report-p0-r0.json", + "bytes": 3963, + "sha256": "36e0aac4fe51dd4a7c8c83fc108c318dd3cceb8f474382f2d5c1ce822a837d7b" + }, + { + "file": "v4-driver.diff", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/driver.diff", + "bytes": 1346, + "sha256": "7051f78db7461150b006cd7c0f848561ff1ad0863fb232ab3cc874b411030348", + "archive": "../oracle-amendment-raw-captures.tar.gz", + "member": "v4-driver.diff" + }, + { + "file": "v4-preparation-manifest.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/preparation-manifest.json", + "bytes": 1823, + "sha256": "fe68b9440c39063253617d9f2f7cbb1ade30ea42c2e038525b4fe1dadcb98787" + }, + { + "file": "v5-primary-model-v5-command.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary-model-v5-command.json", + "bytes": 5953, + "sha256": "1b9e29f87824b0afbe7ee9c8b9b8ed3937ff921dc1b6842439a8846b6c790a5a" + }, + { + "file": "v5-operator-receipt.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/operator-receipt.json", + "bytes": 6601, + "sha256": "43233373d0197d1676c3ef7c43fdfeb921a72013015d4fd7d66e0769326cb3cf" + }, + { + "file": "v5-operator-run.log", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/operator-run.log", + "bytes": 17621, + "sha256": "23ee581a3ee5f256a94673cd7ab8cea13585735f85b746bd1a97d1033ec55319", + "archive": "../oracle-amendment-raw-captures.tar.gz", + "member": "v5-operator-run.log" + }, + { + "file": "v5-report-p0-r0.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/report-p0-r0.json", + "bytes": 2878, + "sha256": "eeb8de007b4695a44635026d6505d1a7558c399031add202624745d411a32990" + }, + { + "file": "v5-driver.diff", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/driver.diff", + "bytes": 2806, + "sha256": "1b6fdcd88a5079a86a78219270024454cd05ba9156fb1aa02037642e4a6c2973", + "archive": "../oracle-amendment-raw-captures.tar.gz", + "member": "v5-driver.diff" + }, + { + "file": "v5-preparation-manifest.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/preparation-manifest.json", + "bytes": 1823, + "sha256": "a0fe212d93036bdba3932296b02d40f43c67b244a517e10cda31987df648f25e" + }, + { + "file": "rope-cpu-probe-result.json", + "source": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/rope-cpu-probe-result.json", + "bytes": 830, + "sha256": "4efcafae067c2decf115e7df4ad119ebe5daa3a603e0bc20fb001598f57bbecb" + } + ], + "archives": [ + { + "file": "../oracle-amendment-raw-captures.tar.gz", + "bytes": 10002, + "sha256": "823e5a2b2909cb6b7b845b818ac15db9ca8115073df646b8ae8a00cbd270de96" + } + ] +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/rope-cpu-probe-result.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/rope-cpu-probe-result.json new file mode 100644 index 0000000000..ead1b5b49f --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/rope-cpu-probe-result.json @@ -0,0 +1,7 @@ +{ + "chunk_id": "f1aaf4", + "wall_time_seconds": 0.00000286, + "exit_code": 0, + "original_token_count": 162, + "output": "original {\"head_dim\": 64, \"partial_rotary_factor\": 0.25, \"rope_parameters\": {\"rope_type\": \"default\", \"rope_theta\": 1000000.0, \"partial_rotary_factor\": 1.0, \"mrope_section\": [16, 8, 8], \"mrope_interleaved\": true}}\nafter_standardize {\"rotary_dim\": 16, \"mrope_section_sum\": 32, \"rope_parameters\": {\"rope_type\": \"default\", \"rope_theta\": 1000000.0, \"partial_rotary_factor\": 0.25, \"mrope_section\": [16, 8, 8], \"mrope_interleaved\": true}}\nwith_override {\"rotary_dim\": 64, \"mrope_section_sum\": 32, \"rope_parameters\": {\"rope_type\": \"default\", \"rope_theta\": 1000000.0, \"partial_rotary_factor\": 1.0, \"mrope_section\": [16, 8, 8], \"mrope_interleaved\": true}}\n" +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-operator-receipt.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-operator-receipt.json new file mode 100644 index 0000000000..ada15bbc12 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-operator-receipt.json @@ -0,0 +1,139 @@ +{ + "command": { + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v3", + "-e", + "TRITON_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/triton", + "-e", + "VLLM_CACHE_ROOT=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/vllm", + "-e", + "TORCHINDUCTOR_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/inductor", + "-e", + "HF_HOME=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/hf", + "-e", + "NUMBA_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/numba", + "-e", + "MAX_JOBS=4", + "-e", + "TORCHINDUCTOR_COMPILE_THREADS=4", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "env": {}, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3", + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary-model-p0-r0.log", + "files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary_model.py": { + "bytes": 4294, + "sha256": "8fd8433d8b8ce41a490af00e9cb58c96ba5a199f250182ab2ed6255a818c7c23" + } + }, + "fixture": { + "origin": "Frozen generated fixture from public native gate", + "gguf_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af", + "table": "Q4_0", + "geometry": "one full-attention layer, H256,4Q/1KV heads,D64,I256,V128" + }, + "config_adaptation": "The unchanged HF text config and GGUF feed pinned Qwen3_5ForCausalLM through the public model_class_overrides argument. The plugin retains its architecture key and unmodified weight adapter. No vision metadata or weights are added.", + "authority": "Root only under GPU0 mutex; no oracle patch, pin change or eager override", + "acceptance": "This is first requestedp0r0. A load/refusal keeps model gatePENDING. On success compare exacttokens and schedule remaining5 fresh-engine requests.", + "version_reason": "v2 reached the renderer but the plugin maps qwen3_5_text to the multimodal architecture. v3 maps that key to the pinned registered text class in every process, using the upstream development/debug configuration option without source edits.", + "preparation": { + "baseline": { + "bytes": 4684, + "sha256": "d5a75fbbaa7bb03ca345b3ab66fd7d422ab6a497429851f82918c1d48fbb85e3" + }, + "original_driver": { + "bytes": 3998, + "sha256": "a8bd0a4c815fb0df6934f69b15bf54179c5204630e617a3cd000bda5100aed2b" + }, + "changes": [ + "model_class_overrides argument selecting pinned text class", + "report harness_adaptations records selection" + ], + "validation": "AST parse, all retained input hashes, and original driver preservation passed; no GPU execution" + } + }, + "exit_code": 1, + "seconds": 28.064705149037763, + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/operator-run.log", + "log_sha256": "c86ac57fd7e6aeb51f8035430bb67cc9bf2e46981c9d55e71993a1fc2a3cc44f", + "verified_inputs_before_after": 2500, + "input_audit_sha256": "27b8734aedc5be3e0be4432b92907f3f967557bf8281763f302468690559ef9b" +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-preparation-manifest.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-preparation-manifest.json new file mode 100644 index 0000000000..781c83112b --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-preparation-manifest.json @@ -0,0 +1,48 @@ +{ + "command": { + "bytes": 5350, + "sha256": "cd76a5f310854b53987785e0904c8c4292020895bf17ccf9802642698ef7ac24" + }, + "driver": { + "bytes": 4294, + "sha256": "8fd8433d8b8ce41a490af00e9cb58c96ba5a199f250182ab2ed6255a818c7c23" + }, + "driver_diff": { + "bytes": 1372, + "sha256": "de80d56a7cddd4d9e6d528a9870905b3bdd4bb953f5ed736e7e488cebdd5b03b" + }, + "primary_import": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "original_driver_preserved": { + "bytes": 3998, + "sha256": "a8bd0a4c815fb0df6934f69b15bf54179c5204630e617a3cd000bda5100aed2b" + }, + "input_files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary_model.py": { + "bytes": 4294, + "sha256": "8fd8433d8b8ce41a490af00e9cb58c96ba5a199f250182ab2ed6255a818c7c23" + } + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-primary-model-v3-command.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-primary-model-v3-command.json new file mode 100644 index 0000000000..3566f6cc58 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-primary-model-v3-command.json @@ -0,0 +1,131 @@ +{ + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v3", + "-e", + "TRITON_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/triton", + "-e", + "VLLM_CACHE_ROOT=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/vllm", + "-e", + "TORCHINDUCTOR_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/inductor", + "-e", + "HF_HOME=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/hf", + "-e", + "NUMBA_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/cache/numba", + "-e", + "MAX_JOBS=4", + "-e", + "TORCHINDUCTOR_COMPILE_THREADS=4", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "env": {}, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3", + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary-model-p0-r0.log", + "files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary_model.py": { + "bytes": 4294, + "sha256": "8fd8433d8b8ce41a490af00e9cb58c96ba5a199f250182ab2ed6255a818c7c23" + } + }, + "fixture": { + "origin": "Frozen generated fixture from public native gate", + "gguf_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af", + "table": "Q4_0", + "geometry": "one full-attention layer, H256,4Q/1KV heads,D64,I256,V128" + }, + "config_adaptation": "The unchanged HF text config and GGUF feed pinned Qwen3_5ForCausalLM through the public model_class_overrides argument. The plugin retains its architecture key and unmodified weight adapter. No vision metadata or weights are added.", + "authority": "Root only under GPU0 mutex; no oracle patch, pin change or eager override", + "acceptance": "This is first requestedp0r0. A load/refusal keeps model gatePENDING. On success compare exacttokens and schedule remaining5 fresh-engine requests.", + "version_reason": "v2 reached the renderer but the plugin maps qwen3_5_text to the multimodal architecture. v3 maps that key to the pinned registered text class in every process, using the upstream development/debug configuration option without source edits.", + "preparation": { + "baseline": { + "bytes": 4684, + "sha256": "d5a75fbbaa7bb03ca345b3ab66fd7d422ab6a497429851f82918c1d48fbb85e3" + }, + "original_driver": { + "bytes": 3998, + "sha256": "a8bd0a4c815fb0df6934f69b15bf54179c5204630e617a3cd000bda5100aed2b" + }, + "changes": [ + "model_class_overrides argument selecting pinned text class", + "report harness_adaptations records selection" + ], + "validation": "AST parse, all retained input hashes, and original driver preservation passed; no GPU execution" + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-report-p0-r0.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-report-p0-r0.json new file mode 100644 index 0000000000..1488291ace --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v3-report-p0-r0.json @@ -0,0 +1,50 @@ +{ + "model": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "config": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "primary_pin": "e126687a9a828d513c01a07cd69f025f27d63280", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "prompt": [ + 1, + 0, + 63, + 127, + 63 + ], + "repeat": 0, + "tokens": [], + "requested": { + "model_dtype": "auto from bfloat16 config", + "block_size": 16, + "num_gpu_blocks_override": 4, + "max_model_len": 64, + "max_num_seqs": 1, + "kv_cache_dtype": "auto", + "seed": 1380926285, + "greedy": true, + "ignore_eos": true, + "max_tokens": 4, + "stop": [], + "stop_token_ids": [] + }, + "harness_adaptations": [ + "pre-tokenized IDs with skip_tokenizer_init=True", + "explicit local HF text config mirrors GGUF geometry", + "pinned model_class_overrides selects the registered text class for the plugin architecture key", + "no oracle patch or eager-mode override" + ], + "status": "PENDING: primary model refused before qualification", + "torch": "2.12.0+git6bbd260", + "torch_git": "6bbd26020da1c6dc198625dfcdd968b1e4e6b1c5", + "hip": "7.2.53211", + "vllm": "0.28.1rc1.dev132+ge126687a9", + "plugin": "0.0.5+d4c1f0d.gfx1100", + "exception_type": "RuntimeError", + "exception": "Engine core initialization failed. See root cause above. Failed core proc(s): {'EngineCore': 1}", + "traceback": "Traceback (most recent call last):\n File \"/home/vikash/.cache/rdna3-gather-impl/primary-model-v3/primary_model.py\", line 53, in main\n model = LLM(model=str(args.model), hf_config_path=str(args.config),\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/entrypoints/llm.py\", line 343, in __init__\n self.llm_engine = LLMEngine.from_engine_args(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py\", line 179, in from_engine_args\n return cls(\n ^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py\", line 105, in __init__\n self.engine_core = EngineCoreClient.make_client(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/core_client.py\", line 110, in make_client\n return SyncMPClient(vllm_config, executor_class, log_stats)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/tracing/otel.py\", line 178, in sync_wrapper\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/core_client.py\", line 813, in __init__\n super().__init__(\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/core_client.py\", line 609, in __init__\n with launch_core_engines(\n ^^^^^^^^^^^^^^^^^^^^\n File \"/usr/lib/python3.12/contextlib.py\", line 144, in __exit__\n next(self.gen)\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/utils.py\", line 1240, in launch_core_engines\n wait_for_engine_startup(\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/utils.py\", line 1320, in wait_for_engine_startup\n raise RuntimeError(\nRuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {'EngineCore': 1}\n" +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-operator-receipt.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-operator-receipt.json new file mode 100644 index 0000000000..40d3b37a3a --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-operator-receipt.json @@ -0,0 +1,139 @@ +{ + "command": { + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v4", + "-e", + "TRITON_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/triton", + "-e", + "VLLM_CACHE_ROOT=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/vllm", + "-e", + "TORCHINDUCTOR_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/inductor", + "-e", + "HF_HOME=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/hf", + "-e", + "NUMBA_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/numba", + "-e", + "MAX_JOBS=4", + "-e", + "TORCHINDUCTOR_COMPILE_THREADS=4", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "env": {}, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4", + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary-model-p0-r0.log", + "files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary_model.py": { + "bytes": 4494, + "sha256": "ba33540ab2c89e2b4dc22b2d84c39e3416a01e3e75d527386a509567db5d32d6" + } + }, + "fixture": { + "origin": "Frozen generated fixture from public native gate", + "gguf_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af", + "table": "Q4_0", + "geometry": "one full-attention layer, H256,4Q/1KV heads,D64,I256,V128" + }, + "config_adaptation": "The unchanged HF text config and GGUF feed pinned Qwen3_5ForCausalLM via model_class_overrides. A top-level hf_overrides partial_rotary_factor=1.0 preserves the GGUF rotary width64 when Transformers standardizes the existing nested RoPE parameters. No model metadata or weights change.", + "authority": "Root only under GPU0 mutex; no oracle patch, pin change or eager override", + "acceptance": "This is first requestedp0r0. A load/refusal keeps model gatePENDING. On success compare exacttokens and schedule remaining5 fresh-engine requests.", + "version_reason": "V3 passes renderer and selects the correct text class, but Qwen3_5TextConfig defaults top-level partial_rotary_factor to0.25, overriding the nested1.0 during standardization. V4 explicitly supplies1.0, restoring rotary_dim64 and sections sum32 required by the original GGUF.", + "preparation": { + "baseline": { + "bytes": 5350, + "sha256": "cd76a5f310854b53987785e0904c8c4292020895bf17ccf9802642698ef7ac24" + }, + "original_driver": { + "bytes": 4294, + "sha256": "8fd8433d8b8ce41a490af00e9cb58c96ba5a199f250182ab2ed6255a818c7c23" + }, + "changes": [ + "hf_overrides top-level partial_rotary_factor=1.0", + "report records the normalized RoPE adaptation" + ], + "validation": "AST parse, all sealed input hashes, and original driver preservation passed; a CPU-only image probe confirms rotary_dim16 before correction and64 after correction, with sections sum32 unchanged; no GPU execution" + } + }, + "exit_code": 1, + "seconds": 43.696692710975185, + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/operator-run.log", + "log_sha256": "1e207fc76cb0bbeb91996e8f17bbf5d473ea3d8d0679a3c7c357aa740513525b", + "verified_inputs_before_after": 2500, + "input_audit_sha256": "0067d718fdf2ec24c5926c36330f25823f0311248fd92a95087d31967483ec9f" +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-preparation-manifest.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-preparation-manifest.json new file mode 100644 index 0000000000..6a70c2e9ba --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-preparation-manifest.json @@ -0,0 +1,48 @@ +{ + "command": { + "bytes": 5546, + "sha256": "1bba00a4b17f63f43ac5e194dd7b47e10a291daca8c74492980acbc4075e569e" + }, + "driver": { + "bytes": 4494, + "sha256": "ba33540ab2c89e2b4dc22b2d84c39e3416a01e3e75d527386a509567db5d32d6" + }, + "driver_diff": { + "bytes": 1346, + "sha256": "7051f78db7461150b006cd7c0f848561ff1ad0863fb232ab3cc874b411030348" + }, + "primary_import": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "original_driver_preserved": { + "bytes": 4294, + "sha256": "8fd8433d8b8ce41a490af00e9cb58c96ba5a199f250182ab2ed6255a818c7c23" + }, + "input_files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary_model.py": { + "bytes": 4494, + "sha256": "ba33540ab2c89e2b4dc22b2d84c39e3416a01e3e75d527386a509567db5d32d6" + } + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-primary-model-v4-command.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-primary-model-v4-command.json new file mode 100644 index 0000000000..426fd85882 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-primary-model-v4-command.json @@ -0,0 +1,131 @@ +{ + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v4", + "-e", + "TRITON_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/triton", + "-e", + "VLLM_CACHE_ROOT=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/vllm", + "-e", + "TORCHINDUCTOR_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/inductor", + "-e", + "HF_HOME=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/hf", + "-e", + "NUMBA_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/cache/numba", + "-e", + "MAX_JOBS=4", + "-e", + "TORCHINDUCTOR_COMPILE_THREADS=4", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "env": {}, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4", + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary-model-p0-r0.log", + "files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary_model.py": { + "bytes": 4494, + "sha256": "ba33540ab2c89e2b4dc22b2d84c39e3416a01e3e75d527386a509567db5d32d6" + } + }, + "fixture": { + "origin": "Frozen generated fixture from public native gate", + "gguf_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af", + "table": "Q4_0", + "geometry": "one full-attention layer, H256,4Q/1KV heads,D64,I256,V128" + }, + "config_adaptation": "The unchanged HF text config and GGUF feed pinned Qwen3_5ForCausalLM via model_class_overrides. A top-level hf_overrides partial_rotary_factor=1.0 preserves the GGUF rotary width64 when Transformers standardizes the existing nested RoPE parameters. No model metadata or weights change.", + "authority": "Root only under GPU0 mutex; no oracle patch, pin change or eager override", + "acceptance": "This is first requestedp0r0. A load/refusal keeps model gatePENDING. On success compare exacttokens and schedule remaining5 fresh-engine requests.", + "version_reason": "V3 passes renderer and selects the correct text class, but Qwen3_5TextConfig defaults top-level partial_rotary_factor to0.25, overriding the nested1.0 during standardization. V4 explicitly supplies1.0, restoring rotary_dim64 and sections sum32 required by the original GGUF.", + "preparation": { + "baseline": { + "bytes": 5350, + "sha256": "cd76a5f310854b53987785e0904c8c4292020895bf17ccf9802642698ef7ac24" + }, + "original_driver": { + "bytes": 4294, + "sha256": "8fd8433d8b8ce41a490af00e9cb58c96ba5a199f250182ab2ed6255a818c7c23" + }, + "changes": [ + "hf_overrides top-level partial_rotary_factor=1.0", + "report records the normalized RoPE adaptation" + ], + "validation": "AST parse, all sealed input hashes, and original driver preservation passed; a CPU-only image probe confirms rotary_dim16 before correction and64 after correction, with sections sum32 unchanged; no GPU execution" + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-report-p0-r0.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-report-p0-r0.json new file mode 100644 index 0000000000..5e758864ab --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v4-report-p0-r0.json @@ -0,0 +1,51 @@ +{ + "model": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "config": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "primary_pin": "e126687a9a828d513c01a07cd69f025f27d63280", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "prompt": [ + 1, + 0, + 63, + 127, + 63 + ], + "repeat": 0, + "tokens": [], + "requested": { + "model_dtype": "auto from bfloat16 config", + "block_size": 16, + "num_gpu_blocks_override": 4, + "max_model_len": 64, + "max_num_seqs": 1, + "kv_cache_dtype": "auto", + "seed": 1380926285, + "greedy": true, + "ignore_eos": true, + "max_tokens": 4, + "stop": [], + "stop_token_ids": [] + }, + "harness_adaptations": [ + "pre-tokenized IDs with skip_tokenizer_init=True", + "explicit local HF text config mirrors GGUF geometry", + "pinned model_class_overrides selects the registered text class for the plugin architecture key", + "top-level partial_rotary_factor=1.0 preserves the GGUF rotary width64 during HF normalization", + "no oracle patch or eager-mode override" + ], + "status": "PENDING: primary model refused before qualification", + "torch": "2.12.0+git6bbd260", + "torch_git": "6bbd26020da1c6dc198625dfcdd968b1e4e6b1c5", + "hip": "7.2.53211", + "vllm": "0.28.1rc1.dev132+ge126687a9", + "plugin": "0.0.5+d4c1f0d.gfx1100", + "exception_type": "RuntimeError", + "exception": "Engine core initialization failed. See root cause above. Failed core proc(s): {'EngineCore': 1}", + "traceback": "Traceback (most recent call last):\n File \"/home/vikash/.cache/rdna3-gather-impl/primary-model-v4/primary_model.py\", line 54, in main\n model = LLM(model=str(args.model), hf_config_path=str(args.config),\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/entrypoints/llm.py\", line 343, in __init__\n self.llm_engine = LLMEngine.from_engine_args(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py\", line 179, in from_engine_args\n return cls(\n ^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py\", line 105, in __init__\n self.engine_core = EngineCoreClient.make_client(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/core_client.py\", line 110, in make_client\n return SyncMPClient(vllm_config, executor_class, log_stats)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/tracing/otel.py\", line 178, in sync_wrapper\n return func(*args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/core_client.py\", line 813, in __init__\n super().__init__(\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/core_client.py\", line 609, in __init__\n with launch_core_engines(\n ^^^^^^^^^^^^^^^^^^^^\n File \"/usr/lib/python3.12/contextlib.py\", line 144, in __exit__\n next(self.gen)\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/utils.py\", line 1240, in launch_core_engines\n wait_for_engine_startup(\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/utils.py\", line 1320, in wait_for_engine_startup\n raise RuntimeError(\nRuntimeError: Engine core initialization failed. See root cause above. Failed core proc(s): {'EngineCore': 1}\n" +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-operator-receipt.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-operator-receipt.json new file mode 100644 index 0000000000..34f2ead832 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-operator-receipt.json @@ -0,0 +1,149 @@ +{ + "command": { + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v5", + "-e", + "TRITON_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/triton", + "-e", + "VLLM_CACHE_ROOT=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/vllm", + "-e", + "TORCHINDUCTOR_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/inductor", + "-e", + "HF_HOME=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/hf", + "-e", + "NUMBA_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/numba", + "-e", + "MAX_JOBS=4", + "-e", + "TORCHINDUCTOR_COMPILE_THREADS=4", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "env": {}, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5", + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary-model-p0-r0.log", + "files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary_model.py": { + "bytes": 5014, + "sha256": "15038bc1d8f2ceb9c309c9092880471adca38adabe16e41223b31319c0b3de4d" + } + }, + "fixture": { + "origin": "Frozen generated fixture from public native gate", + "gguf_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af", + "table": "Q4_0", + "geometry": "one full-attention layer, H256,4Q/1KV heads,D64,I256,V128" + }, + "config_adaptation": "The unchanged HF text config and GGUF feed pinned Qwen3_5ForCausalLM via model_class_overrides. A top-level hf_overrides partial_rotary_factor=1.0 preserves the GGUF rotary width64 when Transformers standardizes the existing nested RoPE parameters. No model metadata or weights change. Physical cache allocation is16 blocks of16 tokens, including the primary null block. Logical max_model_len remains64.", + "authority": "Root only under GPU0 mutex; no oracle patch, pin change or eager override", + "acceptance": "This is first requestedp0r0. A load/refusal keeps model gatePENDING. On success compare exacttokens and schedule remaining5 fresh-engine requests.", + "version_reason": "V4 loads the original model with normal compilation and graph capture, then rejects4 physical blocks because one null block leaves48 usable tokens for logical max_model_len64. V5 uses16 physical blocks, matching the256-cell secondary alignment while preserving all model bytes, prompt IDs, generation counts, sampling, and logical length.", + "preparation": { + "baseline": { + "bytes": 5546, + "sha256": "1bba00a4b17f63f43ac5e194dd7b47e10a291daca8c74492980acbc4075e569e" + }, + "original_driver": { + "bytes": 4494, + "sha256": "ba33540ab2c89e2b4dc22b2d84c39e3416a01e3e75d527386a509567db5d32d6" + }, + "changes": [ + "num_gpu_blocks_override16", + "report records physical and usable cache capacities separately" + ], + "validation": "AST parse, all sealed input hashes, and original driver preservation passed; no GPU execution" + }, + "cache_capacity_contract": { + "logical_max_model_len": 64, + "physical_blocks": 16, + "block_size": 16, + "physical_cells": 256, + "reserved_null_blocks": 1, + "usable_cells": 240, + "bf16_kv_payload_bytes": 65536, + "result": "Requested values plus pinned source semantics; runtime allocation verification remains required" + } + }, + "exit_code": 0, + "seconds": 48.132981712929904, + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/operator-run.log", + "log_sha256": "23ee581a3ee5f256a94673cd7ab8cea13585735f85b746bd1a97d1033ec55319", + "verified_inputs_before_after": 2500, + "input_audit_sha256": "3d6b6fce6b5715bb2c0e17481eef5ebbbdd242ebff37373bba54ea8dc03bf496" +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-preparation-manifest.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-preparation-manifest.json new file mode 100644 index 0000000000..2282d2ed4b --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-preparation-manifest.json @@ -0,0 +1,48 @@ +{ + "command": { + "bytes": 5953, + "sha256": "1b9e29f87824b0afbe7ee9c8b9b8ed3937ff921dc1b6842439a8846b6c790a5a" + }, + "driver": { + "bytes": 5014, + "sha256": "15038bc1d8f2ceb9c309c9092880471adca38adabe16e41223b31319c0b3de4d" + }, + "driver_diff": { + "bytes": 2806, + "sha256": "1b6fdcd88a5079a86a78219270024454cd05ba9156fb1aa02037642e4a6c2973" + }, + "primary_import": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "original_driver_preserved": { + "bytes": 4494, + "sha256": "ba33540ab2c89e2b4dc22b2d84c39e3416a01e3e75d527386a509567db5d32d6" + }, + "input_files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary_model.py": { + "bytes": 5014, + "sha256": "15038bc1d8f2ceb9c309c9092880471adca38adabe16e41223b31319c0b3de4d" + } + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-primary-model-v5-command.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-primary-model-v5-command.json new file mode 100644 index 0000000000..38a640dd10 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-primary-model-v5-command.json @@ -0,0 +1,141 @@ +{ + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v5", + "-e", + "TRITON_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/triton", + "-e", + "VLLM_CACHE_ROOT=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/vllm", + "-e", + "TORCHINDUCTOR_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/inductor", + "-e", + "HF_HOME=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/hf", + "-e", + "NUMBA_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/cache/numba", + "-e", + "MAX_JOBS=4", + "-e", + "TORCHINDUCTOR_COMPILE_THREADS=4", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "env": {}, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5", + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary-model-p0-r0.log", + "files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v5/primary_model.py": { + "bytes": 5014, + "sha256": "15038bc1d8f2ceb9c309c9092880471adca38adabe16e41223b31319c0b3de4d" + } + }, + "fixture": { + "origin": "Frozen generated fixture from public native gate", + "gguf_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af", + "table": "Q4_0", + "geometry": "one full-attention layer, H256,4Q/1KV heads,D64,I256,V128" + }, + "config_adaptation": "The unchanged HF text config and GGUF feed pinned Qwen3_5ForCausalLM via model_class_overrides. A top-level hf_overrides partial_rotary_factor=1.0 preserves the GGUF rotary width64 when Transformers standardizes the existing nested RoPE parameters. No model metadata or weights change. Physical cache allocation is16 blocks of16 tokens, including the primary null block. Logical max_model_len remains64.", + "authority": "Root only under GPU0 mutex; no oracle patch, pin change or eager override", + "acceptance": "This is first requestedp0r0. A load/refusal keeps model gatePENDING. On success compare exacttokens and schedule remaining5 fresh-engine requests.", + "version_reason": "V4 loads the original model with normal compilation and graph capture, then rejects4 physical blocks because one null block leaves48 usable tokens for logical max_model_len64. V5 uses16 physical blocks, matching the256-cell secondary alignment while preserving all model bytes, prompt IDs, generation counts, sampling, and logical length.", + "preparation": { + "baseline": { + "bytes": 5546, + "sha256": "1bba00a4b17f63f43ac5e194dd7b47e10a291daca8c74492980acbc4075e569e" + }, + "original_driver": { + "bytes": 4494, + "sha256": "ba33540ab2c89e2b4dc22b2d84c39e3416a01e3e75d527386a509567db5d32d6" + }, + "changes": [ + "num_gpu_blocks_override16", + "report records physical and usable cache capacities separately" + ], + "validation": "AST parse, all sealed input hashes, and original driver preservation passed; no GPU execution" + }, + "cache_capacity_contract": { + "logical_max_model_len": 64, + "physical_blocks": 16, + "block_size": 16, + "physical_cells": 256, + "reserved_null_blocks": 1, + "usable_cells": 240, + "bf16_kv_payload_bytes": 65536, + "result": "Requested values plus pinned source semantics; runtime allocation verification remains required" + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-report-p0-r0.json b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-report-p0-r0.json new file mode 100644 index 0000000000..e1359be036 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-amendment/v5-report-p0-r0.json @@ -0,0 +1,67 @@ +{ + "model": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "config": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "primary_pin": "e126687a9a828d513c01a07cd69f025f27d63280", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "prompt": [ + 1, + 0, + 63, + 127, + 63 + ], + "repeat": 0, + "tokens": [ + 47, + 19, + 4, + 20 + ], + "requested": { + "model_dtype": "auto from bfloat16 config", + "block_size": 16, + "num_gpu_blocks_override": 16, + "max_model_len": 64, + "max_num_seqs": 1, + "kv_cache_dtype": "auto", + "seed": 1380926285, + "greedy": true, + "ignore_eos": true, + "max_tokens": 4, + "stop": [], + "stop_token_ids": [] + }, + "harness_adaptations": [ + "pre-tokenized IDs with skip_tokenizer_init=True", + "explicit local HF text config mirrors GGUF geometry", + "pinned model_class_overrides selects the registered text class for the plugin architecture key", + "top-level partial_rotary_factor=1.0 preserves the GGUF rotary width64 during HF normalization", + "16 physical KV blocks match the secondary256-cell allocation; the primary reserves one null block and retains logical max_model_len64", + "no oracle patch or eager-mode override" + ], + "status": "EXECUTED; token comparison remains required", + "torch": "2.12.0+git6bbd260", + "torch_git": "6bbd26020da1c6dc198625dfcdd968b1e4e6b1c5", + "hip": "7.2.53211", + "vllm": "0.28.1rc1.dev132+ge126687a9", + "plugin": "0.0.5+d4c1f0d.gfx1100", + "resolved_model_dtype": "torch.bfloat16", + "resolved_cache_config": "CacheConfig(block_size=16, user_specified_block_size=True, user_specified_mamba_block_size=False, kv_cache_layout=None, prefix_match_unit=None, gpu_memory_utilization=0.92, cache_dtype='auto', is_attention_free=False, num_gpu_blocks_override=16, sliding_window=None, enable_prefix_caching=True, prefix_caching_hash_algo='sha256', prefix_cache_retention_interval=0, kv_cache_dtype_skip_layers=[], mamba_page_size_padded=None, skip_page_size_padded=None, mamba_block_size=None, mamba_cache_dtype='auto', mamba_ssm_cache_dtype='auto', mamba_cache_mode='none', replayssm_buffer_len=16, use_replayssm=False, use_kda_recoverssm=False, num_gpu_blocks=16, num_cpu_blocks=None, kv_cache_size_tokens=256, kv_cache_max_concurrency=4.0, kv_sharing_fast_prefill=False, kv_cache_memory_bytes=None, kv_offloading_size=None, kv_offloading_backend='native', _block_size_resolved=True)", + "cache_capacity_contract": { + "block_size": 16, + "physical_blocks": 16, + "physical_cells": 256, + "reserved_null_blocks": 1, + "usable_blocks": 15, + "usable_cells": 240, + "logical_max_model_len": 64, + "bf16_kv_payload_bytes": 65536, + "source": "vllm/v1/core/kv_cache_utils.py:2304-2309 at e126687a9a" + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-audit-arithmetic-explanation.json b/docs/bench-evidence/rocm-quant-gather/oracle-audit-arithmetic-explanation.json new file mode 100644 index 0000000000..e3e167a676 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-audit-arithmetic-explanation.json @@ -0,0 +1,235 @@ +{ + "kind": "Diagnostic transcription of source expressions only. The actual GPU output files remain the evidence.", + "native_source_hashes": { + "rocm_quant_dequant.h": "56b14d63308a6c3d8c19a6619217470fac88dd58599f95b4d9bf919f854f732d", + "rocm_quant_iq_tables.h": "d267d25ff1dd3f67749c251f87047f4b70c9650a99d04b524ff6c3313596a349" + }, + "source_provenance": "Current source snapshot after author comment corrections, not the native-v2 source seal. Comparison uses frozen native-v2 runtime outputs.", + "iq3_s_table_entries_with_difference": 146, + "iq3_s_table_byte_differences": 150, + "iq3_s_only_difference": { + "plugin_raw": 62, + "plugin_effective": 15.5, + "native": 15 + }, + "cases": [ + { + "name": "Q6_K-256", + "elements": 1024, + "arms": { + "native": { + "f32_source_expression_byte_exact": true, + "bf16_source_expression_byte_exact": true + }, + "primary": { + "f32_source_expression_byte_exact": true, + "bf16_source_expression_byte_exact": true + } + }, + "f32_difference_examples": [ + { + "row": 0, + "col": 36, + "d": 0.00048828125, + "scale": 115, + "q": -19, + "integer_product": -2185, + "rounded_half_integer_product": -2184.0, + "native": -1.06689453125, + "primary": -1.06640625, + "absolute_difference": 0.00048828125 + }, + { + "row": 0, + "col": 38, + "d": 0.00048828125, + "scale": 115, + "q": -19, + "integer_product": -2185, + "rounded_half_integer_product": -2184.0, + "native": -1.06689453125, + "primary": -1.06640625, + "absolute_difference": 0.00048828125 + }, + { + "row": 0, + "col": 46, + "d": 0.00048828125, + "scale": 115, + "q": 23, + "integer_product": 2645, + "rounded_half_integer_product": 2644.0, + "native": 1.29150390625, + "primary": 1.291015625, + "absolute_difference": 0.00048828125 + } + ], + "f32_different_elements": 34 + }, + { + "name": "Q6_K-1024", + "elements": 4096, + "arms": { + "native": { + "f32_source_expression_byte_exact": true, + "bf16_source_expression_byte_exact": true + }, + "primary": { + "f32_source_expression_byte_exact": true, + "bf16_source_expression_byte_exact": true + } + }, + "f32_difference_examples": [ + { + "row": 0, + "col": 36, + "d": 0.00048828125, + "scale": 115, + "q": -19, + "integer_product": -2185, + "rounded_half_integer_product": -2184.0, + "native": -1.06689453125, + "primary": -1.06640625, + "absolute_difference": 0.00048828125 + }, + { + "row": 0, + "col": 38, + "d": 0.00048828125, + "scale": 115, + "q": -19, + "integer_product": -2185, + "rounded_half_integer_product": -2184.0, + "native": -1.06689453125, + "primary": -1.06640625, + "absolute_difference": 0.00048828125 + }, + { + "row": 0, + "col": 46, + "d": 0.00048828125, + "scale": 115, + "q": 23, + "integer_product": 2645, + "rounded_half_integer_product": 2644.0, + "native": 1.29150390625, + "primary": 1.291015625, + "absolute_difference": 0.00048828125 + } + ], + "f32_different_elements": 153 + }, + { + "name": "IQ3_S-256", + "elements": 1024, + "arms": { + "native": { + "f32_source_expression_byte_exact": true, + "bf16_source_expression_byte_exact": true + }, + "primary": { + "f32_source_expression_byte_exact": true, + "bf16_source_expression_byte_exact": true + } + }, + "f32_difference_examples": [ + { + "row": 0, + "col": 6, + "d": 0.0078125, + "scale_nibble": 1, + "index": 101, + "grid_byte": 2, + "plugin_grid_value": 62, + "native_grid_value": 15, + "native": -0.3515625, + "primary": -0.36328125, + "absolute_difference": 0.01171875 + }, + { + "row": 0, + "col": 28, + "d": 0.0078125, + "scale_nibble": 1, + "index": 265, + "grid_byte": 0, + "plugin_grid_value": 62, + "native_grid_value": 15, + "native": 0.3515625, + "primary": 0.36328125, + "absolute_difference": 0.01171875 + }, + { + "row": 0, + "col": 39, + "d": 0.0078125, + "scale_nibble": 6, + "index": 509, + "grid_byte": 3, + "plugin_grid_value": 62, + "native_grid_value": 15, + "native": -1.5234375, + "primary": -1.57421875, + "absolute_difference": 0.05078125 + } + ], + "f32_different_elements": 62 + }, + { + "name": "IQ3_S-1024", + "elements": 4096, + "arms": { + "native": { + "f32_source_expression_byte_exact": true, + "bf16_source_expression_byte_exact": true + }, + "primary": { + "f32_source_expression_byte_exact": true, + "bf16_source_expression_byte_exact": true + } + }, + "f32_difference_examples": [ + { + "row": 0, + "col": 6, + "d": 0.0078125, + "scale_nibble": 1, + "index": 101, + "grid_byte": 2, + "plugin_grid_value": 62, + "native_grid_value": 15, + "native": -0.3515625, + "primary": -0.36328125, + "absolute_difference": 0.01171875 + }, + { + "row": 0, + "col": 28, + "d": 0.0078125, + "scale_nibble": 1, + "index": 265, + "grid_byte": 0, + "plugin_grid_value": 62, + "native_grid_value": 15, + "native": 0.3515625, + "primary": 0.36328125, + "absolute_difference": 0.01171875 + }, + { + "row": 0, + "col": 39, + "d": 0.0078125, + "scale_nibble": 6, + "index": 509, + "grid_byte": 3, + "plugin_grid_value": 62, + "native_grid_value": 15, + "native": -1.5234375, + "primary": -1.57421875, + "absolute_difference": 0.05078125 + } + ], + "f32_different_elements": 303 + } + ] +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-audit-artifact-manifest.json b/docs/bench-evidence/rocm-quant-gather/oracle-audit-artifact-manifest.json new file mode 100644 index 0000000000..cef7ff5623 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-audit-artifact-manifest.json @@ -0,0 +1,101 @@ +{ + "scope": "Read-only frozen operation-oracle audit", + "files": { + "arithmetic-explanation.json": { + "bytes": 6572, + "sha256": "c04d32e481ef747672ed1d257206ea36968109e7a8c64bb2c6361d6e301966b7" + }, + "audit_frozen.py": { + "bytes": 5841, + "sha256": "3f18f3da69fd097a993dbd7b9ae075de45ea032a723b81808fd485b72550cd6d" + }, + "executing-source-artifacts.json": { + "bytes": 1951, + "sha256": "4eab933a9e2e85cae2929cdf46e27a95ed0341193551a7db495cca2caec4794d" + }, + "explain_arithmetic-current-source.py": { + "bytes": 6191, + "sha256": "7b10ad36c9bbfa87fc1c274a6b17fcf4cec26fc0a915c69c1e457ddf1ff26f48" + }, + "explain_arithmetic-v2-source-guard.py": { + "bytes": 6008, + "sha256": "21d47f226d555e218dd6b98de89ae4ab01dfc075eadce25eef91da4137d23ba7" + }, + "explain_arithmetic.py": { + "bytes": 6008, + "sha256": "21d47f226d555e218dd6b98de89ae4ab01dfc075eadce25eef91da4137d23ba7" + }, + "fork-binary-symbols.txt": { + "bytes": 240, + "sha256": "26109dc2d2ef7ed403d7fc7cbca45096377de93503ae113c4ffd4c4a0c32443c" + }, + "generated-primary/dequantize_hip.cuh": { + "bytes": 22787, + "sha256": "497842aca4744b903c7fd19b6069be33fa50da9d4bb893634eda46ab9cb35ddd" + }, + "generated-primary/dispatch_utils.h": { + "bytes": 483, + "sha256": "7a17e8ac27b25426fbfd2921d2ebcad6410b703736204e55e832037fd9f250cc" + }, + "generated-primary/ggml-common_hip.h": { + "bytes": 85590, + "sha256": "b7d415d0cb626afb330cc67867418943b1ebe39eb648988f637b4f29af8e8a41" + }, + "generated-primary/gguf_kernel.hip": { + "bytes": 22272, + "sha256": "f714989ddbd4c97bdcd924d5ca3316c60efdebcdffa5c9c818a93579023e4d72" + }, + "generated-primary/torch_bindings.cpp": { + "bytes": 2363, + "sha256": "4fb80d903a1e860a080a04033b6a41eeb840b2bc68220fc509a0db5a5f1cafa7" + }, + "hash-audit.json": { + "bytes": 15914, + "sha256": "86d02da770d682fe11e3360c760c5d27096d5180a17114c2f46f6b2ae04a1341" + }, + "independent-audit.json": { + "bytes": 1591, + "sha256": "747faf446d92f45c645948c7fbda87552ed996736a29eee55069c6893716c26b" + }, + "isolated-agent-onboard-result.json": { + "bytes": 362, + "sha256": "75a643bf8395e6b5f52e9230d50f49f8574e1d1ea3959be697633dd84720608e" + }, + "isolated-agent-onboard.log": { + "bytes": 138, + "sha256": "0fd19afd79de4dc7e059bf9415f7cced93739f51df61127a5ab09c88e92a74ff" + }, + "native-expression-source/rocm_quant_dequant.h": { + "bytes": 24598, + "sha256": "56b14d63308a6c3d8c19a6619217470fac88dd58599f95b4d9bf919f854f732d" + }, + "native-expression-source/rocm_quant_iq_tables.h": { + "bytes": 110865, + "sha256": "d267d25ff1dd3f67749c251f87047f4b70c9650a99d04b524ff6c3313596a349" + }, + "plugin-source-audit.json": { + "bytes": 323, + "sha256": "35b3cd72b0a1fe6c91c641ed169a19326fc052a6354d0c74f2549806a24979ef" + }, + "primary-output-stats.json": { + "bytes": 12439, + "sha256": "88dc6f64749d5d8152b4d128f63e59323e2aa77d51961588cb846e7836e83652" + }, + "report.md": { + "bytes": 9252, + "sha256": "ca646cdd818958214927f5cb5751efc3a111f29a64007432a4506f417bab956a" + }, + "runtime-source-checks.json": { + "bytes": 358, + "sha256": "23e7fd990779705883e0be0da456e9fe22b8c41f45bc5e20906f58861b5f695a" + }, + "secondary-git-identity.json": { + "bytes": 298, + "sha256": "3fd31d2cd85e273181b1dcf97eec92dcf4e43da564fc959b7220cb6b479444af" + }, + "stock-binary-symbols.txt": { + "bytes": 197, + "sha256": "01bc5a8662f9c4a2164444ee140433681d489def952ce6795572becd05c7eab2" + } + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-audit-executing-source-artifacts.json b/docs/bench-evidence/rocm-quant-gather/oracle-audit-executing-source-artifacts.json new file mode 100644 index 0000000000..8d48104d1c --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-audit-executing-source-artifacts.json @@ -0,0 +1,48 @@ +{ + "stock": { + "binary_sha256": "7b9a2e2d0dc95afbf592b5b1678bdc67e0fe2204529f1afe3b0b1f03566f28c2", + "symbols": [ + "00000000003b5a80 T dequantize_row_mxfp4", + "00000000003c3330 T dequantize_row_q8_K", + "000000000035e960 T ggml_get_rows", + "000000000035eaa0 T ggml_get_rows_back", + "000000000028b1c0 T ggml_graph_compute_with_ctx" + ] + }, + "fork": { + "binary_sha256": "cad537f239dc0c4ca70e5b80548f39f297117494269ca2b9b5de8191aa2f4bc1", + "symbols": [ + "00000000003b9d50 T dequantize_row_iq1_xxxs", + "00000000003aaf60 T dequantize_row_mxfp4", + "00000000003bbfe0 T dequantize_row_q8_K", + "000000000034e440 T ggml_get_rows", + "000000000034e580 T ggml_get_rows_back", + "000000000027a0e0 T ggml_graph_compute_with_ctx" + ] + }, + "gguf/gguf_kernel.hip": { + "sha256": "f714989ddbd4c97bdcd924d5ca3316c60efdebcdffa5c9c818a93579023e4d72", + "bytes": 22272, + "copy": "/home/vikash/.cache/rdna3-gather-oracle-review/generated-primary/gguf_kernel.hip" + }, + "gguf/dequantize_hip.cuh": { + "sha256": "497842aca4744b903c7fd19b6069be33fa50da9d4bb893634eda46ab9cb35ddd", + "bytes": 22787, + "copy": "/home/vikash/.cache/rdna3-gather-oracle-review/generated-primary/dequantize_hip.cuh" + }, + "gguf/ggml-common_hip.h": { + "sha256": "b7d415d0cb626afb330cc67867418943b1ebe39eb648988f637b4f29af8e8a41", + "bytes": 85590, + "copy": "/home/vikash/.cache/rdna3-gather-oracle-review/generated-primary/ggml-common_hip.h" + }, + "dispatch_utils.h": { + "sha256": "7a17e8ac27b25426fbfd2921d2ebcad6410b703736204e55e832037fd9f250cc", + "bytes": 483, + "copy": "/home/vikash/.cache/rdna3-gather-oracle-review/generated-primary/dispatch_utils.h" + }, + "torch_bindings.cpp": { + "sha256": "4fb80d903a1e860a080a04033b6a41eeb840b2bc68220fc509a0db5a5f1cafa7", + "bytes": 2363, + "copy": "/home/vikash/.cache/rdna3-gather-oracle-review/generated-primary/torch_bindings.cpp" + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-audit-independent-audit.json b/docs/bench-evidence/rocm-quant-gather/oracle-audit-independent-audit.json new file mode 100644 index 0000000000..cd634175ca --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-audit-independent-audit.json @@ -0,0 +1,52 @@ +{ + "primary-synthetic-v1.json": { + "sha256": "6bc9501cf0abe9a8de36bcc652b636ff5a26df3dbb650ad38dcc0ab098fee9f8", + "verified_files": 80 + }, + "secondary-frozen-v1.json": { + "sha256": "4baa265386ee5d67a0dfb056b42a4650d65a29a8446e3df5a30a7778980b6f21", + "verified_files": 53 + }, + "plugin_artifact": { + "verified_files": 138 + }, + "stock_source": { + "pin": "10bf611e533d81f739128304991c5e133c6aebd8", + "archive": { + "bytes": 171663360, + "sha256": "0f43551d3da94ba0d2af80e202177008a91966de66dbabf87a2042377d16d133" + }, + "verified_git_blobs_and_exported_files": 3425, + "original_artifact_files": 0 + }, + "fork_source": { + "pin": "36fe8e1cc7f2b3b8c92fdda0ab07600141921786", + "archive": { + "bytes": 170752000, + "sha256": "be6761fc65927e10aa1cad664ca5a459fa0f3776c30230cd7acff564c981f93e" + }, + "verified_git_blobs_and_exported_files": 3317, + "original_artifact_files": 0 + }, + "plugin_source": { + "pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "archive": { + "bytes": 952320, + "sha256": "090d8fd5b6cc52404508fbd4d77f38c8f9d04696b98c778e9220447f3310df39" + }, + "verified_git_blobs_and_exported_files": 150, + "original_artifact_files": 122 + }, + "primary_outputs": { + "report": { + "bytes": 44161, + "sha256": "867aaf215887b2c18f46b20b9211592550274099f721b4fff73cefb2f171dd38" + }, + "verified_outputs": 64, + "finite_nontrivial_distinct_selected_rows_and_repeated_id_bytes": true + }, + "secondary_outputs": { + "verified_operations": 6, + "confirmed_model_exit_minus6": 2 + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-audit-report.md b/docs/bench-evidence/rocm-quant-gather/oracle-audit-report.md new file mode 100644 index 0000000000..0220243d11 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-audit-report.md @@ -0,0 +1,59 @@ +**PASS: frozen operation-oracle evidence.** No finding invalidates the 64 primary operation outputs or six secondary F32 operation outputs. This is a bounded source and evidence audit, not fresh review of the gather implementation. Model qualification, production reachability, mutation testing, original fixture coverage, and trace acceptance remain outside this PASS. + +The reviewed spec is `.agents/specs/rocm-quant-gather.md` at `2e5adfb85a255528bffa5d3cdb016efb2bab70d8`. The read-only review worktree is `/home/vikash/vllm.cpp-rdna3-gather-oracle-review`. It remains clean. The auditor performed no GPU run, dependency installation, download, tracked-file edit, publication, merge, or CI action. + +The independent identity audit verified: + +- All 80 files in primary freeze `6bc9501cf0abe9a8de36bcc652b636ff5a26df3dbb650ad38dcc0ab098fee9f8`. +- All 53 files in secondary freeze `4baa265386ee5d67a0dfb056b42a4650d65a29a8446e3df5a30a7778980b6f21`. +- All 138 plugin artifact files, including extension `592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138`. +- All 150 plugin archive files against Git blobs at `d4c1f0d082fc7cd4350da56689109a01c1f29d6c`, the extracted files, and 122 original installed source files. Archive SHA-256: `090d8fd5b6cc52404508fbd4d77f38c8f9d04696b98c778e9220447f3310df39`. +- All 3,425 stock archive files against Git blobs at `10bf611e533d81f739128304991c5e133c6aebd8` and the extracted files. +- All 3,317 fork archive files against Git blobs at `36fe8e1cc7f2b3b8c92fdda0ab07600141921786` and the extracted files. +- The active vLLM source HEAD is `e126687a9a828d513c01a07cd69f025f27d63280`, with no tracked changes. Executing runtime copies of `vllm/utils/torch_utils.py` and `vllm/platforms/rocm.py` equal that source byte-for-byte. + +The primary harness SHA-256 is `1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75`. At `primary.py:88-102`, it loads original packed bytes as uint8, restores four i32 file IDs to a 2-by-2 torch.long tensor, and executes the registered plugin operation. The dense control calls upstream `gguf.dequantize`, converts the dense tensor to the requested output dtype, and calls `torch.embedding`. Its `atol=0.01`, `rtol=0.04`, ID arrangement, widths 256 and 1024, and dtype conversion agree with plugin `tests/test_kernels.py:102-124`. IQ1_M and F16 exclusions are explicit. + +All 64 output hashes match report `867aaf215887b2c18f46b20b9211592550274099f721b4fff73cefb2f171dd38`. Every output is finite and nontrivial. The three distinct selected rows differ, and repeated row 0 reproduces identical bytes. Every input ID vector is `[0,127,64,0]`. Output and selected-packed byte counts agree with the tensor geometry. These are generated 128-row tables. They do not preserve or replace the 32 original upstream GGUF fixtures. + +The actual primary call chain is supported by source and the operator's successful run: + +- Plugin `quantization/vocal_embeds.py:112-118` registers the operation through active vLLM `utils/torch_utils.py:1068-1079`. `platforms/rocm.py:502` supplies the CUDA dispatch key used for HIP tensors. +- Plugin `vocal_embeds.py:79-96` flattens IDs, selects packed rows with `torch.index_select`, dequantizes only those selected rows, and restores the output shape. +- Plugin `ops.py:93-101,181-186` uses exactly the predicate logged as `native_extension`. All 64 recorded values are true. The harness does not alter that predicate or dispatch. +- `csrc/torch_bindings.cpp:43-44` selects `ggml_dequantize`. Generated `csrc/gguf/gguf_kernel.hip:88-102` allocates the requested output dtype and dispatches `ggml_get_to_cuda`. +- Generated `dequantize_hip.cuh:530-569` selects the format decoder. Its launch wrappers are at lines 440-526. Generated source copies and hashes are retained in `generated-primary/` and `executing-source-artifacts.json`. + +This establishes the native HIP path by source-plus-run evidence. It is not a profiler trace. No Triton kernel execution or invocation-parity claim is made. The capture's dense control intentionally expands a full table; total-process memory cannot therefore be attributed entirely to the compressed plugin operation. + +The secondary harness SHA-256 is `09773359f3db39d5c759d03e37e42f4e466f6e33cb063cd2eb9bbf1353b04dbb`. Its CMake target links the exact private CPU oracle builds. No decoder is transcribed in the harness. + +- `secondary.cpp:55-61` calls stock `ggml/src/ggml-quants.c:2807-2815::dequantize_row_q8_K` directly for each selected row. Stock GET_ROWS excludes Q8_K at `ggml/src/ggml-cpu/ops.cpp:5023-5052`. This is the specified adaptation. +- `secondary.cpp:68-80` builds and computes an actual GET_ROWS graph for MXFP4 and IQ1_XXXS. Stock `ggml.c:3891-3909` creates the F32 output. `ggml-cpu.c:1864-1866,3432-3437` executes the CPU graph. +- Stock `ggml-cpu/ops.cpp:4850-4890,5032-5051` selects `to_float`. `ggml.c:751-756` routes MXFP4 to `ggml-quants.c:569-586`. +- Fork `ggml-cpu/ops.cpp:5063-5069` admits IQ1_XXXS. `ggml.c:871-876` routes it to `ggml-quants.c:2727-2751` and the fork's own codebook. + +All six operator output files match their recorded hashes and the author's prior outputs byte-for-byte. The stock binary hash is `7b9a2e2d0dc95afbf592b5b1678bdc67e0fe2204529f1afe3b0b1f03566f28c2`. The fork binary hash is `cad537f239dc0c4ca70e5b80548f39f297117494269ca2b9b5de8191aa2f4bc1`. This capture uses contiguous synthetic tables and IDs. It does not claim the complete upstream test parameter matrix or secondary BF16 execution. + +The operator requested an additional numerical explanation. `arithmetic-explanation.json` reproduces all elements of 16 captured files: Q6_K and IQ3_S, two widths, two output dtypes, and both native and primary arms. These CPU calculations are diagnostic transcriptions of source expressions. They are not replacement oracle results. + +Q6_K differs through intermediate precision. Plugin `dequantize_hip.cuh:248-257` rounds `scale * quant` to FP16 and multiplies in FP16 before output conversion. The local implementation uses F32 products. For example, `115 * -19 = -2185` becomes `-2184` in FP16. With delta `1/2048`, the outputs are `-1.06689453125` and `-1.06640625`. + +IQ3_S differs through its codebook, not rounding. Across 512 words, 150 byte positions in 146 words differ exclusively as plugin `62/4 = 15.5` versus local `15`. Plugin `ggml-common_hip.h:686` defines that table, and `dequantize_hip.cuh:347-353` applies its quarter-scale factor. The local codebook matches the stock representation. The 3.333% difference relative to local magnitude fits the original 4% relative tolerance. It is not byte equality or model token equality. + +The diagnostic initially rejected a changed native header hash. That failed guard remains in `explain_arithmetic-v2-source-guard.py`. The successful calculation uses a copied current header with SHA-256 `56b14d63308a6c3d8c19a6619217470fac88dd58599f95b4d9bf919f854f732d`, after the author's comment corrections, and table hash `d267d25ff1dd3f67749c251f87047f4b70c9650a99d04b524ff6c3313596a349`. Its comparison files are the frozen v2 runtime outputs. It makes no exact-v2 source-seal claim. The operator comparison report is `84bac44682927c28a9921e0dd464c1474039d53633fe53d008a14887cf807024`. + +One medium-severity qualification concern remains: secondary model cache geometry differs from the committed specification. `secondary.cpp:99-106` requests context 64 and BF16 KV. Both logs report 256 KV cells and approximately 0.06 MiB of KV payload. Stock `src/llama-context.cpp:288,294` explicitly pads context to 256. The spec requires 64 cells and 16,384 bytes. Record and resolve this upstream padding adaptation before claiming matched model memory. + +Both secondary bounded models abort on the first decode with SIGABRT. The observed stack includes `llm_graph_input_mem_hybrid::set_input` and `ggml-backend.cpp:194::GGML_ASSERT(buffer)`. Source `llama-graph.cpp:1078-1079` is consistent with an unallocated recurrent `s_copy` input. That object attribution remains an inference. Neither run emits tokens. Primary bounded-model qualification remains PENDING as separately recorded by the operator. The 32 original fixtures, model token equality, model cache adaptation, trace gate, and fork issue #933 remain PENDING under their existing ownership. + +Reproduction commands: + +```sh +python3 /home/vikash/.cache/rdna3-gather-oracle-review/audit_frozen.py +python3 /home/vikash/.cache/rdna3-gather-oracle-review/explain_arithmetic-current-source.py +``` + +Both commands exited 0. The first writes `independent-audit.json`; the second writes `arithmetic-explanation.json`. Neither command uses a GPU or changes tracked files. + +`REV-MUTATION` and product `REV-FULL-GATE` are N/A for this explicitly scoped read-only evidence audit. The startup `scripts/agent-preflight.sh` completed with exit 1: one `test_agent_onboard` failure and 12 skips. The failure expected scratch Git branch `master` but inherited `main`. Its isolated rerun passed all 39 tests with `GIT_CONFIG_GLOBAL=/dev/null`, an external `TMPDIR`, and matching `GIT_CEILING_DIRECTORIES`. Exact command, environment, and output are retained in the isolated-test artifacts. The startup preflight is not reported as green. No second full preflight ran. diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-qualification/README.md b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/README.md new file mode 100644 index 0000000000..b99fa91d3c --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/README.md @@ -0,0 +1,78 @@ +# Bounded model oracle qualification + +Row: `BACKEND-ROCM-QUANT-GATHER`. Issue: [#3093](https://github.com/mudler/vllm.cpp/issues/3093). + +The committed amendment `e35328644b8a0b57f97388965575c20d122a32dc` precedes this harness implementation. +The original 19 GGUF fixtures, projections, prompts, sampling, and four generated tokens remain unchanged. +The native and primary executions use 16 physical blocks of 16 cells and a logical limit of 64 tokens. +The earlier four-block captures remain in the parent evidence directory. +They are superseded only for the matched model-memory comparison. + +## Captures and provenance + +[`capture-manifest.json`](capture-manifest.json) seals each file in +[`qualification-captures.tar.gz`](../qualification-captures.tar.gz). +Each execution directory retains exact argument arrays, environment, input hashes, operator receipts, and raw output. +The root operator executed all GPU and secondary-oracle commands under the shared GPU mutex. +The helper compiled with at most four jobs and ran the CPU-only regression build. + +Primary vLLM stays at `e126687a9a828d513c01a07cd69f025f27d63280`. +The GGUF plugin stays at `d4c1f0d082fc7cd4350da56689109a01c1f29d6c`. +The runtime image is `sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc`. +The primary source and production compiler, graph capture, and kernel dispatch remain unchanged. +The harness selects the pinned text class and preserves the original rotary width through the two amended arguments. + +Stock llama.cpp stays at `10bf611e533d81f739128304991c5e133c6aebd8`. +The IQ1 fork stays at `36fe8e1cc7f2b3b8c92fdda0ab07600141921786`. +The [stock patch](stock-retain-hybrid-copy.md) and [fork patch](fork-retain-hybrid-copy.md) +each retain `s_copy` as one graph leaf in `build_inp_mem_hybrid`. +The raw `.patch` artifacts were re-homed to documents because `.patch` has no admitted evidence class. +Complete pristine and overlay inventories prove that each patch changes only its intended source file. +These overlays qualify this bounded synthetic model only; global oracle pins and issue #933 remain unchanged. + +## Observed failures and corrections + +The original primary harness fails during renderer construction. +The pristine stock and fork models abort during the first decode. +Debugger captures identify a nonnull `s_copy` tensor with null buffer and data immediately before its setter asserts. +The first stock debugger breakpoint missed that setter; its failed witness remains archived beside the corrected capture. + +The secondary overlays execute both original model fixtures. +All 12 stock MXFP4 and fork IQ1_XXXS completions match the native 16-block captures exactly. +Each format covers both prompts and three fresh engines. +The nine pristine codec outputs retain their prior hashes, including the Q8_K direct-decoder adaptation. +Q8_K has no secondary model-execution claim. + +The separate recurrent control contains an actual recurrent layer followed by full attention. +Both pins execute it through pristine and overlay binaries on both prompts and all three repeats. +All 24 captures preserve tokens and every one of the 512 F32 logits byte-for-byte. +Four additional graph controls consume `s_copy` and preserve exact state values through three graph reuses. +The original gather models remain unchanged; the control's shape and tensor hashes are archived separately. + +The first two memory observer attempts hang after the worker's allocator snapshot. +The final observer converts the layout Enum to its name and returns a complete JSON string from the worker. +The v3 capture exits 0, emits `[47, 19, 4, 20]`, and observes the same cache before and after generation. +The failed attempts and operator cleanup receipts remain evidence; neither is an oracle configuration pass. + +## Physical cache and native gates + +The executing primary cache has BF16 dtype, shape `[16, 2, 16, 64]`, and strides `[1024, 16384, 64, 1]`. +Its resolved layout is `LHBNC`, with stride order `[0, 2, 1, 3, 4]`. +The tensor payload, storage, and active allocator block each contain 65,536 bytes. +The allocator backs that block with a 2,097,152-byte segment shared with other allocations. +The process allocator totals are separate fields; they are not KV payload measurements. +Identical-tool model traces remain required to identify the backing allocations independently. + +Native runtime records cover all 228 public engines at BF16, 16 blocks, and 4,096 bytes per page. +The complete focused and ROCm regression command passes all 15 CTests. +The required case with a different ambient device passes independently on two visible GPUs. +The CPU build passes six tests and explicitly skips the two ROCm-only tests. +The initial CPU regex selected no tests; the corrected command requires a nonempty test set. + +## Remaining gates + +The 96 primary model captures, identical-tool model-memory audit, final staged preflight, +fresh mutation review, and operator rerun remain pending at this evidence checkpoint. +The original 32 upstream fixtures still require developer authority for their 289,655,872-byte download. +The combined CUDA/HIP execution remains pending an available combined build. +No throughput, latency, or performance-ratio claim is made by this correctness change. diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-qualification/capture-manifest.json b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/capture-manifest.json new file mode 100644 index 0000000000..324986d31f --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/capture-manifest.json @@ -0,0 +1,2722 @@ +{ + "archive": { + "name": "qualification-captures.tar.gz", + "bytes": 1389587, + "sha256": "9f700062008e24f85aab75d453b368cd043ebfe447f8b97740a99ad7d93a1bc7" + }, + "files": { + "red-primary/commands.json": { + "bytes": 3929, + "sha256": "7cc1a3848ff56e014735380298bdc0a34502c6b79ee2bc558580ed6f80beef91" + }, + "red-primary/operator-input-audit.json": { + "bytes": 494912, + "sha256": "afcfef9451cf354163c324054e2552f02006efb2701b66c8c5520119371bda1b" + }, + "red-primary/operator-receipt.json": { + "bytes": 269, + "sha256": "1d404bc64a1821f7d65e61f776ff906c08bceb0facd4997c7b2478d6c2ab982b" + }, + "red-primary/report-p0-r0.json": { + "bytes": 5143, + "sha256": "8af897f2dc5ee76463728617f8aed7e135d1708b530678fac6d8cf3aa419153d" + }, + "red-primary/run-operator.py": { + "bytes": 1982, + "sha256": "33808ef6ad493d7229debafc24a26efc1a253e1075c0a74c468ff7a4eacf9bc4" + }, + "red-primary/run.log": { + "bytes": 5829, + "sha256": "1652782166a21121a3c3035cccb35556cf4131ab17b06d5919f383601e949acc" + }, + "red-secondary-stock/commands.json": { + "bytes": 3365, + "sha256": "b59fe6d2001e08bab7f6447bee7954417d37e84cfa75e7785ee0db8c75bd49ab" + }, + "red-secondary-stock/gdb.log": { + "bytes": 12467, + "sha256": "1fde3ec093d90d2b6303042edb17b5939bafbcdfe5140f068d3b5d36b2132283" + }, + "red-secondary-stock/operator-receipt.json": { + "bytes": 2672, + "sha256": "f787c63181c6fc599027511df1bda395e65789abe88cbfc72830f83fa670b375" + }, + "red-secondary-stock/pristine.log": { + "bytes": 15567, + "sha256": "1bae3e5955446d7ae04e0e017ae17c316b3a6d55f5b94d4b5d7789c546fa3e52" + }, + "red-secondary-stock-v2/commands.json": { + "bytes": 2856, + "sha256": "4c3027a2b617ac2f6a7832512b72d952df349101b18fe24c682dd07f6e90ac84" + }, + "red-secondary-stock-v2/gdb.log": { + "bytes": 16002, + "sha256": "563a8c4b59d90667decde2e20480d93f753b29ac2b516388fee1c82f77ed6f16" + }, + "red-secondary-stock-v2/operator-receipt.json": { + "bytes": 2200, + "sha256": "e88921866fc7ecb76cfd869357de11159e0fbe79f851ae35f76eda5ccb7bf84b" + }, + "red-secondary-fork/commands.json": { + "bytes": 3335, + "sha256": "012902177f99c262a216c55ad287331ee8dfffffd297e74a48a12931f086318f" + }, + "red-secondary-fork/gdb.log": { + "bytes": 15858, + "sha256": "8d2f3310bd52ede34f698b7b7ea48552bbc152a4a927d602c10b48fe75982ec7" + }, + "red-secondary-fork/operator-receipt.json": { + "bytes": 2666, + "sha256": "a6d2973379e12308c39db348913c4488c93fdcdbaebdf4ef105508fd6ebd9f73" + }, + "red-secondary-fork/pristine.log": { + "bytes": 15426, + "sha256": "3d4e3bdcdd3404b541890be451c10c67ff6f580afd048edbfe6ddbbece8c6a8e" + }, + "recurrent-pristine-v1/commands.json": { + "bytes": 4240, + "sha256": "396dc3f09b0aac7bbf2aa507f5e94ff07f05277970336a912d479800d6a29ce6" + }, + "recurrent-pristine-v1/controlled-states-f32.bin": { + "bytes": 144, + "sha256": "278297accf44028d6b775616bb5f2db11578c7a56ea8eb8172fe45392fb3932d" + }, + "recurrent-pristine-v1/controlled-states.log": { + "bytes": 291, + "sha256": "c80458d757ccd762ba5f2cbd9a2a22ceb54c04fd2c55e2737768d80a42e3dd32" + }, + "recurrent-pristine-v1/fork-p0-r0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "recurrent-pristine-v1/fork-p0-r0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "recurrent-pristine-v1/fork-p0-r0.log": { + "bytes": 13793, + "sha256": "b5aefe197af02aa68842fac9fbf800aea222e094dc7b9e26f29e67a2604720d5" + }, + "recurrent-pristine-v1/operator-input-audit.json": { + "bytes": 3767, + "sha256": "67881d99fdc4c2ddb3706f50bbc4357418a070a08bab672df363332f6a7ec1c3" + }, + "recurrent-pristine-v1/operator-receipt.json": { + "bytes": 756, + "sha256": "1b5634da9d3296292be28f0a4d0f64c06f4fad439fd7ff3ea141e8bbd0e2ee1d" + }, + "recurrent-pristine-v1/stock-p0-r0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "recurrent-pristine-v1/stock-p0-r0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "recurrent-pristine-v1/stock-p0-r0.log": { + "bytes": 13930, + "sha256": "ae2bd191d6f0e37cafe94b22a328e2f0d5db12f2ab2ca589da2a2dd1f85bdce3" + }, + "stock-overlay-green-v1/MXFP4-r0-p0-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9f439d50add79f4b0c78f824695202fb5c86e7f9ee8919d96f3c4c221d46aeb" + }, + "stock-overlay-green-v1/MXFP4-r0-p0-oracle-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "stock-overlay-green-v1/MXFP4-r0-p0-oracle.log": { + "bytes": 12121, + "sha256": "0854c8e5133f17c917748817efc10f5925849ffc8303184aece2fa700069d5f4" + }, + "stock-overlay-green-v1/commands.json": { + "bytes": 3649, + "sha256": "4c38e20be3cd38fe38d98e1dbb26c9ebc5792f4ceca4ba2dd28ae05d819ccfe8" + }, + "stock-overlay-green-v1/operator-input-audit.json": { + "bytes": 3268, + "sha256": "395f75ac086d70152a5a12a37f6e828b40286d9ea82f080ae0f964b48a05c3fa" + }, + "stock-overlay-green-v1/operator-receipt.json": { + "bytes": 490, + "sha256": "b40e720440a4e8bed2b62e419b54247c98255242b793ce295e8079b58ba9f69c" + }, + "stock-overlay-green-v1/recurrent-control-r0-p0-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "stock-overlay-green-v1/recurrent-control-r0-p0-oracle-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "stock-overlay-green-v1/recurrent-control-r0-p0-oracle.log": { + "bytes": 13930, + "sha256": "3c0fc37d66dc63f9d2afc4a046e733be1a0ce8b0897713e389ff69996b3e66e2" + }, + "secondary-matrix-v1/IQ1_XXXS-r0-p0-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "c191d091c96cbc48f7588480110322d2022ff9073e4a861f21b5ee7a2096fbe4" + }, + "secondary-matrix-v1/IQ1_XXXS-r0-p0-oracle-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "secondary-matrix-v1/IQ1_XXXS-r0-p0-oracle.log": { + "bytes": 11996, + "sha256": "2e1c29fa3ddcfe35db233b9abd24e081c140d865794edf483f10e9e5987b44c5" + }, + "secondary-matrix-v1/IQ1_XXXS-r0-p1-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "c110958d0571af6d758810e4e72f3821bd5b126be5162678b2f2749522f3d08a" + }, + "secondary-matrix-v1/IQ1_XXXS-r0-p1-oracle-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "secondary-matrix-v1/IQ1_XXXS-r0-p1-oracle.log": { + "bytes": 11996, + "sha256": "830030c104d00c7c09dfe6589be83b88ede01d2c9e5149905a0726cfabd8ceff" + }, + "secondary-matrix-v1/IQ1_XXXS-r1-p0-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "c191d091c96cbc48f7588480110322d2022ff9073e4a861f21b5ee7a2096fbe4" + }, + "secondary-matrix-v1/IQ1_XXXS-r1-p0-oracle-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "secondary-matrix-v1/IQ1_XXXS-r1-p0-oracle.log": { + "bytes": 11996, + "sha256": "f69a12690cf78254f02be6d26cb53cef2a24159b16428f54bcf7f7e06ec371dd" + }, + "secondary-matrix-v1/IQ1_XXXS-r1-p1-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "c110958d0571af6d758810e4e72f3821bd5b126be5162678b2f2749522f3d08a" + }, + "secondary-matrix-v1/IQ1_XXXS-r1-p1-oracle-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "secondary-matrix-v1/IQ1_XXXS-r1-p1-oracle.log": { + "bytes": 11996, + "sha256": "cb4d48b937f0ff2695ca9fb87673ad836ef56f079f766ec2e83ac927c0fe1c23" + }, + "secondary-matrix-v1/IQ1_XXXS-r2-p0-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "c191d091c96cbc48f7588480110322d2022ff9073e4a861f21b5ee7a2096fbe4" + }, + "secondary-matrix-v1/IQ1_XXXS-r2-p0-oracle-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "secondary-matrix-v1/IQ1_XXXS-r2-p0-oracle.log": { + "bytes": 11996, + "sha256": "78b6723be2e4f473c90129acd8a94f37546300c436c0abdc16f5201670eda302" + }, + "secondary-matrix-v1/IQ1_XXXS-r2-p1-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "c110958d0571af6d758810e4e72f3821bd5b126be5162678b2f2749522f3d08a" + }, + "secondary-matrix-v1/IQ1_XXXS-r2-p1-oracle-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "secondary-matrix-v1/IQ1_XXXS-r2-p1-oracle.log": { + "bytes": 11996, + "sha256": "605baffba564011b8ca8be3c4f8d5da5c70cb75bccc8577dd267b6b4b0c74688" + }, + "secondary-matrix-v1/MXFP4-r0-p0-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9f439d50add79f4b0c78f824695202fb5c86e7f9ee8919d96f3c4c221d46aeb" + }, + "secondary-matrix-v1/MXFP4-r0-p0-oracle-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "secondary-matrix-v1/MXFP4-r0-p0-oracle.log": { + "bytes": 12121, + "sha256": "356bc1a02f183a99662a8022f1bd2bfe77368ebec9d7c5c7e31e1a5256dcdfe0" + }, + "secondary-matrix-v1/MXFP4-r0-p1-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "1515a2e7b710321b931937600687d867cc94aa29cffba3e29bc82213e97a1d85" + }, + "secondary-matrix-v1/MXFP4-r0-p1-oracle-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/MXFP4-r0-p1-oracle.log": { + "bytes": 12122, + "sha256": "f76847c24f38db30bb81e5f00b53dab29c8f6f52f3661d123cd32fda91fc6c9a" + }, + "secondary-matrix-v1/MXFP4-r1-p0-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9f439d50add79f4b0c78f824695202fb5c86e7f9ee8919d96f3c4c221d46aeb" + }, + "secondary-matrix-v1/MXFP4-r1-p0-oracle-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "secondary-matrix-v1/MXFP4-r1-p0-oracle.log": { + "bytes": 12121, + "sha256": "5c399807740a43e381e5ea8ec5a4ea19a9b5c9245baba1040c626ac900c74992" + }, + "secondary-matrix-v1/MXFP4-r1-p1-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "1515a2e7b710321b931937600687d867cc94aa29cffba3e29bc82213e97a1d85" + }, + "secondary-matrix-v1/MXFP4-r1-p1-oracle-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/MXFP4-r1-p1-oracle.log": { + "bytes": 12122, + "sha256": "cd13ccf5de4bd95c33e02d5f28a2baaa1bfc830940b0015dde4c80facd1a9550" + }, + "secondary-matrix-v1/MXFP4-r2-p0-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9f439d50add79f4b0c78f824695202fb5c86e7f9ee8919d96f3c4c221d46aeb" + }, + "secondary-matrix-v1/MXFP4-r2-p0-oracle-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "secondary-matrix-v1/MXFP4-r2-p0-oracle.log": { + "bytes": 12121, + "sha256": "854b335555ee7da1a385b117f829121a09faca1eccbc5d6d5eb91159c553312c" + }, + "secondary-matrix-v1/MXFP4-r2-p1-oracle-logits-f32.bin": { + "bytes": 2048, + "sha256": "1515a2e7b710321b931937600687d867cc94aa29cffba3e29bc82213e97a1d85" + }, + "secondary-matrix-v1/MXFP4-r2-p1-oracle-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/MXFP4-r2-p1-oracle.log": { + "bytes": 12122, + "sha256": "426a09443b00b2b3ec3a7857d9e38c36ae9c78a9f7e7834d7c9a9e20dbe2edd1" + }, + "secondary-matrix-v1/commands.json": { + "bytes": 45298, + "sha256": "db37659cd9310eb3a7aa9ec6cf38a6e9a240584a0a4734c1b0d0f208ce3f84d0" + }, + "secondary-matrix-v1/controls/fork-overlay-r0-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/fork-overlay-r0-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/fork-overlay-r0-p0.log": { + "bytes": 13793, + "sha256": "f156aef86ff14a3b80f9a9c245b1c10d4d6dd264ed1b07186f4818684d3a363d" + }, + "secondary-matrix-v1/controls/fork-overlay-r0-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/fork-overlay-r0-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/fork-overlay-r0-p1.log": { + "bytes": 13793, + "sha256": "491fb142c21ef454a38ae4b8a8f7fc28f6b96829759548fd5813e8efdebed867" + }, + "secondary-matrix-v1/controls/fork-overlay-r1-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/fork-overlay-r1-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/fork-overlay-r1-p0.log": { + "bytes": 13793, + "sha256": "ecdd84ce6c65e662f0a6c6c89ebe575fc0b654e585fbd32b0275af4dae950089" + }, + "secondary-matrix-v1/controls/fork-overlay-r1-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/fork-overlay-r1-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/fork-overlay-r1-p1.log": { + "bytes": 13793, + "sha256": "c40b8ddeda580d6f6062c46fa2295d0626e1d1db0a6da89e9a8b8cea0c1b2425" + }, + "secondary-matrix-v1/controls/fork-overlay-r2-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/fork-overlay-r2-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/fork-overlay-r2-p0.log": { + "bytes": 13793, + "sha256": "39fdb976d0819d002e753c7bcae431888d8ac56311a3a33794cc6fe64918ee23" + }, + "secondary-matrix-v1/controls/fork-overlay-r2-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/fork-overlay-r2-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/fork-overlay-r2-p1.log": { + "bytes": 13793, + "sha256": "176ed72b1bb871cab27858e10fde630a320b4339160274a0a2a6092e015ff985" + }, + "secondary-matrix-v1/controls/fork-overlay-state-reuse.bin": { + "bytes": 144, + "sha256": "278297accf44028d6b775616bb5f2db11578c7a56ea8eb8172fe45392fb3932d" + }, + "secondary-matrix-v1/controls/fork-overlay-state-reuse.log": { + "bytes": 291, + "sha256": "c80458d757ccd762ba5f2cbd9a2a22ceb54c04fd2c55e2737768d80a42e3dd32" + }, + "secondary-matrix-v1/controls/fork-pristine-r0-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/fork-pristine-r0-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/fork-pristine-r0-p0.log": { + "bytes": 13793, + "sha256": "57a409531c63aaa08ca7c0cd73661216030b27291182504dc25f4e584a0743c3" + }, + "secondary-matrix-v1/controls/fork-pristine-r0-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/fork-pristine-r0-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/fork-pristine-r0-p1.log": { + "bytes": 13793, + "sha256": "6ada9b0ba72dabaf9187c1152591fc5956b1bab86a9631cb4a4752498e7666c2" + }, + "secondary-matrix-v1/controls/fork-pristine-r1-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/fork-pristine-r1-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/fork-pristine-r1-p0.log": { + "bytes": 13793, + "sha256": "7768622c32d750a39a7ca3817554cbe4e171b520a94b83180f14323f52453e87" + }, + "secondary-matrix-v1/controls/fork-pristine-r1-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/fork-pristine-r1-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/fork-pristine-r1-p1.log": { + "bytes": 13793, + "sha256": "0be91d4a242162cf646bd035902cb63f29796a9cd828bc00488204152c32110a" + }, + "secondary-matrix-v1/controls/fork-pristine-r2-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/fork-pristine-r2-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/fork-pristine-r2-p0.log": { + "bytes": 13793, + "sha256": "12c725a7cf899cfe6d6fcee27da85753bd4a5594795593244fbb079ecc6d5711" + }, + "secondary-matrix-v1/controls/fork-pristine-r2-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/fork-pristine-r2-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/fork-pristine-r2-p1.log": { + "bytes": 13793, + "sha256": "85de3e0f16eeca3f26f299dffc3cf848a7a8f500c1a1c1871ad1c6b130ded095" + }, + "secondary-matrix-v1/controls/fork-pristine-state-reuse.bin": { + "bytes": 144, + "sha256": "278297accf44028d6b775616bb5f2db11578c7a56ea8eb8172fe45392fb3932d" + }, + "secondary-matrix-v1/controls/fork-pristine-state-reuse.log": { + "bytes": 291, + "sha256": "c80458d757ccd762ba5f2cbd9a2a22ceb54c04fd2c55e2737768d80a42e3dd32" + }, + "secondary-matrix-v1/controls/stock-overlay-r0-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/stock-overlay-r0-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/stock-overlay-r0-p0.log": { + "bytes": 13930, + "sha256": "f8883fbe41d1a894663320a4b28c62c8054789d3caa37b3a2cd69f1ce005f5e6" + }, + "secondary-matrix-v1/controls/stock-overlay-r0-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/stock-overlay-r0-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/stock-overlay-r0-p1.log": { + "bytes": 13930, + "sha256": "1d1cf32ab3ca5ace95cc8c851439e412d98d6f2ecf11cd6124888d4521ec53a4" + }, + "secondary-matrix-v1/controls/stock-overlay-r1-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/stock-overlay-r1-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/stock-overlay-r1-p0.log": { + "bytes": 13930, + "sha256": "daed336ba16f0ca589203dfb664b4fde952b677523a6ecc9a117938caa171155" + }, + "secondary-matrix-v1/controls/stock-overlay-r1-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/stock-overlay-r1-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/stock-overlay-r1-p1.log": { + "bytes": 13930, + "sha256": "3ec1fef75b329a06e438987601dac66260e7131533bfb1db460cf672a73f50a0" + }, + "secondary-matrix-v1/controls/stock-overlay-r2-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/stock-overlay-r2-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/stock-overlay-r2-p0.log": { + "bytes": 13930, + "sha256": "66efff2003e4cdbaa57da7f83ba35c032f622c1657b282bde4aa9c96ec784a9e" + }, + "secondary-matrix-v1/controls/stock-overlay-r2-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/stock-overlay-r2-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/stock-overlay-r2-p1.log": { + "bytes": 13930, + "sha256": "2b95caad355effad56ad9ac76d0586b0c925dbd5c558805d902986ab794c81f1" + }, + "secondary-matrix-v1/controls/stock-overlay-state-reuse.bin": { + "bytes": 144, + "sha256": "278297accf44028d6b775616bb5f2db11578c7a56ea8eb8172fe45392fb3932d" + }, + "secondary-matrix-v1/controls/stock-overlay-state-reuse.log": { + "bytes": 291, + "sha256": "c80458d757ccd762ba5f2cbd9a2a22ceb54c04fd2c55e2737768d80a42e3dd32" + }, + "secondary-matrix-v1/controls/stock-pristine-r0-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/stock-pristine-r0-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/stock-pristine-r0-p0.log": { + "bytes": 13930, + "sha256": "2d14897c73cc54efe1e1dcb7ca06899044c974c73308edc33f755dcdfd937ea9" + }, + "secondary-matrix-v1/controls/stock-pristine-r0-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/stock-pristine-r0-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/stock-pristine-r0-p1.log": { + "bytes": 13930, + "sha256": "f6a30ce78c477282ac871ec4f5c2bb40c2449637f7ebbbdd6b26c711cc51b375" + }, + "secondary-matrix-v1/controls/stock-pristine-r1-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/stock-pristine-r1-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/stock-pristine-r1-p0.log": { + "bytes": 13930, + "sha256": "e6ce2f4b6bfccba5143d396cce5d0b10446247e4ad9cbb72fa5cdbb4cb74aaeb" + }, + "secondary-matrix-v1/controls/stock-pristine-r1-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/stock-pristine-r1-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/stock-pristine-r1-p1.log": { + "bytes": 13930, + "sha256": "e721edf653d920f9effcb123baa9bb98085a82a2966093ada1fad47d98327bdb" + }, + "secondary-matrix-v1/controls/stock-pristine-r2-p0-logits-f32.bin": { + "bytes": 2048, + "sha256": "cecebabad2c80d30f6bc0c34ec1dcef46514b5a7c9d5ad315959eb52941bb980" + }, + "secondary-matrix-v1/controls/stock-pristine-r2-p0-tokens.txt": { + "bytes": 11, + "sha256": "b8f6dbcb7190c31755f395ed236021891e4efa6ea80a4119e829b6dbca6202ea" + }, + "secondary-matrix-v1/controls/stock-pristine-r2-p0.log": { + "bytes": 13930, + "sha256": "b15ece586fa3a7f42636a2a5f5f63ba2f458dd5247e178bf3a8f7e1c62f8a7fd" + }, + "secondary-matrix-v1/controls/stock-pristine-r2-p1-logits-f32.bin": { + "bytes": 2048, + "sha256": "70c60b89cd8043ef1a80236d3c52ffcc100e6bcbd0e3ad669ed6ef3e500070ff" + }, + "secondary-matrix-v1/controls/stock-pristine-r2-p1-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "secondary-matrix-v1/controls/stock-pristine-r2-p1.log": { + "bytes": 13930, + "sha256": "e721edf653d920f9effcb123baa9bb98085a82a2966093ada1fad47d98327bdb" + }, + "secondary-matrix-v1/controls/stock-pristine-state-reuse.bin": { + "bytes": 144, + "sha256": "278297accf44028d6b775616bb5f2db11578c7a56ea8eb8172fe45392fb3932d" + }, + "secondary-matrix-v1/controls/stock-pristine-state-reuse.log": { + "bytes": 291, + "sha256": "c80458d757ccd762ba5f2cbd9a2a22ceb54c04fd2c55e2737768d80a42e3dd32" + }, + "secondary-matrix-v1/operator-input-audit.json": { + "bytes": 42699, + "sha256": "555e967c80ccf637e3c995ca22d5da36683e20ae2ec8041db99cd550ca48ccf4" + }, + "secondary-matrix-v1/operator-output-checks.json": { + "bytes": 5839, + "sha256": "3356c8624ae22e79d1f95ac1e8add96cebcd1f15a9a9245fc4ee08d6c8593949" + }, + "secondary-matrix-v1/operator-receipt.json": { + "bytes": 11561, + "sha256": "3fec16f8442f7420f7d811b56c39081689dd8204308b4b7832eb04290ce335bb" + }, + "secondary-matrix-v1/pristine-codecs/stock-shape-IQ1_XXXS-256-f32.bin": { + "bytes": 4096, + "sha256": "bccfb71bc3b329ff21d4061d4352e0b34958640822a8248ee65045b6af762212" + }, + "secondary-matrix-v1/pristine-codecs/stock-shape-IQ1_XXXS-256-f32.bin.log": { + "bytes": 169, + "sha256": "f8f8d7463ee1473288e775e37d905afbf35c43b7ff6443e3ed90ea6234376c7c" + }, + "secondary-matrix-v1/pristine-codecs/stock-shape-MXFP4-256-f32.bin": { + "bytes": 4096, + "sha256": "da5bd348b0562899e0be685b723f9a26d43b405a46e00d9667f22bd93c73a9b4" + }, + "secondary-matrix-v1/pristine-codecs/stock-shape-MXFP4-256-f32.bin.log": { + "bytes": 166, + "sha256": "6e6c539254bcb2836e72cf3e702ac5c6bb025e60f2e7bf64b8f18beddb3aa68c" + }, + "secondary-matrix-v1/pristine-codecs/stock-shape-Q8_K-256-f32.bin": { + "bytes": 4096, + "sha256": "bc63b12de49c2573302b712ab0528c177ddb486f559d470a0aca9e7f16ba7996" + }, + "secondary-matrix-v1/pristine-codecs/stock-shape-Q8_K-256-f32.bin.log": { + "bytes": 157, + "sha256": "d4ee090db5ac6f6d11a19ca28a6655084d88b9254f9c6fa18433339f17a404c1" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-IQ1_XXXS-1024-f32.bin": { + "bytes": 16384, + "sha256": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-IQ1_XXXS-1024-f32.bin.log": { + "bytes": 175, + "sha256": "599b4d6f937fed29361721ccf5c43f664bee63b25cd188bfdf516504f934ba8a" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-IQ1_XXXS-256-f32.bin": { + "bytes": 4096, + "sha256": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-IQ1_XXXS-256-f32.bin.log": { + "bytes": 172, + "sha256": "56e3aed3c87e44b47a9b7c10ae461147d8a773415fcf38934483d53118f1d1f7" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-MXFP4-1024-f32.bin": { + "bytes": 16384, + "sha256": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-MXFP4-1024-f32.bin.log": { + "bytes": 172, + "sha256": "c5b9d00254be3adbd6fc7e9edede85a276886b03e3bbc9b6a8b9ca39e0f61302" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-MXFP4-256-f32.bin": { + "bytes": 4096, + "sha256": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-MXFP4-256-f32.bin.log": { + "bytes": 170, + "sha256": "ba6cbc0912a63378027e30d26682fa8f735ecbd37b6a01dd20c2b0534ffebc9e" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-Q8_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-Q8_K-1024-f32.bin.log": { + "bytes": 163, + "sha256": "dd9a8f230fb6110feb052fa69a8846c2b11abaf82fa71aaef44bcc91e0b18fd5" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-Q8_K-256-f32.bin": { + "bytes": 4096, + "sha256": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6" + }, + "secondary-matrix-v1/pristine-codecs/synthetic-secondary-Q8_K-256-f32.bin.log": { + "bytes": 160, + "sha256": "45c7ce06e27dd83e7ddb03e8bc2183180162b8c964f3da43de1bb0fcb9ddc51b" + }, + "secondary-matrix-v1/run-operator.py": { + "bytes": 1408, + "sha256": "190e53d72dac1051742d8d7aaf6b53d744c690b5dfeb51cca05e60090a0728e2" + }, + "primary-green-v1/commands.json": { + "bytes": 4042, + "sha256": "d63a6f82feae04a39a947b06b07a041a14d370d516c72c7d442bd439697bfb44" + }, + "primary-green-v1/operator-input-audit.json": { + "bytes": 495039, + "sha256": "08445d4783c3f082f217a47b3383e6accee49aabd7f8a1d6abcdb8a855a455ea" + }, + "primary-green-v1/operator-receipt.json": { + "bytes": 271, + "sha256": "8c5462c30249f3cf0cf17e2a60ff9939b542346ab2946c65da3853bf4ec5425d" + }, + "primary-green-v1/operator-stop.json": { + "bytes": 455, + "sha256": "c57303552964a928bbb99994057ee53b8ad2579a021d2e2cfc10cbc357239ec8" + }, + "primary-green-v1/run-operator.py": { + "bytes": 1982, + "sha256": "46e6ba714f7525df4c2f3a87b436028f250966a275679b8e81f17dafc4d91e2b" + }, + "primary-green-v1/run.log": { + "bytes": 14618, + "sha256": "969bfa585bbd8bb4e3849d320e81297cd7b86b31201237a0211908d4b5cc2d53" + }, + "primary-memory-diagnostic-v2/commands.json": { + "bytes": 4282, + "sha256": "da87b02bee4755b5ba1e23706c18a6583ada22fc4a08f4d68d1780b6e75abb18" + }, + "primary-memory-diagnostic-v2/operator-input-audit.json": { + "bytes": 495480, + "sha256": "44cf0347ebbf4d5baa54ccad3aaca99bf76db60afb2feef73e685fe65704fc83" + }, + "primary-memory-diagnostic-v2/operator-receipt.json": { + "bytes": 272, + "sha256": "7cb465a30cd47d3c7319731f1351b4590b54f9303d7896656bb6736eb4d1fc36" + }, + "primary-memory-diagnostic-v2/operator-stop.json": { + "bytes": 391, + "sha256": "cd33d0e50971bfa8210c63b6d3942dda3898d9ef0f35acfcf3497289923ba545" + }, + "primary-memory-diagnostic-v2/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "primary-memory-diagnostic-v2/primary_model.py": { + "bytes": 8492, + "sha256": "4bd33a876dfc7f298eeee6204194e28f31373c54ec5de40567696b18f4eaf66c" + }, + "primary-memory-diagnostic-v2/run-operator.py": { + "bytes": 1982, + "sha256": "959ae49980b9a730d2be4e7539a56e41c27a11d789f06a4e8f3926c3c86a21bc" + }, + "primary-memory-diagnostic-v2/run.log": { + "bytes": 14475, + "sha256": "283d50637644f15d86fb77a034e334f4dcdc93d4382cf5f5457b0f495b7ed68a" + }, + "primary-memory-diagnostic-v3/commands.json": { + "bytes": 4282, + "sha256": "fbf511613cd4fc9c23a78754f7179edab074e02fa001ff419936fdd38ed7ebd3" + }, + "primary-memory-diagnostic-v3/operator-input-audit.json": { + "bytes": 495480, + "sha256": "803ea0c6ba3d359c409208898b314ab3dc10428909d8f5002f95ff25b5f7d65d" + }, + "primary-memory-diagnostic-v3/operator-receipt.json": { + "bytes": 270, + "sha256": "6aa2e529b2a00b8bba61a15e2254929aead9cd5857208682f3a030ac167d24f5" + }, + "primary-memory-diagnostic-v3/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "primary-memory-diagnostic-v3/primary_model.py": { + "bytes": 9064, + "sha256": "cf451948efca5ab124815ac4134164df7eaf14360157be14fa7c3480f3d52f4c" + }, + "primary-memory-diagnostic-v3/report-p0-r0.json": { + "bytes": 5931, + "sha256": "98529f2f6d24f2a4b3fed962a5c727b76bf6a5a457df3582e9a2255d0736beb6" + }, + "primary-memory-diagnostic-v3/run-operator.py": { + "bytes": 1982, + "sha256": "2d864973eb9d6ee7ad2d2d53d3c82a35cc2b36eb61293f339104c36a9f09e65f" + }, + "primary-memory-diagnostic-v3/run.log": { + "bytes": 20644, + "sha256": "6d30e53d138b5783d18c256a34de59facf852b5bd1a884a7e7685be817a73874" + }, + "native-matched16-v1/ambient-device.log": { + "bytes": 294, + "sha256": "38fec7edbc44a0e557538088a130e98da012892fdec43f29234989d35718209e" + }, + "native-matched16-v1/commands.json": { + "bytes": 15550, + "sha256": "fbedf1a342ca70999ca4574052f4872d23bf5c1ff7950db75d5ff5aa78839018" + }, + "native-matched16-v1/focused-and-regressions.log": { + "bytes": 122914, + "sha256": "4539adff8e6370eb40dfe47e5ba862977f7f6f97a085efd1b4118869c1893de7" + }, + "native-matched16-v1/operator-input-audit.json": { + "bytes": 11823, + "sha256": "05b96c1dce617830ea3794a0dc6807b45ba20e6574ad56648ca55c0751f6eb55" + }, + "native-matched16-v1/operator-output-checks.json": { + "bytes": 23731, + "sha256": "97e0f117e94975645333f18e5f4ba7bcce4ed6ce7e816874e9634b16e50de337" + }, + "native-matched16-v1/operator-receipt.json": { + "bytes": 518, + "sha256": "2417a0ed63aa37f3080c6776a307f247fd8e57296528765019ad42589878a290" + }, + "native-matched16-v1/public-captures/IQ1_S-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "native-matched16-v1/public-captures/IQ1_S-r0-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "native-matched16-v1/public-captures/IQ1_S-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "native-matched16-v1/public-captures/IQ1_S-r0-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "native-matched16-v1/public-captures/IQ1_S-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "native-matched16-v1/public-captures/IQ1_S-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "native-matched16-v1/public-captures/IQ1_S-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "native-matched16-v1/public-captures/IQ1_S-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "native-matched16-v1/public-captures/IQ1_S-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "native-matched16-v1/public-captures/IQ1_S-r1-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "native-matched16-v1/public-captures/IQ1_S-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "native-matched16-v1/public-captures/IQ1_S-r1-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "native-matched16-v1/public-captures/IQ1_S-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "native-matched16-v1/public-captures/IQ1_S-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "native-matched16-v1/public-captures/IQ1_S-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "native-matched16-v1/public-captures/IQ1_S-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "native-matched16-v1/public-captures/IQ1_S-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "native-matched16-v1/public-captures/IQ1_S-r2-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "native-matched16-v1/public-captures/IQ1_S-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "native-matched16-v1/public-captures/IQ1_S-r2-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "native-matched16-v1/public-captures/IQ1_S-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "native-matched16-v1/public-captures/IQ1_S-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "native-matched16-v1/public-captures/IQ1_S-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "native-matched16-v1/public-captures/IQ1_S-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "native-matched16-v1/public-captures/IQ1_XXXS-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "native-matched16-v1/public-captures/IQ2_S-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "native-matched16-v1/public-captures/IQ2_S-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "native-matched16-v1/public-captures/IQ2_S-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "native-matched16-v1/public-captures/IQ2_S-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "native-matched16-v1/public-captures/IQ2_S-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "native-matched16-v1/public-captures/IQ2_S-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "native-matched16-v1/public-captures/IQ2_S-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "native-matched16-v1/public-captures/IQ2_S-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "native-matched16-v1/public-captures/IQ2_S-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "native-matched16-v1/public-captures/IQ2_S-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "native-matched16-v1/public-captures/IQ2_S-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "native-matched16-v1/public-captures/IQ2_S-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "native-matched16-v1/public-captures/IQ2_S-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "native-matched16-v1/public-captures/IQ2_S-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "native-matched16-v1/public-captures/IQ2_S-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "native-matched16-v1/public-captures/IQ2_S-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "native-matched16-v1/public-captures/IQ2_S-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "native-matched16-v1/public-captures/IQ2_S-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "native-matched16-v1/public-captures/IQ2_S-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "native-matched16-v1/public-captures/IQ2_S-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "native-matched16-v1/public-captures/IQ2_S-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "native-matched16-v1/public-captures/IQ2_S-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "native-matched16-v1/public-captures/IQ2_S-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "native-matched16-v1/public-captures/IQ2_S-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "native-matched16-v1/public-captures/IQ2_XS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "native-matched16-v1/public-captures/IQ2_XS-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "native-matched16-v1/public-captures/IQ2_XS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "native-matched16-v1/public-captures/IQ2_XS-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "native-matched16-v1/public-captures/IQ2_XS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "native-matched16-v1/public-captures/IQ2_XS-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "native-matched16-v1/public-captures/IQ2_XS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "native-matched16-v1/public-captures/IQ2_XS-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "native-matched16-v1/public-captures/IQ2_XS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "native-matched16-v1/public-captures/IQ2_XS-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "native-matched16-v1/public-captures/IQ2_XS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "native-matched16-v1/public-captures/IQ2_XS-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "native-matched16-v1/public-captures/IQ2_XS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "native-matched16-v1/public-captures/IQ2_XS-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "native-matched16-v1/public-captures/IQ2_XS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "native-matched16-v1/public-captures/IQ2_XS-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "native-matched16-v1/public-captures/IQ2_XS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "native-matched16-v1/public-captures/IQ2_XS-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "native-matched16-v1/public-captures/IQ2_XS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "native-matched16-v1/public-captures/IQ2_XS-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "native-matched16-v1/public-captures/IQ2_XS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "native-matched16-v1/public-captures/IQ2_XS-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "native-matched16-v1/public-captures/IQ2_XS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "native-matched16-v1/public-captures/IQ2_XS-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "native-matched16-v1/public-captures/IQ2_XXS-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "native-matched16-v1/public-captures/IQ3_S-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "native-matched16-v1/public-captures/IQ3_S-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "native-matched16-v1/public-captures/IQ3_S-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "native-matched16-v1/public-captures/IQ3_S-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "native-matched16-v1/public-captures/IQ3_S-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "native-matched16-v1/public-captures/IQ3_S-r0-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "native-matched16-v1/public-captures/IQ3_S-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "native-matched16-v1/public-captures/IQ3_S-r0-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "native-matched16-v1/public-captures/IQ3_S-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "native-matched16-v1/public-captures/IQ3_S-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "native-matched16-v1/public-captures/IQ3_S-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "native-matched16-v1/public-captures/IQ3_S-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "native-matched16-v1/public-captures/IQ3_S-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "native-matched16-v1/public-captures/IQ3_S-r1-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "native-matched16-v1/public-captures/IQ3_S-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "native-matched16-v1/public-captures/IQ3_S-r1-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "native-matched16-v1/public-captures/IQ3_S-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "native-matched16-v1/public-captures/IQ3_S-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "native-matched16-v1/public-captures/IQ3_S-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "native-matched16-v1/public-captures/IQ3_S-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "native-matched16-v1/public-captures/IQ3_S-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "native-matched16-v1/public-captures/IQ3_S-r2-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "native-matched16-v1/public-captures/IQ3_S-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "native-matched16-v1/public-captures/IQ3_S-r2-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r0-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r0-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r1-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r1-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r2-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "native-matched16-v1/public-captures/IQ3_XXS-r2-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "native-matched16-v1/public-captures/IQ4_NL-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "native-matched16-v1/public-captures/IQ4_NL-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "native-matched16-v1/public-captures/IQ4_NL-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "native-matched16-v1/public-captures/IQ4_NL-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "native-matched16-v1/public-captures/IQ4_NL-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "native-matched16-v1/public-captures/IQ4_NL-r0-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "native-matched16-v1/public-captures/IQ4_NL-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "native-matched16-v1/public-captures/IQ4_NL-r0-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "native-matched16-v1/public-captures/IQ4_NL-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "native-matched16-v1/public-captures/IQ4_NL-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "native-matched16-v1/public-captures/IQ4_NL-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "native-matched16-v1/public-captures/IQ4_NL-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "native-matched16-v1/public-captures/IQ4_NL-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "native-matched16-v1/public-captures/IQ4_NL-r1-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "native-matched16-v1/public-captures/IQ4_NL-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "native-matched16-v1/public-captures/IQ4_NL-r1-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "native-matched16-v1/public-captures/IQ4_NL-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "native-matched16-v1/public-captures/IQ4_NL-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "native-matched16-v1/public-captures/IQ4_NL-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "native-matched16-v1/public-captures/IQ4_NL-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "native-matched16-v1/public-captures/IQ4_NL-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "native-matched16-v1/public-captures/IQ4_NL-r2-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "native-matched16-v1/public-captures/IQ4_NL-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "native-matched16-v1/public-captures/IQ4_NL-r2-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "native-matched16-v1/public-captures/IQ4_XS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "native-matched16-v1/public-captures/IQ4_XS-r0-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "native-matched16-v1/public-captures/IQ4_XS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "native-matched16-v1/public-captures/IQ4_XS-r0-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "native-matched16-v1/public-captures/IQ4_XS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "native-matched16-v1/public-captures/IQ4_XS-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "native-matched16-v1/public-captures/IQ4_XS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "native-matched16-v1/public-captures/IQ4_XS-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "native-matched16-v1/public-captures/IQ4_XS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "native-matched16-v1/public-captures/IQ4_XS-r1-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "native-matched16-v1/public-captures/IQ4_XS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "native-matched16-v1/public-captures/IQ4_XS-r1-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "native-matched16-v1/public-captures/IQ4_XS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "native-matched16-v1/public-captures/IQ4_XS-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "native-matched16-v1/public-captures/IQ4_XS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "native-matched16-v1/public-captures/IQ4_XS-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "native-matched16-v1/public-captures/IQ4_XS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "native-matched16-v1/public-captures/IQ4_XS-r2-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "native-matched16-v1/public-captures/IQ4_XS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "native-matched16-v1/public-captures/IQ4_XS-r2-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "native-matched16-v1/public-captures/IQ4_XS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "native-matched16-v1/public-captures/IQ4_XS-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "native-matched16-v1/public-captures/IQ4_XS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "native-matched16-v1/public-captures/IQ4_XS-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "native-matched16-v1/public-captures/MXFP4-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "native-matched16-v1/public-captures/MXFP4-r0-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "native-matched16-v1/public-captures/MXFP4-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "native-matched16-v1/public-captures/MXFP4-r0-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "native-matched16-v1/public-captures/MXFP4-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "native-matched16-v1/public-captures/MXFP4-r0-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "native-matched16-v1/public-captures/MXFP4-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "native-matched16-v1/public-captures/MXFP4-r0-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "native-matched16-v1/public-captures/MXFP4-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "native-matched16-v1/public-captures/MXFP4-r1-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "native-matched16-v1/public-captures/MXFP4-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "native-matched16-v1/public-captures/MXFP4-r1-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "native-matched16-v1/public-captures/MXFP4-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "native-matched16-v1/public-captures/MXFP4-r1-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "native-matched16-v1/public-captures/MXFP4-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "native-matched16-v1/public-captures/MXFP4-r1-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "native-matched16-v1/public-captures/MXFP4-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "native-matched16-v1/public-captures/MXFP4-r2-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "native-matched16-v1/public-captures/MXFP4-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "native-matched16-v1/public-captures/MXFP4-r2-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "native-matched16-v1/public-captures/MXFP4-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "native-matched16-v1/public-captures/MXFP4-r2-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "native-matched16-v1/public-captures/MXFP4-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "native-matched16-v1/public-captures/MXFP4-r2-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "native-matched16-v1/public-captures/Q2_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "native-matched16-v1/public-captures/Q2_K-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "native-matched16-v1/public-captures/Q2_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "native-matched16-v1/public-captures/Q2_K-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "native-matched16-v1/public-captures/Q2_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "native-matched16-v1/public-captures/Q2_K-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "native-matched16-v1/public-captures/Q2_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "native-matched16-v1/public-captures/Q2_K-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "native-matched16-v1/public-captures/Q2_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "native-matched16-v1/public-captures/Q2_K-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "native-matched16-v1/public-captures/Q2_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "native-matched16-v1/public-captures/Q2_K-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "native-matched16-v1/public-captures/Q2_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "native-matched16-v1/public-captures/Q2_K-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "native-matched16-v1/public-captures/Q2_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "native-matched16-v1/public-captures/Q2_K-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "native-matched16-v1/public-captures/Q2_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "native-matched16-v1/public-captures/Q2_K-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "native-matched16-v1/public-captures/Q2_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "native-matched16-v1/public-captures/Q2_K-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "native-matched16-v1/public-captures/Q2_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "native-matched16-v1/public-captures/Q2_K-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "native-matched16-v1/public-captures/Q2_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "native-matched16-v1/public-captures/Q2_K-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "native-matched16-v1/public-captures/Q3_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "native-matched16-v1/public-captures/Q3_K-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "native-matched16-v1/public-captures/Q3_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "native-matched16-v1/public-captures/Q3_K-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "native-matched16-v1/public-captures/Q3_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "native-matched16-v1/public-captures/Q3_K-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "native-matched16-v1/public-captures/Q3_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "native-matched16-v1/public-captures/Q3_K-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "native-matched16-v1/public-captures/Q3_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "native-matched16-v1/public-captures/Q3_K-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "native-matched16-v1/public-captures/Q3_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "native-matched16-v1/public-captures/Q3_K-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "native-matched16-v1/public-captures/Q3_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "native-matched16-v1/public-captures/Q3_K-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "native-matched16-v1/public-captures/Q3_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "native-matched16-v1/public-captures/Q3_K-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "native-matched16-v1/public-captures/Q3_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "native-matched16-v1/public-captures/Q3_K-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "native-matched16-v1/public-captures/Q3_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "native-matched16-v1/public-captures/Q3_K-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "native-matched16-v1/public-captures/Q3_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "native-matched16-v1/public-captures/Q3_K-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "native-matched16-v1/public-captures/Q3_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "native-matched16-v1/public-captures/Q3_K-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "native-matched16-v1/public-captures/Q4_0-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "native-matched16-v1/public-captures/Q4_0-r0-p0-dense-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "native-matched16-v1/public-captures/Q4_0-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "native-matched16-v1/public-captures/Q4_0-r0-p0-quant-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "native-matched16-v1/public-captures/Q4_0-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "native-matched16-v1/public-captures/Q4_0-r0-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "native-matched16-v1/public-captures/Q4_0-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "native-matched16-v1/public-captures/Q4_0-r0-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "native-matched16-v1/public-captures/Q4_0-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "native-matched16-v1/public-captures/Q4_0-r1-p0-dense-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "native-matched16-v1/public-captures/Q4_0-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "native-matched16-v1/public-captures/Q4_0-r1-p0-quant-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "native-matched16-v1/public-captures/Q4_0-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "native-matched16-v1/public-captures/Q4_0-r1-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "native-matched16-v1/public-captures/Q4_0-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "native-matched16-v1/public-captures/Q4_0-r1-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "native-matched16-v1/public-captures/Q4_0-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "native-matched16-v1/public-captures/Q4_0-r2-p0-dense-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "native-matched16-v1/public-captures/Q4_0-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "native-matched16-v1/public-captures/Q4_0-r2-p0-quant-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "native-matched16-v1/public-captures/Q4_0-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "native-matched16-v1/public-captures/Q4_0-r2-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "native-matched16-v1/public-captures/Q4_0-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "native-matched16-v1/public-captures/Q4_0-r2-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "native-matched16-v1/public-captures/Q4_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "native-matched16-v1/public-captures/Q4_K-r0-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "native-matched16-v1/public-captures/Q4_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "native-matched16-v1/public-captures/Q4_K-r0-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "native-matched16-v1/public-captures/Q4_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "native-matched16-v1/public-captures/Q4_K-r0-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "native-matched16-v1/public-captures/Q4_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "native-matched16-v1/public-captures/Q4_K-r0-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "native-matched16-v1/public-captures/Q4_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "native-matched16-v1/public-captures/Q4_K-r1-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "native-matched16-v1/public-captures/Q4_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "native-matched16-v1/public-captures/Q4_K-r1-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "native-matched16-v1/public-captures/Q4_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "native-matched16-v1/public-captures/Q4_K-r1-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "native-matched16-v1/public-captures/Q4_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "native-matched16-v1/public-captures/Q4_K-r1-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "native-matched16-v1/public-captures/Q4_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "native-matched16-v1/public-captures/Q4_K-r2-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "native-matched16-v1/public-captures/Q4_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "native-matched16-v1/public-captures/Q4_K-r2-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "native-matched16-v1/public-captures/Q4_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "native-matched16-v1/public-captures/Q4_K-r2-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "native-matched16-v1/public-captures/Q4_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "native-matched16-v1/public-captures/Q4_K-r2-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "native-matched16-v1/public-captures/Q5_0-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "native-matched16-v1/public-captures/Q5_0-r0-p0-dense-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "native-matched16-v1/public-captures/Q5_0-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "native-matched16-v1/public-captures/Q5_0-r0-p0-quant-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "native-matched16-v1/public-captures/Q5_0-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "native-matched16-v1/public-captures/Q5_0-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "native-matched16-v1/public-captures/Q5_0-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "native-matched16-v1/public-captures/Q5_0-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "native-matched16-v1/public-captures/Q5_0-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "native-matched16-v1/public-captures/Q5_0-r1-p0-dense-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "native-matched16-v1/public-captures/Q5_0-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "native-matched16-v1/public-captures/Q5_0-r1-p0-quant-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "native-matched16-v1/public-captures/Q5_0-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "native-matched16-v1/public-captures/Q5_0-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "native-matched16-v1/public-captures/Q5_0-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "native-matched16-v1/public-captures/Q5_0-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "native-matched16-v1/public-captures/Q5_0-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "native-matched16-v1/public-captures/Q5_0-r2-p0-dense-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "native-matched16-v1/public-captures/Q5_0-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "native-matched16-v1/public-captures/Q5_0-r2-p0-quant-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "native-matched16-v1/public-captures/Q5_0-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "native-matched16-v1/public-captures/Q5_0-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "native-matched16-v1/public-captures/Q5_0-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "native-matched16-v1/public-captures/Q5_0-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "native-matched16-v1/public-captures/Q5_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "native-matched16-v1/public-captures/Q5_K-r0-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "native-matched16-v1/public-captures/Q5_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "native-matched16-v1/public-captures/Q5_K-r0-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "native-matched16-v1/public-captures/Q5_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "native-matched16-v1/public-captures/Q5_K-r0-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "native-matched16-v1/public-captures/Q5_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "native-matched16-v1/public-captures/Q5_K-r0-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "native-matched16-v1/public-captures/Q5_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "native-matched16-v1/public-captures/Q5_K-r1-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "native-matched16-v1/public-captures/Q5_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "native-matched16-v1/public-captures/Q5_K-r1-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "native-matched16-v1/public-captures/Q5_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "native-matched16-v1/public-captures/Q5_K-r1-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "native-matched16-v1/public-captures/Q5_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "native-matched16-v1/public-captures/Q5_K-r1-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "native-matched16-v1/public-captures/Q5_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "native-matched16-v1/public-captures/Q5_K-r2-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "native-matched16-v1/public-captures/Q5_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "native-matched16-v1/public-captures/Q5_K-r2-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "native-matched16-v1/public-captures/Q5_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "native-matched16-v1/public-captures/Q5_K-r2-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "native-matched16-v1/public-captures/Q5_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "native-matched16-v1/public-captures/Q5_K-r2-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "native-matched16-v1/public-captures/Q6_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "native-matched16-v1/public-captures/Q6_K-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "native-matched16-v1/public-captures/Q6_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "native-matched16-v1/public-captures/Q6_K-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "native-matched16-v1/public-captures/Q6_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "native-matched16-v1/public-captures/Q6_K-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "native-matched16-v1/public-captures/Q6_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "native-matched16-v1/public-captures/Q6_K-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "native-matched16-v1/public-captures/Q6_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "native-matched16-v1/public-captures/Q6_K-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "native-matched16-v1/public-captures/Q6_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "native-matched16-v1/public-captures/Q6_K-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "native-matched16-v1/public-captures/Q6_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "native-matched16-v1/public-captures/Q6_K-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "native-matched16-v1/public-captures/Q6_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "native-matched16-v1/public-captures/Q6_K-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "native-matched16-v1/public-captures/Q6_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "native-matched16-v1/public-captures/Q6_K-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "native-matched16-v1/public-captures/Q6_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "native-matched16-v1/public-captures/Q6_K-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "native-matched16-v1/public-captures/Q6_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "native-matched16-v1/public-captures/Q6_K-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "native-matched16-v1/public-captures/Q6_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "native-matched16-v1/public-captures/Q6_K-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "native-matched16-v1/public-captures/Q8_0-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "native-matched16-v1/public-captures/Q8_0-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "native-matched16-v1/public-captures/Q8_0-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "native-matched16-v1/public-captures/Q8_0-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "native-matched16-v1/public-captures/Q8_0-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "native-matched16-v1/public-captures/Q8_0-r0-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "native-matched16-v1/public-captures/Q8_0-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "native-matched16-v1/public-captures/Q8_0-r0-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "native-matched16-v1/public-captures/Q8_0-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "native-matched16-v1/public-captures/Q8_0-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "native-matched16-v1/public-captures/Q8_0-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "native-matched16-v1/public-captures/Q8_0-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "native-matched16-v1/public-captures/Q8_0-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "native-matched16-v1/public-captures/Q8_0-r1-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "native-matched16-v1/public-captures/Q8_0-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "native-matched16-v1/public-captures/Q8_0-r1-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "native-matched16-v1/public-captures/Q8_0-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "native-matched16-v1/public-captures/Q8_0-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "native-matched16-v1/public-captures/Q8_0-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "native-matched16-v1/public-captures/Q8_0-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "native-matched16-v1/public-captures/Q8_0-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "native-matched16-v1/public-captures/Q8_0-r2-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "native-matched16-v1/public-captures/Q8_0-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "native-matched16-v1/public-captures/Q8_0-r2-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "native-matched16-v1/public-captures/Q8_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "native-matched16-v1/public-captures/Q8_K-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "native-matched16-v1/public-captures/Q8_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "native-matched16-v1/public-captures/Q8_K-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "native-matched16-v1/public-captures/Q8_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "native-matched16-v1/public-captures/Q8_K-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "native-matched16-v1/public-captures/Q8_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "native-matched16-v1/public-captures/Q8_K-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "native-matched16-v1/public-captures/Q8_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "native-matched16-v1/public-captures/Q8_K-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "native-matched16-v1/public-captures/Q8_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "native-matched16-v1/public-captures/Q8_K-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "native-matched16-v1/public-captures/Q8_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "native-matched16-v1/public-captures/Q8_K-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "native-matched16-v1/public-captures/Q8_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "native-matched16-v1/public-captures/Q8_K-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "native-matched16-v1/public-captures/Q8_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "native-matched16-v1/public-captures/Q8_K-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "native-matched16-v1/public-captures/Q8_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "native-matched16-v1/public-captures/Q8_K-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "native-matched16-v1/public-captures/Q8_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "native-matched16-v1/public-captures/Q8_K-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "native-matched16-v1/public-captures/Q8_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "native-matched16-v1/public-captures/Q8_K-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "native-matched16-v1/run-operator.py": { + "bytes": 1644, + "sha256": "cd4f895486066c01e0eacf243cdb025bbb4359636faa5e4215142ac7017ed7ce" + }, + "stock-overlay-evidence/retain-hybrid-copy.patch": { + "bytes": 516, + "sha256": "3a11eff56c0d59e1776cc58577f1d30d9b97327a80281ac0bbae601dac128430" + }, + "stock-overlay-evidence/source-manifest.json": { + "bytes": 811616, + "sha256": "bafc755960047310f4495707e4f17942e532e9ba09ef9e999a1f557ff1581dbc" + }, + "fork-overlay-evidence/retain-hybrid-copy.patch": { + "bytes": 516, + "sha256": "9c203bb04048198b61c42f33ef47aea9230f13074899fdd4bff938f40e3fe270" + }, + "fork-overlay-evidence/source-manifest.json": { + "bytes": 784571, + "sha256": "67b88d3a727213f5b276edad0395d40d89fd22eb99099ac1357b025484571d63" + }, + "recurrent-control-v3.json": { + "bytes": 3526, + "sha256": "7d10ee40f86f24c80f982be87393747456b0b01c4963eb3ff215803bb864079c" + }, + "cpu-configure.log": { + "bytes": 1298, + "sha256": "9737e171c8350e444f0908e277a262119465e0911dfc6a7514445b4ea886ed38" + }, + "cpu-build.log": { + "bytes": 55806, + "sha256": "7c2e8720abdba304a58d80f085b8a184b63998a3b9461adf863f48721446fd7e" + }, + "cpu-focused.log": { + "bytes": 204, + "sha256": "faef3d1c44d2f2697ac20b7315b81c5ac0eeb9d294573ef7166e5af03cda3d84" + }, + "cpu-focused-v2.log": { + "bytes": 1257, + "sha256": "c66e5e7b34243d42e10854366bff94a518da495d74b7e96ebeb14de3a700eacf" + }, + "hip-configure.log": { + "bytes": 1998, + "sha256": "f21c543850e12e25e16071640457b99c736e946ec65dff05a41f37ae71c7f332" + }, + "hip-build.log": { + "bytes": 190986, + "sha256": "fd54d172200d6110bcc239419d793ee5b175563450b250e45a197ab70fe46137" + }, + "native-model-build.log": { + "bytes": 3757, + "sha256": "1204287265537f00582c09426cd1a27f613d6559b48ebc7d2e4b4c40462813a7" + }, + "native-model-build-v2.log": { + "bytes": 0, + "sha256": "e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855" + }, + "stock-overlay-configure.log": { + "bytes": 1565, + "sha256": "551e57b9a5abd417e27d63a2ccc79e3fa96770c073bd182cee41c1c7d2453608" + }, + "stock-overlay-build.log": { + "bytes": 17416, + "sha256": "f399b90658f568116b29de7d173c81184b38d48b1e9f98705fb6acc2764a49cb" + }, + "fork-overlay-configure.log": { + "bytes": 1532, + "sha256": "2fd98a9140ec160c4a0bed58ab424b12897aa366565c73775374328cd5ea0023" + }, + "fork-overlay-build.log": { + "bytes": 16904, + "sha256": "b1dcbf569543d3d48bc473187e750de5481e5b9d4b3c5b059ac4aa0693fc65b7" + } + }, + "omitted": "Generated model binaries, build artifacts and compiler caches remain content-addressed inputs in each frozen command. Original fixtures remain unchanged." +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-qualification/fork-retain-hybrid-copy.md b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/fork-retain-hybrid-copy.md new file mode 100644 index 0000000000..703a4d9c4d --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/fork-retain-hybrid-copy.md @@ -0,0 +1,23 @@ +# IQ1 fork retain-hybrid-copy overlay + +Row: `BACKEND-ROCM-QUANT-GATHER`. Issue: [#3093](https://github.com/mudler/vllm.cpp/issues/3093). + +This document records the minimal diff of the IQ1 fork llama.cpp overlay. +The overlay changes one file, `src/llama-graph.cpp`. +It retains `s_copy` as one graph leaf in `build_inp_mem_hybrid`. + +```diff +--- a/src/llama-graph.cpp ++++ b/src/llama-graph.cpp +@@ -3466,0 +3467 @@ ++ ggml_build_forward_expand(gf, inp_rs->s_copy); +``` + +The sha256 of the original `fork-retain-hybrid-copy.patch` file was +`2a909c7fe0ef2baafe74f637e11ba0ccc13973eb114ba90f7c75b51e6e8fd241`. + +The appliable, manifest-sealed form of this overlay is the archive member +`fork-overlay-evidence/retain-hybrid-copy.patch` inside +[`qualification-captures.tar.gz`](../qualification-captures.tar.gz). +That member carries the change with full context, and the fenced diff above is the +minimal rendering of the same change. diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-qualification/implementer-cpu-mutations.json b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/implementer-cpu-mutations.json new file mode 100644 index 0000000000..f03ac0c315 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/implementer-cpu-mutations.json @@ -0,0 +1,21 @@ +{ + "authority": "Parent explicitly authorized helper CPU mutation executions after initial root-only runtime delegation", + "results": [ + { + "name": "remove-copy-consumption", + "exit_code": 1, + "seconds": 0.003428966971114278, + "observed_failure": "RECURRENT FAILED: consumed recurrent copy has no storage", + "log_sha256": "4e8244705d4f9ad1452e20f6da015daf84f88fc0db83795b12712e7d27f01076" + }, + { + "name": "corrupt-copy-indices", + "exit_code": 1, + "seconds": 0.0035978900268673897, + "observed_failure": "RECURRENT FAILED: recurrent state or reused copy allocation changed", + "log_sha256": "2200cb88bf7a47788357bfe6f8a52b767bce595ddcbb2d42be1b9527a618cb1b" + } + ], + "all_frozen_inputs_unchanged": true, + "gpu_work": false +} diff --git a/docs/bench-evidence/rocm-quant-gather/oracle-qualification/stock-retain-hybrid-copy.md b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/stock-retain-hybrid-copy.md new file mode 100644 index 0000000000..ab4bc0c32f --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/oracle-qualification/stock-retain-hybrid-copy.md @@ -0,0 +1,23 @@ +# Stock retain-hybrid-copy overlay + +Row: `BACKEND-ROCM-QUANT-GATHER`. Issue: [#3093](https://github.com/mudler/vllm.cpp/issues/3093). + +This document records the minimal diff of the stock llama.cpp overlay. +The overlay changes one file, `src/llama-graph.cpp`. +It retains `s_copy` as one graph leaf in `build_inp_mem_hybrid`. + +```diff +--- a/src/llama-graph.cpp ++++ b/src/llama-graph.cpp +@@ -3484,0 +3485 @@ ++ ggml_build_forward_expand(gf, inp_rs->s_copy); +``` + +The sha256 of the original `stock-retain-hybrid-copy.patch` file was +`5fc4996ffd57bc105c42be6689b55e4786d72fb8481bdd059b740429f407b8e0`. + +The appliable, manifest-sealed form of this overlay is the archive member +`stock-overlay-evidence/retain-hybrid-copy.patch` inside +[`qualification-captures.tar.gz`](../qualification-captures.tar.gz). +That member carries the change with full context, and the fenced diff above is the +minimal rendering of the same change. diff --git a/docs/bench-evidence/rocm-quant-gather/plugin-artifact-sha256.json b/docs/bench-evidence/rocm-quant-gather/plugin-artifact-sha256.json new file mode 100644 index 0000000000..62a81fd61d --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/plugin-artifact-sha256.json @@ -0,0 +1,140 @@ +{ + "vllm_gguf_plugin/_C_gguf.abi3.so": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138", + "vllm_gguf_plugin/__init__.py": "3e047fe518dc5b3f7ca2e2068cc6921b8b82c84cc87d23e79807a4cc6ccb0a04", + "vllm_gguf_plugin/config_parser.py": "a49b946f6374aa12a14a6517cd2f0140271818e59881a87f0238024b211f7984", + "vllm_gguf_plugin/csrc/cuda_compat.h": "959bc9e0e73e020d777530b18bf12160d35cc4eb976335955395729700b2f975", + "vllm_gguf_plugin/csrc/dispatch_utils.h": "7a17e8ac27b25426fbfd2921d2ebcad6410b703736204e55e832037fd9f250cc", + "vllm_gguf_plugin/csrc/gguf/dequantize.cuh": "96c25644ffba8ec6b9beffb7b1c30435e9b0e0466def1d4721f7f8aab401c7e5", + "vllm_gguf_plugin/csrc/gguf/dequantize_hip.cuh": "497842aca4744b903c7fd19b6069be33fa50da9d4bb893634eda46ab9cb35ddd", + "vllm_gguf_plugin/csrc/gguf/ggml-common.h": "990b6aca44615eaf39a3fa76b65992e1101cf1c4d27ec3a36070074f9f83136b", + "vllm_gguf_plugin/csrc/gguf/ggml-common_hip.h": "b7d415d0cb626afb330cc67867418943b1ebe39eb648988f637b4f29af8e8a41", + "vllm_gguf_plugin/csrc/gguf/gguf_kernel.cu": "4da2c6a6c0f9b0e60661f8334f2d356322f6f4f443b51dcfa570c4da57291e38", + "vllm_gguf_plugin/csrc/gguf/gguf_kernel.hip": "f714989ddbd4c97bdcd924d5ca3316c60efdebcdffa5c9c818a93579023e4d72", + "vllm_gguf_plugin/csrc/gguf/mmq.cuh": "1a3a8fb8cd9d1682a421bba53df92fdb323f7ba00956ab04bced4865cf9ad030", + "vllm_gguf_plugin/csrc/gguf/mmq_hip.cuh": "0e4a23bb4a8f2f9f7769ead653d9d2d72d16901ad9974d3aa3ba9972ee80ca34", + "vllm_gguf_plugin/csrc/gguf/mmvq.cuh": "f46667f8c09a7ca8b90fd4b7c89a5eca54cca99372762c3c2853870874d6e829", + "vllm_gguf_plugin/csrc/gguf/mmvq_hip.cuh": "38231c4faa337935c65562cf7f336761d13dd06bb19309e3e5a8f68a9f1ad448", + "vllm_gguf_plugin/csrc/gguf/moe.cuh": "9e7510e259ed6a4ce9ff18743628b16a9a10f1fcac44553c450bee1684f6bf25", + "vllm_gguf_plugin/csrc/gguf/moe_hip.cuh": "9e3358731eb607a97272f13a02314deb3db6f76026e77622570bfd2a2ae2b7bb", + "vllm_gguf_plugin/csrc/gguf/moe_vec.cuh": "dbe05b1a3515ebf3b2f3d07264f742566eb9f01bef769f46b2f90442a18633da", + "vllm_gguf_plugin/csrc/gguf/moe_vec_hip.cuh": "acbe6bf9625fe73ac96c0f87cdf43def66dc23643155977aedaafd84348192c9", + "vllm_gguf_plugin/csrc/gguf/vecdotq.cuh": "4972eddb514db4a12a996e667905a86fe8ce0f0f42e9839dd2db75774ad5d326", + "vllm_gguf_plugin/csrc/gguf/vecdotq_hip.cuh": "38caef027a395cf9d0285f67beb051e0432d74ef194a8d44b4cb820e927fa013", + "vllm_gguf_plugin/csrc/hip_compat.h": "25d7d6b1dab03d8d85a71976e5955dd6b91ea6ed6442713145770ebcc5d91abe", + "vllm_gguf_plugin/csrc/torch_bindings.cpp": "4fb80d903a1e860a080a04033b6a41eeb840b2bc68220fc509a0db5a5f1cafa7", + "vllm_gguf_plugin/gguf_files.py": "82e003351d59c7abd78477c6df244d1cd06cd8834c0a8db4a5fbf43144d57c3a", + "vllm_gguf_plugin/gguf_utils.py": "d790422af5161db802845cfdb3ff293bd34b9dab50461c0e477c1f1edea5ee8c", + "vllm_gguf_plugin/loader.py": "8c80ecb5fadecf60c603274ceeb3cd84500a8c6f915ea52b86d307e4775f2b2f", + "vllm_gguf_plugin/ops.py": "daac821e871ff73df30b1836c17a7ef9574f8be6e4bedd5f6456c442b69c90db", + "vllm_gguf_plugin/plugin.py": "f414bfc427b5e9142408bb1be23a71d23fb451327ff9182142f1aa5f91f05f1a", + "vllm_gguf_plugin/quantization/__init__.py": "b6755dbcdb6ec7aba9434271dc162d0d7c31e813d6b9e434699f21164ac18d1b", + "vllm_gguf_plugin/quantization/config.py": "18553e34317ffdab94b193439db428aec1af0d000444998c18eafb9397714370", + "vllm_gguf_plugin/quantization/diffusion_config.py": "6970775f14c91286d061fb63e4d8353713108de374782c8d51dfc7785e08b359", + "vllm_gguf_plugin/quantization/fused_moe.py": "bf195f8586bf6f0231e6b4e80c392fab1e39674590f89b482d6c8a279c4f80ab", + "vllm_gguf_plugin/quantization/layout.py": "6ffb4c25b6e862a3953ff8d00fba3c9a59febbafd791cb15ae0c44f0404ef799", + "vllm_gguf_plugin/quantization/linear.py": "b40556bc026632e610adaee2855d42ddc3d00d99ba07fdfee7cae171ec180efe", + "vllm_gguf_plugin/quantization/params.py": "18032548702812fe5180e2eefe7139a49ec990cf30159672021b36d8345e693b", + "vllm_gguf_plugin/quantization/utils.py": "9ffd3ca8f69d5fe0f5121576d4dfa1264c3d93bf277a8044a54e6c4471fbe9e5", + "vllm_gguf_plugin/quantization/vocal_embeds.py": "8312ad417b7d680d200f6f277011230ca7d2ea6f424c9aadab583b70ed032cb9", + "vllm_gguf_plugin/triton/__init__.py": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1", + "vllm_gguf_plugin/triton/dequantize/__init__.py": "249b99a046d4aa0d60a34b8dd2695ea64836d02e5c7cee2b2682b3578a0fd1fe", + "vllm_gguf_plugin/triton/dequantize/interface.py": "fa7f5cd35eae43068f798ba154c787e0719ca0401ace3af72754c3be0e7e4705", + "vllm_gguf_plugin/triton/dequantize/iq_quant/__init__.py": "484c1f66ed156f26dc558dee8111ec6aef969d0af6bed784193ceb148e1d01fd", + "vllm_gguf_plugin/triton/dequantize/iq_quant/iq1_m.py": "735412106ed9d36db56cf7394516ca2f56aae2f94d7ad9ee033f5552e65fc473", + "vllm_gguf_plugin/triton/dequantize/iq_quant/iq1_s.py": "e4619535a6776d8fbbf4ac1c451a7773b866d07f3c40a1231112bcc2c436a4c7", + "vllm_gguf_plugin/triton/dequantize/iq_quant/iq2_s.py": "2102e3f5293c3aefd0a0352e68ddccae9adc5e0d339e3d901629a6a3e3ed7b65", + "vllm_gguf_plugin/triton/dequantize/iq_quant/iq2_xs.py": "30cf8761d25401603849a71ca6f98467d418c4832e15fa485bde037ca96aea4e", + "vllm_gguf_plugin/triton/dequantize/iq_quant/iq2_xxs.py": "2cc8610c83de7087e8bf542dce2d47c2a89d9d88c7fbd9092a4b4753e57409c6", + "vllm_gguf_plugin/triton/dequantize/iq_quant/iq3_s.py": "841ab0215c7c61be31068f634b3c2d26903bbd9639438eaa9d0e4c6871e8a374", + "vllm_gguf_plugin/triton/dequantize/iq_quant/iq3_xxs.py": "414627369715ab31aec9ac5103b3ef18ecb8bfb45670af2e877418821ea62210", + "vllm_gguf_plugin/triton/dequantize/iq_quant/iq4_nl.py": "b8e640287eb0929f83dcdc3bf01d7bf8ac0ac2b63cadc5b7d17f83bb77daa4db", + "vllm_gguf_plugin/triton/dequantize/iq_quant/iq4_xs.py": "91529ddc95d9e7da4824364b4308d0aa8c5d9712ea3d7c4a644f5a396c7ff4f0", + "vllm_gguf_plugin/triton/dequantize/k_quant/__init__.py": "3c42c6eb160150dc781258ddf2103d7b0de03b674297646f42929f707694bb20", + "vllm_gguf_plugin/triton/dequantize/k_quant/q2_k.py": "4dff47db3970b03709b22ed67cb29fddfb99faec5fe26980f0683b3bd7759ef1", + "vllm_gguf_plugin/triton/dequantize/k_quant/q3_k.py": "e97247b9ead180afa31d896bcb72e2df8ef1ea5f326e9988d3d3c36a9caa58a1", + "vllm_gguf_plugin/triton/dequantize/k_quant/q4_k.py": "4a1d1323fab0a463fc33fa9abc32a8d2a43b30be9f3d24de95b4f008c3253811", + "vllm_gguf_plugin/triton/dequantize/k_quant/q5_k.py": "c74345d0972a37777f33c013758e868b2b835f5805b721a88c1ad40fd92e0238", + "vllm_gguf_plugin/triton/dequantize/k_quant/q6_k.py": "1c705c7052286cd179d4793d9fa08d57a20145bb24f7da49511503e5fe0eab04", + "vllm_gguf_plugin/triton/dequantize/standard_quant/__init__.py": "9727990b5e8d5f051f572cd8999c28f22b2be99c2babeb1dee17c081b1853ac9", + "vllm_gguf_plugin/triton/dequantize/standard_quant/q4_0.py": "4aab047fbf9abf6234ef6fd64f2a3aee4023cb91c850579997e9827d5b997643", + "vllm_gguf_plugin/triton/dequantize/standard_quant/q4_1.py": "5d0d85223f69082951a5301b87235e74c1c212e1c34f987871bbdf0a523e93c3", + "vllm_gguf_plugin/triton/dequantize/standard_quant/q5_0.py": "ae4e2c494b227a6d6534631b6ee2bdbc313c46ecd2a7e8af8a582575338c796e", + "vllm_gguf_plugin/triton/dequantize/standard_quant/q5_1.py": "d6d526993a0f26cb0da9030d24456cd6a163d13a66205c2f47f4557b9443daec", + "vllm_gguf_plugin/triton/dequantize/standard_quant/q8_0.py": "274008decd5ddf15fa6741d1422201d6831c59d439df0a7b44fe7939eaafbc8d", + "vllm_gguf_plugin/triton/dequantize/standard_quant/q8_1.py": "d2b56f6ba4b0018240b81bc354d805917a83f0f26838cd45021821e50b7b39fc", + "vllm_gguf_plugin/triton/dequantize/utils.py": "3b46512556ee6ef55d5c7acd32ce289f41640c022252ab0f65ec961e9a854750", + "vllm_gguf_plugin/triton/fused_moe/__init__.py": "b466c73479b97f45e40d1d50d7a534a8f6f05f5965a03e084a363f0773d0d73f", + "vllm_gguf_plugin/triton/fused_moe/interface.py": "5761acc437eb5f3f49b84d235ed45adcf5b1ae2b0a3650c3b7fe4b4e1430852d", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/__init__.py": "4c66e819f5765037507a598f69a0edc019dbc41ddd79fde78bfeb087878cc4e4", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/iq1_m.py": "ac3134f134f5f4f8cd667b407f5e9b126f1a72805d6e54a5b64fc711f76a6e75", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/iq1_s.py": "2245b0ee042fd2ebd9928f71e4b69b3c64a06d685030226b632038ca344a8a3a", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/iq2_s.py": "df9c1abca21fb85df4f28122521e0e62aa31fa7c7bd9eca9b58edec80e4c2f77", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/iq2_xs.py": "b064d651e2cf630b44d5c5231fa2dc9bc44a72dfc4b2497baaf670c87b610791", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/iq2_xxs.py": "72862bfa799739e40185fa7671fce86bbf176a28329a78973aa5e735d025bb63", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/iq3_s.py": "adf4ec93760bfb5b327310ac7c257c8091f522c30c56014c43fa743c42e1c2cb", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/iq3_xxs.py": "2471ca802e675a6d92e091f9970e3745ffcd64307a042a1e91af61e51834f5bf", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/iq4_nl.py": "78b14baaf6d976e72635bea06962eff3757b17d23d6d90c842399d6981dbc56d", + "vllm_gguf_plugin/triton/fused_moe/iq_quant/iq4_xs.py": "16d18b9fc71b70dfce2044d6938488ba83eed9daf7202c144da92cfa88cbbeda", + "vllm_gguf_plugin/triton/fused_moe/k_quant/__init__.py": "ac57f723d1726db7abb41898cb9e8bceaf50209510fcbda9ac50fa118c1fd8a2", + "vllm_gguf_plugin/triton/fused_moe/k_quant/q2_k.py": "96def7c24bb30de179ee1864b065be0e6fd1045f9bdab8ad391f1be28593d575", + "vllm_gguf_plugin/triton/fused_moe/k_quant/q3_k.py": "691e036c396c7125de73cbcd1993023a96cdc136e1ec1599f3d970430058b38d", + "vllm_gguf_plugin/triton/fused_moe/k_quant/q4_k.py": "0216c060cd8cd7a8668863417ec45b5ee9198c384402192cfa2537b078d45349", + "vllm_gguf_plugin/triton/fused_moe/k_quant/q5_k.py": "2298c8dac8434c4910c017ac1db2e631a38b33cd2c66a7c22893ab93ecc9a94f", + "vllm_gguf_plugin/triton/fused_moe/k_quant/q6_k.py": "7a3b30ddafda65e321c2b9091d1b2ac05c2ae9ec1cfb281672817bf2c23c7ee8", + "vllm_gguf_plugin/triton/fused_moe/standard_quant/__init__.py": "2e0626f087aaf64495beafa674ea9bb5c96ab4dca080e27ee1dec0f19ea7ea63", + "vllm_gguf_plugin/triton/fused_moe/standard_quant/q4_0.py": "981ca0dfb18e5afb6264b524a90bef6368648bcac359c83510ca17f7e3516e87", + "vllm_gguf_plugin/triton/fused_moe/standard_quant/q4_1.py": "7cfb86144a6a4d8e36b58ec036d5b2e7a41078f40cc8f856f27ca5d20fc42c7e", + "vllm_gguf_plugin/triton/fused_moe/standard_quant/q5_0.py": "109bc28e34be425884b038c1424a38b1529e1faabe1bcdfffb188eed70ad4313", + "vllm_gguf_plugin/triton/fused_moe/standard_quant/q5_1.py": "583b8d10f93c45530caef0be15b95faf8157746f5095fe066672f73d7260ad2c", + "vllm_gguf_plugin/triton/fused_moe/standard_quant/q8_0.py": "83d7255b105a94216d0f5d5d99116924de251bbc703b0d4a25b25e4aad77d004", + "vllm_gguf_plugin/triton/fused_moe/standard_quant/q8_1.py": "d15cf10adaa916f0ebe9ff3004bef2e12272e3c712436391867276b6ea5f28ad", + "vllm_gguf_plugin/triton/fused_moe/utils.py": "fc31d25eafd9c796a0df0c59e11cdd4e881efda7f9dec22bf7b79f09f2b7b79d", + "vllm_gguf_plugin/triton/gemm/__init__.py": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1", + "vllm_gguf_plugin/triton/gemm/interface.py": "4797dd1ead0968eed05a7a4d700288e9e9500a194719b9457e03287d4021b4b5", + "vllm_gguf_plugin/triton/gemm/iq_quant/__init__.py": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq1_m.py": "a085d73390d934bb9a080d52e6240e00d78c9157853a852166fded297bb32610", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq1_s.py": "1fa2ac641d986803fc8f5dd8e3e81334580ddf287b106cb01bcb6fb10989936e", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq2_s.py": "3c0ffa66e08bf1e38f950be2e53f39303c8bb0e11087caf6ae639eea83e52838", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq2_xs.py": "a9259737326c7e4ed6559285355c822f972921d454e0cce3dd91a2d718dd218a", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq2_xxs.py": "f7b240c3d764b82805d68b6108d618dfa49594cddba4d0af11873e5fbeb4c061", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq3_s.py": "85b029e372f7c025320965cc98283f7556982bd0195b429173c094d4eb261d68", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq3_xxs.py": "cce28674e735a9ccec6a503604790e915e723783551c90fd80dcebce5535d4c0", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq4_nl.py": "8da067cc8a21a58bab73a7c5af4a7eaf293d7287033069e303d99511b0153fbf", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq4_xs.py": "b420a17e6ea301a3c0ef7e732c71b720cda92481aadc08d229505495a7566afe", + "vllm_gguf_plugin/triton/gemm/iq_quant/iq_tables.py": "237e8f35e00611aa5f1ae3c176a440fec6cd672aeb54b8ced4fc2aaad019e693", + "vllm_gguf_plugin/triton/gemm/k_quant/__init__.py": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1", + "vllm_gguf_plugin/triton/gemm/k_quant/q2_k.py": "0e3c07ca796f20ef2af8bb0c7f9675c9a56256a180731e92df9802fa157ea95b", + "vllm_gguf_plugin/triton/gemm/k_quant/q3_k.py": "9b08b1190036c3f6f9f76c5df47d651139de1cc275b4e0506924d58e45303fe9", + "vllm_gguf_plugin/triton/gemm/k_quant/q4_k.py": "8a3f708e6efb9351125ac4388c9b1cccc545a4e0b10929ec6d19e1178c99e96d", + "vllm_gguf_plugin/triton/gemm/k_quant/q5_k.py": "eb981367e11f71068ba16e6b42a39c49f272b01291b51dd1a2867f887b64a879", + "vllm_gguf_plugin/triton/gemm/k_quant/q6_k.py": "1f5eb00e017b24b8651cfe814856747a1b86e6b0e2b140d318c66fd48fd0a80c", + "vllm_gguf_plugin/triton/gemm/standard_quant/__init__.py": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1", + "vllm_gguf_plugin/triton/gemm/standard_quant/q4_0.py": "e3a80371bfbfd13946ff4ac0f5a0dcb10c780de40d76a3309246632c70c6c8c1", + "vllm_gguf_plugin/triton/gemm/standard_quant/q4_1.py": "7dc5dbb13ba58810129faf71f5dd4dd24ee2a0cd2bf633dc9d1dddf27fa355ba", + "vllm_gguf_plugin/triton/gemm/standard_quant/q5_0.py": "7966b6d565b7285e0268aa0c2fdef6cfb464234ca32d34481e80672cf906c26e", + "vllm_gguf_plugin/triton/gemm/standard_quant/q5_1.py": "c625c7bd20992b45c5c3f42d4be1f83b7d3d6843299aeaaa74417f858bc13ece", + "vllm_gguf_plugin/triton/gemm/standard_quant/q8_0.py": "75de808a1017fd6e4787086b9ffa42acfbc685e5ce1c8255a9d3e890ae61fa4e", + "vllm_gguf_plugin/triton/gemm/standard_quant/q8_1.py": "84dcf565d25c1190b6d8984810118a66049ac57d54aaf04920f0e1025b946885", + "vllm_gguf_plugin/triton/gemm/utils.py": "6ee2b74c572dafc1511d006e5a08cebae06138f6a2b51aa59939edd88ebf051f", + "vllm_gguf_plugin/weight_utils.py": "03cdf368cc6fc58b4751f7471dd1d89a27647b4266e8a5bda30ed857de61df9d", + "vllm_gguf_plugin/weights_adapter/__init__.py": "081d701f52e2388ae38efce6d24030298881c8b4065ac5df40f19fb216980ead", + "vllm_gguf_plugin/weights_adapter/base.py": "899b76daac1cd263e606b96474df53353da55101e3bb61edc12e66aacaf6923f", + "vllm_gguf_plugin/weights_adapter/diffusion/__init__.py": "1af8199a0e542315efc0b79fe741c9d32c103191c27a7562c0ca7d53909b2aed", + "vllm_gguf_plugin/weights_adapter/diffusion/base.py": "c2678ceae3e2296bd0b9f4fefc77374b464e687236ab65b0cc8055a981eeeda5", + "vllm_gguf_plugin/weights_adapter/diffusion/flux2_klein.py": "d5007751145b7a85c075486a704134ae1f11c569d2be78374f945737f2acb8a8", + "vllm_gguf_plugin/weights_adapter/diffusion/integration.py": "a1ee6919b529831fd44e608b71a674d6c13052970713c200cb4a8e3e9636d368", + "vllm_gguf_plugin/weights_adapter/diffusion/loader.py": "a7044349ea18645c6d3f4e36c9ea30e07b7e9c47aa3d38bc5b7fd716846d3a26", + "vllm_gguf_plugin/weights_adapter/diffusion/qwen_image.py": "6be01743e2c1a13008ca4e89b0a88784fd7eebc00c69583455099882cfccca8e", + "vllm_gguf_plugin/weights_adapter/diffusion/z_image.py": "5abbd262ab6b6f97b745e1534dc9f44ea4d0e75e2444d0b193f28ed8a89ab487", + "vllm_gguf_plugin/weights_adapter/gemma3.py": "2b5011dccbb976dd83d85ff9bdd358e5f17039c8543cfb592a47157f6ac74465", + "vllm_gguf_plugin/weights_adapter/gemma4.py": "dc89fff10a58bc9539e5ee201c5c231839965b8a04cb5e6f155a91f5875874da", + "vllm_gguf_plugin/weights_adapter/olmoe.py": "47a77b4ef1ed1fadcc5507f00a5eb1cc861b0a3111e7ead817d26a737ac7dc7c", + "vllm_gguf_plugin/weights_adapter/qwen3_5.py": "ed19007102c518a50e89a92fa2084c4034980f59aabd0881ce13df78df1fa2d5", + "vllm_gguf_plugin/weights_adapter/transformers.py": "f252fabd0e31f7e408b347fc686584c568dec840e7d17c9b7f729a7d0425342b", + "vllm_gguf_plugin.egg-info/PKG-INFO": "ac6e584fcd033f0578f2ded9f368ec1ca90d63c9268d3521901ae706a8537079", + "vllm_gguf_plugin.egg-info/SOURCES.txt": "1a897e9037ed45581c2d126d788e363b7e0041d4ec03edb1477c683fbf2f2605", + "vllm_gguf_plugin.egg-info/dependency_links.txt": "01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b", + "vllm_gguf_plugin.egg-info/entry_points.txt": "6087369aa91ab6f5c0bdc80a76daed8a92175bb0a1db7a03aa89a24cd89ff241", + "vllm_gguf_plugin.egg-info/requires.txt": "89642c00a45edc69b5157d59b942c030999737af3b590acaf2735bf4264a74d5", + "vllm_gguf_plugin.egg-info/top_level.txt": "6b3fa0d9d555c7095f09dfd0c51e4ab37aaf29e23b0fe783abd7e4dbd0357be3" +} diff --git a/docs/bench-evidence/rocm-quant-gather/plugin-build-command.json b/docs/bench-evidence/rocm-quant-gather/plugin-build-command.json new file mode 100644 index 0000000000..24c5945276 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/plugin-build-command.json @@ -0,0 +1,54 @@ +[ + "docker", + "run", + "--rm", + "--pull=never", + "--name", + "rdna3-gather-plugin-build", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773/runtime:/home/vikash/oracle/gfx1100-active-2773/runtime:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-w", + "/home/vikash/.cache/rdna3-gather-impl/plugin-source", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HIP_VISIBLE_DEVICES=", + "-e", + "ROCR_VISIBLE_DEVICES=", + "-e", + "PYTORCH_ROCM_ARCH=gfx1100", + "-e", + "MAX_JOBS=4", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "VLLM_GGUF_PLUGIN_LOCAL_VERSION_SUFFIX=d4c1f0d.gfx1100", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "setup.py", + "build", + "--build-base", + "/home/vikash/.cache/rdna3-gather-impl/plugin-build", + "--build-lib", + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact" +] diff --git a/docs/bench-evidence/rocm-quant-gather/preflight-final-staged-result.json b/docs/bench-evidence/rocm-quant-gather/preflight-final-staged-result.json new file mode 100644 index 0000000000..1ee3e689fb --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/preflight-final-staged-result.json @@ -0,0 +1,48 @@ +{ + "worktree": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "base_commit": "b46852d13f7c8776557dab170e02ebea50cdfdec", + "staged_tree": "57d80b867e348f6a61ecb3b369fae4430e175057", + "scope": "The final full preflight reads this staged source and record state. Subsequent result-only record edits require focused record checks; frozen product bytes remain identical.", + "argv": [ + "env", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "GIT_CEILING_DIRECTORIES=/home/vikash/.cache/rdna3-gather-impl/tmp", + "GIT_CONFIG_GLOBAL=/dev/null", + "PYTHONPATH=/home/vikash/.cache/hrx-evaluation-3080-preflight/python", + "bash", + "scripts/agent-preflight.sh", + "--staged", + "--quiet" + ], + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/preflight-final-staged.log", + "source_manifest": "/home/vikash/.cache/rdna3-gather-impl/evidence/fresh-review-source-manifest.json", + "source_manifest_sha256": "780329dbc81e8ba1eeef575e1f321e30c232e4109bf0718edeb94c9d6b25a2fa", + "exit_status": 0, + "log_sha256": "23e50cc8260928a06e42faa0d7f63ef0b27f5d9aa04883a50028ddf413441833", + "reported_ok_calls": 158, + "skipped_argument_dependent_gates": [ + "check-arm-isa-build.py", + "check-cpu-isa-build.py", + "check-cuda-fat-gencode.py", + "check-pr-size.py", + "check-triton-aot-multiarch.py" + ], + "all_green": false, + "explicit_cpu_isa": { + "argv": [ + "python3", + "scripts/check-cpu-isa-build.py", + "--compile-commands", + "build-cpu-rocm-quant-gather/compile_commands.json" + ], + "exit_status": 0, + "log_sha256": "4f54e0b5f7fb96d55768af81286498f80a23d0d70cae9b32e62cc2a8e6bcaf16" + }, + "scope_of_unsupplied_inputs": { + "check-arm-isa-build.py": "No ARM compilation in this x86_64 HIP build", + "check-cuda-fat-gencode.py": "VLLM_CPP_CUDA=OFF for this HIP build", + "check-triton-aot-multiarch.py": "CUDA AOT artifacts are disabled in this HIP build", + "check-pr-size.py": "Exact commit classification runs after committing; staged paths were independently classified" + }, + "combined_cuda_hip_execution": "PENDING: no combined build was available for this qualification" +} diff --git a/docs/bench-evidence/rocm-quant-gather/preflight-python-environment.json b/docs/bench-evidence/rocm-quant-gather/preflight-python-environment.json new file mode 100644 index 0000000000..418552043c --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/preflight-python-environment.json @@ -0,0 +1,9 @@ +{ + "python": "3.12.3 (main, Jul 15 2026, 23:46:41) [GCC 13.3.0]", + "platform": "Linux-7.0.0-28-generic-x86_64-with-glibc2.39", + "numpy": "2.4.6", + "numpy_path": "/home/vikash/.cache/hrx-evaluation-3080-preflight/python/numpy/__init__.py", + "msgspec": "0.21.1", + "msgspec_path": "/home/vikash/.cache/hrx-evaluation-3080-preflight/python/msgspec/__init__.py", + "purpose": "Existing task-local Python dependencies used by record mutation suites; no package installation or runtime modification in this row." +} diff --git a/docs/bench-evidence/rocm-quant-gather/primary-model-config.json b/docs/bench-evidence/rocm-quant-gather/primary-model-config.json new file mode 100644 index 0000000000..006d7637c7 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/primary-model-config.json @@ -0,0 +1,44 @@ +{ + "model_type": "qwen3_5_text", + "architectures": [ + "Qwen3_5ForCausalLM" + ], + "dtype": "bfloat16", + "hidden_size": 256, + "num_hidden_layers": 1, + "num_attention_heads": 4, + "num_key_value_heads": 1, + "head_dim": 64, + "intermediate_size": 256, + "vocab_size": 128, + "max_position_embeddings": 64, + "full_attention_interval": 1, + "layer_types": [ + "full_attention" + ], + "rms_norm_eps": 1e-06, + "attention_bias": false, + "attention_dropout": 0.0, + "attn_output_gate": true, + "hidden_act": "silu", + "tie_word_embeddings": false, + "bos_token_id": 1, + "eos_token_id": 2, + "unk_token_id": 0, + "linear_conv_kernel_dim": 4, + "linear_key_head_dim": 64, + "linear_value_head_dim": 64, + "linear_num_key_heads": 1, + "linear_num_value_heads": 1, + "rope_parameters": { + "rope_type": "default", + "rope_theta": 1000000.0, + "partial_rotary_factor": 1.0, + "mrope_section": [ + 16, + 8, + 8 + ], + "mrope_interleaved": true + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/primary-model-v1-command.json b/docs/bench-evidence/rocm-quant-gather/primary-model-v1-command.json new file mode 100644 index 0000000000..3a285b5d9d --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/primary-model-v1-command.json @@ -0,0 +1,101 @@ +{ + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v1", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "env": {}, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1", + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary-model-p0-r0.log", + "files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary_model.py": { + "bytes": 3998, + "sha256": "a8bd0a4c815fb0df6934f69b15bf54179c5204630e617a3cd000bda5100aed2b" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + } + }, + "fixture": { + "origin": "Frozen generated fixture from public native gate", + "gguf_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af", + "table": "Q4_0", + "geometry": "one full-attention layer, H256,4Q/1KV heads,D64,I256,V128" + }, + "config_adaptation": "Explicit text-only HF config mirrors GGUF metadata; plugin resolves architecture using its unchanged pinned adapter. No vision weights/config invented.", + "authority": "Root only under GPU0 mutex; no oracle patch, pin change or eager override", + "acceptance": "This is first requestedp0r0. A load/refusal keeps model gatePENDING. On success compare exacttokens and schedule remaining5 fresh-engine requests." +} diff --git a/docs/bench-evidence/rocm-quant-gather/primary-model-v1-operator-receipt.json b/docs/bench-evidence/rocm-quant-gather/primary-model-v1-operator-receipt.json new file mode 100644 index 0000000000..0be2f9bc3e --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/primary-model-v1-operator-receipt.json @@ -0,0 +1,72 @@ +{ + "manifest_sha256": "adbd68d083f11d765fe6adca161acfcb7ebe968ca1bdfefd210ed4ef5abc5749", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v1", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "exit_code": 1, + "seconds": 24.120296001434326, + "log_sha256": "01b26816b2eb24fcd88eeab00c15ccd8643361dabfef8b6529aea42db6bd548b", + "input_count": 340, + "all_inputs_verified_before_after": true +} diff --git a/docs/bench-evidence/rocm-quant-gather/primary-model-v1-report-p0-r0.json b/docs/bench-evidence/rocm-quant-gather/primary-model-v1-report-p0-r0.json new file mode 100644 index 0000000000..476622289a --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/primary-model-v1-report-p0-r0.json @@ -0,0 +1,49 @@ +{ + "model": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "config": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "primary_pin": "e126687a9a828d513c01a07cd69f025f27d63280", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "prompt": [ + 1, + 0, + 63, + 127, + 63 + ], + "repeat": 0, + "tokens": [], + "requested": { + "model_dtype": "auto from bfloat16 config", + "block_size": 16, + "num_gpu_blocks_override": 4, + "max_model_len": 64, + "max_num_seqs": 1, + "kv_cache_dtype": "auto", + "seed": 1380926285, + "greedy": true, + "ignore_eos": true, + "max_tokens": 4, + "stop": [], + "stop_token_ids": [] + }, + "harness_adaptations": [ + "pre-tokenized IDs with skip_tokenizer_init=True", + "explicit local HF text config mirrors GGUF geometry", + "no oracle patch or eager-mode override" + ], + "status": "PENDING: primary model refused before qualification", + "torch": "2.12.0+git6bbd260", + "torch_git": "6bbd26020da1c6dc198625dfcdd968b1e4e6b1c5", + "hip": "7.2.53211", + "vllm": "0.28.1rc1.dev132+ge126687a9", + "plugin": "0.0.5+d4c1f0d.gfx1100", + "exception_type": "ValidationError", + "exception": "1 validation error for ModelConfig\n Value error, Model architectures ['Qwen3_5ForConditionalGeneration'] failed to be inspected. Please check the logs for more details. [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]\n For further information visit https://errors.pydantic.dev/2.13/v/value_error", + "traceback": "Traceback (most recent call last):\n File \"/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary_model.py\", line 52, in main\n model = LLM(model=str(args.model), hf_config_path=str(args.config),\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/entrypoints/llm.py\", line 343, in __init__\n self.llm_engine = LLMEngine.from_engine_args(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py\", line 171, in from_engine_args\n vllm_config = engine_args.create_engine_config(usage_context)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/engine/arg_utils.py\", line 2001, in create_engine_config\n model_config = self.create_model_config()\n ^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/usr/local/lib/python3.12/dist-packages/quark/online_quantization/vllm/plugin.py\", line 193, in patched_create_model_config\n return orig_create_model_config(self, *args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/plugin.py\", line 76, in create_model_config\n return original_create_model_config(self, *args, **kwargs)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/engine/arg_utils.py\", line 1757, in create_model_config\n return ModelConfig(\n ^^^^^^^^^^^^\n File \"/usr/local/lib/python3.12/dist-packages/pydantic/_internal/_dataclasses.py\", line 121, in __init__\n s.__pydantic_validator__.validate_python(ArgsKwargs(args, kwargs), self_instance=s)\npydantic_core._pydantic_core.ValidationError: 1 validation error for ModelConfig\n Value error, Model architectures ['Qwen3_5ForConditionalGeneration'] failed to be inspected. Please check the logs for more details. [type=value_error, input_value=ArgsKwargs((), {'model': ...nderer_num_workers': 1}), input_type=ArgsKwargs]\n For further information visit https://errors.pydantic.dev/2.13/v/value_error\n" +} diff --git a/docs/bench-evidence/rocm-quant-gather/primary-model-v2-command.json b/docs/bench-evidence/rocm-quant-gather/primary-model-v2-command.json new file mode 100644 index 0000000000..44e9609a93 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/primary-model-v2-command.json @@ -0,0 +1,116 @@ +{ + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v2", + "-e", + "TRITON_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/triton", + "-e", + "VLLM_CACHE_ROOT=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/vllm", + "-e", + "TORCHINDUCTOR_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/inductor", + "-e", + "HF_HOME=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/hf", + "-e", + "NUMBA_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/numba", + "-e", + "MAX_JOBS=4", + "-e", + "TORCHINDUCTOR_COMPILE_THREADS=4", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "env": {}, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1", + "log": "/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/primary-model-p0-r0.log", + "files": { + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/config.json": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary_model.py": { + "bytes": 3998, + "sha256": "a8bd0a4c815fb0df6934f69b15bf54179c5204630e617a3cd000bda5100aed2b" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + } + }, + "fixture": { + "origin": "Frozen generated fixture from public native gate", + "gguf_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af", + "table": "Q4_0", + "geometry": "one full-attention layer, H256,4Q/1KV heads,D64,I256,V128" + }, + "config_adaptation": "Explicit text-only HF config mirrors GGUF metadata; plugin resolves architecture using its unchanged pinned adapter. No vision weights/config invented.", + "authority": "Root only under GPU0 mutex; no oracle patch, pin change or eager override", + "acceptance": "This is first requestedp0r0. A load/refusal keeps model gatePENDING. On success compare exacttokens and schedule remaining5 fresh-engine requests.", + "version_reason": "v1 failed during Triton cache initialization at unwritable /.triton; v2 sets explicit task cache directories and build parallelism without changing HOME, model, configuration, inputs, sampling or runtime" +} diff --git a/docs/bench-evidence/rocm-quant-gather/primary-model-v2-operator-receipt.json b/docs/bench-evidence/rocm-quant-gather/primary-model-v2-operator-receipt.json new file mode 100644 index 0000000000..591e63f6af --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/primary-model-v2-operator-receipt.json @@ -0,0 +1,86 @@ +{ + "manifest_sha256": "d5a75fbbaa7bb03ca345b3ab66fd7d422ab6a497429851f82918c1d48fbb85e3", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "HF_HUB_OFFLINE=1", + "-e", + "TRANSFORMERS_OFFLINE=1", + "-e", + "VLLM_NO_USAGE_STATS=1", + "--name", + "rdna3-gather-primary-model-v2", + "-e", + "TRITON_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/triton", + "-e", + "VLLM_CACHE_ROOT=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/vllm", + "-e", + "TORCHINDUCTOR_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/inductor", + "-e", + "HF_HOME=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/hf", + "-e", + "NUMBA_CACHE_DIR=/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/cache/numba", + "-e", + "MAX_JOBS=4", + "-e", + "TORCHINDUCTOR_COMPILE_THREADS=4", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary_model.py", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model/Q4_0.gguf", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/model", + "/home/vikash/.cache/rdna3-gather-impl/primary-model-v2/report-p0-r0.json", + "--prompt-index", + "0", + "--repeat", + "0" + ], + "exit_code": 1, + "seconds": 25.378447771072388, + "log_sha256": "2e72479519ed22c90b7895aa3fe57b2cf6bbd5ea52108762390d21464b018945", + "input_count": 340, + "all_inputs_verified_before_after": true +} diff --git a/docs/bench-evidence/rocm-quant-gather/primary-model-v2-report-p0-r0.json b/docs/bench-evidence/rocm-quant-gather/primary-model-v2-report-p0-r0.json new file mode 100644 index 0000000000..09d2084459 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/primary-model-v2-report-p0-r0.json @@ -0,0 +1,49 @@ +{ + "model": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "config": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "primary_pin": "e126687a9a828d513c01a07cd69f025f27d63280", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "prompt": [ + 1, + 0, + 63, + 127, + 63 + ], + "repeat": 0, + "tokens": [], + "requested": { + "model_dtype": "auto from bfloat16 config", + "block_size": 16, + "num_gpu_blocks_override": 4, + "max_model_len": 64, + "max_num_seqs": 1, + "kv_cache_dtype": "auto", + "seed": 1380926285, + "greedy": true, + "ignore_eos": true, + "max_tokens": 4, + "stop": [], + "stop_token_ids": [] + }, + "harness_adaptations": [ + "pre-tokenized IDs with skip_tokenizer_init=True", + "explicit local HF text config mirrors GGUF geometry", + "no oracle patch or eager-mode override" + ], + "status": "PENDING: primary model refused before qualification", + "torch": "2.12.0+git6bbd260", + "torch_git": "6bbd26020da1c6dc198625dfcdd968b1e4e6b1c5", + "hip": "7.2.53211", + "vllm": "0.28.1rc1.dev132+ge126687a9", + "plugin": "0.0.5+d4c1f0d.gfx1100", + "exception_type": "TypeError", + "exception": "Invalid type of HuggingFace config. Expected type: , but found type: ", + "traceback": "Traceback (most recent call last):\n File \"/home/vikash/.cache/rdna3-gather-impl/primary-model-v1/primary_model.py\", line 52, in main\n model = LLM(model=str(args.model), hf_config_path=str(args.config),\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/entrypoints/llm.py\", line 343, in __init__\n self.llm_engine = LLMEngine.from_engine_args(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py\", line 179, in from_engine_args\n return cls(\n ^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/v1/engine/llm_engine.py\", line 91, in __init__\n self.renderer = renderer = renderer_from_config(self.vllm_config)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/renderers/registry.py\", line 88, in renderer_from_config\n return RENDERER_REGISTRY.load_renderer(renderer_mode, config, tokenizer)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/renderers/registry.py\", line 70, in load_renderer\n return renderer_cls(config, tokenizer)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/renderers/hf.py\", line 916, in __init__\n super().__init__(config, tokenizer)\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/renderers/base.py\", line 127, in __init__\n self.mm_processor = mm_registry.create_processor(\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/multimodal/registry.py\", line 237, in create_processor\n return factories.build_processor(ctx, cache=cache)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/multimodal/registry.py\", line 95, in build_processor\n return self.processor(info, dummy_inputs_builder, cache=cache)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/multimodal/processing/processor.py\", line 1049, in __init__\n self.data_parser = self.info.get_data_parser()\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/model_executor/models/qwen3_vl.py\", line 929, in get_data_parser\n self.get_hf_config().vision_config.spatial_merge_size,\n ^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/model_executor/models/qwen3_5.py\", line 108, in get_hf_config\n return self.ctx.get_hf_config(Qwen3_5Config)\n ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\n File \"/home/vikash/oracle/gfx1100-active-2773/runtime/lib/python3.12/site-packages/vllm/multimodal/processing/context.py\", line 140, in get_hf_config\n raise TypeError(\nTypeError: Invalid type of HuggingFace config. Expected type: , but found type: \n" +} diff --git a/docs/bench-evidence/rocm-quant-gather/primary-operation-captures.tar.gz b/docs/bench-evidence/rocm-quant-gather/primary-operation-captures.tar.gz new file mode 100644 index 0000000000..0679c57e20 Binary files /dev/null and b/docs/bench-evidence/rocm-quant-gather/primary-operation-captures.tar.gz differ diff --git a/docs/bench-evidence/rocm-quant-gather/primary-synthetic-v1.json b/docs/bench-evidence/rocm-quant-gather/primary-synthetic-v1.json new file mode 100644 index 0000000000..b0ffb3c6ea --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/primary-synthetic-v1.json @@ -0,0 +1,378 @@ +{ + "version": 1, + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "docker", + "run", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/vllm.cpp-rdna3-gather-impl/tools/rocm_quant_gather/primary.py", + "capture", + "/home/vikash/.cache/rdna3-gather-impl/corpus/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1" + ], + "authority": "Operator executes under global GPU mutex; helper ran CPU-only imports with no GPU devices exposed", + "expected": "Capture 16 primary codecs, widths256/1024, f32/bf16:64 outputs. Preserve upstream embedding IDs and atol0.01/rtol0.04. Q8_K/MXFP4/IQ1_XXXS remain secondary.", + "gate_limit": "Synthetic operation evidence only. Primary bounded-model refusal and missing original32fixtures remain independently PENDING.", + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/primary-synthetic-v1.log", + "files": { + "/home/vikash/vllm.cpp-rdna3-gather-impl/tools/rocm_quant_gather/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/manifest.json": { + "bytes": 6329, + "sha256": "78bc808e19b40ea745f5e767b72618dbd8087bb685333345826257dd4371ccfd" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/plugin-artifact-sha256.json": { + "bytes": 16561, + "sha256": "b4def029f9693f89654fa2c60433c9a82dd9924909bced7b772ec609e648f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_0-1024.packed": { + "bytes": 73728, + "sha256": "4e50d50e97e08c448cfff4fbed121ef24d657cfd30c2e09f66e70c647ce545b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XS-256.packed": { + "bytes": 9472, + "sha256": "3281c7397dde054ea6c9c8a661cab43dbd0c982eb23b9a31523a616565c827e7" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_S-1024.packed": { + "bytes": 56320, + "sha256": "a94d30560f904618cead2f0618aea6455d5183eb3ed443ca5c12e48fe5b31b34" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_XXS-256.packed": { + "bytes": 12544, + "sha256": "d0f775f8e88d87ae542de6d16122a7a1cea09ebba91203f6513b74727082b6db" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_0-1024.packed": { + "bytes": 139264, + "sha256": "c4435564bf5fd2dc23e0c78107f28ebd033ec741c417d41fa9dcbd89dd2909ba" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_XXS-1024.packed": { + "bytes": 50176, + "sha256": "2d3a0d835fe7a47d802b59e8e412e4084de78faa2aaca59d1ce565fad11ac34a" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XS-1024.packed": { + "bytes": 37888, + "sha256": "8fbc716e220d386b0c27f1cca003384091cd0e522c1442656677af4ac780de5f" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q3_K-1024.packed": { + "bytes": 56320, + "sha256": "6118b4b2f10678fb09a2cb3324d69443ff3b33805f403b79ed3432d5bc1b2732" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_0-256.packed": { + "bytes": 18432, + "sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XXS-1024.packed": { + "bytes": 33792, + "sha256": "2fbe6e6ffcbc32fda24c5d63d4a4b678283e2bd571f491210097a27b88cc1fd4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q6_K-1024.packed": { + "bytes": 107520, + "sha256": "2487007668582b61b3c59fdbaf0e92c2f5c74a61781daba95c13704983643ab1" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-256.packed": { + "bytes": 4864, + "sha256": "af0b5e336987746fd49cd33c790b99eb1479c26f64b6679fa6c685e5934ba0c4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q3_K-256.packed": { + "bytes": 14080, + "sha256": "e4c3a11c1bbff806ae2729033910b7db606452d1b9aa499aa2ced46740ad85d4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_S-256.packed": { + "bytes": 14080, + "sha256": "d4eebe1369e21b0efd6ec5c89f54c785849c02edde957ebd302dad376f63c14c" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-256.packed": { + "bytes": 17408, + "sha256": "baa828a753cc6f496f356271dad644976cd57d1218517e3f641fe56b7ec8db4a" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_K-1024.packed": { + "bytes": 73728, + "sha256": "d15d1bb953010f1ecdac424a54c365308e3b058a1b818bafb89cfc9b9b1678cc" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-256.packed": { + "bytes": 37376, + "sha256": "bcf6f6691a787fe3f2a9bcc3d7923eeb4953862c30f904a6b10fc07410f8d657" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_K-256.packed": { + "bytes": 18432, + "sha256": "b0c107c4b353e2a169247e0406c1eb015d7a6d97edd5906f710868b8c2cecce1" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_XS-256.packed": { + "bytes": 17408, + "sha256": "35b2a41e738a6cb0a86deb4fd366f71ae2dc2a6fad7169f5a9afb53f523e919c" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q2_K-1024.packed": { + "bytes": 43008, + "sha256": "58cb93e09423c7151fac47084de7e38bff98f51e5f304de243b41a17c47ac4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_XS-1024.packed": { + "bytes": 69632, + "sha256": "189f75d03197100afafd3fe92183e47a2d73665d64f89b36d8b10daa05adf2c4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_S-1024.packed": { + "bytes": 25600, + "sha256": "6bf00a4b02da43e5505823b12d9e368aa1ded7f87427eea02ec03ebc68288c33" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_K-1024.packed": { + "bytes": 90112, + "sha256": "6dd54fbc62a9818334e50347fe185247156ecc833a5655ce1920651a4dc537b5" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-1024.packed": { + "bytes": 149504, + "sha256": "78e39e7ef7317d842a6fd965b356ceb1959a5ce03da824fbc699689ea2ef4877" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_0-256.packed": { + "bytes": 34816, + "sha256": "8be599e194405f9b5429e578a03a5717e4d78f53ab4c9c11a52659495ce805ee" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-1024.packed": { + "bytes": 19456, + "sha256": "ae8e061519977e2ba7a77ae9d30e21193b1a009b5ed38739974e945de09b94c6" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_S-256.packed": { + "bytes": 10496, + "sha256": "6eb1966662d9e424fefc5c3625282e7f33fdfab3af9cb4c0cbac6bdc593711ea" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_NL-1024.packed": { + "bytes": 73728, + "sha256": "4e50d50e97e08c448cfff4fbed121ef24d657cfd30c2e09f66e70c647ce545b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_0-256.packed": { + "bytes": 22528, + "sha256": "b3531b1abd263bc4c6a3174b22fa1fd54a4b89b66e737f2eb3ba36be771d7528" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_K-256.packed": { + "bytes": 22528, + "sha256": "5e63d961f36486c9636dd2657e9069bd83ece65e466c56c657265fbd97353b07" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XXS-256.packed": { + "bytes": 8448, + "sha256": "4a3c10afb47315763ce507edcb5814e06d1be93a21b178cf73183302a3f1ceb9" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_S-1024.packed": { + "bytes": 41984, + "sha256": "2cc5c79aef24da7bb363c72b7ac91f05c38b87fdc96627253203926f4465d73e" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q6_K-256.packed": { + "bytes": 26880, + "sha256": "134bba0f8076d43d31fcbbaeb37c3d8191b0d383e6595f787d3387e2e016738a" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_NL-256.packed": { + "bytes": 18432, + "sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_0-1024.packed": { + "bytes": 90112, + "sha256": "b1d241ee00677d6d55a5e774916608aa4479e6c549d8ed1e4803c0938cd6a855" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_S-256.packed": { + "bytes": 6400, + "sha256": "754f0d0b9e7d3656a1f669cf756779352cccb246bf6aa0b25e7f362180cf0882" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-1024.packed": { + "bytes": 69632, + "sha256": "929ef7d789f210f60cede40156f689734aaf36519cd1f43a82d44d17ae0adfb4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q2_K-256.packed": { + "bytes": 10752, + "sha256": "655f77ebb32d974da7907da4a217c45bbbcd0b127df73a731f85afe427bb7dcc" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_S-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_0-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q2_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XXS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_S-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q3_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_XS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_0-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_S-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_S-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_XXS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_S-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q6_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q6_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_S-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q2_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q3_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ3_XXS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_0-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XXS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_0-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_0-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ2_XS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_NL-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_NL-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q5_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_0-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ4_XS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q4_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + } + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/primary-synthetic-v1.operator-receipt.json b/docs/bench-evidence/rocm-quant-gather/primary-synthetic-v1.operator-receipt.json new file mode 100644 index 0000000000..df077baf9b --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/primary-synthetic-v1.operator-receipt.json @@ -0,0 +1,64 @@ +{ + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "docker", + "run", + "--name", + "rdna3-gather-primary-synthetic-v1", + "--rm", + "--device=/dev/kfd", + "--device=/dev/dri", + "--pull=never", + "--network=none", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "-v", + "/home/vikash/vllm.cpp-rdna3-gather-impl:/home/vikash/vllm.cpp-rdna3-gather-impl:ro", + "-e", + "PYTHONUNBUFFERED=1", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "/home/vikash/vllm.cpp-rdna3-gather-impl/tools/rocm_quant_gather/primary.py", + "capture", + "/home/vikash/.cache/rdna3-gather-impl/corpus/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1" + ], + "exit_code": 0, + "elapsed_seconds": 12.55049991607666, + "timeout": false, + "manifest_sha256": "6bc9501cf0abe9a8de36bcc652b636ff5a26df3dbb650ad38dcc0ab098fee9f8", + "log_sha256": "90b105e4334e7549cf97fc4943ff1db0f873fa1424000b0e2e108597a53c058d", + "all_inputs_and_138_plugin_files_verified_before_after": true, + "report_sha256": "867aaf215887b2c18f46b20b9211592550274099f721b4fff73cefb2f171dd38", + "output_count": 64 +} diff --git a/docs/bench-evidence/rocm-quant-gather/public-model-trace-captures.tar.gz b/docs/bench-evidence/rocm-quant-gather/public-model-trace-captures.tar.gz new file mode 100644 index 0000000000..5454aa8fd1 Binary files /dev/null and b/docs/bench-evidence/rocm-quant-gather/public-model-trace-captures.tar.gz differ diff --git a/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-command.json b/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-command.json new file mode 100644 index 0000000000..146c3c9e1c --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-command.json @@ -0,0 +1,557 @@ +{ + "row": "BACKEND-ROCM-QUANT-GATHER", + "argv": [ + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/bin/rocprofv3", + "--hip-trace", + "--kernel-trace", + "--memory-allocation-trace", + "--output-format", + "csv", + "--output-file", + "trace_%pid%", + "--rocm-root", + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix", + "--output-directory", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/profiler", + "--", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/test_capi_rocm_embedding_quant" + ], + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_PUBLIC_EVIDENCE": "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures" + }, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1", + "log": "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public.log", + "files": { + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/test_capi_rocm_embedding_quant": { + "bytes": 30926000, + "sha256": "6a8f7e1029b904c94c0d76561f0291cd17fd7b3e4db29e2d95e3b33894c74efd" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/source.tar": { + "bytes": 1013760, + "sha256": "77601cd1020533ecbf080fe73a5bfb6020611f0dbe762b07dce3c05d45a36de5" + }, + "/home/vikash/.cache/rdna3-gather-impl/native-final-v3/commands.json": { + "bytes": 9311, + "sha256": "2d05f18ea3a17739b441cd28f2c3b32c580f5cb48ff220ef547caa4148ef8f7d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-attach.so": { + "bytes": 897296, + "sha256": "742848d814c8c852aba6e5df960c7797968bcf19ce6a546ffde29c9f5a21ae4e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprof-trace-decoder.so.0.2": { + "bytes": 407656, + "sha256": "f4c158cdf49b46c15392decb3fef4065fc0654867de6ed26ffbcd1303e8301b2" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocpd.so.1": { + "bytes": 1325128, + "sha256": "378e816d72ce4b0d7f0958b574be85e51b491cd1067e80372f27491f07081d6a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/libamd_comgr.so": { + "bytes": 13181720, + "sha256": "ff98e63e6b8677eb51376816f3e1169e377ac446e7bf4c5d247683a72f2f59eb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprof-trace-decoder.so": { + "bytes": 407656, + "sha256": "f4c158cdf49b46c15392decb3fef4065fc0654867de6ed26ffbcd1303e8301b2" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-register.so": { + "bytes": 967760, + "sha256": "7166282aacab9f6c8fdcc28ad884e57b6b09cd4e08aead98b8e862a61010e85c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-attach.so.1": { + "bytes": 897296, + "sha256": "742848d814c8c852aba6e5df960c7797968bcf19ce6a546ffde29c9f5a21ae4e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocattach.so": { + "bytes": 1076520, + "sha256": "e86594e2368aacbecc77d39da44323150fb39f1a346862f264cfcf6a0645a99d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk.so": { + "bytes": 15823336, + "sha256": "807f890e2cb6661000845250358c3939f41573e88fc3aac6d529bb2d5fa6b55c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocpd.so": { + "bytes": 1325128, + "sha256": "378e816d72ce4b0d7f0958b574be85e51b491cd1067e80372f27491f07081d6a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/libamd_comgr.so.3.3.0": { + "bytes": 13181720, + "sha256": "ff98e63e6b8677eb51376816f3e1169e377ac446e7bf4c5d247683a72f2f59eb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocpd.so.1.3.5": { + "bytes": 1325128, + "sha256": "378e816d72ce4b0d7f0958b574be85e51b491cd1067e80372f27491f07081d6a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-roctx.so.1": { + "bytes": 845320, + "sha256": "07ab9ebdd0785c8a2f01b06c54bdebfcd70a9aa3ac7f6fb791045362dce6395c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/libamd_comgr.so.3": { + "bytes": 13181720, + "sha256": "ff98e63e6b8677eb51376816f3e1169e377ac446e7bf4c5d247683a72f2f59eb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocattach.so.1.3.5": { + "bytes": 1076520, + "sha256": "e86594e2368aacbecc77d39da44323150fb39f1a346862f264cfcf6a0645a99d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-roctx.so.1.3.5": { + "bytes": 845320, + "sha256": "07ab9ebdd0785c8a2f01b06c54bdebfcd70a9aa3ac7f6fb791045362dce6395c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-roctx.so": { + "bytes": 845320, + "sha256": "07ab9ebdd0785c8a2f01b06c54bdebfcd70a9aa3ac7f6fb791045362dce6395c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-register.so.0.6.0": { + "bytes": 967760, + "sha256": "7166282aacab9f6c8fdcc28ad884e57b6b09cd4e08aead98b8e862a61010e85c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk.so.1": { + "bytes": 15823336, + "sha256": "807f890e2cb6661000845250358c3939f41573e88fc3aac6d529bb2d5fa6b55c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-register.so.0": { + "bytes": 967760, + "sha256": "7166282aacab9f6c8fdcc28ad884e57b6b09cd4e08aead98b8e862a61010e85c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocattach.so.1": { + "bytes": 1076520, + "sha256": "e86594e2368aacbecc77d39da44323150fb39f1a346862f264cfcf6a0645a99d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk.so.1.3.5": { + "bytes": 15823336, + "sha256": "807f890e2cb6661000845250358c3939f41573e88fc3aac6d529bb2d5fa6b55c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprof-trace-decoder.so.0.2.0": { + "bytes": 407656, + "sha256": "f4c158cdf49b46c15392decb3fef4065fc0654867de6ed26ffbcd1303e8301b2" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-attach.so.1.3.5": { + "bytes": 897296, + "sha256": "742848d814c8c852aba6e5df960c7797968bcf19ce6a546ffde29c9f5a21ae4e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/bin/rocprofv3": { + "bytes": 95727, + "sha256": "2d220ded2167097e480e6866a916b575548d56bed8a823a4e505f7133868f0a6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofv3-list-avail.so": { + "bytes": 1270072, + "sha256": "acee4c6e3ea5cb365b4cc65ee3dc5ce07412aa75f5211cb58edf13d962de7fb6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so.1.3.5": { + "bytes": 11752584, + "sha256": "5c4bf4f4bff332fd7713f2c25fd7605bc80b1a5befdee7949b845b57cee87b0e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so.1": { + "bytes": 11752584, + "sha256": "5c4bf4f4bff332fd7713f2c25fd7605bc80b1a5befdee7949b845b57cee87b0e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so": { + "bytes": 11752584, + "sha256": "5c4bf4f4bff332fd7713f2c25fd7605bc80b1a5befdee7949b845b57cee87b0e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool-kokkosp.so": { + "bytes": 796176, + "sha256": "cd3c7cbed051159d3966a71be00934c7a7de1963233a891684b2484007845c1a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofv3-list-avail.so.1": { + "bytes": 1270072, + "sha256": "acee4c6e3ea5cb365b4cc65ee3dc5ce07412aa75f5211cb58edf13d962de7fb6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofv3-list-avail.so.1.3.5": { + "bytes": 1270072, + "sha256": "acee4c6e3ea5cb365b4cc65ee3dc5ce07412aa75f5211cb58edf13d962de7fb6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool-kokkosp.so.1.3.5": { + "bytes": 796176, + "sha256": "cd3c7cbed051159d3966a71be00934c7a7de1963233a891684b2484007845c1a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool-kokkosp.so.1": { + "bytes": 796176, + "sha256": "cd3c7cbed051159d3966a71be00934c7a7de1963233a891684b2484007845c1a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libclang-cpp.so.23.0git": { + "bytes": 89897976, + "sha256": "0119f76826759e3f0534bb95bcc28e3e788e3a041e47b89310b3d324197b8ad3" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libLLVM.so.23.0git": { + "bytes": 132274504, + "sha256": "1f258534e3b14393fb2d65d6cb6a93fac239727cc1df5b698b6d2b51f78a65d9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libclang-cpp.so": { + "bytes": 89897976, + "sha256": "0119f76826759e3f0534bb95bcc28e3e788e3a041e47b89310b3d324197b8ad3" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libLLVM-23git.so": { + "bytes": 132274504, + "sha256": "1f258534e3b14393fb2d65d6cb6a93fac239727cc1df5b698b6d2b51f78a65d9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libLLVM.so": { + "bytes": 132274504, + "sha256": "1f258534e3b14393fb2d65d6cb6a93fac239727cc1df5b698b6d2b51f78a65d9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_dw.so.1": { + "bytes": 5679360, + "sha256": "f9ad555cea1480082fd89dedaf5e0626a6deea0cb296c39d44b020adb56a0c49" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_elf.so.1": { + "bytes": 1198616, + "sha256": "a51543f71efce0c552ae606fe103b6e4fa4f6b283ba48a762f6d57b880401832" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_zstd.so.1": { + "bytes": 879168, + "sha256": "5fc4ab0e6899318ece9bea43d3d397d20053418a5e40cb6bc86efe19a801c75e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libdw.so": { + "bytes": 5679360, + "sha256": "f9ad555cea1480082fd89dedaf5e0626a6deea0cb296c39d44b020adb56a0c49" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_zstd.so.1.5.7": { + "bytes": 879168, + "sha256": "5fc4ab0e6899318ece9bea43d3d397d20053418a5e40cb6bc86efe19a801c75e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libelf.so": { + "bytes": 1198616, + "sha256": "a51543f71efce0c552ae606fe103b6e4fa4f6b283ba48a762f6d57b880401832" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libzstd.so": { + "bytes": 879168, + "sha256": "5fc4ab0e6899318ece9bea43d3d397d20053418a5e40cb6bc86efe19a801c75e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libdrm.so": { + "bytes": 360336, + "sha256": "f1b274ac80ac1bcd241d70e1f13f9586e3fc83545a357b312890c1722ca68dcf" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/liblzma.so": { + "bytes": 218464, + "sha256": "15402b36e2fdb0762e0b6590c3470fccec5e80cc8010c84ac4835573d592c02d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libsqlite3.so": { + "bytes": 1479528, + "sha256": "44234a62c1ca345fd1a7886b4eeb82b6396ff7737967446c1627d4154c69a69a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libdrm_amdgpu.so": { + "bytes": 251416, + "sha256": "78c1af14962d9ee0e43602d422eb8619b6c825a9b7c8f532308ea1f4108a466f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_liblzma.so.5.8.1": { + "bytes": 218464, + "sha256": "15402b36e2fdb0762e0b6590c3470fccec5e80cc8010c84ac4835573d592c02d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libbz2.so": { + "bytes": 104016, + "sha256": "628d4871ea18e82dcc73b28f45a587f00a7abd4b208b371a713eb2e9d5b95dec" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_bz2.so": { + "bytes": 104016, + "sha256": "628d4871ea18e82dcc73b28f45a587f00a7abd4b208b371a713eb2e9d5b95dec" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_liblzma.so.5": { + "bytes": 218464, + "sha256": "15402b36e2fdb0762e0b6590c3470fccec5e80cc8010c84ac4835573d592c02d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_drm_amdgpu.so.1": { + "bytes": 251416, + "sha256": "78c1af14962d9ee0e43602d422eb8619b6c825a9b7c8f532308ea1f4108a466f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_z.so.1.3.2": { + "bytes": 129576, + "sha256": "27361c0c552d2afd5ecf9079f356309ed827e9a27d35f2fc68c7b5e6b5c83199" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_drm.so.2": { + "bytes": 360336, + "sha256": "f1b274ac80ac1bcd241d70e1f13f9586e3fc83545a357b312890c1722ca68dcf" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_sqlite3.so": { + "bytes": 1479528, + "sha256": "44234a62c1ca345fd1a7886b4eeb82b6396ff7737967446c1627d4154c69a69a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libz.so": { + "bytes": 129576, + "sha256": "27361c0c552d2afd5ecf9079f356309ed827e9a27d35f2fc68c7b5e6b5c83199" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_z.so.1": { + "bytes": 129576, + "sha256": "27361c0c552d2afd5ecf9079f356309ed827e9a27d35f2fc68c7b5e6b5c83199" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/config.yaml": { + "bytes": 443032, + "sha256": "63542b5a2a285098f6ccb1d355574c8559ca75d3d5f80c82c5f02be7e3c8f960" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/convert-counters-collection-format.py": { + "bytes": 6513, + "sha256": "933afa0416346c97755e1895cd4c482fe3414eca02401885842e8a8f4963edc3" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/setup-env.sh": { + "bytes": 1384, + "sha256": "bcfd4c15a443cfa4e11f064e5dee1ee3e8793b7dc61513c1f2c5a9bf44c1b89f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/derived_counters.xml": { + "bytes": 95217, + "sha256": "774ba31a494d9135faf2ac876d0c1e6222358166c7a2b2019bb34e1e546095dc" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/basic_counters.xml": { + "bytes": 107538, + "sha256": "510cb2f0bd7400b5e2ac7dd3ff473d39ecb77d126042b99c1fcd4b0a38ea39f9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/CMakeLists.txt": { + "bytes": 1459, + "sha256": "809258de222cffb7e49af96c68a93f449a916d35a1a6f8868cf25f06add41cc0" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/thread_trace/agent.cpp": { + "bytes": 15667, + "sha256": "6d2eee1aea16dc7c851a7ccd9b8ebe50cdbbf5a512db15646334f42bdb39dba6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/thread_trace/CMakeLists.txt": { + "bytes": 2492, + "sha256": "4c2f34441922ef0c55eaf486af8303f24987f9ae23594262a2183c368775c007" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/thread_trace/main.cpp": { + "bytes": 5386, + "sha256": "cd7dd11518b692aa2982a74984eb1a0953a1f3131e2d8eb24b7211a294a41903" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/thread_trace/README.md": { + "bytes": 1081, + "sha256": "a88cc99102c22045939f37d1eab4bda802ba9b6cbfe5271c6c652dddfe8c2614" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_tracing/client.cpp": { + "bytes": 13389, + "sha256": "ef5c1d5768b8566bad9299d03ee619d744d1ed4bf12e70fda9aece8a9345876b" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_tracing/CMakeLists.txt": { + "bytes": 1732, + "sha256": "672138d4cb7014418a91e5e8caf615cbf93df765647cd8e52de78cb5523f2f33" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_tracing/main.cpp": { + "bytes": 9024, + "sha256": "cbb02ba874c4c20ce2c00d42d57113f20625eacbe1e7f04dbdd0003000443819" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_tracing/README.md": { + "bytes": 551, + "sha256": "f25e5590cb1860609e9c761257155c9901e0fab218a9e0ecd4cdc8b3c0a1687e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/client.cpp": { + "bytes": 11294, + "sha256": "6267a50614de3307b9c0b0623b16645b1d09733d9f930a02276d5ed0a300df8f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/CMakeLists.txt": { + "bytes": 2048, + "sha256": "48e08db456719fba451da3b73034c08089d9c83061a7916762d4265decd431a9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/main.cpp": { + "bytes": 8659, + "sha256": "1807978bb1f4ec6bcfdffb9a18367557364b926d25d5f6e79688e44397b1b32d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/client.hpp": { + "bytes": 1418, + "sha256": "1d37987fd6f8f34bb9f9f1df5188b0ecaee5ee2f1a99b2b3fc4fa0cb3cfca19e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/README.md": { + "bytes": 356, + "sha256": "d569ac790862a3699175f097d29d9725bbfa662b84d50cc5b7a74e13306e87c7" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/CMakeLists.txt": { + "bytes": 3539, + "sha256": "b554903b80d735b5488a4dc2a80b81928d3758678037d3d72f20b8bd85d8aad5" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/call_stack.hpp": { + "bytes": 2937, + "sha256": "bc95aaada5e341c82eb5213df84d81068d2fbca2ed506d5c2c2cabfb54d3131e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/filesystem.hpp": { + "bytes": 3096, + "sha256": "5fc38fcabffcec84d87fbc2be55ab528ea7b4328e53aa885a9340aa4948691a5" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/name_info.hpp": { + "bytes": 1849, + "sha256": "9f8bb3663608c13f8844da6322b13f8154f74842e7d58a38ee8a3764eb55651c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/defines.hpp": { + "bytes": 5493, + "sha256": "ee3401d2ef7131b3a2523e254653a9317e19aa2be00f5a796ff8afe90c7104cb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/client.cpp": { + "bytes": 9503, + "sha256": "7937856b638b1b24060aaab7441f09e85c8746a95b19704952ab9156ec0dc14a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/CMakeLists.txt": { + "bytes": 1679, + "sha256": "71d1bb17f48511ffa43b7b844d15c8e7c7f4fe60f182af9420c94c67f49b4d61" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/main.cpp": { + "bytes": 7927, + "sha256": "8c09efef58659fa03823f4b30fbd3b94a712fee31fa6cca2edcce751256c6929" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/transpose_kernels.hpp": { + "bytes": 4643, + "sha256": "90175f330ec0f40a83bf1528f768f0c2aba21810a0ee12063f6da8aa21f4cfcd" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/README.md": { + "bytes": 423, + "sha256": "dc1813f6980788e01498d4b4b7a9cab4ca857a68948c7b2e3970783011409c92" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/print_functional_counters_client.cpp": { + "bytes": 17500, + "sha256": "cda871f9bce62f0fca2294b324ce7e6e30fef6450ac7e571aa6d836c9a4452ed" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/CMakeLists.txt": { + "bytes": 10081, + "sha256": "ea491ec6f527f73551f6b31b5fde27ee5f777b9c645621656416c641ba78243a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/main.cpp": { + "bytes": 5632, + "sha256": "ecf46f16fa6243db4d51b5021b533f63ec451bf4f4aae44af89471a641c5e3ed" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/device_counting_async_client.cpp": { + "bytes": 14612, + "sha256": "e2df28c4ce6253ec1a9d56def57f3d4b3830bb887b9efdbe26fb584a799f7b54" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/device_serialized_main.cpp": { + "bytes": 3516, + "sha256": "459bc84019b9ed6df32a1fe850187a12562fe74a4b212269f321ae22ae9ae39c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/client.hpp": { + "bytes": 1289, + "sha256": "2fd28605c5ad6616ca882444e8085c2c717da2d99ca98dc61e3d6f874462e792" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/callback_client.cpp": { + "bytes": 10883, + "sha256": "56a6a787c30b9124cc242480e0bb11f061eff3f07b39ffdf8d9288849f49fa3b" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/README.md": { + "bytes": 739, + "sha256": "39a6d215564afd103ebe7a2eb3a1b483af5211b4c9775f22a4ce5a76bcbbd149" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/device_counting_sync_client.cpp": { + "bytes": 19505, + "sha256": "0d6d5e573dedcf5d00b2fdcd672e4ea76ab835093dc47a403dd9f9ec5734f793" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/buffered_client.cpp": { + "bytes": 18653, + "sha256": "cb9292901935ada027dc63586ee1c898be34909e4e1c5c40b477a9c0c7fae9d2" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/client.cpp": { + "bytes": 27153, + "sha256": "45a54e1af1a2aa0ea0cc99e9382d417472334f6df7d6b4465efaa8b5c57237a4" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/CMakeLists.txt": { + "bytes": 3359, + "sha256": "fb7e7988e4976e06d428100390de9b9eaf44dc1a825ff00ef467dbd2663d4ec7" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/main.cpp": { + "bytes": 4408, + "sha256": "63ef6ca870fecbd2911c67fb73715323abbc6de700728e6c6dd2b0bdd9d98638" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/client.hpp": { + "bytes": 1474, + "sha256": "1d62c3450c2830b7f711563ae48d088ddac82259430e6f2dddef22500d635e32" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/README.md": { + "bytes": 418, + "sha256": "2f9722fc01ad229cabae82a11d9e3141d96942a95141443361e8b2039430f230" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/utils.hpp": { + "bytes": 1391, + "sha256": "339eb666c0c1d9e803186263ad705c68c59c92549c23113c10be5b15874e46cb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/client.cpp": { + "bytes": 7641, + "sha256": "4558e1e2d6aab085ef4fbfcfd6c4f433908acaf32511d3d71ab7621fd6df4510" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/CMakeLists.txt": { + "bytes": 2247, + "sha256": "c5ea84e14d54ab79f6e573f9bb1702a86d37ffe3506c89c88086b159370127a9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/main.cpp": { + "bytes": 13378, + "sha256": "6e215e947a9d034e352bce5a8518b233dea45fb9e0848a7338f92874cde1b4d8" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/pcs.hpp": { + "bytes": 3389, + "sha256": "ad2225d4adee003b0fd78ffc9b59d8c96a0c96253bbc9d6c43d5e350066f4e5a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/README.md": { + "bytes": 691, + "sha256": "e3c906783a57916c30bfd0667e42b1b4638999099d0428c9cbaca04e5b7922bb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/utils.cpp": { + "bytes": 1656, + "sha256": "3442d1c93882b1798adc5d9120c1e17a675460ab28c36d674ab483bf41456cb1" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/pcs.cpp": { + "bytes": 18676, + "sha256": "d09f3f28eba6ad68a30825fa21f33f96f0062bc466761e9889bcb429b7ff8e24" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/client.cpp": { + "bytes": 13875, + "sha256": "cb3fb9f13e4f263eab0037d6bc66e9b03be56f6916dc662fad6663dbe5cc0884" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/CMakeLists.txt": { + "bytes": 2089, + "sha256": "b5fa1810951a7afbcf817a07749edbcc2a790f1f1933546a85db771c480e62ca" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/main.cpp": { + "bytes": 9807, + "sha256": "6fb5c5e9b416842df480470af4307b9571ec0fb95c2f1e9fad1fa879ccfc7d0d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/client.hpp": { + "bytes": 1474, + "sha256": "51d1b853f62a7bbf4d18854a071c1378e57b527605f27b563d1b3cc9cc5d3718" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/README.md": { + "bytes": 431, + "sha256": "1adfbb032bd9f3ed877d35216952018ceb63d6070b00848da9fc3f5300ac3726" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/client.cpp": { + "bytes": 27923, + "sha256": "8b22ae1af6673ccf4c28a8ff9e5b57826e04447f83fb9cddd5cb7e201cf8538e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/CMakeLists.txt": { + "bytes": 4630, + "sha256": "130a15dc43bdf117a58fd4cfbf65f7806a1325bcf31b71fe6ddc6fe1b793c0af" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/main.cpp": { + "bytes": 13378, + "sha256": "6e215e947a9d034e352bce5a8518b233dea45fb9e0848a7338f92874cde1b4d8" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/client.hpp": { + "bytes": 1539, + "sha256": "726d87b15436e2b3192f4468f8678d209ada3a7c4ca2b353bbde1ad705244f6f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/README.md": { + "bytes": 894, + "sha256": "037a6c03ab82a3dee1bb91a7f687c6e38c29bd1f93e0dbe24d64cd1da530c2e0" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/client.cpp": { + "bytes": 24708, + "sha256": "9067a403adb0db206c0d94bc2baea3d568f8a26d6ef0bd2d61112d85330f4660" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/CMakeLists.txt": { + "bytes": 1961, + "sha256": "b8e7c7e91a8dd46c6f6ec780da4e29c0aaf4ee14c696de8ed283b3bb15ac8215" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/main.cpp": { + "bytes": 13671, + "sha256": "3fe9ca538bac7db58a8e51bce8a805d63c02c3ca0160bdf31904a4d21bac3018" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/client.hpp": { + "bytes": 1539, + "sha256": "726d87b15436e2b3192f4468f8678d209ada3a7c4ca2b353bbde1ad705244f6f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/README.md": { + "bytes": 529, + "sha256": "b26868a670abe00bc06c66f866271908ea485978f24a772ec7945a68c87f8b9c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/tests/cmake/rocprofiler_test_gpu_targets.cmake": { + "bytes": 563, + "sha256": "586a3b79e5a2823511e87a74ad3c3b042ce9c8ed696022983a48604d5874b145" + } + }, + "scope": "Native model KV dtype/allocation evidence through unchanged complete public test;228engines, no filter, no performanceclaim", + "source_version": "Exactly native-final-v3 candidate preserved before trailing-space-only table cleanup; frozen source.tar and binary", + "authority": "Root operator underGPU0 mutex", + "expected": "Publictestpasses all19codecs2prompts3repeats2arms; check executedBF16KVtemplates andcacheallocation sizes through sourcechain; oraclemodelparityremainsPENDING" +} diff --git a/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-operator-receipt.json b/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-operator-receipt.json new file mode 100644 index 0000000000..5cc40696aa --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-operator-receipt.json @@ -0,0 +1,33 @@ +{ + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/bin/rocprofv3", + "--hip-trace", + "--kernel-trace", + "--memory-allocation-trace", + "--output-format", + "csv", + "--output-file", + "trace_%pid%", + "--rocm-root", + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix", + "--output-directory", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/profiler", + "--", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/test_capi_rocm_embedding_quant" + ], + "cwd": "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1", + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_PUBLIC_EVIDENCE": "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures" + }, + "exit": 0, + "seconds": 8.711418390274048, + "log_sha256": "6bc27e9dc1f38e70b00c816f94a2c9b50c4c0c9d88b05a31bcfae76c740ea582", + "inputs_unchanged": true +} diff --git a/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-operator-trace-audit.json b/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-operator-trace-audit.json new file mode 100644 index 0000000000..9dd598f7df --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-operator-trace-audit.json @@ -0,0 +1,492 @@ +{ + "result": "PASS", + "scope": "Native public completion memory format and repeatability; no oracle model parity or performance claim", + "engines": 228, + "assertions": 2850, + "api_records": 211083, + "kernels": 32658, + "runtime_correlation_missing": 0, + "kv_kernels": { + "void vt::rocm::ReshapeAndCacheK(unsigned short const*, unsigned short const*, unsigned short*, unsigned short*, long const*, long, long, long, long, long, long, long, long)": 912, + "void vt::rocm::(anonymous namespace)::PagedAttnOnline(float*, float const*, __hip_bfloat16 const*, __hip_bfloat16 const*, int const*, int const*, int const*, long, long, long, long, long, long, long, long, long, long, long, long, long, float, float, bool, int, int, float, float)": 912 + }, + "gpu_allocations": 4841, + "hipMalloc_16384_bytes": 228, + "cache_source_formula": "4 blocks * 16 cells * 1 KV head * (64 K + 64 V) * 2 BF16 bytes = 16384 bytes for one layer", + "allocation_attribution": "Executed BF16 attention and 228 exact-size allocations agree with the runner cache formula. CSV omits kernel argument pointers, so allocation-to-cache pointer attribution is a source-supported inference.", + "source_anchors": [ + "tests/capi/test_rocm_embedding_quant.cpp:49", + "tests/capi/rocm_quant_gather_fixture.h:126", + "src/vllm/v1/worker/gpu/runner.cpp:1572", + "src/vllm/v1/kv_cache_interface.cpp:107" + ], + "captures_checked": 456, + "quant_dense_and_three_repeats": "byte exact tokens and all512 finite F32 logits per completion", + "oracle_model_gate": "PENDING: primary Qwen3_5TextConfig renderer refusal; secondary GGML buffer assertion and256-cell padding mismatch", + "artifacts": { + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/operator.log": "6bc27e9dc1f38e70b00c816f94a2c9b50c4c0c9d88b05a31bcfae76c740ea582", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/operator-receipt.json": "68d6f848fd89089e8598c58371f441818196b5be216e8d1d1a617c72c09c4d2e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/command.json": "e2d40fa6b84c822c95a8a221507daf4735996277f7d06fa6d39cdd1d4b326600", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/source.tar": "77601cd1020533ecbf080fe73a5bfb6020611f0dbe762b07dce3c05d45a36de5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/profiler/trace_1253121_agent_info.csv": "90f47b57110e910cd39ed10f58a95a8f16bd1dadc12bb286363a2014e0ea9f40", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/profiler/trace_1253121_hip_api_trace.csv": "ee8cb2a85284839f93019e909b972888a4443d2c9e5899d53cb14a6ab7f414db", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/profiler/trace_1253121_kernel_trace.csv": "9885b2818a116f3b1875e34eacda0091a903e79111eaa09234ce51d8aa47602b", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/profiler/trace_1253121_memory_allocation_trace.csv": "033b8f3eae261a41cac48f9c2fb23db9625b99d4d842e40b2ed8cdd4c19c3699", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p0-dense-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p0-dense-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p0-quant-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p0-quant-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p1-dense-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p1-dense-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p1-quant-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p1-quant-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p0-dense-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p0-dense-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p0-quant-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p0-quant-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p1-dense-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p1-dense-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p1-quant-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p1-quant-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p0-dense-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p0-dense-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p0-quant-logits-f32.bin": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p0-quant-tokens.txt": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p1-dense-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p1-dense-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p1-quant-logits-f32.bin": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p1-quant-tokens.txt": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p0-dense-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p0-dense-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p0-quant-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p0-quant-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p1-dense-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p1-dense-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p1-quant-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p1-quant-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p0-dense-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p0-dense-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p0-quant-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p0-quant-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p1-dense-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p1-dense-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p1-quant-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p1-quant-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p0-dense-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p0-dense-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p0-quant-logits-f32.bin": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p0-quant-tokens.txt": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p1-dense-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p1-dense-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p1-quant-logits-f32.bin": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p1-quant-tokens.txt": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p0-dense-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p0-dense-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p0-quant-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p0-quant-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p1-dense-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p1-dense-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p1-quant-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p1-quant-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p0-dense-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p0-dense-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p0-quant-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p0-quant-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p1-dense-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p1-dense-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p1-quant-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p1-quant-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p0-dense-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p0-dense-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p0-quant-logits-f32.bin": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p0-quant-tokens.txt": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p1-dense-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p1-dense-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p1-quant-logits-f32.bin": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p1-quant-tokens.txt": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p0-dense-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p0-dense-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p0-quant-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p0-quant-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p1-dense-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p1-dense-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p1-quant-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p1-quant-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p0-dense-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p0-dense-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p0-quant-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p0-quant-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p1-dense-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p1-dense-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p1-quant-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p1-quant-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p0-dense-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p0-dense-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p0-quant-logits-f32.bin": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p0-quant-tokens.txt": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p1-dense-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p1-dense-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p1-quant-logits-f32.bin": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p1-quant-tokens.txt": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p0-dense-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p0-dense-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p0-quant-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p0-quant-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p1-dense-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p1-dense-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p1-quant-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p1-quant-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p0-dense-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p0-dense-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p0-quant-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p0-quant-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p1-dense-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p1-dense-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p1-quant-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p1-quant-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p0-dense-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p0-dense-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p0-quant-logits-f32.bin": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p0-quant-tokens.txt": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p1-dense-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p1-dense-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p1-quant-logits-f32.bin": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p1-quant-tokens.txt": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p0-dense-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p0-dense-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p0-quant-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p0-quant-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p1-dense-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p1-dense-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p1-quant-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p1-quant-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p0-dense-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p0-dense-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p0-quant-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p0-quant-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p1-dense-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p1-dense-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p1-quant-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p1-quant-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p0-dense-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p0-dense-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p0-quant-logits-f32.bin": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p0-quant-tokens.txt": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p1-dense-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p1-dense-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p1-quant-logits-f32.bin": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p1-quant-tokens.txt": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p0-dense-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p0-dense-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p0-quant-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p0-quant-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p1-dense-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p1-dense-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p1-quant-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p1-quant-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p0-dense-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p0-dense-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p0-quant-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p0-quant-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p1-dense-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p1-dense-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p1-quant-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p1-quant-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p0-dense-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p0-dense-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p0-quant-logits-f32.bin": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p0-quant-tokens.txt": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p1-dense-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p1-dense-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p1-quant-logits-f32.bin": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p1-quant-tokens.txt": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p0-dense-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p0-dense-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p0-quant-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p0-quant-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p1-dense-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p1-dense-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p1-quant-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p1-quant-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p0-dense-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p0-dense-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p0-quant-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p0-quant-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p1-dense-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p1-dense-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p1-quant-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p1-quant-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p0-dense-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p0-dense-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p0-quant-logits-f32.bin": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p0-quant-tokens.txt": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p1-dense-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p1-dense-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p1-quant-logits-f32.bin": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p1-quant-tokens.txt": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p0-dense-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p0-dense-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p0-quant-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p0-quant-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p1-dense-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p1-dense-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p1-quant-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p1-quant-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p0-dense-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p0-dense-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p0-quant-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p0-quant-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p1-dense-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p1-dense-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p1-quant-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p1-quant-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p0-dense-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p0-dense-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p0-quant-logits-f32.bin": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p0-quant-tokens.txt": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p1-dense-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p1-dense-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p1-quant-logits-f32.bin": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p1-quant-tokens.txt": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p0-dense-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p0-dense-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p0-quant-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p0-quant-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p1-dense-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p1-dense-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p1-quant-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p1-quant-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p0-dense-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p0-dense-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p0-quant-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p0-quant-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p1-dense-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p1-dense-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p1-quant-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p1-quant-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p0-dense-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p0-dense-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p0-quant-logits-f32.bin": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p0-quant-tokens.txt": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p1-dense-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p1-dense-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p1-quant-logits-f32.bin": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p1-quant-tokens.txt": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p0-dense-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p0-dense-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p0-quant-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p0-quant-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p1-dense-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p1-dense-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p1-quant-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p1-quant-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p0-dense-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p0-dense-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p0-quant-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p0-quant-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p1-dense-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p1-dense-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p1-quant-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p1-quant-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p0-dense-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p0-dense-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p0-quant-logits-f32.bin": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p0-quant-tokens.txt": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p1-dense-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p1-dense-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p1-quant-logits-f32.bin": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p1-quant-tokens.txt": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p0-dense-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p0-dense-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p0-quant-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p0-quant-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p1-dense-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p1-dense-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p1-quant-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p1-quant-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p0-dense-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p0-dense-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p0-quant-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p0-quant-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p1-dense-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p1-dense-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p1-quant-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p1-quant-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p0-dense-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p0-dense-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p0-quant-logits-f32.bin": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p0-quant-tokens.txt": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p1-dense-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p1-dense-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p1-quant-logits-f32.bin": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p1-quant-tokens.txt": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p0-dense-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p0-dense-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p0-quant-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p0-quant-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p1-dense-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p1-dense-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p1-quant-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p1-quant-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p0-dense-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p0-dense-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p0-quant-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p0-quant-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p1-dense-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p1-dense-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p1-quant-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p1-quant-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p0-dense-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p0-dense-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p0-quant-logits-f32.bin": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p0-quant-tokens.txt": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p1-dense-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p1-dense-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p1-quant-logits-f32.bin": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p1-quant-tokens.txt": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p0-dense-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p0-dense-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p0-quant-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p0-quant-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p1-dense-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p1-dense-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p1-quant-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p1-quant-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p0-dense-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p0-dense-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p0-quant-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p0-quant-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p1-dense-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p1-dense-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p1-quant-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p1-quant-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p0-dense-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p0-dense-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p0-quant-logits-f32.bin": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p0-quant-tokens.txt": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p1-dense-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p1-dense-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p1-quant-logits-f32.bin": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p1-quant-tokens.txt": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p0-dense-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p0-dense-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p0-quant-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p0-quant-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p1-dense-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p1-dense-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p1-quant-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p1-quant-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p0-dense-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p0-dense-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p0-quant-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p0-quant-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p1-dense-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p1-dense-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p1-quant-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p1-quant-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p0-dense-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p0-dense-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p0-quant-logits-f32.bin": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p0-quant-tokens.txt": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p1-dense-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p1-dense-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p1-quant-logits-f32.bin": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p1-quant-tokens.txt": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p0-dense-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p0-dense-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p0-quant-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p0-quant-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p1-dense-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p1-dense-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p1-quant-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p1-quant-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p0-dense-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p0-dense-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p0-quant-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p0-quant-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p1-dense-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p1-dense-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p1-quant-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p1-quant-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p0-dense-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p0-dense-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p0-quant-logits-f32.bin": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p0-quant-tokens.txt": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p1-dense-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p1-dense-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p1-quant-logits-f32.bin": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p1-quant-tokens.txt": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p0-dense-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p0-dense-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p0-quant-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p0-quant-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p1-dense-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p1-dense-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p1-quant-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p1-quant-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p0-dense-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p0-dense-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p0-quant-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p0-quant-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p1-dense-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p1-dense-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p1-quant-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p1-quant-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p0-dense-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p0-dense-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p0-quant-logits-f32.bin": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p0-quant-tokens.txt": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p1-dense-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p1-dense-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p1-quant-logits-f32.bin": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p1-quant-tokens.txt": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p0-dense-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p0-dense-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p0-quant-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p0-quant-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p1-dense-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p1-dense-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p1-quant-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p1-quant-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p0-dense-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p0-dense-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p0-quant-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p0-quant-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p1-dense-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p1-dense-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p1-quant-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p1-quant-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p0-dense-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p0-dense-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p0-quant-logits-f32.bin": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p0-quant-tokens.txt": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p1-dense-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p1-dense-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p1-quant-logits-f32.bin": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p1-quant-tokens.txt": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p0-dense-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p0-dense-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p0-quant-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p0-quant-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p1-dense-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p1-dense-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p1-quant-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p1-quant-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p0-dense-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p0-dense-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p0-quant-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p0-quant-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p1-dense-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p1-dense-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p1-quant-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p1-quant-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p0-dense-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p0-dense-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p0-quant-logits-f32.bin": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p0-quant-tokens.txt": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p1-dense-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p1-dense-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p1-quant-logits-f32.bin": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8", + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p1-quant-tokens.txt": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-public-audit.json b/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-public-audit.json new file mode 100644 index 0000000000..fac48b6d94 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/public-model-trace-v1-public-audit.json @@ -0,0 +1,3223 @@ +{ + "scope": "same-binary public quantized/dense completion equality; not oracle model token equality", + "formats": 19, + "completions": 228, + "comparisons": [ + { + "format": "IQ1_S", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 1, + 7, + 25, + 75 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_S", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 40, + 36, + 34, + 59 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_S", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 1, + 7, + 25, + 75 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_S", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 40, + 36, + 34, + 59 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_S", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 1, + 7, + 25, + 75 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_S", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 40, + 36, + 34, + 59 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_XXXS", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 31, + 60, + 87, + 37 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_XXXS", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 94, + 54, + 93, + 88 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_XXXS", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 31, + 60, + 87, + 37 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_XXXS", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 94, + 54, + 93, + 88 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_XXXS", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 31, + 60, + 87, + 37 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ1_XXXS", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 94, + 54, + 93, + 88 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_S", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 29, + 55, + 68, + 87 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_S", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 33, + 81, + 18, + 34 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_S", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 29, + 55, + 68, + 87 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_S", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 33, + 81, + 18, + 34 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_S", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 29, + 55, + 68, + 87 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_S", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 33, + 81, + 18, + 34 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XS", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 103, + 71, + 43, + 23 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XS", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 45, + 91, + 61, + 61 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XS", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 103, + 71, + 43, + 23 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XS", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 45, + 91, + 61, + 61 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XS", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 103, + 71, + 43, + 23 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XS", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 45, + 91, + 61, + 61 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XXS", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 73, + 113, + 43, + 41 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XXS", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 74, + 24, + 89, + 64 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XXS", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 73, + 113, + 43, + 41 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XXS", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 74, + 24, + 89, + 64 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XXS", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 73, + 113, + 43, + 41 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ2_XXS", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 74, + 24, + 89, + 64 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_S", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 109, + 57, + 38, + 96 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_S", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 18, + 101, + 112, + 89 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_S", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 109, + 57, + 38, + 96 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_S", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 18, + 101, + 112, + 89 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_S", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 109, + 57, + 38, + 96 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_S", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 18, + 101, + 112, + 89 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_XXS", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 126, + 93, + 43, + 45 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_XXS", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 73, + 83, + 122, + 118 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_XXS", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 126, + 93, + 43, + 45 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_XXS", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 73, + 83, + 122, + 118 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_XXS", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 126, + 93, + 43, + 45 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ3_XXS", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 73, + 83, + 122, + 118 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_NL", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 47, + 62, + 117, + 37 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_NL", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 11, + 28, + 104, + 27 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_NL", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 47, + 62, + 117, + 37 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_NL", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 11, + 28, + 104, + 27 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_NL", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 47, + 62, + 117, + 37 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_NL", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 11, + 28, + 104, + 27 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_XS", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 60, + 116, + 75, + 114 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_XS", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 49, + 29, + 55, + 35 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_XS", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 60, + 116, + 75, + 114 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_XS", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 49, + 29, + 55, + 35 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_XS", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 60, + 116, + 75, + 114 + ], + "dense_and_repeat_exact": true + }, + { + "format": "IQ4_XS", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 49, + 29, + 55, + 35 + ], + "dense_and_repeat_exact": true + }, + { + "format": "MXFP4", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 6, + 3, + 11, + 96 + ], + "dense_and_repeat_exact": true + }, + { + "format": "MXFP4", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 20, + 74, + 2, + 51 + ], + "dense_and_repeat_exact": true + }, + { + "format": "MXFP4", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 6, + 3, + 11, + 96 + ], + "dense_and_repeat_exact": true + }, + { + "format": "MXFP4", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 20, + 74, + 2, + 51 + ], + "dense_and_repeat_exact": true + }, + { + "format": "MXFP4", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 6, + 3, + 11, + 96 + ], + "dense_and_repeat_exact": true + }, + { + "format": "MXFP4", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 20, + 74, + 2, + 51 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q2_K", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 46, + 110, + 85, + 3 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q2_K", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 29, + 78, + 71, + 65 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q2_K", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 46, + 110, + 85, + 3 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q2_K", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 29, + 78, + 71, + 65 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q2_K", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 46, + 110, + 85, + 3 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q2_K", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 29, + 78, + 71, + 65 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q3_K", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 71, + 26, + 89, + 13 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q3_K", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 16, + 51, + 79, + 19 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q3_K", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 71, + 26, + 89, + 13 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q3_K", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 16, + 51, + 79, + 19 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q3_K", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 71, + 26, + 89, + 13 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q3_K", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 16, + 51, + 79, + 19 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_0", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 47, + 19, + 4, + 20 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_0", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 11, + 28, + 104, + 78 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_0", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 47, + 19, + 4, + 20 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_0", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 11, + 28, + 104, + 78 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_0", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 47, + 19, + 4, + 20 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_0", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 11, + 28, + 104, + 78 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_K", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 125, + 23, + 85, + 103 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_K", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 103, + 21, + 114, + 29 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_K", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 125, + 23, + 85, + 103 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_K", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 103, + 21, + 114, + 29 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_K", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 125, + 23, + 85, + 103 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q4_K", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 103, + 21, + 114, + 29 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_0", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 122, + 18, + 117, + 112 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_0", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 98, + 39, + 67, + 52 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_0", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 122, + 18, + 117, + 112 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_0", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 98, + 39, + 67, + 52 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_0", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 122, + 18, + 117, + 112 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_0", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 98, + 39, + 67, + 52 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_K", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 7, + 2, + 22, + 46 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_K", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 29, + 2, + 22, + 46 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_K", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 7, + 2, + 22, + 46 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_K", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 29, + 2, + 22, + 46 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_K", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 7, + 2, + 22, + 46 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q5_K", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 29, + 2, + 22, + 46 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q6_K", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 84, + 23, + 48, + 53 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q6_K", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 10, + 86, + 39, + 77 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q6_K", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 84, + 23, + 48, + 53 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q6_K", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 10, + 86, + 39, + 77 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q6_K", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 84, + 23, + 48, + 53 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q6_K", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 10, + 86, + 39, + 77 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_0", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 34, + 7, + 116, + 101 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_0", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 55, + 74, + 95, + 113 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_0", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 34, + 7, + 116, + 101 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_0", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 55, + 74, + 95, + 113 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_0", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 34, + 7, + 116, + 101 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_0", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 55, + 74, + 95, + 113 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_K", + "repeat": 0, + "prompt_index": 0, + "tokens": [ + 80, + 63, + 80, + 63 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_K", + "repeat": 0, + "prompt_index": 1, + "tokens": [ + 40, + 14, + 115, + 5 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_K", + "repeat": 1, + "prompt_index": 0, + "tokens": [ + 80, + 63, + 80, + 63 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_K", + "repeat": 1, + "prompt_index": 1, + "tokens": [ + 40, + 14, + 115, + 5 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_K", + "repeat": 2, + "prompt_index": 0, + "tokens": [ + 80, + 63, + 80, + 63 + ], + "dense_and_repeat_exact": true + }, + { + "format": "Q8_K", + "repeat": 2, + "prompt_index": 1, + "tokens": [ + 40, + 14, + 115, + 5 + ], + "dense_and_repeat_exact": true + } + ], + "files": { + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "9a7e87be2f90b2b51878e3424b3db457b410e5d5930fa22755d5534c1967cf08" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a12eac02c8cb84a149c02e369eb4d29fa4ee4c89eeebaddeda5094c23ff5f2cb" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d7e6f2acae98af061483b4dd0722cbcf784e9683720ef4e509fb05574219d7bd" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_S-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "d9850e483046535eee14e396c7321e91c4aa4e9fcd7ebc7a20336b6277152522" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "4f6b598fc3142001dd0d3a933236c33e84e8fc4d0aebdeee00f2ba561604f386" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0fcf54423ce21a21cb3a7cd3d78416a09cff71c384209cacd47348347f4f00a4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "b4985b0433260dd2dbff82591a824a5daaef6afac7e83265573bc7c67b7d4baa" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ1_XXXS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "93569bd6ea15cbc8b61fd0f43bfc3ae89fc13d7e568dd47d497fbfe373e56d69" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "875c78829fd811d7a96f406ddae171e8ab6f9da5d3abaf7a58fe1591ac37d09c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "3a74972628df200ac04f4ce356f5f150f592e937fed7ea866f9125006c1999e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "5a60ff63d8a76bd112296973a0d2347198187605b555bdaff55aca4709774f17" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_S-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "44d929b443672c840e4beaf127704b7ffc61ac84432b4198d6c001d8d8a606ca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "f56786f89cb750e27989ce8fbe94c0fbe2979ff20b3ad9bca475de6077c019e4" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "36df7f97f56e8f060f479812f9886fbff833c71272afb286d63e0516f9e7f4b9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "0d739dc6493c26f87bef42fa11ceca6d4070430e42edb7eab3131be2bc8d0d21" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "548f8e0e2c754c3e0e84d5c69e907c7a506bae4f65942f66d48e454b1f4dc93c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "58a2878ab37c154e1fe91e7e1bc12c7286ce7d49308111c694f74578a00bf005" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f0d5fd5743bded857c178438485b67b8375bc0bb34c147399d5627a55432b65f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "54fbd51d02319038c7a63142ffa3181793eda4aa2fbb66b38a8015273cf5064c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ2_XXS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eeeb557f78f3e6a475eccbc15a4da924b22df8f78e5be6a8d2705eeab9af5b79" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "eec0fb4c95ed29a3f891875812aca630adf32ec6a766606e119966f24968f36e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "2bb6e26b95a8769c188508d6dce0af84836eed0549539c6a1411fecae31bdeca" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "6ffc3fa9bde8b1ceb93084fd6935c87eb598f2dcbc82fedfe30ab49993cb752a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_S-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "23ea09894c993dc9396570b3f478cbbcb1720910e9d59484913c0783874f8634" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "4943727e005a07b88e1d3db4bc7abce36e199e114b454d2ec3d62d9faf60abe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "b768a528c46498035f34a3ac95cab600ba61f4d56cea952df5f6c3d27613b5e8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "94a991cc39d4778e6ae402f021b9e5b60bb74c6d35371fdde93fa9293f467ed1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ3_XXS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "67846e04c1eaea0fe3852b28e2de589413806f458b1b4fe7195fd30dc619014e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "22e18ce184d52df1b65612ccd102ad67da3a84a9113d0a44a024417af5bcfb96" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9b403f07e9feb65f83b0172ca46b071149eb906af0e5d36a034597936d377140" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "a63747c494ced4489b012152ec3679571588809e20674b40827f5a874229ade0" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_NL-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "9f126f54b75e1662b2f05dd3a27d7cd0f94cd0f5b1f613ef9981b8221b2c2473" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "ef283a43caf4f8405f5e904413a57710312f4e12a336152f31109ed76574a44d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bac4f07548c176f275a0269531759617db69c9eb17a773e001cc2e77b2346498" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "d4369c9904e0f80f86964a593f74108543f9f76ff1817010ec007448a58ce8f5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/IQ4_XS-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ac2ec0e11ede8681b22e93e0cb81919d659188a560671b12e468d3b87f2a71a2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "64b38996e1f2c242ddb8d62371ea481ac25b14c58ef551059ca4088f795607fe" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "e33ea09097900b44138ae433200515096377bbd3357756f3db005eb2db9dd8d9" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "6c192a29861569bdc7b0a2bed1803e5182f7ec343ff07ef7bebe90e37e466219" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/MXFP4-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "98fd81184c9f74af0c19eba787ae63a4d437b0ffc0b36c3c351a88c3e3187307" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "73c1682ca9381abc586e1250d23d462bd2f22a818f9c39e70d054d622f0db012" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "eb857003fdc2acab5e54ef4dfe66d2082a47bf48979956df07bb530b9b235eac" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "f21e668e2d4c939025118a530d8921ff3efdc7f7e87f47a092811345a7cb5cfc" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q2_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "c0b6a4f54e05aa528ae7cc198985a5302163913cfdfaaae291d26b939989a10f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "215b4d0bb3d0a2093449e92943f4a781e97cd390fe14532747df279fa19f5053" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "a7ded008caba30e7ad44549a6399f7766dc66959abe3a0ba4d803ec47d39fc50" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "8efd532e2ba609398099add4119a772f55deb5f8b43aee77b1c06ebf692c8a19" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q3_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d1deb01a916a62f73c242ff0c8aa1fff22c9327bc2662b234faca6b647cfe6" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p0-quant-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p0-dense-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p0-quant-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p0-dense-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p0-quant-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p0-dense-tokens.txt": { + "bytes": 11, + "sha256": "1a96f3b52d51bf85a5c2da89c0132bba345cfbe6822d28fde4df078bc2cfa91e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "6a29993af6d6ca6fa96ae46168a2a440c5b55a714613b34054138928aa16c150" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "22500293108517f09d87796a3f7dbd21fc2fe3dce64b8055561bb52bbe4c62e3" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_0-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f2ddfbe1d64ce169916fcaaa4965f81dc1c8b259a062987c7caba5f66fafb101" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p0-quant-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p0-dense-tokens.txt": { + "bytes": 14, + "sha256": "90d2fcdc8e20f50bbd988f2d23ee2dd7eba08f1c420f12c993a5e0dbcd758208" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "ba7793887d37de36034092d2857b137937dd9e13c418f4f654c22399485fb1e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p1-quant-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p1-dense-tokens.txt": { + "bytes": 14, + "sha256": "ec0ebc3beeceeacf89c91be6151c16306b55412197d0f981606139853ee3741e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q4_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "1322e504646e3e15b283321eada9374f13a0828c974e7ca557eaf86633791818" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p0-quant-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p0-dense-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p0-quant-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p0-dense-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p0-quant-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p0-dense-tokens.txt": { + "bytes": 15, + "sha256": "427052e9db2f5e8f6eddfb6a98154c16eff216b251ba8f1840952fb7c7209637" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "cf2ccd95a7b8375519261c639be7467e40203cafb3cde030bbafc5aef5963d2f" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "e2b25d5e20be62588e06d09c515205068040405642be6ed02e2540fcc74b1f2c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_0-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "457e6c0d4286b64fb0dd97aabe20e4107be09dee93a09546ba0e7fa1cadffa6a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p0-quant-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p0-dense-tokens.txt": { + "bytes": 10, + "sha256": "ec1982beff344ea4cf333e6232679cb7ca76428940bc61c051c1cf3cbec1163d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "bf4285247160b2c71f5991b24a0fea641179b0165bb9793d53756252e714e1d2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p1-quant-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p1-dense-tokens.txt": { + "bytes": 11, + "sha256": "89ec82c13f27a3a4b16da852d26341e7b93a1e3254b9bba9e408ee670f344528" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q5_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "449c119478742aa639c1445fd367cb45b6c483cf23116725d7c761e5ca3a5945" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "c4f67126f67de6cdba3146cbbeef84e1038d5a67ec2f89d10e1f45872723ec09" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "753f7878a49cef082ff4b1077f944f345e1b7bf1c15e78e83f48ed2897f5b17e" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "66a4cbd59120ab4ebc084cec015d9169567b98f55466834df7004d3c0c89a88d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q6_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "7a6301766b7b20e272697e0b012a621230906944fdae62e8880bea21672c40c7" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p0-quant-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p0-dense-tokens.txt": { + "bytes": 13, + "sha256": "57e1f1162073a7d35395c3bae7aae54353fcc138556583b8d13fb58eecab874d" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "0a531dd1fa8b88198108431c2b5b18b716c54c3ca21009988fdf3b124f8c8b34" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p1-quant-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p1-dense-tokens.txt": { + "bytes": 13, + "sha256": "aa2f972d1e3e49df1d62c1d7826445b1e903c676cf99ba5ceb09260d73c0c06a" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_0-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "66dc92bde7f36bfe38e063e65cb6839de5f8c6bc68fb8bb1b917a4973eaceb89" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r0-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r1-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p0-quant-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p0-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p0-dense-tokens.txt": { + "bytes": 12, + "sha256": "0e6d7a84c0e87f20a39e21f1e53934d4f181853528a5790bf913310a04f867b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p0-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "68d9adf067366233488d694005a9a12d4dc24f1cf7f8d2e2d99037836a7623c2" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p1-quant-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p1-quant-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p1-dense-tokens.txt": { + "bytes": 12, + "sha256": "aad9614465f58ebdd11b2a41a463dfcdba65061f548a10f6d97f2d72d60dc49c" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/public-captures/Q8_K-r2-p1-dense-logits-f32.bin": { + "bytes": 2048, + "sha256": "f529263c252e85898137bc6bd682e77a0f8c17e35f051d81b6234537a3ab63c8" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/profiler/trace_1253121_kernel_trace.csv": { + "bytes": 7862752, + "sha256": "9885b2818a116f3b1875e34eacda0091a903e79111eaa09234ce51d8aa47602b" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/profiler/trace_1253121_hip_api_trace.csv": { + "bytes": 21364430, + "sha256": "ee8cb2a85284839f93019e909b972888a4443d2c9e5899d53cb14a6ab7f414db" + }, + "/home/vikash/.cache/rdna3-gather-impl/public-model-trace-v1/profiler/trace_1253121_memory_allocation_trace.csv": { + "bytes": 1449190, + "sha256": "033b8f3eae261a41cac48f9c2fb23db9625b99d4d842e40b2ed8cdd4c19c3699" + } + }, + "native_model_memory": { + "observed_bf16_cache_reads": 912, + "observed_two_byte_cache_writes": 912, + "observed_direct_bf16_quantized_gathers": 456, + "observed_16384_byte_hipmalloc_allocations": 228, + "source_attribution": "runner.cpp:1571-1574 allocates num_blocks*page_size; kv_cache_interface.cpp:114-116 sizes16*1*(64+64)*2=4096 bytes/page; four pages=16384", + "attribution_limit": "The profiler records allocation size, not its semantic buffer name; its one-per-engine match is read with the cache source chain", + "oracle_model_memory_parity": "PENDING: primary refuses, secondary models abort and pad64cells to256" + }, + "pass": true +} diff --git a/docs/bench-evidence/rocm-quant-gather/public-red-after-prerequisite-v1.json b/docs/bench-evidence/rocm-quant-gather/public-red-after-prerequisite-v1.json new file mode 100644 index 0000000000..fa4b20a6d2 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/public-red-after-prerequisite-v1.json @@ -0,0 +1,58 @@ +{ + "version": 1, + "row": "BACKEND-ROCM-QUANT-GATHER", + "head": "4fc300f93cfb075c0afddccdd476a552fa900cdb", + "prerequisite_original_head": "6a7bcb77637e66df34429208e3a4055e0945a875", + "files": { + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/capi/rocm_quant_gather_fixture.h": { + "bytes": 8283, + "sha256": "339414f93e59e6be9a4ca545d5e762bb2816714343f466e218dbc2b69483bdfd" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/capi/test_rocm_embedding_quant.cpp": { + "bytes": 6579, + "sha256": "add58ac23eef26588fb358d7679feeac961baee3a1da3d5ddd3adedaf9a1a825" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/CMakeLists.txt": { + "bytes": 274633, + "sha256": "c97dc1e2fa7e70bb8c579a71d22fc7884d98bf6275d0c1ac6d8c5908ee4e4b93" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vllm/model_executor/models/qwen3_5.cpp": { + "bytes": 646312, + "sha256": "899913b8aeff65a08eb5289f6f08777ee78638e37722f9d8138daff723f8f498" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vt/rocm/rocm_ops.hip": { + "bytes": 22510, + "sha256": "c05bddd75520fa8e418313ce3bd05957b7135e106bab16f47fb3fd6220d8eb0f" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/libvllm.a": { + "bytes": 94251376, + "sha256": "a6173ff1ce88eff74be323584295f7e07abd52142e559190dd213916f38f0bc3" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_capi_rocm_embedding_quant": { + "bytes": 30196792, + "sha256": "067d0c228b22ed88b84dc1bc5d53f5fa5aacd44731e618b19d9a343673930885" + }, + "/home/vikash/.cache/rdna3-gather-impl/fixtures/Q4_0.gguf": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/fixtures/Q4_0-dense.gguf": { + "bytes": 991296, + "sha256": "0e6554ba521edfde00d4d025a3058eabaaacdce6b24344f959e83d8dde35df7f" + } + }, + "cwd": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp" + }, + "argv": [ + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_capi_rocm_embedding_quant", + "--abort-after=1" + ], + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/public-red-after-prerequisite.log", + "authority": "Operator alone executes under the recorded GPU 0 mutex", + "expected_red": "Public load and completion succeed, then positive kEmbeddingQuant ROCm selection assertion fails because registration is absent. Any failure before successful completion is a wrong-reason red.", + "control": "Prerequisite operator rerun passed this exact Q4_0-dense.gguf geometry and bytes. This red test checks quant provider stats before entering its dense Run arm, so --abort-after=1 does not itself rerun that dense control. The green test later executes both arms for all formats." +} diff --git a/docs/bench-evidence/rocm-quant-gather/public-red-after-prerequisite.operator-receipt.json b/docs/bench-evidence/rocm-quant-gather/public-red-after-prerequisite.operator-receipt.json new file mode 100644 index 0000000000..561d2c6ded --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/public-red-after-prerequisite.operator-receipt.json @@ -0,0 +1,8 @@ +{ + "recipe_sha256": "903b67613b2af4c102960e297c1f47ec19c809cd13ce2f57e1fd16dcdbe7ef71", + "head": "4fc300f93cfb075c0afddccdd476a552fa900cdb", + "exit_code": 1, + "elapsed_seconds": 1.344334363937378, + "files_verified_before_after": true, + "log_sha256": "943fd719efb6ea84e4e833043c7e2aa376843ce458d3bf8ffab7936e3d98a5db" +} diff --git a/docs/bench-evidence/rocm-quant-gather/q8k-reader-red.json b/docs/bench-evidence/rocm-quant-gather/q8k-reader-red.json new file mode 100644 index 0000000000..8ef0320e13 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/q8k-reader-red.json @@ -0,0 +1,36 @@ +{ + "version": 1, + "head": "2e5adfb85a255528bffa5d3cdb016efb2bab70d8", + "argv": [ + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_keep_quant", + "--test-case=Q8_K GGUF reader preserves block geometry and packed bytes" + ], + "env": { + "HIP_VISIBLE_DEVICES": "-1", + "ROCR_VISIBLE_DEVICES": "-1" + }, + "exit": 1, + "failure": "GgufFile::Open rejects token_embd.weight GGML type15 before any value assertion", + "files": { + "/home/vikash/vllm.cpp-rdna3-gather-impl/src/vllm/model_executor/model_loader/gguf_reader.cpp": { + "bytes": 29750, + "sha256": "fe7e9033ea40452c96a66d6e71080ebb422edcad38c56845ed2769bdb3eecb77" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/vllm/test_gguf_keep_quant.cpp": { + "bytes": 134876, + "sha256": "65a5e5a40f19d8e264bc1bb341c2a8276a3719c59c7faa2c91b4f4b7bfe9dd92" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/libvllm.a": { + "bytes": 95078628, + "sha256": "6cdf9bf1e69b9061e798eb0f6a44675a96158e4473d53b3badebd9bef04789b0" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/build-rocm-quant-gather/tests/test_gguf_keep_quant": { + "bytes": 31354464, + "sha256": "196026cc9e0a5f90ce90bc3a4f9d2e23ba34b0822df3e8be108e4e29e0030c96" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/q8k-reader-red.log": { + "bytes": 751, + "sha256": "b11156a6174c030ddf8deccb4df8e809bd2c2d107c7fec1f9c9bdfae529eeea8" + } + } +} diff --git a/docs/bench-evidence/rocm-quant-gather/qualification-captures.tar.gz b/docs/bench-evidence/rocm-quant-gather/qualification-captures.tar.gz new file mode 100644 index 0000000000..659cad3b04 Binary files /dev/null and b/docs/bench-evidence/rocm-quant-gather/qualification-captures.tar.gz differ diff --git a/docs/bench-evidence/rocm-quant-gather/raw-logs.tar.gz b/docs/bench-evidence/rocm-quant-gather/raw-logs.tar.gz new file mode 100644 index 0000000000..3bb82937b9 Binary files /dev/null and b/docs/bench-evidence/rocm-quant-gather/raw-logs.tar.gz differ diff --git a/docs/bench-evidence/rocm-quant-gather/record-anchor-repair.json b/docs/bench-evidence/rocm-quant-gather/record-anchor-repair.json new file mode 100644 index 0000000000..9b46f59719 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/record-anchor-repair.json @@ -0,0 +1,12 @@ +{ + "base": "b46852d13f7c8776557dab170e02ebea50cdfdec", + "path": ".agents/engine-matrix.md", + "changed_rows": [ + "SPEC-MTP-GGUF", + "SPEC-DFLASH-GGUF" + ], + "unrelated_lines_byte_identical": true, + "baseline_unchanged": true, + "before_sha256": "699166f84e6987612df8f12f235758169f0da86ffbf305a2aaedf05c61d59ce9", + "after_sha256": "c621448ac7fcf479a1187ab7f33bd63dce5da48109a467e1a8712f9c008b9147" +} diff --git a/docs/bench-evidence/rocm-quant-gather/secondary-frozen-v1.json b/docs/bench-evidence/rocm-quant-gather/secondary-frozen-v1.json new file mode 100644 index 0000000000..3e10f075f3 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/secondary-frozen-v1.json @@ -0,0 +1,399 @@ +{ + "version": 1, + "row": "BACKEND-ROCM-QUANT-GATHER", + "source_checks": { + "stock": { + "pin": "10bf611e533d81f739128304991c5e133c6aebd8", + "verified_archive_members": 3425, + "all_source_bytes_match_pin": true + }, + "fork": { + "pin": "36fe8e1cc7f2b3b8c92fdda0ab07600141921786", + "verified_archive_members": 3317, + "all_source_bytes_match_pin": true + } + }, + "cwd": "/home/vikash/vllm.cpp-rdna3-gather-impl", + "env": { + "HIP_VISIBLE_DEVICES": "-1", + "ROCR_VISIBLE_DEVICES": "-1" + }, + "build_recipe": { + "source": "/home/vikash/vllm.cpp-rdna3-gather-impl/tools/rocm_quant_gather", + "configuration": "Release, CPU only, GGML_NATIVE=OFF, GGML_AVX2=ON, OpenMP=OFF, -j 4", + "stock_configure_log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-stock-configure.log", + "fork_configure_log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-fork-configure.log" + }, + "files": { + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-1024.packed": { + "bytes": 19456, + "sha256": "ae8e061519977e2ba7a77ae9d30e21193b1a009b5ed38739974e945de09b94c6" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-256.packed": { + "bytes": 4864, + "sha256": "af0b5e336987746fd49cd33c790b99eb1479c26f64b6679fa6c685e5934ba0c4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-1024.packed": { + "bytes": 69632, + "sha256": "929ef7d789f210f60cede40156f689734aaf36519cd1f43a82d44d17ae0adfb4" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-256.packed": { + "bytes": 17408, + "sha256": "baa828a753cc6f496f356271dad644976cd57d1218517e3f641fe56b7ec8db4a" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-1024.packed": { + "bytes": 149504, + "sha256": "78e39e7ef7317d842a6fd965b356ceb1959a5ce03da824fbc699689ea2ef4877" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-256.packed": { + "bytes": 37376, + "sha256": "bcf6f6691a787fe3f2a9bcc3d7923eeb4953862c30f904a6b10fc07410f8d657" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-IQ1_XXXS-1024-f32.bin": { + "bytes": 16384, + "sha256": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-IQ1_XXXS-1024-gather.log": { + "bytes": 175, + "sha256": "599b4d6f937fed29361721ccf5c43f664bee63b25cd188bfdf516504f934ba8a" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-IQ1_XXXS-256-f32.bin": { + "bytes": 4096, + "sha256": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-IQ1_XXXS-256-gather.log": { + "bytes": 172, + "sha256": "56e3aed3c87e44b47a9b7c10ae461147d8a773415fcf38934483d53118f1d1f7" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-MXFP4-1024-f32.bin": { + "bytes": 16384, + "sha256": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-MXFP4-1024-gather.log": { + "bytes": 172, + "sha256": "c5b9d00254be3adbd6fc7e9edede85a276886b03e3bbc9b6a8b9ca39e0f61302" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-MXFP4-256-f32.bin": { + "bytes": 4096, + "sha256": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-MXFP4-256-gather.log": { + "bytes": 170, + "sha256": "ba6cbc0912a63378027e30d26682fa8f735ecbd37b6a01dd20c2b0534ffebc9e" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-Q8_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-Q8_K-1024-gather.log": { + "bytes": 163, + "sha256": "dd9a8f230fb6110feb052fa69a8846c2b11abaf82fa71aaef44bcc91e0b18fd5" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-Q8_K-256-f32.bin": { + "bytes": 4096, + "sha256": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-Q8_K-256-gather.log": { + "bytes": 160, + "sha256": "45c7ce06e27dd83e7ddb03e8bc2183180162b8c964f3da43de1bb0fcb9ddc51b" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-fork-model-iq1xxxs-p0-r0.log": { + "bytes": 12874, + "sha256": "5a1d15aa6566013e50f6e7103aa3910669de04534875c31bfb97f4b02f5a4096" + }, + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-stock-model-mxfp4-p0-r0.log": { + "bytes": 13005, + "sha256": "189f067f48a023502aba88ffac394b07df6ba4bf3af0d228a728a834ea58d822" + }, + "/home/vikash/.cache/rdna3-gather-impl/fixtures/IQ1_XXXS.gguf": { + "bytes": 930624, + "sha256": "3d16b36474ae4e4663f1bc4842d9bfe8e62843491d3e6f5724173c04495e2c72" + }, + "/home/vikash/.cache/rdna3-gather-impl/fixtures/MXFP4.gguf": { + "bytes": 943168, + "sha256": "e60f444269c54001579cb5bd2ad652b00ac6070984f62ca4a88450e36e8260de" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/oracle/ggml/src/libggml-base.a": { + "bytes": 1395314, + "sha256": "907d9cb9052e243f2024009355c927ad5ef70d6cbfdec7bbd9b061615ca26f48" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/oracle/ggml/src/libggml-cpu.a": { + "bytes": 1495086, + "sha256": "4d6349f1557b0e6d46ca6a1ad4de2608fc60a584a62eaef836865ef86e4c5ba8" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/oracle/ggml/src/libggml.a": { + "bytes": 61932, + "sha256": "713ed875969a0f4a38bf41cb916c88e2a49539949c29759164006a1bfcd664dc" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/oracle/src/libllama.a": { + "bytes": 9770360, + "sha256": "1e02daf9bdf3c7b822181c1e3f3d1b8f7bdfaae68a9c72ab00a0bf3045693145" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/rocm_gather_secondary": { + "bytes": 5715184, + "sha256": "cad537f239dc0c4ca70e5b80548f39f297117494269ca2b9b5de8191aa2f4bc1" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-source/ggml/src/ggml-backend.cpp": { + "bytes": 96019, + "sha256": "507577061d22e673a4f002bb215c65aa9c63b88a4e6d1586f52c74ca0b98fa07" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-source/ggml/src/ggml-cpu/ops.cpp": { + "bytes": 411897, + "sha256": "42f40dbddddef1cfee2c441b7a2522aa2b0f2e11d76037a4d8fc49364a13cb2b" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-source/ggml/src/ggml-quants.c": { + "bytes": 243330, + "sha256": "f60b82ed4387ed73cc7a42ad57180cc084ea79f72a7211f9f6a2b861a710624b" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-source/src/llama-graph.cpp": { + "bytes": 132879, + "sha256": "e7d4915cc66bc6187f42ad016f9b1288f8e1702e15da4e63d9862cf8afcb8267" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-source/src/models/qwen35.cpp": { + "bytes": 28474, + "sha256": "b61c5813fc80c54269d45cbbdbbc34b090500e8017dc190483605b37249d007c" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-source.tar": { + "bytes": 170752000, + "sha256": "be6761fc65927e10aa1cad664ca5a459fa0f3776c30230cd7acff564c981f93e" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/oracle/ggml/src/libggml-base.a": { + "bytes": 1324458, + "sha256": "62f639da8b2865775065084f8a38134604b75416204d664cd0991d019f4faba1" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/oracle/ggml/src/libggml-cpu.a": { + "bytes": 1476372, + "sha256": "4118662b842e15bf9d7ab2e018182012027a604ed97836a82a153e04a6d14fa0" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/oracle/ggml/src/libggml.a": { + "bytes": 61932, + "sha256": "cef0c83e3f08e03fd19650f35d6bb39666eecf1ab88ade63aaa3ec2170a95b59" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/oracle/src/libllama.a": { + "bytes": 10039328, + "sha256": "b3672ed0aae46e1c6cf2dfd500f6fcf3f37f1be7ca372a336a43583f4686f7f7" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary": { + "bytes": 5736280, + "sha256": "7b9a2e2d0dc95afbf592b5b1678bdc67e0fe2204529f1afe3b0b1f03566f28c2" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/ggml/src/ggml-backend.cpp": { + "bytes": 96019, + "sha256": "507577061d22e673a4f002bb215c65aa9c63b88a4e6d1586f52c74ca0b98fa07" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/ggml/src/ggml-cpu/ops.cpp": { + "bytes": 411743, + "sha256": "740701e585872545650edfd08c9114619a9ad1e67dbab19f6b2060b6290fdf88" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/ggml/src/ggml-quants.c": { + "bytes": 228250, + "sha256": "07143d7068936ae46b3c528b2f3d4bbb666e74d88992165716174d243573965d" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/src/llama-graph.cpp": { + "bytes": 134601, + "sha256": "bc1c497c4da0951a4f98de50e397bcb632b2e40d4f24b8c58203c964d4d6183b" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/src/models/qwen35.cpp": { + "bytes": 28474, + "sha256": "b61c5813fc80c54269d45cbbdbbc34b090500e8017dc190483605b37249d007c" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-source.tar": { + "bytes": 171663360, + "sha256": "0f43551d3da94ba0d2af80e202177008a91966de66dbabf87a2042377d16d133" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tests/capi/rocm_quant_gather_fixture.h": { + "bytes": 8283, + "sha256": "339414f93e59e6be9a4ca545d5e762bb2816714343f466e218dbc2b69483bdfd" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tools/rocm_quant_gather/CMakeLists.txt": { + "bytes": 1179, + "sha256": "c90a83b40ffd31db1228324827422eda62d4a8ab62907d9ba866dca4cc106104" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tools/rocm_quant_gather/secondary.cpp": { + "bytes": 6817, + "sha256": "09773359f3db39d5c759d03e37e42f4e466f6e33cb063cd2eb9bbf1353b04dbb" + } + }, + "cases": [ + { + "name": "Q8_K-256", + "oracle": "stock", + "kind": "operation", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary", + "gather", + "15", + "128", + "256", + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-256.packed", + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-256.ids", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-Q8_K-256-f32.bin" + ], + "observed_exit": 0, + "expected": "Successful upstream graph or exact Q8_K decoder; identical output SHA256", + "operator_output_note": "Use a new output path to preserve implementer output bytes", + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-Q8_K-256-gather.log" + }, + { + "name": "Q8_K-1024", + "oracle": "stock", + "kind": "operation", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary", + "gather", + "15", + "128", + "1024", + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-1024.packed", + "/home/vikash/.cache/rdna3-gather-impl/corpus/Q8_K-1024.ids", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-Q8_K-1024-f32.bin" + ], + "observed_exit": 0, + "expected": "Successful upstream graph or exact Q8_K decoder; identical output SHA256", + "operator_output_note": "Use a new output path to preserve implementer output bytes", + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-Q8_K-1024-gather.log" + }, + { + "name": "MXFP4-256", + "oracle": "stock", + "kind": "operation", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary", + "gather", + "39", + "128", + "256", + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-256.packed", + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-256.ids", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-MXFP4-256-f32.bin" + ], + "observed_exit": 0, + "expected": "Successful upstream graph or exact Q8_K decoder; identical output SHA256", + "operator_output_note": "Use a new output path to preserve implementer output bytes", + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-MXFP4-256-gather.log" + }, + { + "name": "MXFP4-1024", + "oracle": "stock", + "kind": "operation", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary", + "gather", + "39", + "128", + "1024", + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-1024.packed", + "/home/vikash/.cache/rdna3-gather-impl/corpus/MXFP4-1024.ids", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-MXFP4-1024-f32.bin" + ], + "observed_exit": 0, + "expected": "Successful upstream graph or exact Q8_K decoder; identical output SHA256", + "operator_output_note": "Use a new output path to preserve implementer output bytes", + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-MXFP4-1024-gather.log" + }, + { + "name": "IQ1_XXXS-256", + "oracle": "fork", + "kind": "operation", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/rocm_gather_secondary", + "gather", + "66", + "128", + "256", + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-256.packed", + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-256.ids", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-IQ1_XXXS-256-f32.bin" + ], + "observed_exit": 0, + "expected": "Successful upstream graph or exact Q8_K decoder; identical output SHA256", + "operator_output_note": "Use a new output path to preserve implementer output bytes", + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-IQ1_XXXS-256-gather.log" + }, + { + "name": "IQ1_XXXS-1024", + "oracle": "fork", + "kind": "operation", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/rocm_gather_secondary", + "gather", + "66", + "128", + "1024", + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-1024.packed", + "/home/vikash/.cache/rdna3-gather-impl/corpus/IQ1_XXXS-1024.ids", + "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-IQ1_XXXS-1024-f32.bin" + ], + "observed_exit": 0, + "expected": "Successful upstream graph or exact Q8_K decoder; identical output SHA256", + "operator_output_note": "Use a new output path to preserve implementer output bytes", + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-IQ1_XXXS-1024-gather.log" + }, + { + "name": "MXFP4-model-p0-r0", + "oracle": "stock", + "kind": "model", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary", + "model", + "/home/vikash/.cache/rdna3-gather-impl/fixtures/MXFP4.gguf", + "1", + "0", + "63", + "127", + "63" + ], + "observed_shell_exit": 134, + "observed_signal": 6, + "expected": "Model loads; first llama_decode aborts at ggml-backend.cpp:194 GGML_ASSERT(buffer)", + "inference": "Unallocated recurrent inp_rs->s_copy->buffer, reached by llm_graph_input_mem_hybrid::set_input; the full-attention graph does not consume this input", + "gate": "PENDING under #3093; no model token result", + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-stock-model-mxfp4-p0-r0.log" + }, + { + "name": "IQ1_XXXS-model-p0-r0", + "oracle": "fork", + "kind": "model", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/rocm_gather_secondary", + "model", + "/home/vikash/.cache/rdna3-gather-impl/fixtures/IQ1_XXXS.gguf", + "1", + "0", + "63", + "127", + "63" + ], + "observed_shell_exit": 134, + "observed_signal": 6, + "expected": "Model loads; first llama_decode aborts at ggml-backend.cpp:194 GGML_ASSERT(buffer)", + "inference": "Unallocated recurrent inp_rs->s_copy->buffer, reached by llm_graph_input_mem_hybrid::set_input; the full-attention graph does not consume this input", + "gate": "PENDING under #3093; no model token result", + "log": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-fork-model-iq1xxxs-p0-r0.log" + } + ] +} diff --git a/docs/bench-evidence/rocm-quant-gather/secondary-operation-captures.tar.gz b/docs/bench-evidence/rocm-quant-gather/secondary-operation-captures.tar.gz new file mode 100644 index 0000000000..ac684665f7 Binary files /dev/null and b/docs/bench-evidence/rocm-quant-gather/secondary-operation-captures.tar.gz differ diff --git a/docs/bench-evidence/rocm-quant-gather/stock-shapes-v1-commands.json b/docs/bench-evidence/rocm-quant-gather/stock-shapes-v1-commands.json new file mode 100644 index 0000000000..fc388c0208 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/stock-shapes-v1-commands.json @@ -0,0 +1,108 @@ +{ + "files": { + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/manifest.json": { + "bytes": 1126, + "sha256": "9e1536b648a31c31262f0dfa3f01ce3693b747f7472972310649a4d19ce9cc4e" + }, + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/IQ1_XXXS-256.packed": { + "bytes": 190, + "sha256": "689284b70f0d1d4377bd6bc5cb607377862cdcce28b1bdaa3ee1b41c51e5c127" + }, + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/MXFP4-256.packed": { + "bytes": 680, + "sha256": "5f887e137c3fe120e2044e675622696b5ccd6603d5b46a18b640ba5abb8068e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/Q8_K-256.packed": { + "bytes": 1460, + "sha256": "81c780fb2f98e9d3ec7add68960cc5856b10de665f3e8b3a88b21e944664c97e" + }, + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/Q8_K-256.ids": { + "bytes": 16, + "sha256": "0b975a0d33e8d060ec1659248177222f2cf938ebf8b0439a70996d3a7f8fdf89" + }, + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/IQ1_XXXS-256.ids": { + "bytes": 16, + "sha256": "0b975a0d33e8d060ec1659248177222f2cf938ebf8b0439a70996d3a7f8fdf89" + }, + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/MXFP4-256.ids": { + "bytes": 16, + "sha256": "0b975a0d33e8d060ec1659248177222f2cf938ebf8b0439a70996d3a7f8fdf89" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native-capture": { + "bytes": 30693264, + "sha256": "2569a80c1bddec3ebd9332e1ca2cfb9d316df5db6e8afe17a51ab66d09438698" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary": { + "bytes": 5736280, + "sha256": "7b9a2e2d0dc95afbf592b5b1678bdc67e0fe2204529f1afe3b0b1f03566f28c2" + }, + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/rocm_gather_secondary": { + "bytes": 5715184, + "sha256": "cad537f239dc0c4ca70e5b80548f39f297117494269ca2b9b5de8191aa2f4bc1" + }, + "/home/vikash/vllm.cpp-rdna3-gather-impl/tools/rocm_quant_gather/stock_shapes.py": { + "bytes": 1961, + "sha256": "e09a1ae74a1e55193bcae982606e68ec14a86ff9e98192c729d0aafe487c5321" + } + }, + "secondary_runs": [ + { + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary", + "gather", + "15", + "5", + "256", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/Q8_K-256.packed", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/Q8_K-256.ids", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/secondary/Q8_K-256-f32.bin" + ], + "exit_code": 0, + "log": "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/secondary/Q8_K-256.log" + }, + { + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary", + "gather", + "39", + "5", + "256", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/MXFP4-256.packed", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/MXFP4-256.ids", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/secondary/MXFP4-256-f32.bin" + ], + "exit_code": 0, + "log": "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/secondary/MXFP4-256.log" + }, + { + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/rocm_gather_secondary", + "gather", + "66", + "5", + "256", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/IQ1_XXXS-256.packed", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/IQ1_XXXS-256.ids", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/secondary/IQ1_XXXS-256-f32.bin" + ], + "exit_code": 0, + "log": "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/secondary/IQ1_XXXS-256.log" + } + ], + "native": { + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native-capture", + "capture", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/native" + ], + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0" + }, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1", + "log": "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/native.log" + }, + "gate": "Stock test_get_rows applicable width256 rows5 four IDs contiguous unbatched. Q8_K uses upstream direct decoder; MXFP4 and IQ1_XXXS use real graphs. f32 exact; bf16 explicit RNE.", + "authority": "Only root operator runs native under GPU0 mutex; secondary recipes are CPU only" +} diff --git a/docs/bench-evidence/rocm-quant-gather/stock-shapes-v1-operator-results.json b/docs/bench-evidence/rocm-quant-gather/stock-shapes-v1-operator-results.json new file mode 100644 index 0000000000..d1c2497910 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/stock-shapes-v1-operator-results.json @@ -0,0 +1,118 @@ +{ + "manifest_sha256": "35448ef5b3fc457ff3738963de27bc11a312c3d92ceefb58ba91b51083c966a5", + "input_count": 11, + "runs": [ + { + "name": "Q8_K-256-f32", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary", + "gather", + "15", + "5", + "256", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/Q8_K-256.packed", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/Q8_K-256.ids", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/operator/Q8_K-256-f32.bin" + ], + "exit_code": 0, + "seconds": 0.03152322769165039, + "log_sha256": "d4ee090db5ac6f6d11a19ca28a6655084d88b9254f9c6fa18433339f17a404c1", + "all_input_hashes_verified_before_after": true + }, + { + "name": "MXFP4-256-f32", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-stock-build/rocm_gather_secondary", + "gather", + "39", + "5", + "256", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/MXFP4-256.packed", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/MXFP4-256.ids", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/operator/MXFP4-256-f32.bin" + ], + "exit_code": 0, + "seconds": 0.0353548526763916, + "log_sha256": "6e6c539254bcb2836e72cf3e702ac5c6bb025e60f2e7bf64b8f18beddb3aa68c", + "all_input_hashes_verified_before_after": true + }, + { + "name": "IQ1_XXXS-256-f32", + "argv": [ + "/home/vikash/.cache/rdna3-gather-impl/llama-fork-build/rocm_gather_secondary", + "gather", + "66", + "5", + "256", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/IQ1_XXXS-256.packed", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/IQ1_XXXS-256.ids", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/operator/IQ1_XXXS-256-f32.bin" + ], + "exit_code": 0, + "seconds": 0.043924570083618164, + "log_sha256": "f8f8d7463ee1473288e775e37d905afbf35c43b7ff6443e3ed90ea6234376c7c", + "all_input_hashes_verified_before_after": true + }, + { + "name": "native", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native-capture", + "capture", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/inputs/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/stock-shapes-v1/operator/native" + ], + "exit_code": 0, + "seconds": 0.3421025276184082, + "log_sha256": "7740d959f71ecb2717d160fa60769c2191ffa1d73e906b3a10e584d203f9c928", + "all_input_hashes_verified_before_after": true + } + ], + "comparison": [ + { + "name": "Q8_K-256-f32.bin", + "native_sha256": "bc63b12de49c2573302b712ab0528c177ddb486f559d470a0aca9e7f16ba7996", + "upstream_f32_sha256": "bc63b12de49c2573302b712ab0528c177ddb486f559d470a0aca9e7f16ba7996", + "exact": true, + "adaptation": null + }, + { + "name": "Q8_K-256-bf16.bin", + "native_sha256": "4da3baa7408806c94c0fef1baedbe59dcae9d85d00567880c8f689b951d7e683", + "upstream_f32_sha256": "bc63b12de49c2573302b712ab0528c177ddb486f559d470a0aca9e7f16ba7996", + "exact": true, + "adaptation": "F32 upstream rounded once to BF16 RNE" + }, + { + "name": "MXFP4-256-f32.bin", + "native_sha256": "da5bd348b0562899e0be685b723f9a26d43b405a46e00d9667f22bd93c73a9b4", + "upstream_f32_sha256": "da5bd348b0562899e0be685b723f9a26d43b405a46e00d9667f22bd93c73a9b4", + "exact": true, + "adaptation": null + }, + { + "name": "MXFP4-256-bf16.bin", + "native_sha256": "ef2052053825cf4308309213f6bc146915e499716bf44434cdc421957379b90c", + "upstream_f32_sha256": "da5bd348b0562899e0be685b723f9a26d43b405a46e00d9667f22bd93c73a9b4", + "exact": true, + "adaptation": "F32 upstream rounded once to BF16 RNE" + }, + { + "name": "IQ1_XXXS-256-f32.bin", + "native_sha256": "bccfb71bc3b329ff21d4061d4352e0b34958640822a8248ee65045b6af762212", + "upstream_f32_sha256": "bccfb71bc3b329ff21d4061d4352e0b34958640822a8248ee65045b6af762212", + "exact": true, + "adaptation": null + }, + { + "name": "IQ1_XXXS-256-bf16.bin", + "native_sha256": "813221d5039f686ec01be2513e48df24bd21a0a20466efe08407e65dc5837062", + "upstream_f32_sha256": "bccfb71bc3b329ff21d4061d4352e0b34958640822a8248ee65045b6af762212", + "exact": true, + "adaptation": "F32 upstream rounded once to BF16 RNE" + } + ] +} diff --git a/docs/bench-evidence/rocm-quant-gather/synthetic-parity-v2.json b/docs/bench-evidence/rocm-quant-gather/synthetic-parity-v2.json new file mode 100644 index 0000000000..a7844717bd --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/synthetic-parity-v2.json @@ -0,0 +1,1835 @@ +{ + "manifest": { + "bytes": 6329, + "sha256": "78bc808e19b40ea745f5e767b72618dbd8087bb685333345826257dd4371ccfd" + }, + "cases": [ + { + "name": "Q4_0-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 18432, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-256-f32.bin", + "bytes": 4096, + "sha256": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-256-f32.bin", + "bytes": 4096, + "sha256": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4" + } + }, + { + "name": "Q4_0-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 18432, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-256-bf16.bin", + "bytes": 2048, + "sha256": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-256-bf16.bin", + "bytes": 2048, + "sha256": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7" + } + }, + { + "name": "Q4_0-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 73728, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-1024-f32.bin", + "bytes": 16384, + "sha256": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-1024-f32.bin", + "bytes": 16384, + "sha256": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32" + } + }, + { + "name": "Q4_0-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 73728, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-1024-bf16.bin", + "bytes": 8192, + "sha256": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-1024-bf16.bin", + "bytes": 8192, + "sha256": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca" + } + }, + { + "name": "Q5_0-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 22528, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-256-f32.bin", + "bytes": 4096, + "sha256": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-256-f32.bin", + "bytes": 4096, + "sha256": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d" + } + }, + { + "name": "Q5_0-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 22528, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-256-bf16.bin", + "bytes": 2048, + "sha256": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-256-bf16.bin", + "bytes": 2048, + "sha256": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b" + } + }, + { + "name": "Q5_0-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 90112, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-1024-f32.bin", + "bytes": 16384, + "sha256": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-1024-f32.bin", + "bytes": 16384, + "sha256": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3" + } + }, + { + "name": "Q5_0-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 90112, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-1024-bf16.bin", + "bytes": 8192, + "sha256": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-1024-bf16.bin", + "bytes": 8192, + "sha256": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed" + } + }, + { + "name": "Q8_0-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 34816, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-256-f32.bin", + "bytes": 4096, + "sha256": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-256-f32.bin", + "bytes": 4096, + "sha256": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc" + } + }, + { + "name": "Q8_0-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 34816, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-256-bf16.bin", + "bytes": 2048, + "sha256": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-256-bf16.bin", + "bytes": 2048, + "sha256": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb" + } + }, + { + "name": "Q8_0-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 139264, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-1024-f32.bin", + "bytes": 16384, + "sha256": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-1024-f32.bin", + "bytes": 16384, + "sha256": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc" + } + }, + { + "name": "Q8_0-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 139264, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-1024-bf16.bin", + "bytes": 8192, + "sha256": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-1024-bf16.bin", + "bytes": 8192, + "sha256": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f" + } + }, + { + "name": "Q2_K-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 10752, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-256-f32.bin", + "bytes": 4096, + "sha256": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-256-f32.bin", + "bytes": 4096, + "sha256": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4" + } + }, + { + "name": "Q2_K-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 10752, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-256-bf16.bin", + "bytes": 2048, + "sha256": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-256-bf16.bin", + "bytes": 2048, + "sha256": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e" + } + }, + { + "name": "Q2_K-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 43008, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-1024-f32.bin", + "bytes": 16384, + "sha256": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-1024-f32.bin", + "bytes": 16384, + "sha256": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b" + } + }, + { + "name": "Q2_K-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 43008, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1" + } + }, + { + "name": "Q3_K-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 14080, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-256-f32.bin", + "bytes": 4096, + "sha256": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-256-f32.bin", + "bytes": 4096, + "sha256": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c" + } + }, + { + "name": "Q3_K-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 14080, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-256-bf16.bin", + "bytes": 2048, + "sha256": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-256-bf16.bin", + "bytes": 2048, + "sha256": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d" + } + }, + { + "name": "Q3_K-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 56320, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-1024-f32.bin", + "bytes": 16384, + "sha256": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-1024-f32.bin", + "bytes": 16384, + "sha256": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d" + } + }, + { + "name": "Q3_K-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 56320, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e" + } + }, + { + "name": "Q4_K-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 18432, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-256-f32.bin", + "bytes": 4096, + "sha256": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-256-f32.bin", + "bytes": 4096, + "sha256": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780" + } + }, + { + "name": "Q4_K-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 18432, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-256-bf16.bin", + "bytes": 2048, + "sha256": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-256-bf16.bin", + "bytes": 2048, + "sha256": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023" + } + }, + { + "name": "Q4_K-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 73728, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-1024-f32.bin", + "bytes": 16384, + "sha256": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-1024-f32.bin", + "bytes": 16384, + "sha256": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76" + } + }, + { + "name": "Q4_K-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 73728, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370" + } + }, + { + "name": "Q5_K-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 22528, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-256-f32.bin", + "bytes": 4096, + "sha256": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-256-f32.bin", + "bytes": 4096, + "sha256": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb" + } + }, + { + "name": "Q5_K-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 22528, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-256-bf16.bin", + "bytes": 2048, + "sha256": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-256-bf16.bin", + "bytes": 2048, + "sha256": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a" + } + }, + { + "name": "Q5_K-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 90112, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-1024-f32.bin", + "bytes": 16384, + "sha256": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-1024-f32.bin", + "bytes": 16384, + "sha256": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14" + } + }, + { + "name": "Q5_K-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 90112, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab" + } + }, + { + "name": "Q6_K-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 26880, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": false, + "max_absolute_error": 0.00048828125, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-256-f32.bin", + "bytes": 4096, + "sha256": "3e1665a3167a552c47493bdc538166acfa6cbd43a1e7dd30230c40e3c5b11207" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-256-f32.bin", + "bytes": 4096, + "sha256": "955ba972e48a303803cdacca928e7f60176dfaa1b7f8590038bdb91830658428" + } + }, + { + "name": "Q6_K-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 26880, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": false, + "max_absolute_error": 0.0078125, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-256-bf16.bin", + "bytes": 2048, + "sha256": "242d1b322588618517ee3d4d62e393d08b2faca2262414ab00d89d092598e8ee" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-256-bf16.bin", + "bytes": 2048, + "sha256": "c65412c7615787b9103207f55776d49dfda767740f4d6e1106cb711a99404bc6" + } + }, + { + "name": "Q6_K-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 107520, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": false, + "max_absolute_error": 0.00048828125, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-1024-f32.bin", + "bytes": 16384, + "sha256": "158400e054e65af2adad3e7d348b2fc3e082fcfc065706db200704d3c86a2581" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-1024-f32.bin", + "bytes": 16384, + "sha256": "5a0f2bb0da03f1eb5c689c6f95be636890aec44873bffe77dd039c2b5bc6c20a" + } + }, + { + "name": "Q6_K-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 107520, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": false, + "max_absolute_error": 0.0078125, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "54c4ea60561a742634c5fdf24c4dd372f0e101a48bf29b53d53a0b7e88f31aea" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "5955c913676e2e5003f84ad0dca7062d3f7106188a13e4844b13774a2764c238" + } + }, + { + "name": "Q8_K-256", + "dtype": "f32", + "oracle": "stock", + "elements": 1024, + "table_bytes": 37376, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_K-256-f32.bin", + "bytes": 4096, + "sha256": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/Q8_K-256-f32.bin", + "bytes": 4096, + "sha256": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6" + } + }, + { + "name": "Q8_K-256", + "dtype": "bf16", + "oracle": "stock", + "elements": 1024, + "table_bytes": 37376, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_K-256-bf16.bin", + "bytes": 2048, + "sha256": "f5da512dcaf2f4165abd5b4b1377d275a5945d2ab8946d2d8266600c5b03d31f" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/Q8_K-256-f32.bin", + "bytes": 4096, + "sha256": "42d365370201b210a4c646cc5718a3bf7becff7ef20bb5f381a1736ba22850e6" + } + }, + { + "name": "Q8_K-1024", + "dtype": "f32", + "oracle": "stock", + "elements": 4096, + "table_bytes": 149504, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_K-1024-f32.bin", + "bytes": 16384, + "sha256": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/Q8_K-1024-f32.bin", + "bytes": 16384, + "sha256": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c" + } + }, + { + "name": "Q8_K-1024", + "dtype": "bf16", + "oracle": "stock", + "elements": 4096, + "table_bytes": 149504, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_K-1024-bf16.bin", + "bytes": 8192, + "sha256": "6a1aa0cd72dca3f6ae3f6b835eeba50a87bf7b143ad607baa30482f0835f8c22" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/Q8_K-1024-f32.bin", + "bytes": 16384, + "sha256": "bc4c56d33b3d53d928f21b39953ba08f191aec465ead263abb832820f6ccd29c" + } + }, + { + "name": "IQ2_XXS-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 8448, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-256-f32.bin", + "bytes": 4096, + "sha256": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-256-f32.bin", + "bytes": 4096, + "sha256": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432" + } + }, + { + "name": "IQ2_XXS-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 8448, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-256-bf16.bin", + "bytes": 2048, + "sha256": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-256-bf16.bin", + "bytes": 2048, + "sha256": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b" + } + }, + { + "name": "IQ2_XXS-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 33792, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-1024-f32.bin", + "bytes": 16384, + "sha256": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-1024-f32.bin", + "bytes": 16384, + "sha256": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2" + } + }, + { + "name": "IQ2_XXS-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 33792, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-1024-bf16.bin", + "bytes": 8192, + "sha256": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-1024-bf16.bin", + "bytes": 8192, + "sha256": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c" + } + }, + { + "name": "IQ2_XS-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 9472, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-256-f32.bin", + "bytes": 4096, + "sha256": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-256-f32.bin", + "bytes": 4096, + "sha256": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa" + } + }, + { + "name": "IQ2_XS-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 9472, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-256-bf16.bin", + "bytes": 2048, + "sha256": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-256-bf16.bin", + "bytes": 2048, + "sha256": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db" + } + }, + { + "name": "IQ2_XS-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 37888, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-1024-f32.bin", + "bytes": 16384, + "sha256": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-1024-f32.bin", + "bytes": 16384, + "sha256": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd" + } + }, + { + "name": "IQ2_XS-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 37888, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-1024-bf16.bin", + "bytes": 8192, + "sha256": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-1024-bf16.bin", + "bytes": 8192, + "sha256": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5" + } + }, + { + "name": "IQ3_XXS-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 12544, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-256-f32.bin", + "bytes": 4096, + "sha256": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-256-f32.bin", + "bytes": 4096, + "sha256": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda" + } + }, + { + "name": "IQ3_XXS-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 12544, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-256-bf16.bin", + "bytes": 2048, + "sha256": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-256-bf16.bin", + "bytes": 2048, + "sha256": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631" + } + }, + { + "name": "IQ3_XXS-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 50176, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-1024-f32.bin", + "bytes": 16384, + "sha256": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-1024-f32.bin", + "bytes": 16384, + "sha256": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1" + } + }, + { + "name": "IQ3_XXS-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 50176, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-1024-bf16.bin", + "bytes": 8192, + "sha256": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-1024-bf16.bin", + "bytes": 8192, + "sha256": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44" + } + }, + { + "name": "IQ1_S-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 6400, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-256-f32.bin", + "bytes": 4096, + "sha256": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-256-f32.bin", + "bytes": 4096, + "sha256": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1" + } + }, + { + "name": "IQ1_S-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 6400, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-256-bf16.bin", + "bytes": 2048, + "sha256": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-256-bf16.bin", + "bytes": 2048, + "sha256": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090" + } + }, + { + "name": "IQ1_S-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 25600, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-1024-f32.bin", + "bytes": 16384, + "sha256": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-1024-f32.bin", + "bytes": 16384, + "sha256": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02" + } + }, + { + "name": "IQ1_S-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 25600, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-1024-bf16.bin", + "bytes": 8192, + "sha256": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-1024-bf16.bin", + "bytes": 8192, + "sha256": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9" + } + }, + { + "name": "IQ4_NL-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 18432, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-256-f32.bin", + "bytes": 4096, + "sha256": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-256-f32.bin", + "bytes": 4096, + "sha256": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e" + } + }, + { + "name": "IQ4_NL-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 18432, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-256-bf16.bin", + "bytes": 2048, + "sha256": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-256-bf16.bin", + "bytes": 2048, + "sha256": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9" + } + }, + { + "name": "IQ4_NL-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 73728, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-1024-f32.bin", + "bytes": 16384, + "sha256": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-1024-f32.bin", + "bytes": 16384, + "sha256": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516" + } + }, + { + "name": "IQ4_NL-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 73728, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-1024-bf16.bin", + "bytes": 8192, + "sha256": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-1024-bf16.bin", + "bytes": 8192, + "sha256": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3" + } + }, + { + "name": "IQ3_S-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 14080, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": false, + "max_absolute_error": 0.12109375, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-256-f32.bin", + "bytes": 4096, + "sha256": "7843ff13e1c7f1475f949583ec0466b7cd3020723994d7c725e7c618c11c34f0" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-256-f32.bin", + "bytes": 4096, + "sha256": "8cad98f9fecbfcac883cb11a91df1ac703846dfe88bac12792ab3066420243ad" + } + }, + { + "name": "IQ3_S-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 14080, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": false, + "max_absolute_error": 0.125, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-256-bf16.bin", + "bytes": 2048, + "sha256": "281888da0636914c257cd25407adc1465b2d40e96aa7694eb5c3bd0df4036b73" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-256-bf16.bin", + "bytes": 2048, + "sha256": "7f36f4a4ba9f0977f68d452e8ec1d26ea299ea9ddbc82fd0ede324abc94d6595" + } + }, + { + "name": "IQ3_S-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 56320, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": false, + "max_absolute_error": 0.12109375, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-1024-f32.bin", + "bytes": 16384, + "sha256": "8932dbeb65d3716cfb60014ad471ab0fb5d2d2d4b037e8745464e65f77b936de" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-1024-f32.bin", + "bytes": 16384, + "sha256": "ebcadd200e64857ed483aa7176172dc94b966198fb81d2910ab723260fb46fdc" + } + }, + { + "name": "IQ3_S-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 56320, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": false, + "max_absolute_error": 0.125, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-1024-bf16.bin", + "bytes": 8192, + "sha256": "11a954e23ccc31f98a3398902b7ce96ec39f0fc0119d41202732630299d9a623" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-1024-bf16.bin", + "bytes": 8192, + "sha256": "88e55dfa1c9378436bd431db09fca5bf5aad9910dc311b493d6f94d1ec98dc7c" + } + }, + { + "name": "IQ2_S-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 10496, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-256-f32.bin", + "bytes": 4096, + "sha256": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-256-f32.bin", + "bytes": 4096, + "sha256": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f" + } + }, + { + "name": "IQ2_S-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 10496, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-256-bf16.bin", + "bytes": 2048, + "sha256": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-256-bf16.bin", + "bytes": 2048, + "sha256": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195" + } + }, + { + "name": "IQ2_S-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 41984, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-1024-f32.bin", + "bytes": 16384, + "sha256": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-1024-f32.bin", + "bytes": 16384, + "sha256": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4" + } + }, + { + "name": "IQ2_S-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 41984, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-1024-bf16.bin", + "bytes": 8192, + "sha256": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-1024-bf16.bin", + "bytes": 8192, + "sha256": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907" + } + }, + { + "name": "IQ4_XS-256", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 17408, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-256-f32.bin", + "bytes": 4096, + "sha256": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-256-f32.bin", + "bytes": 4096, + "sha256": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2" + } + }, + { + "name": "IQ4_XS-256", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 1024, + "table_bytes": 17408, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-256-bf16.bin", + "bytes": 2048, + "sha256": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-256-bf16.bin", + "bytes": 2048, + "sha256": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc" + } + }, + { + "name": "IQ4_XS-1024", + "dtype": "f32", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 69632, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-1024-f32.bin", + "bytes": 16384, + "sha256": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-1024-f32.bin", + "bytes": 16384, + "sha256": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889" + } + }, + { + "name": "IQ4_XS-1024", + "dtype": "bf16", + "oracle": "primary-plugin", + "elements": 4096, + "table_bytes": 69632, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "atol=0.01 rtol=0.04, pinned upstream embedding test", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-1024-bf16.bin", + "bytes": 8192, + "sha256": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-1024-bf16.bin", + "bytes": 8192, + "sha256": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d" + } + }, + { + "name": "MXFP4-256", + "dtype": "f32", + "oracle": "stock", + "elements": 1024, + "table_bytes": 17408, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/MXFP4-256-f32.bin", + "bytes": 4096, + "sha256": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/MXFP4-256-f32.bin", + "bytes": 4096, + "sha256": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2" + } + }, + { + "name": "MXFP4-256", + "dtype": "bf16", + "oracle": "stock", + "elements": 1024, + "table_bytes": 17408, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/MXFP4-256-bf16.bin", + "bytes": 2048, + "sha256": "4b66106696144b4970929d2f5734d3ddf8e6e4580a2fe8910b500ebaab5431a9" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/MXFP4-256-f32.bin", + "bytes": 4096, + "sha256": "aca27b230e9675ea214c31723306036025295c570903f8804d4d767b4fb0bdb2" + } + }, + { + "name": "MXFP4-1024", + "dtype": "f32", + "oracle": "stock", + "elements": 4096, + "table_bytes": 69632, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/MXFP4-1024-f32.bin", + "bytes": 16384, + "sha256": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/MXFP4-1024-f32.bin", + "bytes": 16384, + "sha256": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec" + } + }, + { + "name": "MXFP4-1024", + "dtype": "bf16", + "oracle": "stock", + "elements": 4096, + "table_bytes": 69632, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/MXFP4-1024-bf16.bin", + "bytes": 8192, + "sha256": "cdd6e95bc3bc3e18bc014dee2cdb125cb179b670256896536b0e3822b0adfa83" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/MXFP4-1024-f32.bin", + "bytes": 16384, + "sha256": "a2688e4f8d766f642b8fbdebcbfff7e4e2720ae9fc57409812eb0a21f4bb6fec" + } + }, + { + "name": "IQ1_XXXS-256", + "dtype": "f32", + "oracle": "fork", + "elements": 1024, + "table_bytes": 4864, + "output_bytes": 4096, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_XXXS-256-f32.bin", + "bytes": 4096, + "sha256": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/IQ1_XXXS-256-f32.bin", + "bytes": 4096, + "sha256": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce" + } + }, + { + "name": "IQ1_XXXS-256", + "dtype": "bf16", + "oracle": "fork", + "elements": 1024, + "table_bytes": 4864, + "output_bytes": 2048, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_XXXS-256-bf16.bin", + "bytes": 2048, + "sha256": "9de69ca2d01fda18f05e9ee7ac531f3cb806d39a3af8d10e64ba0c53dab1b08a" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/IQ1_XXXS-256-f32.bin", + "bytes": 4096, + "sha256": "b4707ef68fb11c9d3720db82722924de4004e75c97d07cdbd0b24c08f3677fce" + } + }, + { + "name": "IQ1_XXXS-1024", + "dtype": "f32", + "oracle": "fork", + "elements": 4096, + "table_bytes": 19456, + "output_bytes": 16384, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_XXXS-1024-f32.bin", + "bytes": 16384, + "sha256": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/IQ1_XXXS-1024-f32.bin", + "bytes": 16384, + "sha256": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0" + } + }, + { + "name": "IQ1_XXXS-1024", + "dtype": "bf16", + "oracle": "fork", + "elements": 4096, + "table_bytes": 19456, + "output_bytes": 8192, + "scratch_bytes": 16, + "tolerance": "byte-exact, with explicit f32-to-bf16 RNE for bf16 output", + "byte_exact": true, + "max_absolute_error": 0.0, + "finite": true, + "pass": true, + "native": { + "file": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_XXXS-1024-bf16.bin", + "bytes": 8192, + "sha256": "14e07510990f222b0656fdbacd1a3d6271c28e00526587b67c4a1b399674c3d5" + }, + "oracle_output": { + "file": "/home/vikash/.cache/rdna3-gather-impl/evidence/secondary-operator-v1/IQ1_XXXS-1024-f32.bin", + "bytes": 16384, + "sha256": "10b121b53bde30b8e24ea1fb6ef8bb4f3123952900c2c66886d2e09b40ee62c0" + } + } + ], + "outputs": 76, + "all_pass": true, + "scope": "synthetic operation parity; original fixture and model gates remain separate" +} diff --git a/docs/bench-evidence/rocm-quant-gather/table-whitespace-only.json b/docs/bench-evidence/rocm-quant-gather/table-whitespace-only.json new file mode 100644 index 0000000000..0a2844c4cd --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/table-whitespace-only.json @@ -0,0 +1,6 @@ +{ + "old_sha256": "d267d25ff1dd3f67749c251f87047f4b70c9650a99d04b524ff6c3313596a349", + "new_sha256": "bdee96a810ce7ceab6102025ea053ca7ecdb3caea5118727e02bbe69d7d784f7", + "same_lines_after_rstrip": true, + "arithmetic_and_table_values_changed": false +} diff --git a/docs/bench-evidence/rocm-quant-gather/trace-v2-commands.json b/docs/bench-evidence/rocm-quant-gather/trace-v2-commands.json new file mode 100644 index 0000000000..98819db07c --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/trace-v2-commands.json @@ -0,0 +1,1495 @@ +{ + "version": 2, + "row": "BACKEND-ROCM-QUANT-GATHER", + "runs": [ + { + "name": "native", + "argv": [ + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/bin/rocprofv3", + "--memory-allocation-trace", + "--hip-trace", + "--kernel-trace", + "--memory-copy-trace", + "--output-format", + "csv", + "--output-file", + "trace_%pid%", + "--rocm-root", + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix", + "--output-directory", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native-profiler", + "--", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native-capture", + "capture", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native" + ], + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_TRACE_I64_IDS": "1" + }, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/trace-v1", + "log": "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native.log" + }, + { + "name": "primary", + "argv": [ + "docker", + "run", + "--rm", + "--pull=never", + "--network=none", + "--name", + "rdna3-gather-trace-v2", + "--device=/dev/kfd", + "--device=/dev/dri", + "--security-opt", + "seccomp=unconfined", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/host-rocprof/bin/rocprofv3", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-v", + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix:/host-rocprof:ro", + "-w", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "--memory-allocation-trace", + "--hip-trace", + "--kernel-trace", + "--memory-copy-trace", + "--output-format", + "csv", + "--output-file", + "trace_%pid%", + "--rocm-root", + "/host-rocprof", + "--output-directory", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary-profiler", + "--", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary_trace.py", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary" + ], + "env": {}, + "cwd": "/home/vikash/.cache/rdna3-gather-impl/trace-v1", + "log": "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary.log" + } + ], + "files": { + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native-capture": { + "bytes": 30693264, + "sha256": "2569a80c1bddec3ebd9332e1ca2cfb9d316df5db6e8afe17a51ab66d09438698" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/source.tar": { + "bytes": 204800, + "sha256": "0a34af72425548242ebbf8dce48fb066182b87c7d58681c83b87a0e0a68ea5b6" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary.py": { + "bytes": 7227, + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary_trace.py": { + "bytes": 4048, + "sha256": "dfd4c7068ea117c89ae894d5cd0dadc5946b32476373c64684687f0c46cf6437" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ1_S-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ1_S-1024.packed": { + "bytes": 25600, + "sha256": "6bf00a4b02da43e5505823b12d9e368aa1ded7f87427eea02ec03ebc68288c33" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ1_S-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ1_S-256.packed": { + "bytes": 6400, + "sha256": "754f0d0b9e7d3656a1f669cf756779352cccb246bf6aa0b25e7f362180cf0882" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_S-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_S-1024.packed": { + "bytes": 41984, + "sha256": "2cc5c79aef24da7bb363c72b7ac91f05c38b87fdc96627253203926f4465d73e" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_S-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_S-256.packed": { + "bytes": 10496, + "sha256": "6eb1966662d9e424fefc5c3625282e7f33fdfab3af9cb4c0cbac6bdc593711ea" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_XS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_XS-1024.packed": { + "bytes": 37888, + "sha256": "8fbc716e220d386b0c27f1cca003384091cd0e522c1442656677af4ac780de5f" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_XS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_XS-256.packed": { + "bytes": 9472, + "sha256": "3281c7397dde054ea6c9c8a661cab43dbd0c982eb23b9a31523a616565c827e7" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_XXS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_XXS-1024.packed": { + "bytes": 33792, + "sha256": "2fbe6e6ffcbc32fda24c5d63d4a4b678283e2bd571f491210097a27b88cc1fd4" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_XXS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ2_XXS-256.packed": { + "bytes": 8448, + "sha256": "4a3c10afb47315763ce507edcb5814e06d1be93a21b178cf73183302a3f1ceb9" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ3_S-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ3_S-1024.packed": { + "bytes": 56320, + "sha256": "a94d30560f904618cead2f0618aea6455d5183eb3ed443ca5c12e48fe5b31b34" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ3_S-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ3_S-256.packed": { + "bytes": 14080, + "sha256": "d4eebe1369e21b0efd6ec5c89f54c785849c02edde957ebd302dad376f63c14c" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ3_XXS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ3_XXS-1024.packed": { + "bytes": 50176, + "sha256": "2d3a0d835fe7a47d802b59e8e412e4084de78faa2aaca59d1ce565fad11ac34a" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ3_XXS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ3_XXS-256.packed": { + "bytes": 12544, + "sha256": "d0f775f8e88d87ae542de6d16122a7a1cea09ebba91203f6513b74727082b6db" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ4_NL-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ4_NL-1024.packed": { + "bytes": 73728, + "sha256": "4e50d50e97e08c448cfff4fbed121ef24d657cfd30c2e09f66e70c647ce545b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ4_NL-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ4_NL-256.packed": { + "bytes": 18432, + "sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ4_XS-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ4_XS-1024.packed": { + "bytes": 69632, + "sha256": "189f75d03197100afafd3fe92183e47a2d73665d64f89b36d8b10daa05adf2c4" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ4_XS-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/IQ4_XS-256.packed": { + "bytes": 17408, + "sha256": "35b2a41e738a6cb0a86deb4fd366f71ae2dc2a6fad7169f5a9afb53f523e919c" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q2_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q2_K-1024.packed": { + "bytes": 43008, + "sha256": "58cb93e09423c7151fac47084de7e38bff98f51e5f304de243b41a17c47ac4af" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q2_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q2_K-256.packed": { + "bytes": 10752, + "sha256": "655f77ebb32d974da7907da4a217c45bbbcd0b127df73a731f85afe427bb7dcc" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q3_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q3_K-1024.packed": { + "bytes": 56320, + "sha256": "6118b4b2f10678fb09a2cb3324d69443ff3b33805f403b79ed3432d5bc1b2732" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q3_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q3_K-256.packed": { + "bytes": 14080, + "sha256": "e4c3a11c1bbff806ae2729033910b7db606452d1b9aa499aa2ced46740ad85d4" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q4_0-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q4_0-1024.packed": { + "bytes": 73728, + "sha256": "4e50d50e97e08c448cfff4fbed121ef24d657cfd30c2e09f66e70c647ce545b1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q4_0-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q4_0-256.packed": { + "bytes": 18432, + "sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q4_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q4_K-1024.packed": { + "bytes": 73728, + "sha256": "d15d1bb953010f1ecdac424a54c365308e3b058a1b818bafb89cfc9b9b1678cc" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q4_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q4_K-256.packed": { + "bytes": 18432, + "sha256": "b0c107c4b353e2a169247e0406c1eb015d7a6d97edd5906f710868b8c2cecce1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q5_0-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q5_0-1024.packed": { + "bytes": 90112, + "sha256": "b1d241ee00677d6d55a5e774916608aa4479e6c549d8ed1e4803c0938cd6a855" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q5_0-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q5_0-256.packed": { + "bytes": 22528, + "sha256": "b3531b1abd263bc4c6a3174b22fa1fd54a4b89b66e737f2eb3ba36be771d7528" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q5_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q5_K-1024.packed": { + "bytes": 90112, + "sha256": "6dd54fbc62a9818334e50347fe185247156ecc833a5655ce1920651a4dc537b5" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q5_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q5_K-256.packed": { + "bytes": 22528, + "sha256": "5e63d961f36486c9636dd2657e9069bd83ece65e466c56c657265fbd97353b07" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q6_K-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q6_K-1024.packed": { + "bytes": 107520, + "sha256": "2487007668582b61b3c59fdbaf0e92c2f5c74a61781daba95c13704983643ab1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q6_K-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q6_K-256.packed": { + "bytes": 26880, + "sha256": "134bba0f8076d43d31fcbbaeb37c3d8191b0d383e6595f787d3387e2e016738a" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q8_0-1024.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q8_0-1024.packed": { + "bytes": 139264, + "sha256": "c4435564bf5fd2dc23e0c78107f28ebd033ec741c417d41fa9dcbd89dd2909ba" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q8_0-256.ids": { + "bytes": 16, + "sha256": "519c1b2fc6da05a94eab0732b8b998418d87e1a465ca3560bd9e75b87c20aa15" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/Q8_0-256.packed": { + "bytes": 34816, + "sha256": "8be599e194405f9b5429e578a03a5717e4d78f53ab4c9c11a52659495ce805ee" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/manifest.json": { + "bytes": 5333, + "sha256": "f9e59c0da596f728e28c95892d9046f457400041a76368a8cd5f4ed6ad1a4c6b" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/_C_gguf.abi3.so": { + "bytes": 5175208, + "sha256": "592af79c210496e96eaa7bd409ae0a67e7de8c334aa9f782082c7639ba300138" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/__init__.py": { + "bytes": 411, + "sha256": "3e047fe518dc5b3f7ca2e2068cc6921b8b82c84cc87d23e79807a4cc6ccb0a04" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/config_parser.py": { + "bytes": 2238, + "sha256": "a49b946f6374aa12a14a6517cd2f0140271818e59881a87f0238024b211f7984" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/cuda_compat.h": { + "bytes": 1896, + "sha256": "959bc9e0e73e020d777530b18bf12160d35cc4eb976335955395729700b2f975" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/dispatch_utils.h": { + "bytes": 483, + "sha256": "7a17e8ac27b25426fbfd2921d2ebcad6410b703736204e55e832037fd9f250cc" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/dequantize.cuh": { + "bytes": 22264, + "sha256": "96c25644ffba8ec6b9beffb7b1c30435e9b0e0466def1d4721f7f8aab401c7e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/dequantize_hip.cuh": { + "bytes": 22787, + "sha256": "497842aca4744b903c7fd19b6069be33fa50da9d4bb893634eda46ab9cb35ddd" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/ggml-common.h": { + "bytes": 85532, + "sha256": "990b6aca44615eaf39a3fa76b65992e1101cf1c4d27ec3a36070074f9f83136b" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/ggml-common_hip.h": { + "bytes": 85590, + "sha256": "b7d415d0cb626afb330cc67867418943b1ebe39eb648988f637b4f29af8e8a41" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/gguf_kernel.cu": { + "bytes": 22161, + "sha256": "4da2c6a6c0f9b0e60661f8334f2d356322f6f4f443b51dcfa570c4da57291e38" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/gguf_kernel.hip": { + "bytes": 22272, + "sha256": "f714989ddbd4c97bdcd924d5ca3316c60efdebcdffa5c9c818a93579023e4d72" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/mmq.cuh": { + "bytes": 23925, + "sha256": "1a3a8fb8cd9d1682a421bba53df92fdb323f7ba00956ab04bced4865cf9ad030" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/mmq_hip.cuh": { + "bytes": 24603, + "sha256": "0e4a23bb4a8f2f9f7769ead653d9d2d72d16901ad9974d3aa3ba9972ee80ca34" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/mmvq.cuh": { + "bytes": 11978, + "sha256": "f46667f8c09a7ca8b90fd4b7c89a5eca54cca99372762c3c2853870874d6e829" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/mmvq_hip.cuh": { + "bytes": 12617, + "sha256": "38231c4faa337935c65562cf7f336761d13dd06bb19309e3e5a8f68a9f1ad448" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/moe.cuh": { + "bytes": 29528, + "sha256": "9e7510e259ed6a4ce9ff18743628b16a9a10f1fcac44553c450bee1684f6bf25" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/moe_hip.cuh": { + "bytes": 30206, + "sha256": "9e3358731eb607a97272f13a02314deb3db6f76026e77622570bfd2a2ae2b7bb" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/moe_vec.cuh": { + "bytes": 17570, + "sha256": "dbe05b1a3515ebf3b2f3d07264f742566eb9f01bef769f46b2f90442a18633da" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/moe_vec_hip.cuh": { + "bytes": 18209, + "sha256": "acbe6bf9625fe73ac96c0f87cdf43def66dc23643155977aedaafd84348192c9" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/vecdotq.cuh": { + "bytes": 70609, + "sha256": "4972eddb514db4a12a996e667905a86fe8ce0f0f42e9839dd2db75774ad5d326" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/gguf/vecdotq_hip.cuh": { + "bytes": 70697, + "sha256": "38caef027a395cf9d0285f67beb051e0432d74ef194a8d44b4cb820e927fa013" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/hip_compat.h": { + "bytes": 1952, + "sha256": "25d7d6b1dab03d8d85a71976e5955dd6b91ea6ed6442713145770ebcc5d91abe" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/csrc/torch_bindings.cpp": { + "bytes": 2363, + "sha256": "4fb80d903a1e860a080a04033b6a41eeb840b2bc68220fc509a0db5a5f1cafa7" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/gguf_files.py": { + "bytes": 774, + "sha256": "82e003351d59c7abd78477c6df244d1cd06cd8834c0a8db4a5fbf43144d57c3a" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/gguf_utils.py": { + "bytes": 15828, + "sha256": "d790422af5161db802845cfdb3ff293bd34b9dab50461c0e477c1f1edea5ee8c" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/loader.py": { + "bytes": 8882, + "sha256": "8c80ecb5fadecf60c603274ceeb3cd84500a8c6f915ea52b86d307e4775f2b2f" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/ops.py": { + "bytes": 7597, + "sha256": "daac821e871ff73df30b1836c17a7ef9574f8be6e4bedd5f6456c442b69c90db" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/plugin.py": { + "bytes": 5337, + "sha256": "f414bfc427b5e9142408bb1be23a71d23fb451327ff9182142f1aa5f91f05f1a" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/quantization/__init__.py": { + "bytes": 1555, + "sha256": "b6755dbcdb6ec7aba9434271dc162d0d7c31e813d6b9e434699f21164ac18d1b" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/quantization/config.py": { + "bytes": 4213, + "sha256": "18553e34317ffdab94b193439db428aec1af0d000444998c18eafb9397714370" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/quantization/diffusion_config.py": { + "bytes": 3757, + "sha256": "6970775f14c91286d061fb63e4d8353713108de374782c8d51dfc7785e08b359" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/quantization/fused_moe.py": { + "bytes": 8762, + "sha256": "bf195f8586bf6f0231e6b4e80c392fab1e39674590f89b482d6c8a279c4f80ab" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/quantization/layout.py": { + "bytes": 5211, + "sha256": "6ffb4c25b6e862a3953ff8d00fba3c9a59febbafd791cb15ae0c44f0404ef799" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/quantization/linear.py": { + "bytes": 9991, + "sha256": "b40556bc026632e610adaee2855d42ddc3d00d99ba07fdfee7cae171ec180efe" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/quantization/params.py": { + "bytes": 15207, + "sha256": "18032548702812fe5180e2eefe7139a49ec990cf30159672021b36d8345e693b" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/quantization/utils.py": { + "bytes": 2187, + "sha256": "9ffd3ca8f69d5fe0f5121576d4dfa1264c3d93bf277a8044a54e6c4471fbe9e5" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/quantization/vocal_embeds.py": { + "bytes": 6928, + "sha256": "8312ad417b7d680d200f6f277011230ca7d2ea6f424c9aadab583b70ed032cb9" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/__init__.py": { + "bytes": 38, + "sha256": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/__init__.py": { + "bytes": 84, + "sha256": "249b99a046d4aa0d60a34b8dd2695ea64836d02e5c7cee2b2682b3578a0fd1fe" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/interface.py": { + "bytes": 2643, + "sha256": "fa7f5cd35eae43068f798ba154c787e0719ca0401ace3af72754c3be0e7e4705" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/__init__.py": { + "bytes": 792, + "sha256": "484c1f66ed156f26dc558dee8111ec6aef969d0af6bed784193ceb148e1d01fd" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/iq1_m.py": { + "bytes": 2090, + "sha256": "735412106ed9d36db56cf7394516ca2f56aae2f94d7ad9ee033f5552e65fc473" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/iq1_s.py": { + "bytes": 1497, + "sha256": "e4619535a6776d8fbbf4ac1c451a7773b866d07f3c40a1231112bcc2c436a4c7" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/iq2_s.py": { + "bytes": 1634, + "sha256": "2102e3f5293c3aefd0a0352e68ddccae9adc5e0d339e3d901629a6a3e3ed7b65" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/iq2_xs.py": { + "bytes": 1578, + "sha256": "30cf8761d25401603849a71ca6f98467d418c4832e15fa485bde037ca96aea4e" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/iq2_xxs.py": { + "bytes": 1559, + "sha256": "2cc8610c83de7087e8bf542dce2d47c2a89d9d88c7fbd9092a4b4753e57409c6" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/iq3_s.py": { + "bytes": 1778, + "sha256": "841ab0215c7c61be31068f634b3c2d26903bbd9639438eaa9d0e4c6871e8a374" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/iq3_xxs.py": { + "bytes": 1598, + "sha256": "414627369715ab31aec9ac5103b3ef18ecb8bfb45670af2e877418821ea62210" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/iq4_nl.py": { + "bytes": 1209, + "sha256": "b8e640287eb0929f83dcdc3bf01d7bf8ac0ac2b63cadc5b7d17f83bb77daa4db" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/iq_quant/iq4_xs.py": { + "bytes": 1605, + "sha256": "91529ddc95d9e7da4824364b4308d0aa8c5d9712ea3d7c4a644f5a396c7ff4f0" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/k_quant/__init__.py": { + "bytes": 420, + "sha256": "3c42c6eb160150dc781258ddf2103d7b0de03b674297646f42929f707694bb20" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/k_quant/q2_k.py": { + "bytes": 1246, + "sha256": "4dff47db3970b03709b22ed67cb29fddfb99faec5fe26980f0683b3bd7759ef1" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/k_quant/q3_k.py": { + "bytes": 1353, + "sha256": "e97247b9ead180afa31d896bcb72e2df8ef1ea5f326e9988d3d3c36a9caa58a1" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/k_quant/q4_k.py": { + "bytes": 1229, + "sha256": "4a1d1323fab0a463fc33fa9abc32a8d2a43b30be9f3d24de95b4f008c3253811" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/k_quant/q5_k.py": { + "bytes": 1365, + "sha256": "c74345d0972a37777f33c013758e868b2b835f5805b721a88c1ad40fd92e0238" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/k_quant/q6_k.py": { + "bytes": 1423, + "sha256": "1c705c7052286cd179d4793d9fa08d57a20145bb24f7da49511503e5fe0eab04" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/standard_quant/__init__.py": { + "bytes": 501, + "sha256": "9727990b5e8d5f051f572cd8999c28f22b2be99c2babeb1dee17c081b1853ac9" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/standard_quant/q4_0.py": { + "bytes": 883, + "sha256": "4aab047fbf9abf6234ef6fd64f2a3aee4023cb91c850579997e9827d5b997643" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/standard_quant/q4_1.py": { + "bytes": 943, + "sha256": "5d0d85223f69082951a5301b87235e74c1c212e1c34f987871bbdf0a523e93c3" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/standard_quant/q5_0.py": { + "bytes": 990, + "sha256": "ae4e2c494b227a6d6534631b6ee2bdbc313c46ecd2a7e8af8a582575338c796e" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/standard_quant/q5_1.py": { + "bytes": 1049, + "sha256": "d6d526993a0f26cb0da9030d24456cd6a163d13a66205c2f47f4557b9443daec" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/standard_quant/q8_0.py": { + "bytes": 831, + "sha256": "274008decd5ddf15fa6741d1422201d6831c59d439df0a7b44fe7939eaafbc8d" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/standard_quant/q8_1.py": { + "bytes": 831, + "sha256": "d2b56f6ba4b0018240b81bc354d805917a83f0f26838cd45021821e50b7b39fc" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/dequantize/utils.py": { + "bytes": 4319, + "sha256": "3b46512556ee6ef55d5c7acd32ce289f41640c022252ab0f65ec961e9a854750" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/__init__.py": { + "bytes": 173, + "sha256": "b466c73479b97f45e40d1d50d7a534a8f6f05f5965a03e084a363f0773d0d73f" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/interface.py": { + "bytes": 3153, + "sha256": "5761acc437eb5f3f49b84d235ed45adcf5b1ae2b0a3650c3b7fe4b4e1430852d" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/__init__.py": { + "bytes": 705, + "sha256": "4c66e819f5765037507a598f69a0edc019dbc41ddd79fde78bfeb087878cc4e4" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/iq1_m.py": { + "bytes": 4458, + "sha256": "ac3134f134f5f4f8cd667b407f5e9b126f1a72805d6e54a5b64fc711f76a6e75" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/iq1_s.py": { + "bytes": 3565, + "sha256": "2245b0ee042fd2ebd9928f71e4b69b3c64a06d685030226b632038ca344a8a3a" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/iq2_s.py": { + "bytes": 3735, + "sha256": "df9c1abca21fb85df4f28122521e0e62aa31fa7c7bd9eca9b58edec80e4c2f77" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/iq2_xs.py": { + "bytes": 3673, + "sha256": "b064d651e2cf630b44d5c5231fa2dc9bc44a72dfc4b2497baaf670c87b610791" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/iq2_xxs.py": { + "bytes": 3610, + "sha256": "72862bfa799739e40185fa7671fce86bbf176a28329a78973aa5e735d025bb63" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/iq3_s.py": { + "bytes": 4820, + "sha256": "adf4ec93760bfb5b327310ac7c257c8091f522c30c56014c43fa743c42e1c2cb" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/iq3_xxs.py": { + "bytes": 4697, + "sha256": "2471ca802e675a6d92e091f9970e3745ffcd64307a042a1e91af61e51834f5bf" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/iq4_nl.py": { + "bytes": 3335, + "sha256": "78b14baaf6d976e72635bea06962eff3757b17d23d6d90c842399d6981dbc56d" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/iq_quant/iq4_xs.py": { + "bytes": 3846, + "sha256": "16d18b9fc71b70dfce2044d6938488ba83eed9daf7202c144da92cfa88cbbeda" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/k_quant/__init__.py": { + "bytes": 389, + "sha256": "ac57f723d1726db7abb41898cb9e8bceaf50209510fcbda9ac50fa118c1fd8a2" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/k_quant/q2_k.py": { + "bytes": 3319, + "sha256": "96def7c24bb30de179ee1864b065be0e6fd1045f9bdab8ad391f1be28593d575" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/k_quant/q3_k.py": { + "bytes": 4252, + "sha256": "691e036c396c7125de73cbcd1993023a96cdc136e1ec1599f3d970430058b38d" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/k_quant/q4_k.py": { + "bytes": 3556, + "sha256": "0216c060cd8cd7a8668863417ec45b5ee9198c384402192cfa2537b078d45349" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/k_quant/q5_k.py": { + "bytes": 3783, + "sha256": "2298c8dac8434c4910c017ac1db2e631a38b33cd2c66a7c22893ab93ecc9a94f" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/k_quant/q6_k.py": { + "bytes": 3454, + "sha256": "7a3b30ddafda65e321c2b9091d1b2ac05c2ae9ec1cfb281672817bf2c23c7ee8" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/standard_quant/__init__.py": { + "bytes": 456, + "sha256": "2e0626f087aaf64495beafa674ea9bb5c96ab4dca080e27ee1dec0f19ea7ea63" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/standard_quant/q4_0.py": { + "bytes": 3464, + "sha256": "981ca0dfb18e5afb6264b524a90bef6368648bcac359c83510ca17f7e3516e87" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/standard_quant/q4_1.py": { + "bytes": 3114, + "sha256": "7cfb86144a6a4d8e36b58ec036d5b2e7a41078f40cc8f856f27ca5d20fc42c7e" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/standard_quant/q5_0.py": { + "bytes": 3441, + "sha256": "109bc28e34be425884b038c1424a38b1529e1faabe1bcdfffb188eed70ad4313" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/standard_quant/q5_1.py": { + "bytes": 3505, + "sha256": "583b8d10f93c45530caef0be15b95faf8157746f5095fe066672f73d7260ad2c" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/standard_quant/q8_0.py": { + "bytes": 3289, + "sha256": "83d7255b105a94216d0f5d5d99116924de251bbc703b0d4a25b25e4aad77d004" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/standard_quant/q8_1.py": { + "bytes": 3289, + "sha256": "d15cf10adaa916f0ebe9ff3004bef2e12272e3c712436391867276b6ea5f28ad" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/fused_moe/utils.py": { + "bytes": 8831, + "sha256": "fc31d25eafd9c796a0df0c59e11cdd4e881efda7f9dec22bf7b79f09f2b7b79d" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/__init__.py": { + "bytes": 38, + "sha256": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/interface.py": { + "bytes": 2748, + "sha256": "4797dd1ead0968eed05a7a4d700288e9e9500a194719b9457e03287d4021b4b5" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/__init__.py": { + "bytes": 38, + "sha256": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq1_m.py": { + "bytes": 3656, + "sha256": "a085d73390d934bb9a080d52e6240e00d78c9157853a852166fded297bb32610" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq1_s.py": { + "bytes": 2784, + "sha256": "1fa2ac641d986803fc8f5dd8e3e81334580ddf287b106cb01bcb6fb10989936e" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq2_s.py": { + "bytes": 2933, + "sha256": "3c0ffa66e08bf1e38f950be2e53f39303c8bb0e11087caf6ae639eea83e52838" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq2_xs.py": { + "bytes": 2892, + "sha256": "a9259737326c7e4ed6559285355c822f972921d454e0cce3dd91a2d718dd218a" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq2_xxs.py": { + "bytes": 2835, + "sha256": "f7b240c3d764b82805d68b6108d618dfa49594cddba4d0af11873e5fbeb4c061" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq3_s.py": { + "bytes": 4001, + "sha256": "85b029e372f7c025320965cc98283f7556982bd0195b429173c094d4eb261d68" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq3_xxs.py": { + "bytes": 3905, + "sha256": "cce28674e735a9ccec6a503604790e915e723783551c90fd80dcebce5535d4c0" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq4_nl.py": { + "bytes": 2529, + "sha256": "8da067cc8a21a58bab73a7c5af4a7eaf293d7287033069e303d99511b0153fbf" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq4_xs.py": { + "bytes": 3048, + "sha256": "b420a17e6ea301a3c0ef7e732c71b720cda92481aadc08d229505495a7566afe" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/iq_quant/iq_tables.py": { + "bytes": 1672, + "sha256": "237e8f35e00611aa5f1ae3c176a440fec6cd672aeb54b8ced4fc2aaad019e693" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/k_quant/__init__.py": { + "bytes": 38, + "sha256": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/k_quant/q2_k.py": { + "bytes": 2489, + "sha256": "0e3c07ca796f20ef2af8bb0c7f9675c9a56256a180731e92df9802fa157ea95b" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/k_quant/q3_k.py": { + "bytes": 3424, + "sha256": "9b08b1190036c3f6f9f76c5df47d651139de1cc275b4e0506924d58e45303fe9" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/k_quant/q4_k.py": { + "bytes": 2745, + "sha256": "8a3f708e6efb9351125ac4388c9b1cccc545a4e0b10929ec6d19e1178c99e96d" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/k_quant/q5_k.py": { + "bytes": 2972, + "sha256": "eb981367e11f71068ba16e6b42a39c49f272b01291b51dd1a2867f887b64a879" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/k_quant/q6_k.py": { + "bytes": 2622, + "sha256": "1f5eb00e017b24b8651cfe814856747a1b86e6b0e2b140d318c66fd48fd0a80c" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/standard_quant/__init__.py": { + "bytes": 38, + "sha256": "6c6ec57de9f82ef42d82760f144a4571cb16b3cd6de33aaa7aa9fd65e8ab1fa1" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/standard_quant/q4_0.py": { + "bytes": 5005, + "sha256": "e3a80371bfbfd13946ff4ac0f5a0dcb10c780de40d76a3309246632c70c6c8c1" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/standard_quant/q4_1.py": { + "bytes": 2283, + "sha256": "7dc5dbb13ba58810129faf71f5dd4dd24ee2a0cd2bf633dc9d1dddf27fa355ba" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/standard_quant/q5_0.py": { + "bytes": 2631, + "sha256": "7966b6d565b7285e0268aa0c2fdef6cfb464234ca32d34481e80672cf906c26e" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/standard_quant/q5_1.py": { + "bytes": 2695, + "sha256": "c625c7bd20992b45c5c3f42d4be1f83b7d3d6843299aeaaa74417f858bc13ece" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/standard_quant/q8_0.py": { + "bytes": 2458, + "sha256": "75de808a1017fd6e4787086b9ffa42acfbc685e5ce1c8255a9d3e890ae61fa4e" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/standard_quant/q8_1.py": { + "bytes": 2458, + "sha256": "84dcf565d25c1190b6d8984810118a66049ac57d54aaf04920f0e1025b946885" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/triton/gemm/utils.py": { + "bytes": 8553, + "sha256": "6ee2b74c572dafc1511d006e5a08cebae06138f6a2b51aa59939edd88ebf051f" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weight_utils.py": { + "bytes": 7706, + "sha256": "03cdf368cc6fc58b4751f7471dd1d89a27647b4266e8a5bda30ed857de61df9d" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/__init__.py": { + "bytes": 1782, + "sha256": "081d701f52e2388ae38efce6d24030298881c8b4065ac5df40f19fb216980ead" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/base.py": { + "bytes": 2194, + "sha256": "899b76daac1cd263e606b96474df53353da55101e3bb61edc12e66aacaf6923f" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/diffusion/__init__.py": { + "bytes": 1617, + "sha256": "1af8199a0e542315efc0b79fe741c9d32c103191c27a7562c0ca7d53909b2aed" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/diffusion/base.py": { + "bytes": 3712, + "sha256": "c2678ceae3e2296bd0b9f4fefc77374b464e687236ab65b0cc8055a981eeeda5" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/diffusion/flux2_klein.py": { + "bytes": 3886, + "sha256": "d5007751145b7a85c075486a704134ae1f11c569d2be78374f945737f2acb8a8" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/diffusion/integration.py": { + "bytes": 8527, + "sha256": "a1ee6919b529831fd44e608b71a674d6c13052970713c200cb4a8e3e9636d368" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/diffusion/loader.py": { + "bytes": 9204, + "sha256": "a7044349ea18645c6d3f4e36c9ea30e07b7e9c47aa3d38bc5b7fd716846d3a26" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/diffusion/qwen_image.py": { + "bytes": 736, + "sha256": "6be01743e2c1a13008ca4e89b0a88784fd7eebc00c69583455099882cfccca8e" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/diffusion/z_image.py": { + "bytes": 1385, + "sha256": "5abbd262ab6b6f97b745e1534dc9f44ea4d0e75e2444d0b193f28ed8a89ab487" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/gemma3.py": { + "bytes": 4056, + "sha256": "2b5011dccbb976dd83d85ff9bdd358e5f17039c8543cfb592a47157f6ac74465" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/gemma4.py": { + "bytes": 6652, + "sha256": "dc89fff10a58bc9539e5ee201c5c231839965b8a04cb5e6f155a91f5875874da" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/olmoe.py": { + "bytes": 2943, + "sha256": "47a77b4ef1ed1fadcc5507f00a5eb1cc861b0a3111e7ead817d26a737ac7dc7c" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/qwen3_5.py": { + "bytes": 17170, + "sha256": "ed19007102c518a50e89a92fa2084c4034980f59aabd0881ce13df78df1fa2d5" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin/weights_adapter/transformers.py": { + "bytes": 9456, + "sha256": "f252fabd0e31f7e408b347fc686584c568dec840e7d17c9b7f729a7d0425342b" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin.egg-info/PKG-INFO": { + "bytes": 3505, + "sha256": "ac6e584fcd033f0578f2ded9f368ec1ca90d63c9268d3521901ae706a8537079" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin.egg-info/SOURCES.txt": { + "bytes": 6689, + "sha256": "1a897e9037ed45581c2d126d788e363b7e0041d4ec03edb1477c683fbf2f2605" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin.egg-info/dependency_links.txt": { + "bytes": 1, + "sha256": "01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin.egg-info/entry_points.txt": { + "bytes": 56, + "sha256": "6087369aa91ab6f5c0bdc80a76daed8a92175bb0a1db7a03aa89a24cd89ff241" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin.egg-info/requires.txt": { + "bytes": 71, + "sha256": "89642c00a45edc69b5157d59b942c030999737af3b590acaf2735bf4264a74d5" + }, + "/home/vikash/.cache/rdna3-gather-impl/plugin-artifact/vllm_gguf_plugin.egg-info/top_level.txt": { + "bytes": 17, + "sha256": "6b3fa0d9d555c7095f09dfd0c51e4ab37aaf29e23b0fe783abd7e4dbd0357be3" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/bin/rocprofv3": { + "bytes": 95727, + "sha256": "2d220ded2167097e480e6866a916b575548d56bed8a823a4e505f7133868f0a6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/libamd_comgr.so": { + "bytes": 13181720, + "sha256": "ff98e63e6b8677eb51376816f3e1169e377ac446e7bf4c5d247683a72f2f59eb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/libamd_comgr.so.3": { + "bytes": 13181720, + "sha256": "ff98e63e6b8677eb51376816f3e1169e377ac446e7bf4c5d247683a72f2f59eb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/libamd_comgr.so.3.3.0": { + "bytes": 13181720, + "sha256": "ff98e63e6b8677eb51376816f3e1169e377ac446e7bf4c5d247683a72f2f59eb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprof-trace-decoder.so": { + "bytes": 407656, + "sha256": "f4c158cdf49b46c15392decb3fef4065fc0654867de6ed26ffbcd1303e8301b2" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprof-trace-decoder.so.0.2": { + "bytes": 407656, + "sha256": "f4c158cdf49b46c15392decb3fef4065fc0654867de6ed26ffbcd1303e8301b2" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprof-trace-decoder.so.0.2.0": { + "bytes": 407656, + "sha256": "f4c158cdf49b46c15392decb3fef4065fc0654867de6ed26ffbcd1303e8301b2" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-register.so": { + "bytes": 967760, + "sha256": "7166282aacab9f6c8fdcc28ad884e57b6b09cd4e08aead98b8e862a61010e85c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-register.so.0": { + "bytes": 967760, + "sha256": "7166282aacab9f6c8fdcc28ad884e57b6b09cd4e08aead98b8e862a61010e85c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-register.so.0.6.0": { + "bytes": 967760, + "sha256": "7166282aacab9f6c8fdcc28ad884e57b6b09cd4e08aead98b8e862a61010e85c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-attach.so": { + "bytes": 897296, + "sha256": "742848d814c8c852aba6e5df960c7797968bcf19ce6a546ffde29c9f5a21ae4e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-attach.so.1": { + "bytes": 897296, + "sha256": "742848d814c8c852aba6e5df960c7797968bcf19ce6a546ffde29c9f5a21ae4e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-attach.so.1.3.5": { + "bytes": 897296, + "sha256": "742848d814c8c852aba6e5df960c7797968bcf19ce6a546ffde29c9f5a21ae4e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocattach.so": { + "bytes": 1076520, + "sha256": "e86594e2368aacbecc77d39da44323150fb39f1a346862f264cfcf6a0645a99d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocattach.so.1": { + "bytes": 1076520, + "sha256": "e86594e2368aacbecc77d39da44323150fb39f1a346862f264cfcf6a0645a99d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocattach.so.1.3.5": { + "bytes": 1076520, + "sha256": "e86594e2368aacbecc77d39da44323150fb39f1a346862f264cfcf6a0645a99d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocpd.so": { + "bytes": 1325128, + "sha256": "378e816d72ce4b0d7f0958b574be85e51b491cd1067e80372f27491f07081d6a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocpd.so.1": { + "bytes": 1325128, + "sha256": "378e816d72ce4b0d7f0958b574be85e51b491cd1067e80372f27491f07081d6a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-rocpd.so.1.3.5": { + "bytes": 1325128, + "sha256": "378e816d72ce4b0d7f0958b574be85e51b491cd1067e80372f27491f07081d6a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-roctx.so": { + "bytes": 845320, + "sha256": "07ab9ebdd0785c8a2f01b06c54bdebfcd70a9aa3ac7f6fb791045362dce6395c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-roctx.so.1": { + "bytes": 845320, + "sha256": "07ab9ebdd0785c8a2f01b06c54bdebfcd70a9aa3ac7f6fb791045362dce6395c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk-roctx.so.1.3.5": { + "bytes": 845320, + "sha256": "07ab9ebdd0785c8a2f01b06c54bdebfcd70a9aa3ac7f6fb791045362dce6395c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk.so": { + "bytes": 15823336, + "sha256": "807f890e2cb6661000845250358c3939f41573e88fc3aac6d529bb2d5fa6b55c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk.so.1": { + "bytes": 15823336, + "sha256": "807f890e2cb6661000845250358c3939f41573e88fc3aac6d529bb2d5fa6b55c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/librocprofiler-sdk.so.1.3.5": { + "bytes": 15823336, + "sha256": "807f890e2cb6661000845250358c3939f41573e88fc3aac6d529bb2d5fa6b55c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libLLVM-23git.so": { + "bytes": 132274504, + "sha256": "1f258534e3b14393fb2d65d6cb6a93fac239727cc1df5b698b6d2b51f78a65d9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libLLVM.so": { + "bytes": 132274504, + "sha256": "1f258534e3b14393fb2d65d6cb6a93fac239727cc1df5b698b6d2b51f78a65d9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libLLVM.so.23.0git": { + "bytes": 132274504, + "sha256": "1f258534e3b14393fb2d65d6cb6a93fac239727cc1df5b698b6d2b51f78a65d9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libclang-cpp.so": { + "bytes": 89897976, + "sha256": "0119f76826759e3f0534bb95bcc28e3e788e3a041e47b89310b3d324197b8ad3" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/llvm/lib/libclang-cpp.so.23.0git": { + "bytes": 89897976, + "sha256": "0119f76826759e3f0534bb95bcc28e3e788e3a041e47b89310b3d324197b8ad3" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libbz2.so": { + "bytes": 104016, + "sha256": "628d4871ea18e82dcc73b28f45a587f00a7abd4b208b371a713eb2e9d5b95dec" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libdrm.so": { + "bytes": 360336, + "sha256": "f1b274ac80ac1bcd241d70e1f13f9586e3fc83545a357b312890c1722ca68dcf" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libdrm_amdgpu.so": { + "bytes": 251416, + "sha256": "78c1af14962d9ee0e43602d422eb8619b6c825a9b7c8f532308ea1f4108a466f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libdw.so": { + "bytes": 5679360, + "sha256": "f9ad555cea1480082fd89dedaf5e0626a6deea0cb296c39d44b020adb56a0c49" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libelf.so": { + "bytes": 1198616, + "sha256": "a51543f71efce0c552ae606fe103b6e4fa4f6b283ba48a762f6d57b880401832" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/liblzma.so": { + "bytes": 218464, + "sha256": "15402b36e2fdb0762e0b6590c3470fccec5e80cc8010c84ac4835573d592c02d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_bz2.so": { + "bytes": 104016, + "sha256": "628d4871ea18e82dcc73b28f45a587f00a7abd4b208b371a713eb2e9d5b95dec" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_drm.so.2": { + "bytes": 360336, + "sha256": "f1b274ac80ac1bcd241d70e1f13f9586e3fc83545a357b312890c1722ca68dcf" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_drm_amdgpu.so.1": { + "bytes": 251416, + "sha256": "78c1af14962d9ee0e43602d422eb8619b6c825a9b7c8f532308ea1f4108a466f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_dw.so.1": { + "bytes": 5679360, + "sha256": "f9ad555cea1480082fd89dedaf5e0626a6deea0cb296c39d44b020adb56a0c49" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_elf.so.1": { + "bytes": 1198616, + "sha256": "a51543f71efce0c552ae606fe103b6e4fa4f6b283ba48a762f6d57b880401832" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_liblzma.so.5": { + "bytes": 218464, + "sha256": "15402b36e2fdb0762e0b6590c3470fccec5e80cc8010c84ac4835573d592c02d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_liblzma.so.5.8.1": { + "bytes": 218464, + "sha256": "15402b36e2fdb0762e0b6590c3470fccec5e80cc8010c84ac4835573d592c02d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_sqlite3.so": { + "bytes": 1479528, + "sha256": "44234a62c1ca345fd1a7886b4eeb82b6396ff7737967446c1627d4154c69a69a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_z.so.1": { + "bytes": 129576, + "sha256": "27361c0c552d2afd5ecf9079f356309ed827e9a27d35f2fc68c7b5e6b5c83199" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_z.so.1.3.2": { + "bytes": 129576, + "sha256": "27361c0c552d2afd5ecf9079f356309ed827e9a27d35f2fc68c7b5e6b5c83199" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_zstd.so.1": { + "bytes": 879168, + "sha256": "5fc4ab0e6899318ece9bea43d3d397d20053418a5e40cb6bc86efe19a801c75e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/librocm_sysdeps_zstd.so.1.5.7": { + "bytes": 879168, + "sha256": "5fc4ab0e6899318ece9bea43d3d397d20053418a5e40cb6bc86efe19a801c75e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libsqlite3.so": { + "bytes": 1479528, + "sha256": "44234a62c1ca345fd1a7886b4eeb82b6396ff7737967446c1627d4154c69a69a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libz.so": { + "bytes": 129576, + "sha256": "27361c0c552d2afd5ecf9079f356309ed827e9a27d35f2fc68c7b5e6b5c83199" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocm_sysdeps/lib/libzstd.so": { + "bytes": 879168, + "sha256": "5fc4ab0e6899318ece9bea43d3d397d20053418a5e40cb6bc86efe19a801c75e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool-kokkosp.so": { + "bytes": 796176, + "sha256": "cd3c7cbed051159d3966a71be00934c7a7de1963233a891684b2484007845c1a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool-kokkosp.so.1": { + "bytes": 796176, + "sha256": "cd3c7cbed051159d3966a71be00934c7a7de1963233a891684b2484007845c1a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool-kokkosp.so.1.3.5": { + "bytes": 796176, + "sha256": "cd3c7cbed051159d3966a71be00934c7a7de1963233a891684b2484007845c1a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so": { + "bytes": 11752584, + "sha256": "5c4bf4f4bff332fd7713f2c25fd7605bc80b1a5befdee7949b845b57cee87b0e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so.1": { + "bytes": 11752584, + "sha256": "5c4bf4f4bff332fd7713f2c25fd7605bc80b1a5befdee7949b845b57cee87b0e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofiler-sdk-tool.so.1.3.5": { + "bytes": 11752584, + "sha256": "5c4bf4f4bff332fd7713f2c25fd7605bc80b1a5befdee7949b845b57cee87b0e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofv3-list-avail.so": { + "bytes": 1270072, + "sha256": "acee4c6e3ea5cb365b4cc65ee3dc5ce07412aa75f5211cb58edf13d962de7fb6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofv3-list-avail.so.1": { + "bytes": 1270072, + "sha256": "acee4c6e3ea5cb365b4cc65ee3dc5ce07412aa75f5211cb58edf13d962de7fb6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/lib/rocprofiler-sdk/librocprofv3-list-avail.so.1.3.5": { + "bytes": 1270072, + "sha256": "acee4c6e3ea5cb365b4cc65ee3dc5ce07412aa75f5211cb58edf13d962de7fb6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/basic_counters.xml": { + "bytes": 107538, + "sha256": "510cb2f0bd7400b5e2ac7dd3ff473d39ecb77d126042b99c1fcd4b0a38ea39f9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/config.yaml": { + "bytes": 443032, + "sha256": "63542b5a2a285098f6ccb1d355574c8559ca75d3d5f80c82c5f02be7e3c8f960" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/convert-counters-collection-format.py": { + "bytes": 6513, + "sha256": "933afa0416346c97755e1895cd4c482fe3414eca02401885842e8a8f4963edc3" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/derived_counters.xml": { + "bytes": 95217, + "sha256": "774ba31a494d9135faf2ac876d0c1e6222358166c7a2b2019bb34e1e546095dc" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/CMakeLists.txt": { + "bytes": 1459, + "sha256": "809258de222cffb7e49af96c68a93f449a916d35a1a6f8868cf25f06add41cc0" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/CMakeLists.txt": { + "bytes": 1961, + "sha256": "b8e7c7e91a8dd46c6f6ec780da4e29c0aaf4ee14c696de8ed283b3bb15ac8215" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/README.md": { + "bytes": 529, + "sha256": "b26868a670abe00bc06c66f866271908ea485978f24a772ec7945a68c87f8b9c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/client.cpp": { + "bytes": 24708, + "sha256": "9067a403adb0db206c0d94bc2baea3d568f8a26d6ef0bd2d61112d85330f4660" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/client.hpp": { + "bytes": 1539, + "sha256": "726d87b15436e2b3192f4468f8678d209ada3a7c4ca2b353bbde1ad705244f6f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_buffered_tracing/main.cpp": { + "bytes": 13671, + "sha256": "3fe9ca538bac7db58a8e51bce8a805d63c02c3ca0160bdf31904a4d21bac3018" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/CMakeLists.txt": { + "bytes": 2089, + "sha256": "b5fa1810951a7afbcf817a07749edbcc2a790f1f1933546a85db771c480e62ca" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/README.md": { + "bytes": 431, + "sha256": "1adfbb032bd9f3ed877d35216952018ceb63d6070b00848da9fc3f5300ac3726" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/client.cpp": { + "bytes": 13875, + "sha256": "cb3fb9f13e4f263eab0037d6bc66e9b03be56f6916dc662fad6663dbe5cc0884" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/client.hpp": { + "bytes": 1474, + "sha256": "51d1b853f62a7bbf4d18854a071c1378e57b527605f27b563d1b3cc9cc5d3718" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/api_callback_tracing/main.cpp": { + "bytes": 9807, + "sha256": "6fb5c5e9b416842df480470af4307b9571ec0fb95c2f1e9fad1fa879ccfc7d0d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/CMakeLists.txt": { + "bytes": 1679, + "sha256": "71d1bb17f48511ffa43b7b844d15c8e7c7f4fe60f182af9420c94c67f49b4d61" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/README.md": { + "bytes": 423, + "sha256": "dc1813f6980788e01498d4b4b7a9cab4ca857a68948c7b2e3970783011409c92" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/client.cpp": { + "bytes": 9503, + "sha256": "7937856b638b1b24060aaab7441f09e85c8746a95b19704952ab9156ec0dc14a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/main.cpp": { + "bytes": 7927, + "sha256": "8c09efef58659fa03823f4b30fbd3b94a712fee31fa6cca2edcce751256c6929" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_isa_decode/transpose_kernels.hpp": { + "bytes": 4643, + "sha256": "90175f330ec0f40a83bf1528f768f0c2aba21810a0ee12063f6da8aa21f4cfcd" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_tracing/CMakeLists.txt": { + "bytes": 1732, + "sha256": "672138d4cb7014418a91e5e8caf615cbf93df765647cd8e52de78cb5523f2f33" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_tracing/README.md": { + "bytes": 551, + "sha256": "f25e5590cb1860609e9c761257155c9901e0fab218a9e0ecd4cdc8b3c0a1687e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_tracing/client.cpp": { + "bytes": 13389, + "sha256": "ef5c1d5768b8566bad9299d03ee619d744d1ed4bf12e70fda9aece8a9345876b" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/code_object_tracing/main.cpp": { + "bytes": 9024, + "sha256": "cbb02ba874c4c20ce2c00d42d57113f20625eacbe1e7f04dbdd0003000443819" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/CMakeLists.txt": { + "bytes": 3539, + "sha256": "b554903b80d735b5488a4dc2a80b81928d3758678037d3d72f20b8bd85d8aad5" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/call_stack.hpp": { + "bytes": 2937, + "sha256": "bc95aaada5e341c82eb5213df84d81068d2fbca2ed506d5c2c2cabfb54d3131e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/defines.hpp": { + "bytes": 5493, + "sha256": "ee3401d2ef7131b3a2523e254653a9317e19aa2be00f5a796ff8afe90c7104cb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/filesystem.hpp": { + "bytes": 3096, + "sha256": "5fc38fcabffcec84d87fbc2be55ab528ea7b4328e53aa885a9340aa4948691a5" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/common/name_info.hpp": { + "bytes": 1849, + "sha256": "9f8bb3663608c13f8844da6322b13f8154f74842e7d58a38ee8a3764eb55651c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/CMakeLists.txt": { + "bytes": 10081, + "sha256": "ea491ec6f527f73551f6b31b5fde27ee5f777b9c645621656416c641ba78243a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/README.md": { + "bytes": 739, + "sha256": "39a6d215564afd103ebe7a2eb3a1b483af5211b4c9775f22a4ce5a76bcbbd149" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/buffered_client.cpp": { + "bytes": 18653, + "sha256": "cb9292901935ada027dc63586ee1c898be34909e4e1c5c40b477a9c0c7fae9d2" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/callback_client.cpp": { + "bytes": 10883, + "sha256": "56a6a787c30b9124cc242480e0bb11f061eff3f07b39ffdf8d9288849f49fa3b" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/client.hpp": { + "bytes": 1289, + "sha256": "2fd28605c5ad6616ca882444e8085c2c717da2d99ca98dc61e3d6f874462e792" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/device_counting_async_client.cpp": { + "bytes": 14612, + "sha256": "e2df28c4ce6253ec1a9d56def57f3d4b3830bb887b9efdbe26fb584a799f7b54" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/device_counting_sync_client.cpp": { + "bytes": 19505, + "sha256": "0d6d5e573dedcf5d00b2fdcd672e4ea76ab835093dc47a403dd9f9ec5734f793" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/device_serialized_main.cpp": { + "bytes": 3516, + "sha256": "459bc84019b9ed6df32a1fe850187a12562fe74a4b212269f321ae22ae9ae39c" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/main.cpp": { + "bytes": 5632, + "sha256": "ecf46f16fa6243db4d51b5021b533f63ec451bf4f4aae44af89471a641c5e3ed" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/counter_collection/print_functional_counters_client.cpp": { + "bytes": 17500, + "sha256": "cda871f9bce62f0fca2294b324ce7e6e30fef6450ac7e571aa6d836c9a4452ed" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/CMakeLists.txt": { + "bytes": 4630, + "sha256": "130a15dc43bdf117a58fd4cfbf65f7806a1325bcf31b71fe6ddc6fe1b793c0af" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/README.md": { + "bytes": 894, + "sha256": "037a6c03ab82a3dee1bb91a7f687c6e38c29bd1f93e0dbe24d64cd1da530c2e0" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/client.cpp": { + "bytes": 27923, + "sha256": "8b22ae1af6673ccf4c28a8ff9e5b57826e04447f83fb9cddd5cb7e201cf8538e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/client.hpp": { + "bytes": 1539, + "sha256": "726d87b15436e2b3192f4468f8678d209ada3a7c4ca2b353bbde1ad705244f6f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/external_correlation_id_request/main.cpp": { + "bytes": 13378, + "sha256": "6e215e947a9d034e352bce5a8518b233dea45fb9e0848a7338f92874cde1b4d8" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/CMakeLists.txt": { + "bytes": 2048, + "sha256": "48e08db456719fba451da3b73034c08089d9c83061a7916762d4265decd431a9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/README.md": { + "bytes": 356, + "sha256": "d569ac790862a3699175f097d29d9725bbfa662b84d50cc5b7a74e13306e87c7" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/client.cpp": { + "bytes": 11294, + "sha256": "6267a50614de3307b9c0b0623b16645b1d09733d9f930a02276d5ed0a300df8f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/client.hpp": { + "bytes": 1418, + "sha256": "1d37987fd6f8f34bb9f9f1df5188b0ecaee5ee2f1a99b2b3fc4fa0cb3cfca19e" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/intercept_table/main.cpp": { + "bytes": 8659, + "sha256": "1807978bb1f4ec6bcfdffb9a18367557364b926d25d5f6e79688e44397b1b32d" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/CMakeLists.txt": { + "bytes": 3359, + "sha256": "fb7e7988e4976e06d428100390de9b9eaf44dc1a825ff00ef467dbd2663d4ec7" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/README.md": { + "bytes": 418, + "sha256": "2f9722fc01ad229cabae82a11d9e3141d96942a95141443361e8b2039430f230" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/client.cpp": { + "bytes": 27153, + "sha256": "45a54e1af1a2aa0ea0cc99e9382d417472334f6df7d6b4465efaa8b5c57237a4" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/client.hpp": { + "bytes": 1474, + "sha256": "1d62c3450c2830b7f711563ae48d088ddac82259430e6f2dddef22500d635e32" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/openmp_target/main.cpp": { + "bytes": 4408, + "sha256": "63ef6ca870fecbd2911c67fb73715323abbc6de700728e6c6dd2b0bdd9d98638" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/CMakeLists.txt": { + "bytes": 2247, + "sha256": "c5ea84e14d54ab79f6e573f9bb1702a86d37ffe3506c89c88086b159370127a9" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/README.md": { + "bytes": 691, + "sha256": "e3c906783a57916c30bfd0667e42b1b4638999099d0428c9cbaca04e5b7922bb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/client.cpp": { + "bytes": 7641, + "sha256": "4558e1e2d6aab085ef4fbfcfd6c4f433908acaf32511d3d71ab7621fd6df4510" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/main.cpp": { + "bytes": 13378, + "sha256": "6e215e947a9d034e352bce5a8518b233dea45fb9e0848a7338f92874cde1b4d8" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/pcs.cpp": { + "bytes": 18676, + "sha256": "d09f3f28eba6ad68a30825fa21f33f96f0062bc466761e9889bcb429b7ff8e24" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/pcs.hpp": { + "bytes": 3389, + "sha256": "ad2225d4adee003b0fd78ffc9b59d8c96a0c96253bbc9d6c43d5e350066f4e5a" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/utils.cpp": { + "bytes": 1656, + "sha256": "3442d1c93882b1798adc5d9120c1e17a675460ab28c36d674ab483bf41456cb1" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/pc_sampling/utils.hpp": { + "bytes": 1391, + "sha256": "339eb666c0c1d9e803186263ad705c68c59c92549c23113c10be5b15874e46cb" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/thread_trace/CMakeLists.txt": { + "bytes": 2492, + "sha256": "4c2f34441922ef0c55eaf486af8303f24987f9ae23594262a2183c368775c007" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/thread_trace/README.md": { + "bytes": 1081, + "sha256": "a88cc99102c22045939f37d1eab4bda802ba9b6cbfe5271c6c652dddfe8c2614" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/thread_trace/agent.cpp": { + "bytes": 15667, + "sha256": "6d2eee1aea16dc7c851a7ccd9b8ebe50cdbbf5a512db15646334f42bdb39dba6" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/samples/thread_trace/main.cpp": { + "bytes": 5386, + "sha256": "cd7dd11518b692aa2982a74984eb1a0953a1f3131e2d8eb24b7211a294a41903" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/setup-env.sh": { + "bytes": 1384, + "sha256": "bcfd4c15a443cfa4e11f064e5dee1ee3e8793b7dc61513c1f2c5a9bf44c1b89f" + }, + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/share/rocprofiler-sdk/tests/cmake/rocprofiler_test_gpu_targets.cmake": { + "bytes": 563, + "sha256": "586a3b79e5a2823511e87a74ad3c3b042ce9c8ed696022983a48604d5874b145" + } + }, + "profiler_version": "1.3.5", + "profiler_revision": "6b0e43f341195e203754e08f850e437ff2fc09f9", + "authority": "Root operator alone runs both commands under GPU0 file mutex", + "scope": "Same-byte operation dtype/residency evidence; no performance or model qualification claim", + "workload": { + "tables": 32, + "outputs": 64, + "formats": 16, + "widths": [ + 256, + 1024 + ], + "rows": 128, + "ids": [ + 0, + 127, + 64, + 0 + ], + "id_dtype": "i64", + "id_bytes": 32, + "output_dtypes": [ + "f32", + "bf16" + ] + }, + "adaptations": [ + "Native 1D IDs represent the same sequence as upstream 2D [2,2] IDs", + "primary_trace.py excludes prior primary.py dense validation; exact production method unchanged", + "Native I64 indices widen the same signed32 manifest IDs without changing values", + "PyTorch allocator telemetry is labeled separately from profiler memory-copy/API evidence", + "Both tools include input uploads and output downloads; decode provenance must be read with the executed source chain" + ], + "expectations": [ + "Both commands exit0 with64 outputs", + "Every output matches its same-implementation untraced capture byte-for-byte", + "Native has64 GatherRows kernels and no decode/embedding host fallback", + "Primary selects packed rows before64 native dequantize kernels; no full-table float decode in this harness", + "Native constant16-byte error scratch, no selected packed-row allocation and no f32 staging for bf16 outputs" + ], + "untraced_native": "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2", + "untraced_primary": "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1", + "change": "Add profiler allocation events on both sides. Reuse all frozenv1 inputs, native executable and primary harness unchanged. No performance claim." +} diff --git a/docs/bench-evidence/rocm-quant-gather/trace-v2-operator-input-audit.json b/docs/bench-evidence/rocm-quant-gather/trace-v2-operator-input-audit.json new file mode 100644 index 0000000000..904f87e54b --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/trace-v2-operator-input-audit.json @@ -0,0 +1,40 @@ +{ + "manifest_sha256": "8bc6f8b8ced9848eb5dd575d5f77972778568b163548f586552d1233a8f37087", + "verified_input_count": 335, + "archived_sources": [ + { + "path": "src/vt/rocm/rocm_embedding_quant.h", + "sha256": "1b2b31617d660b64f622e692089423ce1657df102322558a4ab731f521952bc5" + }, + { + "path": "src/vt/rocm/rocm_embedding_quant.hip", + "sha256": "d3989353ece6a0c09e1f68ac16c5dbe8c949cd32c3831f41d4c2af835a3512e5" + }, + { + "path": "src/vt/rocm/rocm_quant_dequant.h", + "sha256": "a00ad5d98ae63fbb37683c1e26aa5494ca48a95f10c98bb817b6e780de481546" + }, + { + "path": "src/vt/rocm/rocm_quant_iq_tables.h", + "sha256": "d267d25ff1dd3f67749c251f87047f4b70c9650a99d04b524ff6c3313596a349" + }, + { + "path": "src/vt/rocm/rocm_ops.hip", + "sha256": "650a7d033b3ce3fc7a6c9a2e6bcfa80963c172ad5ef58421e2956892aac80935" + }, + { + "path": "tools/rocm_quant_gather/capture.cpp", + "sha256": "647167efc6da3e89157d888f7febd2a69ed1e8781fa0cd3c2439d99154d79b81" + }, + { + "path": "tools/rocm_quant_gather/primary.py", + "sha256": "1ddd29da328db6817b5b15119c228cad55b1f2d4e5acc143c80d70966a5c2d75" + }, + { + "path": "tools/rocm_quant_gather/primary_trace.py", + "sha256": "dfd4c7068ea117c89ae894d5cd0dadc5946b32476373c64684687f0c46cf6437" + } + ], + "profiler_contains_no_gpu_runtime": true, + "gpu_mutex": "/home/vikash/gpu.lock" +} diff --git a/docs/bench-evidence/rocm-quant-gather/trace-v2-operator-results.json b/docs/bench-evidence/rocm-quant-gather/trace-v2-operator-results.json new file mode 100644 index 0000000000..71e8e5977f --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/trace-v2-operator-results.json @@ -0,0 +1,122 @@ +[ + { + "name": "native", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix/bin/rocprofv3", + "--memory-allocation-trace", + "--hip-trace", + "--kernel-trace", + "--memory-copy-trace", + "--output-format", + "csv", + "--output-file", + "trace_%pid%", + "--rocm-root", + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix", + "--output-directory", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native-profiler", + "--", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native-capture", + "capture", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native" + ], + "cwd": "/home/vikash/.cache/rdna3-gather-impl/trace-v1", + "env": { + "HIP_VISIBLE_DEVICES": "0", + "ROCR_VISIBLE_DEVICES": "0", + "TMPDIR": "/home/vikash/.cache/rdna3-gather-impl/tmp", + "VT_ROCM_GATHER_TRACE_I64_IDS": "1" + }, + "exit_code": 0, + "seconds": 1.7153351306915283, + "log": "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native.log", + "log_sha256": "cbd29ede0f79ae1eceefbd819fcfa9f71a0709b92c7158655a2f0ba9d5015d2d", + "all335_input_hashes_verified_before_after": true + }, + { + "name": "primary", + "argv": [ + "flock", + "-n", + "-F", + "/home/vikash/gpu.lock", + "docker", + "run", + "--rm", + "--pull=never", + "--network=none", + "--name", + "rdna3-gather-trace-v2", + "--device=/dev/kfd", + "--device=/dev/dri", + "--security-opt", + "seccomp=unconfined", + "--ipc=private", + "--shm-size=2g", + "--user", + "1000:1000", + "--group-add", + "44", + "--group-add", + "992", + "--entrypoint", + "/host-rocprof/bin/rocprofv3", + "-v", + "/home/vikash/oracle/gfx1100-active-2773:/home/vikash/oracle/gfx1100-active-2773:ro", + "-v", + "/home/vikash/.cache/rdna3-gather-impl:/home/vikash/.cache/rdna3-gather-impl", + "-v", + "/home/vikash/vllm.cpp-rdna3-f16-impl/build-rocm-f16-evidence/g5-traces-v1/profiler-prefix:/host-rocprof:ro", + "-w", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1", + "-e", + "HIP_VISIBLE_DEVICES=0", + "-e", + "ROCR_VISIBLE_DEVICES=0", + "-e", + "TMPDIR=/home/vikash/.cache/rdna3-gather-impl/tmp", + "-e", + "PYTHONDONTWRITEBYTECODE=1", + "-e", + "PYTHONUNBUFFERED=1", + "-e", + "PYTHONPATH=/home/vikash/.cache/rdna3-gather-impl/plugin-artifact:/home/vikash/.cache/rdna3-gather-impl/llama-stock-source/gguf-py", + "-e", + "USER=vikash", + "-e", + "LOGNAME=vikash", + "-e", + "XDG_CACHE_HOME=/home/vikash/.cache/rdna3-gather-impl/plugin-cache", + "sha256:80aab4c182a1f3eeebe286173977e57fcaf10a049b41f475655b35d285de31dc", + "--memory-allocation-trace", + "--hip-trace", + "--kernel-trace", + "--memory-copy-trace", + "--output-format", + "csv", + "--output-file", + "trace_%pid%", + "--rocm-root", + "/host-rocprof", + "--output-directory", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary-profiler", + "--", + "/home/vikash/oracle/gfx1100-active-2773/runtime/bin/python", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary_trace.py", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/inputs/manifest.json", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary" + ], + "cwd": "/home/vikash/.cache/rdna3-gather-impl/trace-v1", + "env": {}, + "exit_code": 0, + "seconds": 12.011876583099365, + "log": "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary.log", + "log_sha256": "a9d28d83887375bbeae27b735e094de947bca79d5712805b2c7c353d5bf5167f", + "all335_input_hashes_verified_before_after": true + } +] diff --git a/docs/bench-evidence/rocm-quant-gather/trace-v2-operator-trace-audit.json b/docs/bench-evidence/rocm-quant-gather/trace-v2-operator-trace-audit.json new file mode 100644 index 0000000000..299164c36a --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/trace-v2-operator-trace-audit.json @@ -0,0 +1,529 @@ +{ + "scope": "Synthetic native/primary operation memory and dispatch; no model acceptance, performance or invocation equality", + "inputs": { + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q4_0-256-f32.bin": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-256-f32.bin": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q4_0-256-bf16.bin": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-256-bf16.bin": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q4_0-1024-f32.bin": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-1024-f32.bin": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q4_0-1024-bf16.bin": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_0-1024-bf16.bin": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q5_0-256-f32.bin": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-256-f32.bin": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q5_0-256-bf16.bin": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-256-bf16.bin": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q5_0-1024-f32.bin": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-1024-f32.bin": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q5_0-1024-bf16.bin": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_0-1024-bf16.bin": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q8_0-256-f32.bin": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-256-f32.bin": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q8_0-256-bf16.bin": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-256-bf16.bin": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q8_0-1024-f32.bin": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-1024-f32.bin": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q8_0-1024-bf16.bin": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q8_0-1024-bf16.bin": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q2_K-256-f32.bin": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-256-f32.bin": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q2_K-256-bf16.bin": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-256-bf16.bin": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q2_K-1024-f32.bin": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-1024-f32.bin": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q2_K-1024-bf16.bin": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q2_K-1024-bf16.bin": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q3_K-256-f32.bin": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-256-f32.bin": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q3_K-256-bf16.bin": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-256-bf16.bin": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q3_K-1024-f32.bin": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-1024-f32.bin": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q3_K-1024-bf16.bin": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q3_K-1024-bf16.bin": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q4_K-256-f32.bin": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-256-f32.bin": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q4_K-256-bf16.bin": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-256-bf16.bin": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q4_K-1024-f32.bin": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-1024-f32.bin": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q4_K-1024-bf16.bin": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q4_K-1024-bf16.bin": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q5_K-256-f32.bin": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-256-f32.bin": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q5_K-256-bf16.bin": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-256-bf16.bin": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q5_K-1024-f32.bin": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-1024-f32.bin": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q5_K-1024-bf16.bin": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q5_K-1024-bf16.bin": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q6_K-256-f32.bin": "3e1665a3167a552c47493bdc538166acfa6cbd43a1e7dd30230c40e3c5b11207", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-256-f32.bin": "3e1665a3167a552c47493bdc538166acfa6cbd43a1e7dd30230c40e3c5b11207", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q6_K-256-bf16.bin": "242d1b322588618517ee3d4d62e393d08b2faca2262414ab00d89d092598e8ee", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-256-bf16.bin": "242d1b322588618517ee3d4d62e393d08b2faca2262414ab00d89d092598e8ee", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q6_K-1024-f32.bin": "158400e054e65af2adad3e7d348b2fc3e082fcfc065706db200704d3c86a2581", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-1024-f32.bin": "158400e054e65af2adad3e7d348b2fc3e082fcfc065706db200704d3c86a2581", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/Q6_K-1024-bf16.bin": "54c4ea60561a742634c5fdf24c4dd372f0e101a48bf29b53d53a0b7e88f31aea", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/Q6_K-1024-bf16.bin": "54c4ea60561a742634c5fdf24c4dd372f0e101a48bf29b53d53a0b7e88f31aea", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_XXS-256-f32.bin": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-256-f32.bin": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_XXS-256-bf16.bin": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-256-bf16.bin": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_XXS-1024-f32.bin": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-1024-f32.bin": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_XXS-1024-bf16.bin": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XXS-1024-bf16.bin": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_XS-256-f32.bin": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-256-f32.bin": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_XS-256-bf16.bin": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-256-bf16.bin": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_XS-1024-f32.bin": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-1024-f32.bin": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_XS-1024-bf16.bin": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_XS-1024-bf16.bin": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ3_XXS-256-f32.bin": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-256-f32.bin": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ3_XXS-256-bf16.bin": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-256-bf16.bin": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ3_XXS-1024-f32.bin": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-1024-f32.bin": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ3_XXS-1024-bf16.bin": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_XXS-1024-bf16.bin": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ1_S-256-f32.bin": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-256-f32.bin": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ1_S-256-bf16.bin": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-256-bf16.bin": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ1_S-1024-f32.bin": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-1024-f32.bin": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ1_S-1024-bf16.bin": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ1_S-1024-bf16.bin": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ4_NL-256-f32.bin": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-256-f32.bin": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ4_NL-256-bf16.bin": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-256-bf16.bin": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ4_NL-1024-f32.bin": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-1024-f32.bin": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ4_NL-1024-bf16.bin": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_NL-1024-bf16.bin": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ3_S-256-f32.bin": "7843ff13e1c7f1475f949583ec0466b7cd3020723994d7c725e7c618c11c34f0", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-256-f32.bin": "7843ff13e1c7f1475f949583ec0466b7cd3020723994d7c725e7c618c11c34f0", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ3_S-256-bf16.bin": "281888da0636914c257cd25407adc1465b2d40e96aa7694eb5c3bd0df4036b73", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-256-bf16.bin": "281888da0636914c257cd25407adc1465b2d40e96aa7694eb5c3bd0df4036b73", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ3_S-1024-f32.bin": "8932dbeb65d3716cfb60014ad471ab0fb5d2d2d4b037e8745464e65f77b936de", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-1024-f32.bin": "8932dbeb65d3716cfb60014ad471ab0fb5d2d2d4b037e8745464e65f77b936de", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ3_S-1024-bf16.bin": "11a954e23ccc31f98a3398902b7ce96ec39f0fc0119d41202732630299d9a623", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ3_S-1024-bf16.bin": "11a954e23ccc31f98a3398902b7ce96ec39f0fc0119d41202732630299d9a623", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_S-256-f32.bin": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-256-f32.bin": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_S-256-bf16.bin": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-256-bf16.bin": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_S-1024-f32.bin": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-1024-f32.bin": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ2_S-1024-bf16.bin": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ2_S-1024-bf16.bin": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ4_XS-256-f32.bin": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-256-f32.bin": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ4_XS-256-bf16.bin": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-256-bf16.bin": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ4_XS-1024-f32.bin": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-1024-f32.bin": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/native/IQ4_XS-1024-bf16.bin": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "/home/vikash/.cache/rdna3-gather-impl/native-synthetic-v2/IQ4_XS-1024-bf16.bin": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_0-256-f32.bin": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_0-256-bf16.bin": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_0-1024-f32.bin": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_0-1024-bf16.bin": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_0-256-f32.bin": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_0-256-bf16.bin": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_0-1024-f32.bin": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_0-1024-bf16.bin": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q8_0-256-f32.bin": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q8_0-256-bf16.bin": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q8_0-1024-f32.bin": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q8_0-1024-bf16.bin": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q2_K-256-f32.bin": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q2_K-256-bf16.bin": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q2_K-1024-f32.bin": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q2_K-1024-bf16.bin": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q3_K-256-f32.bin": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q3_K-256-bf16.bin": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q3_K-1024-f32.bin": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q3_K-1024-bf16.bin": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_K-256-f32.bin": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_K-256-bf16.bin": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_K-1024-f32.bin": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_K-1024-bf16.bin": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_K-256-f32.bin": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_K-256-bf16.bin": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_K-1024-f32.bin": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_K-1024-bf16.bin": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q6_K-256-f32.bin": "3e1665a3167a552c47493bdc538166acfa6cbd43a1e7dd30230c40e3c5b11207", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q6_K-256-bf16.bin": "242d1b322588618517ee3d4d62e393d08b2faca2262414ab00d89d092598e8ee", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q6_K-1024-f32.bin": "158400e054e65af2adad3e7d348b2fc3e082fcfc065706db200704d3c86a2581", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q6_K-1024-bf16.bin": "54c4ea60561a742634c5fdf24c4dd372f0e101a48bf29b53d53a0b7e88f31aea", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XXS-256-f32.bin": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XXS-256-bf16.bin": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XXS-1024-f32.bin": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XXS-1024-bf16.bin": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XS-256-f32.bin": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XS-256-bf16.bin": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XS-1024-f32.bin": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XS-1024-bf16.bin": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_XXS-256-f32.bin": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_XXS-256-bf16.bin": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_XXS-1024-f32.bin": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_XXS-1024-bf16.bin": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ1_S-256-f32.bin": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ1_S-256-bf16.bin": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ1_S-1024-f32.bin": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ1_S-1024-bf16.bin": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_NL-256-f32.bin": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_NL-256-bf16.bin": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_NL-1024-f32.bin": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_NL-1024-bf16.bin": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_S-256-f32.bin": "7843ff13e1c7f1475f949583ec0466b7cd3020723994d7c725e7c618c11c34f0", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_S-256-bf16.bin": "281888da0636914c257cd25407adc1465b2d40e96aa7694eb5c3bd0df4036b73", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_S-1024-f32.bin": "8932dbeb65d3716cfb60014ad471ab0fb5d2d2d4b037e8745464e65f77b936de", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_S-1024-bf16.bin": "11a954e23ccc31f98a3398902b7ce96ec39f0fc0119d41202732630299d9a623", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_S-256-f32.bin": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_S-256-bf16.bin": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_S-1024-f32.bin": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_S-1024-bf16.bin": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_XS-256-f32.bin": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_XS-256-bf16.bin": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_XS-1024-f32.bin": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_XS-1024-bf16.bin": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native-profiler/trace_991006_hip_api_trace.csv": "71f39a3ce41265b5e64976285467c30997a284ec6cdfeeaf7fc0427a9397c365", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native-profiler/trace_991006_kernel_trace.csv": "b3143e465b295e47689ce08f5576328105631c5fde581ae1362213062f1e7326", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native-profiler/trace_991006_memory_allocation_trace.csv": "c5a2597359009327c76bcd3b04f62edb73075a3de29abf78f4cfac4aee791456", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native-profiler/trace_991006_memory_copy_trace.csv": "5ccd025d4662683a4d4691713f1f020222e90d387286349e05d6e6d940599c22", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/report.json": "340582db5488267bf86603f32482e7f43f5ec4c2e3deff88337887e4b171906d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q4_0-256-f32.bin": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-256-f32.bin": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q4_0-256-bf16.bin": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-256-bf16.bin": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q4_0-1024-f32.bin": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-1024-f32.bin": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q4_0-1024-bf16.bin": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_0-1024-bf16.bin": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q5_0-256-f32.bin": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-256-f32.bin": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q5_0-256-bf16.bin": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-256-bf16.bin": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q5_0-1024-f32.bin": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-1024-f32.bin": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q5_0-1024-bf16.bin": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_0-1024-bf16.bin": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q8_0-256-f32.bin": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-256-f32.bin": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q8_0-256-bf16.bin": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-256-bf16.bin": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q8_0-1024-f32.bin": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-1024-f32.bin": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q8_0-1024-bf16.bin": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q8_0-1024-bf16.bin": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q2_K-256-f32.bin": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-256-f32.bin": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q2_K-256-bf16.bin": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-256-bf16.bin": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q2_K-1024-f32.bin": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-1024-f32.bin": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q2_K-1024-bf16.bin": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q2_K-1024-bf16.bin": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q3_K-256-f32.bin": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-256-f32.bin": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q3_K-256-bf16.bin": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-256-bf16.bin": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q3_K-1024-f32.bin": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-1024-f32.bin": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q3_K-1024-bf16.bin": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q3_K-1024-bf16.bin": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q4_K-256-f32.bin": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-256-f32.bin": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q4_K-256-bf16.bin": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-256-bf16.bin": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q4_K-1024-f32.bin": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-1024-f32.bin": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q4_K-1024-bf16.bin": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q4_K-1024-bf16.bin": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q5_K-256-f32.bin": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-256-f32.bin": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q5_K-256-bf16.bin": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-256-bf16.bin": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q5_K-1024-f32.bin": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-1024-f32.bin": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q5_K-1024-bf16.bin": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q5_K-1024-bf16.bin": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q6_K-256-f32.bin": "955ba972e48a303803cdacca928e7f60176dfaa1b7f8590038bdb91830658428", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-256-f32.bin": "955ba972e48a303803cdacca928e7f60176dfaa1b7f8590038bdb91830658428", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q6_K-256-bf16.bin": "c65412c7615787b9103207f55776d49dfda767740f4d6e1106cb711a99404bc6", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-256-bf16.bin": "c65412c7615787b9103207f55776d49dfda767740f4d6e1106cb711a99404bc6", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q6_K-1024-f32.bin": "5a0f2bb0da03f1eb5c689c6f95be636890aec44873bffe77dd039c2b5bc6c20a", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-1024-f32.bin": "5a0f2bb0da03f1eb5c689c6f95be636890aec44873bffe77dd039c2b5bc6c20a", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/Q6_K-1024-bf16.bin": "5955c913676e2e5003f84ad0dca7062d3f7106188a13e4844b13774a2764c238", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/Q6_K-1024-bf16.bin": "5955c913676e2e5003f84ad0dca7062d3f7106188a13e4844b13774a2764c238", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_XXS-256-f32.bin": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-256-f32.bin": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_XXS-256-bf16.bin": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-256-bf16.bin": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_XXS-1024-f32.bin": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-1024-f32.bin": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_XXS-1024-bf16.bin": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XXS-1024-bf16.bin": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_XS-256-f32.bin": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-256-f32.bin": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_XS-256-bf16.bin": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-256-bf16.bin": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_XS-1024-f32.bin": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-1024-f32.bin": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_XS-1024-bf16.bin": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_XS-1024-bf16.bin": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ3_XXS-256-f32.bin": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-256-f32.bin": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ3_XXS-256-bf16.bin": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-256-bf16.bin": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ3_XXS-1024-f32.bin": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-1024-f32.bin": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ3_XXS-1024-bf16.bin": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_XXS-1024-bf16.bin": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ1_S-256-f32.bin": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-256-f32.bin": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ1_S-256-bf16.bin": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-256-bf16.bin": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ1_S-1024-f32.bin": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-1024-f32.bin": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ1_S-1024-bf16.bin": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ1_S-1024-bf16.bin": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ4_NL-256-f32.bin": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-256-f32.bin": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ4_NL-256-bf16.bin": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-256-bf16.bin": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ4_NL-1024-f32.bin": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-1024-f32.bin": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ4_NL-1024-bf16.bin": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_NL-1024-bf16.bin": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ3_S-256-f32.bin": "8cad98f9fecbfcac883cb11a91df1ac703846dfe88bac12792ab3066420243ad", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-256-f32.bin": "8cad98f9fecbfcac883cb11a91df1ac703846dfe88bac12792ab3066420243ad", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ3_S-256-bf16.bin": "7f36f4a4ba9f0977f68d452e8ec1d26ea299ea9ddbc82fd0ede324abc94d6595", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-256-bf16.bin": "7f36f4a4ba9f0977f68d452e8ec1d26ea299ea9ddbc82fd0ede324abc94d6595", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ3_S-1024-f32.bin": "ebcadd200e64857ed483aa7176172dc94b966198fb81d2910ab723260fb46fdc", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-1024-f32.bin": "ebcadd200e64857ed483aa7176172dc94b966198fb81d2910ab723260fb46fdc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ3_S-1024-bf16.bin": "88e55dfa1c9378436bd431db09fca5bf5aad9910dc311b493d6f94d1ec98dc7c", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ3_S-1024-bf16.bin": "88e55dfa1c9378436bd431db09fca5bf5aad9910dc311b493d6f94d1ec98dc7c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_S-256-f32.bin": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-256-f32.bin": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_S-256-bf16.bin": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-256-bf16.bin": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_S-1024-f32.bin": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-1024-f32.bin": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ2_S-1024-bf16.bin": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ2_S-1024-bf16.bin": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ4_XS-256-f32.bin": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-256-f32.bin": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ4_XS-256-bf16.bin": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-256-bf16.bin": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ4_XS-1024-f32.bin": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-1024-f32.bin": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "/home/vikash/.cache/rdna3-gather-impl/trace-v1/primary/IQ4_XS-1024-bf16.bin": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "/home/vikash/.cache/rdna3-gather-impl/primary-synthetic-v1/IQ4_XS-1024-bf16.bin": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_0-256-f32.bin": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_0-256-bf16.bin": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_0-1024-f32.bin": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_0-1024-bf16.bin": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_0-256-f32.bin": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_0-256-bf16.bin": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_0-1024-f32.bin": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_0-1024-bf16.bin": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q8_0-256-f32.bin": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q8_0-256-bf16.bin": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q8_0-1024-f32.bin": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q8_0-1024-bf16.bin": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q2_K-256-f32.bin": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q2_K-256-bf16.bin": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q2_K-1024-f32.bin": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q2_K-1024-bf16.bin": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q3_K-256-f32.bin": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q3_K-256-bf16.bin": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q3_K-1024-f32.bin": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q3_K-1024-bf16.bin": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_K-256-f32.bin": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_K-256-bf16.bin": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_K-1024-f32.bin": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_K-1024-bf16.bin": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_K-256-f32.bin": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_K-256-bf16.bin": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_K-1024-f32.bin": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_K-1024-bf16.bin": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q6_K-256-f32.bin": "955ba972e48a303803cdacca928e7f60176dfaa1b7f8590038bdb91830658428", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q6_K-256-bf16.bin": "c65412c7615787b9103207f55776d49dfda767740f4d6e1106cb711a99404bc6", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q6_K-1024-f32.bin": "5a0f2bb0da03f1eb5c689c6f95be636890aec44873bffe77dd039c2b5bc6c20a", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q6_K-1024-bf16.bin": "5955c913676e2e5003f84ad0dca7062d3f7106188a13e4844b13774a2764c238", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XXS-256-f32.bin": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XXS-256-bf16.bin": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XXS-1024-f32.bin": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XXS-1024-bf16.bin": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XS-256-f32.bin": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XS-256-bf16.bin": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XS-1024-f32.bin": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XS-1024-bf16.bin": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_XXS-256-f32.bin": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_XXS-256-bf16.bin": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_XXS-1024-f32.bin": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_XXS-1024-bf16.bin": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ1_S-256-f32.bin": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ1_S-256-bf16.bin": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ1_S-1024-f32.bin": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ1_S-1024-bf16.bin": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_NL-256-f32.bin": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_NL-256-bf16.bin": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_NL-1024-f32.bin": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_NL-1024-bf16.bin": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_S-256-f32.bin": "8cad98f9fecbfcac883cb11a91df1ac703846dfe88bac12792ab3066420243ad", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_S-256-bf16.bin": "7f36f4a4ba9f0977f68d452e8ec1d26ea299ea9ddbc82fd0ede324abc94d6595", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_S-1024-f32.bin": "ebcadd200e64857ed483aa7176172dc94b966198fb81d2910ab723260fb46fdc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_S-1024-bf16.bin": "88e55dfa1c9378436bd431db09fca5bf5aad9910dc311b493d6f94d1ec98dc7c", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_S-256-f32.bin": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_S-256-bf16.bin": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_S-1024-f32.bin": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_S-1024-bf16.bin": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_XS-256-f32.bin": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_XS-256-bf16.bin": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_XS-1024-f32.bin": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_XS-1024-bf16.bin": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary-profiler/trace_1_hip_api_trace.csv": "658d51c64da619f38a0a62b5124fa3671fbb270fa8904f78f7612a743cbc2ef2", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary-profiler/trace_1_kernel_trace.csv": "ce69aaa3539a969f03593a32df1773e3a1e6c97b2cad1b2766ebab53c3ffb46a", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary-profiler/trace_1_memory_allocation_trace.csv": "73b305d403ecd027af00c78f53d65b584d698585fe15e6339e7340250a308e8f", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary-profiler/trace_1_memory_copy_trace.csv": "542a694b8459e640c8db5753bec29f66cb758cd44a325cbbcc7d7f98abe41dff", + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/report.json": "4bcfa3c78603128b6d05f0f285bff507e1eb1d9b3ddcb5ea7218df374bbb34fe" + }, + "arms": { + "native": { + "outputs_byte_exact_across_two_traces_and_untraced": 64, + "runtime_api_events": 1033, + "kernel_events": 296, + "copy_events": 24, + "unmatched_kernel_or_copy_correlations": 0, + "gpu_decoders": 64, + "f32_templates": 32, + "bf16_templates": 32, + "memory_api_fields": [ + "Kind", + "Operation", + "Agent_Id", + "Allocation_Size", + "Address", + "Correlation_Id", + "Start_Timestamp", + "End_Timestamp" + ], + "kernel_histogram": { + "__amd_rocclr_copyBuffer": 168, + "__amd_rocclr_fillBufferUnAligned": 64, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2 + }, + "gpu_allocations": 192, + "table_allocations": 32, + "id_allocations": 32, + "selected_output_allocations": 64, + "bounds_scratch_allocations": 64, + "bounds_scratch_bytes_each": 16, + "extra_gpu_allocations": 0, + "host_runtime_allocations": 66 + }, + "primary": { + "outputs_byte_exact_across_two_traces_and_untraced": 64, + "runtime_api_events": 4304, + "kernel_events": 296, + "copy_events": 24, + "unmatched_kernel_or_copy_correlations": 0, + "gpu_decoders": 64, + "f32_templates": 32, + "bf16_templates": 32, + "memory_api_fields": [ + "Kind", + "Operation", + "Agent_Id", + "Allocation_Size", + "Address", + "Correlation_Id", + "Start_Timestamp", + "End_Timestamp" + ], + "kernel_histogram": { + "__amd_rocclr_copyBuffer": 104, + "void at::native::(anonymous namespace)::indexSelectSmallIndex(at::cuda::detail::TensorInfo, at::cuda::detail::TensorInfo, at::cuda::detail::TensorInfo, int, int, unsigned int, long)": 64, + "void at::native::vectorized_elementwise_kernel<4, at::native::FillFunctor, std::array >(int, at::native::FillFunctor, std::array)": 32, + "void dequantize_block<32, 2, &(dequantize_q4_0(void const*, int, int, __half2&)), float>(void const*, float*, long)": 2, + "void at::native::vectorized_elementwise_kernel<8, at::native::FillFunctor, std::array >(int, at::native::FillFunctor, std::array)": 32, + "void dequantize_block<32, 2, &(dequantize_q4_0(void const*, int, int, __half2&)), c10::BFloat16>(void const*, c10::BFloat16*, long)": 2, + "void dequantize_block<32, 2, &(dequantize_q5_0(void const*, int, int, __half2&)), float>(void const*, float*, long)": 2, + "void dequantize_block<32, 2, &(dequantize_q5_0(void const*, int, int, __half2&)), c10::BFloat16>(void const*, c10::BFloat16*, long)": 2, + "void dequantize_block<32, 1, &(dequantize_q8_0(void const*, int, int, __half2&)), float>(void const*, float*, long)": 2, + "void dequantize_block<32, 1, &(dequantize_q8_0(void const*, int, int, __half2&)), c10::BFloat16>(void const*, c10::BFloat16*, long)": 2, + "void dequantize_block_q2_K(void const*, float*)": 2, + "void dequantize_block_q2_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_q3_K(void const*, float*)": 2, + "void dequantize_block_q3_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_q4_K(void const*, float*)": 2, + "void dequantize_block_q4_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_q5_K(void const*, float*)": 2, + "void dequantize_block_q5_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_q6_K(void const*, float*)": 2, + "void dequantize_block_q6_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq2_xxs(void const*, float*)": 2, + "void dequantize_block_iq2_xxs(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq2_xs(void const*, float*)": 2, + "void dequantize_block_iq2_xs(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq3_xxs(void const*, float*)": 2, + "void dequantize_block_iq3_xxs(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq1_s(void const*, float*)": 2, + "void dequantize_block_iq1_s(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq4_nl(void const*, float*)": 2, + "void dequantize_block_iq4_nl(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq3_s(void const*, float*)": 2, + "void dequantize_block_iq3_s(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq2_s(void const*, float*)": 2, + "void dequantize_block_iq2_s(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq4_xs(void const*, float*)": 2, + "void dequantize_block_iq4_xs(void const*, c10::BFloat16*)": 2 + }, + "gpu_packed_row_selections": 64, + "index_select_precedes_each_decoder": true, + "profiler_gpu_pool_bytes": 2097152, + "tensor_allocation_evidence": "PyTorch telemetry, separately verified: exact output delta and selected packed rows rounded to 512-byte allocation units" + } + }, + "limitation": "Memory-copy CSV has no byte arguments. Native allocation sizes are measured device allocations. Primary tensor sizes use allocator telemetry because rocprof sees its backing pool. Source inspection complements traces for no host decode.", + "result": "PASS" +} diff --git a/docs/bench-evidence/rocm-quant-gather/trace-v2-profiler-captures.tar.gz b/docs/bench-evidence/rocm-quant-gather/trace-v2-profiler-captures.tar.gz new file mode 100644 index 0000000000..0b7101b6d6 Binary files /dev/null and b/docs/bench-evidence/rocm-quant-gather/trace-v2-profiler-captures.tar.gz differ diff --git a/docs/bench-evidence/rocm-quant-gather/trace-v2-trace-audit.json b/docs/bench-evidence/rocm-quant-gather/trace-v2-trace-audit.json new file mode 100644 index 0000000000..52bbaaae37 --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/trace-v2-trace-audit.json @@ -0,0 +1,654 @@ +{ + "manifest": { + "bytes": 5333, + "sha256": "f9e59c0da596f728e28c95892d9046f457400041a76368a8cd5f4ed6ad1a4c6b" + }, + "outputs_per_arm": 64, + "scope": "same-byte operation dispatch and memory; no model or performance claim", + "arms": { + "native": { + "unchanged_output_bytes": 64, + "gpu_decoders": 64, + "decoder_template_dtypes": { + "f32": 32, + "bf16": 32 + }, + "kernel_histogram": { + "__amd_rocclr_copyBuffer": 168, + "__amd_rocclr_fillBufferUnAligned": 64, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(float*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2, + "void vt::rocm::(anonymous namespace)::EmbeddingQuantGatherKernel(unsigned short*, unsigned char const*, long const*, long, long, long, long, unsigned long, vt::rocm::EmbeddingQuantError*)": 2 + }, + "profiler_hipmalloc_allocations": 192, + "scratch_allocations_16_bytes": 64, + "extra_device_allocations": 0, + "allocation_evidence": "rocprofv3 sizes correlated with hipMalloc; runtime host allocations excluded" + }, + "primary": { + "unchanged_output_bytes": 64, + "gpu_decoders": 64, + "decoder_template_dtypes": { + "f32": 32, + "bf16": 32 + }, + "kernel_histogram": { + "__amd_rocclr_copyBuffer": 104, + "void at::native::(anonymous namespace)::indexSelectSmallIndex(at::cuda::detail::TensorInfo, at::cuda::detail::TensorInfo, at::cuda::detail::TensorInfo, int, int, unsigned int, long)": 64, + "void at::native::vectorized_elementwise_kernel<4, at::native::FillFunctor, std::array >(int, at::native::FillFunctor, std::array)": 32, + "void dequantize_block<32, 2, &(dequantize_q4_0(void const*, int, int, __half2&)), float>(void const*, float*, long)": 2, + "void at::native::vectorized_elementwise_kernel<8, at::native::FillFunctor, std::array >(int, at::native::FillFunctor, std::array)": 32, + "void dequantize_block<32, 2, &(dequantize_q4_0(void const*, int, int, __half2&)), c10::BFloat16>(void const*, c10::BFloat16*, long)": 2, + "void dequantize_block<32, 2, &(dequantize_q5_0(void const*, int, int, __half2&)), float>(void const*, float*, long)": 2, + "void dequantize_block<32, 2, &(dequantize_q5_0(void const*, int, int, __half2&)), c10::BFloat16>(void const*, c10::BFloat16*, long)": 2, + "void dequantize_block<32, 1, &(dequantize_q8_0(void const*, int, int, __half2&)), float>(void const*, float*, long)": 2, + "void dequantize_block<32, 1, &(dequantize_q8_0(void const*, int, int, __half2&)), c10::BFloat16>(void const*, c10::BFloat16*, long)": 2, + "void dequantize_block_q2_K(void const*, float*)": 2, + "void dequantize_block_q2_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_q3_K(void const*, float*)": 2, + "void dequantize_block_q3_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_q4_K(void const*, float*)": 2, + "void dequantize_block_q4_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_q5_K(void const*, float*)": 2, + "void dequantize_block_q5_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_q6_K(void const*, float*)": 2, + "void dequantize_block_q6_K(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq2_xxs(void const*, float*)": 2, + "void dequantize_block_iq2_xxs(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq2_xs(void const*, float*)": 2, + "void dequantize_block_iq2_xs(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq3_xxs(void const*, float*)": 2, + "void dequantize_block_iq3_xxs(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq1_s(void const*, float*)": 2, + "void dequantize_block_iq1_s(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq4_nl(void const*, float*)": 2, + "void dequantize_block_iq4_nl(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq3_s(void const*, float*)": 2, + "void dequantize_block_iq3_s(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq2_s(void const*, float*)": 2, + "void dequantize_block_iq2_s(void const*, c10::BFloat16*)": 2, + "void dequantize_block_iq4_xs(void const*, float*)": 2, + "void dequantize_block_iq4_xs(void const*, c10::BFloat16*)": 2 + }, + "packed_gpu_selections": 64, + "tensor_memory_evidence": "PyTorch telemetry: output plus selected rows rounded to observed512-byte allocator units", + "profiler_allocation_scope": "backing allocator pool; does not measure individual PyTorch tensors" + } + }, + "files": { + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_0-256-f32.bin": { + "bytes": 4096, + "sha256": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_0-256-bf16.bin": { + "bytes": 2048, + "sha256": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_0-1024-f32.bin": { + "bytes": 16384, + "sha256": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_0-1024-bf16.bin": { + "bytes": 8192, + "sha256": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_0-256-f32.bin": { + "bytes": 4096, + "sha256": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_0-256-bf16.bin": { + "bytes": 2048, + "sha256": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_0-1024-f32.bin": { + "bytes": 16384, + "sha256": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_0-1024-bf16.bin": { + "bytes": 8192, + "sha256": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q8_0-256-f32.bin": { + "bytes": 4096, + "sha256": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q8_0-256-bf16.bin": { + "bytes": 2048, + "sha256": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q8_0-1024-f32.bin": { + "bytes": 16384, + "sha256": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q8_0-1024-bf16.bin": { + "bytes": 8192, + "sha256": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q2_K-256-f32.bin": { + "bytes": 4096, + "sha256": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q2_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q2_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q2_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q3_K-256-f32.bin": { + "bytes": 4096, + "sha256": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q3_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q3_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q3_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_K-256-f32.bin": { + "bytes": 4096, + "sha256": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q4_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_K-256-f32.bin": { + "bytes": 4096, + "sha256": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q5_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q6_K-256-f32.bin": { + "bytes": 4096, + "sha256": "3e1665a3167a552c47493bdc538166acfa6cbd43a1e7dd30230c40e3c5b11207" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q6_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "242d1b322588618517ee3d4d62e393d08b2faca2262414ab00d89d092598e8ee" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q6_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "158400e054e65af2adad3e7d348b2fc3e082fcfc065706db200704d3c86a2581" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/Q6_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "54c4ea60561a742634c5fdf24c4dd372f0e101a48bf29b53d53a0b7e88f31aea" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XXS-256-f32.bin": { + "bytes": 4096, + "sha256": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XXS-256-bf16.bin": { + "bytes": 2048, + "sha256": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XXS-1024-f32.bin": { + "bytes": 16384, + "sha256": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XXS-1024-bf16.bin": { + "bytes": 8192, + "sha256": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XS-256-f32.bin": { + "bytes": 4096, + "sha256": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XS-256-bf16.bin": { + "bytes": 2048, + "sha256": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XS-1024-f32.bin": { + "bytes": 16384, + "sha256": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_XS-1024-bf16.bin": { + "bytes": 8192, + "sha256": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_XXS-256-f32.bin": { + "bytes": 4096, + "sha256": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_XXS-256-bf16.bin": { + "bytes": 2048, + "sha256": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_XXS-1024-f32.bin": { + "bytes": 16384, + "sha256": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_XXS-1024-bf16.bin": { + "bytes": 8192, + "sha256": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ1_S-256-f32.bin": { + "bytes": 4096, + "sha256": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ1_S-256-bf16.bin": { + "bytes": 2048, + "sha256": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ1_S-1024-f32.bin": { + "bytes": 16384, + "sha256": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ1_S-1024-bf16.bin": { + "bytes": 8192, + "sha256": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_NL-256-f32.bin": { + "bytes": 4096, + "sha256": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_NL-256-bf16.bin": { + "bytes": 2048, + "sha256": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_NL-1024-f32.bin": { + "bytes": 16384, + "sha256": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_NL-1024-bf16.bin": { + "bytes": 8192, + "sha256": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_S-256-f32.bin": { + "bytes": 4096, + "sha256": "7843ff13e1c7f1475f949583ec0466b7cd3020723994d7c725e7c618c11c34f0" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_S-256-bf16.bin": { + "bytes": 2048, + "sha256": "281888da0636914c257cd25407adc1465b2d40e96aa7694eb5c3bd0df4036b73" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_S-1024-f32.bin": { + "bytes": 16384, + "sha256": "8932dbeb65d3716cfb60014ad471ab0fb5d2d2d4b037e8745464e65f77b936de" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ3_S-1024-bf16.bin": { + "bytes": 8192, + "sha256": "11a954e23ccc31f98a3398902b7ce96ec39f0fc0119d41202732630299d9a623" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_S-256-f32.bin": { + "bytes": 4096, + "sha256": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_S-256-bf16.bin": { + "bytes": 2048, + "sha256": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_S-1024-f32.bin": { + "bytes": 16384, + "sha256": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ2_S-1024-bf16.bin": { + "bytes": 8192, + "sha256": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_XS-256-f32.bin": { + "bytes": 4096, + "sha256": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_XS-256-bf16.bin": { + "bytes": 2048, + "sha256": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_XS-1024-f32.bin": { + "bytes": 16384, + "sha256": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/IQ4_XS-1024-bf16.bin": { + "bytes": 8192, + "sha256": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native-profiler/trace_991006_kernel_trace.csv": { + "bytes": 56542, + "sha256": "b3143e465b295e47689ce08f5576328105631c5fde581ae1362213062f1e7326" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native-profiler/trace_991006_hip_api_trace.csv": { + "bytes": 2732597, + "sha256": "71f39a3ce41265b5e64976285467c30997a284ec6cdfeeaf7fc0427a9397c365" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native-profiler/trace_991006_memory_allocation_trace.csv": { + "bytes": 60823, + "sha256": "c5a2597359009327c76bcd3b04f62edb73075a3de29abf78f4cfac4aee791456" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/native/report.json": { + "bytes": 21534, + "sha256": "340582db5488267bf86603f32482e7f43f5ec4c2e3deff88337887e4b171906d" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_0-256-f32.bin": { + "bytes": 4096, + "sha256": "9966480cdeef7ea0007e83a432b0003915042615cec988bdc78aa81a9e05c4d4" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_0-256-bf16.bin": { + "bytes": 2048, + "sha256": "634ad1adfae1205b50995644230abe47d006f04afa3de126a908edc6d52a41c7" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_0-1024-f32.bin": { + "bytes": 16384, + "sha256": "73d07ea6da5385e2701b402832ecb32426cc902eabfa489644cc5b6b0a458e32" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_0-1024-bf16.bin": { + "bytes": 8192, + "sha256": "1cad407983d06f622ef3f0448bdfdde5e11aa2c816a1ecb1ee8f6cfbd717b2ca" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_0-256-f32.bin": { + "bytes": 4096, + "sha256": "cddca55a921b2cc41118c57912cf30b394138a33e6d59bfbb24e5c7351f9b50d" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_0-256-bf16.bin": { + "bytes": 2048, + "sha256": "6eab30eecc3c325dd86839aba260930ca8bf7d38710e5f1b51591f1a0b4af63b" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_0-1024-f32.bin": { + "bytes": 16384, + "sha256": "495d23a83140b03ada09177c6f4b9bc31faadb610b1098e7687af410404acdf3" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_0-1024-bf16.bin": { + "bytes": 8192, + "sha256": "92b78a76767ce3cfe53b5aa42b23120dffd9ca588f4f7637e9b93717ff72e9ed" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q8_0-256-f32.bin": { + "bytes": 4096, + "sha256": "1c8493a1edd90c3a205727544e432228f90487b57d836e04aa2c70dbde2b18dc" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q8_0-256-bf16.bin": { + "bytes": 2048, + "sha256": "cf1659dbd4ec5c173833e93ed5ce6ef4f5267a5f9946e2fd2ddc5b735ad1dabb" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q8_0-1024-f32.bin": { + "bytes": 16384, + "sha256": "834f8078f7ac197ee65dba962fcdf461ae7df8b2a4f5d32e4e84e9bd4be1fedc" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q8_0-1024-bf16.bin": { + "bytes": 8192, + "sha256": "5c6c831f0038680adcbc353b74b9a7f2ccb61c2a8c08f2aa65800fee663a882f" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q2_K-256-f32.bin": { + "bytes": 4096, + "sha256": "e24b791187d6821a4c2a75c762dbdfe17964a3290ad6ebe409102b98f42301a4" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q2_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "bc5d0b2852039a8f373b4c77d3c6e37a64f89b38850aaba1291396beb404ac4e" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q2_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "55a37b1031ff271501b735e019d48c5b16651a28d096abc9cf66ae9b7e13032b" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q2_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "355660442b0a1b33e66868cd480fe19ad3cbef94d291cf96a0ba3a846b7bfff1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q3_K-256-f32.bin": { + "bytes": 4096, + "sha256": "8abfd966f1a8d1cfe3a56ffe70951cf74861cec4937fe6cec311955c3bb8097c" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q3_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "cdf4fecc71e5f51885720bdba5f669eac9f5526ce37d21cc0e4983943398219d" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q3_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "c5db632dc69564c309db1e55800627d303353b322039d70bc64ad60a65e4544d" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q3_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "3c2a56e17bcd792a99c1d429d04872d530e3f35e5e59a40f0a2cb733e3c5881e" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_K-256-f32.bin": { + "bytes": 4096, + "sha256": "828bc6bd954f80b2bd5e90c4dead3785cc3f619a3e0cf85bbdd8977889e93780" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "1183adfc4a8cf6fbdeed1a024991b6b703d67e9254f2547f8be74070ba311023" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "818de25fdc842d287a28a404fe68291bed1a4f6f92d692be1697b63942d5de76" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q4_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "edb9258354cc7e85a80f0ac0f4d2372b7586152ea51ac9b7a5d05c1ddd5e7370" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_K-256-f32.bin": { + "bytes": 4096, + "sha256": "cccaa4367b4a06faba242e84ef5aef302d58ce73c58c51415337ca33058b7dbb" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "fa74aaf35aaea5781445d59d0d0dada1584a47fa079a374fb60a0d00dc52751a" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "8abdaf31ff6e869553fd22f6f57b15dd5699ac83921422291c22fb5979734c14" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q5_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "d448a2b8bf36fe9817b6ab4447bab0b32cf052268cdbe5d4a0b1166cb93dabab" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q6_K-256-f32.bin": { + "bytes": 4096, + "sha256": "955ba972e48a303803cdacca928e7f60176dfaa1b7f8590038bdb91830658428" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q6_K-256-bf16.bin": { + "bytes": 2048, + "sha256": "c65412c7615787b9103207f55776d49dfda767740f4d6e1106cb711a99404bc6" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q6_K-1024-f32.bin": { + "bytes": 16384, + "sha256": "5a0f2bb0da03f1eb5c689c6f95be636890aec44873bffe77dd039c2b5bc6c20a" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/Q6_K-1024-bf16.bin": { + "bytes": 8192, + "sha256": "5955c913676e2e5003f84ad0dca7062d3f7106188a13e4844b13774a2764c238" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XXS-256-f32.bin": { + "bytes": 4096, + "sha256": "2d205710a3efc0745c45410d91ff4e86d43f486a8366714b93586957c69c1432" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XXS-256-bf16.bin": { + "bytes": 2048, + "sha256": "7cfe27efb61e2bacbb37365e0375af69a42afacb9ad6bd20ed8a5732b710204b" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XXS-1024-f32.bin": { + "bytes": 16384, + "sha256": "7e5ac721517328088f4c4ff43e30aec9413c5e18047d91b3651b486d33ca45e2" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XXS-1024-bf16.bin": { + "bytes": 8192, + "sha256": "d94a7d1d127389dccf61f4d39c484e8a38c6c6b8eb5cf1f2a2e1c2a5b0c5240c" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XS-256-f32.bin": { + "bytes": 4096, + "sha256": "81f10e413631a1188b1b74dabba22f95812c2d7c8d5b34b2276cf69bf30b92aa" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XS-256-bf16.bin": { + "bytes": 2048, + "sha256": "5d1ebc762d9b8c8160b7246ad2701e404d04fccac8a42fb8e1d12739fc7023db" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XS-1024-f32.bin": { + "bytes": 16384, + "sha256": "74837cd35b8970c160e6785ca19ad9163178e039b3f63d280be46003710430bd" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_XS-1024-bf16.bin": { + "bytes": 8192, + "sha256": "ea2f9564900a1bb82888a86bd5e1605682e30e1cc33fa0453815dc30a22712a5" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_XXS-256-f32.bin": { + "bytes": 4096, + "sha256": "292689c91164b2c7dc6f225d06eacbbc19b005d40dd8992c19dbe8bd55b0fdda" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_XXS-256-bf16.bin": { + "bytes": 2048, + "sha256": "92bc21816f28827df760ea52f25de72ee085b53c63b3d691e3d42a7c09496631" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_XXS-1024-f32.bin": { + "bytes": 16384, + "sha256": "7e044ef28ae859e13df96388c97c06f50e1df645cdf7eb9fec9a373a7b7745e1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_XXS-1024-bf16.bin": { + "bytes": 8192, + "sha256": "812efe38d269cceafbc23f54913b3d3e3ef2177730f0bfcf9915287bb804bf44" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ1_S-256-f32.bin": { + "bytes": 4096, + "sha256": "4e7645670f8d1e86c3aa84d9b63435bdbd241228eeea56cf39686a6280d3eaf1" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ1_S-256-bf16.bin": { + "bytes": 2048, + "sha256": "f2a6a1a52667cf9cf4f822f9af52408880f21d9a1955d46e7704a3a21d191090" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ1_S-1024-f32.bin": { + "bytes": 16384, + "sha256": "7d93653c10b72ce36a7bc4a6a46ec21eefd3d4704128f0281b8eeb4b018f1a02" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ1_S-1024-bf16.bin": { + "bytes": 8192, + "sha256": "5972de407abdee675d416358801507600103e60c5518d5a7e3dc61362d3e4fe9" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_NL-256-f32.bin": { + "bytes": 4096, + "sha256": "16184a3758d6b6f361b16cf741d5fe0d05ff8f7d866050ebdc3354915c57ad9e" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_NL-256-bf16.bin": { + "bytes": 2048, + "sha256": "4f602b6088ab68c2631b7293a3e884785828a135de210d6fd8eb2b82b7bc69d9" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_NL-1024-f32.bin": { + "bytes": 16384, + "sha256": "d7f5fb54d2e2c52429c669245858945ae1b0a6fcee09f24c8a21c7c7d71ab516" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_NL-1024-bf16.bin": { + "bytes": 8192, + "sha256": "2bf0d166828994d5497b4cb20400fbbc45c1fc0fd94a3380d525df16450b81a3" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_S-256-f32.bin": { + "bytes": 4096, + "sha256": "8cad98f9fecbfcac883cb11a91df1ac703846dfe88bac12792ab3066420243ad" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_S-256-bf16.bin": { + "bytes": 2048, + "sha256": "7f36f4a4ba9f0977f68d452e8ec1d26ea299ea9ddbc82fd0ede324abc94d6595" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_S-1024-f32.bin": { + "bytes": 16384, + "sha256": "ebcadd200e64857ed483aa7176172dc94b966198fb81d2910ab723260fb46fdc" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ3_S-1024-bf16.bin": { + "bytes": 8192, + "sha256": "88e55dfa1c9378436bd431db09fca5bf5aad9910dc311b493d6f94d1ec98dc7c" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_S-256-f32.bin": { + "bytes": 4096, + "sha256": "2c2908ecf0382e9e8d6414ca7bbacd8fe8ac95cc15dd5483d32293883801714f" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_S-256-bf16.bin": { + "bytes": 2048, + "sha256": "22e25316adaf5931c49ef737972a2f1d612b90acb69a3ca76002effd2a43d195" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_S-1024-f32.bin": { + "bytes": 16384, + "sha256": "90c3f52ffccbc2f83e5b66edaf5805f2e23967aea551d55d1216142a2e3cc9f4" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ2_S-1024-bf16.bin": { + "bytes": 8192, + "sha256": "68f4582c895e47c648f5a3e3972d991d38a49404f5d3dd1434a2e27b90096907" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_XS-256-f32.bin": { + "bytes": 4096, + "sha256": "bc2868543de19f14c38589e427629f77c0fd02a8a24d22d47570e7fa03250ae2" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_XS-256-bf16.bin": { + "bytes": 2048, + "sha256": "1d664b63884d614512b6544cb085eb91626e7e20ebd05d9afca3e662bcbdeccc" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_XS-1024-f32.bin": { + "bytes": 16384, + "sha256": "c5d5b46955ae2c26887b015601e67cdaa3c4712910e1d743f58a0910f2cef889" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/IQ4_XS-1024-bf16.bin": { + "bytes": 8192, + "sha256": "b5d820d40868cc699f6d83fa2cc63f773b7c01c42af2fc961952261e4198a48d" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary-profiler/trace_1_kernel_trace.csv": { + "bytes": 72665, + "sha256": "ce69aaa3539a969f03593a32df1773e3a1e6c97b2cad1b2766ebab53c3ffb46a" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary-profiler/trace_1_hip_api_trace.csv": { + "bytes": 13717297, + "sha256": "658d51c64da619f38a0a62b5124fa3671fbb270fa8904f78f7612a743cbc2ef2" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary-profiler/trace_1_memory_allocation_trace.csv": { + "bytes": 749, + "sha256": "73b305d403ecd027af00c78f53d65b584d698585fe15e6339e7340250a308e8f" + }, + "/home/vikash/.cache/rdna3-gather-impl/trace-v2/primary/report.json": { + "bytes": 50780, + "sha256": "4bcfa3c78603128b6d05f0f285bff507e1eb1d9b3ddcb5ea7218df374bbb34fe" + } + }, + "pass": true +} diff --git a/docs/bench-evidence/rocm-quant-gather/two-regeneration-model-hashes.json b/docs/bench-evidence/rocm-quant-gather/two-regeneration-model-hashes.json new file mode 100644 index 0000000000..0ef7dc44ce --- /dev/null +++ b/docs/bench-evidence/rocm-quant-gather/two-regeneration-model-hashes.json @@ -0,0 +1,347 @@ +{ + "files": [ + { + "file": "IQ1_S-dense.gguf", + "bytes": 991296, + "first_sha256": "c67f7dcb52d65f3751c0e06b37e0e61b188c2b093f2c5f9e92190955391a1666", + "second_sha256": "c67f7dcb52d65f3751c0e06b37e0e61b188c2b093f2c5f9e92190955391a1666" + }, + { + "file": "IQ1_S.gguf", + "bytes": 932160, + "first_sha256": "30ce542d730ee89631229d7e51fe7fadaf14a23bca4b656ba3d6000b280901d8", + "second_sha256": "30ce542d730ee89631229d7e51fe7fadaf14a23bca4b656ba3d6000b280901d8" + }, + { + "file": "IQ1_S.packed", + "bytes": 6400, + "first_sha256": "754f0d0b9e7d3656a1f669cf756779352cccb246bf6aa0b25e7f362180cf0882", + "second_sha256": "754f0d0b9e7d3656a1f669cf756779352cccb246bf6aa0b25e7f362180cf0882" + }, + { + "file": "IQ1_XXXS-dense.gguf", + "bytes": 991296, + "first_sha256": "55e0de6d6b5fb16baaf534ca9fa2f259f94b676f831b745929b7e997dffc930f", + "second_sha256": "55e0de6d6b5fb16baaf534ca9fa2f259f94b676f831b745929b7e997dffc930f" + }, + { + "file": "IQ1_XXXS.gguf", + "bytes": 930624, + "first_sha256": "3d16b36474ae4e4663f1bc4842d9bfe8e62843491d3e6f5724173c04495e2c72", + "second_sha256": "3d16b36474ae4e4663f1bc4842d9bfe8e62843491d3e6f5724173c04495e2c72" + }, + { + "file": "IQ1_XXXS.packed", + "bytes": 4864, + "first_sha256": "af0b5e336987746fd49cd33c790b99eb1479c26f64b6679fa6c685e5934ba0c4", + "second_sha256": "af0b5e336987746fd49cd33c790b99eb1479c26f64b6679fa6c685e5934ba0c4" + }, + { + "file": "IQ2_S-dense.gguf", + "bytes": 991296, + "first_sha256": "dadcca3abfae7e710a1ed9bd6e679a92a02206cd41ee6709dd6713de201ff0a7", + "second_sha256": "dadcca3abfae7e710a1ed9bd6e679a92a02206cd41ee6709dd6713de201ff0a7" + }, + { + "file": "IQ2_S.gguf", + "bytes": 936256, + "first_sha256": "fa8f65a209f9f838f79523ca3a8baf60d244099b321610f1b224dd52f2d6d18a", + "second_sha256": "fa8f65a209f9f838f79523ca3a8baf60d244099b321610f1b224dd52f2d6d18a" + }, + { + "file": "IQ2_S.packed", + "bytes": 10496, + "first_sha256": "6eb1966662d9e424fefc5c3625282e7f33fdfab3af9cb4c0cbac6bdc593711ea", + "second_sha256": "6eb1966662d9e424fefc5c3625282e7f33fdfab3af9cb4c0cbac6bdc593711ea" + }, + { + "file": "IQ2_XS-dense.gguf", + "bytes": 991296, + "first_sha256": "341a57b2dca5fce0c9554ae388fb3ccc04c6d2ac180b7201fdd85427c79360a5", + "second_sha256": "341a57b2dca5fce0c9554ae388fb3ccc04c6d2ac180b7201fdd85427c79360a5" + }, + { + "file": "IQ2_XS.gguf", + "bytes": 935232, + "first_sha256": "91301828d93a083a1a0d8aba9cbbad2d0ba4f8cbef6ee6499b8ed2fdba6a074b", + "second_sha256": "91301828d93a083a1a0d8aba9cbbad2d0ba4f8cbef6ee6499b8ed2fdba6a074b" + }, + { + "file": "IQ2_XS.packed", + "bytes": 9472, + "first_sha256": "3281c7397dde054ea6c9c8a661cab43dbd0c982eb23b9a31523a616565c827e7", + "second_sha256": "3281c7397dde054ea6c9c8a661cab43dbd0c982eb23b9a31523a616565c827e7" + }, + { + "file": "IQ2_XXS-dense.gguf", + "bytes": 991296, + "first_sha256": "ed1114ef1967ad0f750758e42d3818b0ac48ce0948b3ba7bfc79c0491390882e", + "second_sha256": "ed1114ef1967ad0f750758e42d3818b0ac48ce0948b3ba7bfc79c0491390882e" + }, + { + "file": "IQ2_XXS.gguf", + "bytes": 934208, + "first_sha256": "e8c8da741d84aae97b3acdda1543d5a47c889998609ad37d5fdffff2c5e1440e", + "second_sha256": "e8c8da741d84aae97b3acdda1543d5a47c889998609ad37d5fdffff2c5e1440e" + }, + { + "file": "IQ2_XXS.packed", + "bytes": 8448, + "first_sha256": "4a3c10afb47315763ce507edcb5814e06d1be93a21b178cf73183302a3f1ceb9", + "second_sha256": "4a3c10afb47315763ce507edcb5814e06d1be93a21b178cf73183302a3f1ceb9" + }, + { + "file": "IQ3_S-dense.gguf", + "bytes": 991296, + "first_sha256": "2767698af39a4c7a62137500a9c29b5946d043814b482a5b6e691253cb6fd0d7", + "second_sha256": "2767698af39a4c7a62137500a9c29b5946d043814b482a5b6e691253cb6fd0d7" + }, + { + "file": "IQ3_S.gguf", + "bytes": 939840, + "first_sha256": "788e8878029509075a038030e94618ec5f01f98e5f822b18433f46db2f68ae48", + "second_sha256": "788e8878029509075a038030e94618ec5f01f98e5f822b18433f46db2f68ae48" + }, + { + "file": "IQ3_S.packed", + "bytes": 14080, + "first_sha256": "d4eebe1369e21b0efd6ec5c89f54c785849c02edde957ebd302dad376f63c14c", + "second_sha256": "d4eebe1369e21b0efd6ec5c89f54c785849c02edde957ebd302dad376f63c14c" + }, + { + "file": "IQ3_XXS-dense.gguf", + "bytes": 991296, + "first_sha256": "441678a7caac93dbb5da84ec3e245849b7e26184b027dc03016ce58d81c6e78e", + "second_sha256": "441678a7caac93dbb5da84ec3e245849b7e26184b027dc03016ce58d81c6e78e" + }, + { + "file": "IQ3_XXS.gguf", + "bytes": 938304, + "first_sha256": "124e98d2acba39dea81d04807f2d5d3863b47556245b6994a22b9ee52b60ad48", + "second_sha256": "124e98d2acba39dea81d04807f2d5d3863b47556245b6994a22b9ee52b60ad48" + }, + { + "file": "IQ3_XXS.packed", + "bytes": 12544, + "first_sha256": "d0f775f8e88d87ae542de6d16122a7a1cea09ebba91203f6513b74727082b6db", + "second_sha256": "d0f775f8e88d87ae542de6d16122a7a1cea09ebba91203f6513b74727082b6db" + }, + { + "file": "IQ4_NL-dense.gguf", + "bytes": 991296, + "first_sha256": "5eba44b7fed6a00a9e7644874cf7d973a55d3d06e2d953d12928ca1560fa3442", + "second_sha256": "5eba44b7fed6a00a9e7644874cf7d973a55d3d06e2d953d12928ca1560fa3442" + }, + { + "file": "IQ4_NL.gguf", + "bytes": 944192, + "first_sha256": "7dd47db4741f71965cb512e9965d9c7aed638f482d87a4132fbd34d8f7d3aad8", + "second_sha256": "7dd47db4741f71965cb512e9965d9c7aed638f482d87a4132fbd34d8f7d3aad8" + }, + { + "file": "IQ4_NL.packed", + "bytes": 18432, + "first_sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19", + "second_sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19" + }, + { + "file": "IQ4_XS-dense.gguf", + "bytes": 991296, + "first_sha256": "365673ef465a02cccca10fdc9c90080c9350afa408249e4582a12804d0ad7221", + "second_sha256": "365673ef465a02cccca10fdc9c90080c9350afa408249e4582a12804d0ad7221" + }, + { + "file": "IQ4_XS.gguf", + "bytes": 943168, + "first_sha256": "d5a51e4ed1629f705aa9b7edf761cf6a868db8ee73b881ed1d4f23af774e1c91", + "second_sha256": "d5a51e4ed1629f705aa9b7edf761cf6a868db8ee73b881ed1d4f23af774e1c91" + }, + { + "file": "IQ4_XS.packed", + "bytes": 17408, + "first_sha256": "35b2a41e738a6cb0a86deb4fd366f71ae2dc2a6fad7169f5a9afb53f523e919c", + "second_sha256": "35b2a41e738a6cb0a86deb4fd366f71ae2dc2a6fad7169f5a9afb53f523e919c" + }, + { + "file": "MXFP4-dense.gguf", + "bytes": 991296, + "first_sha256": "d2014d4edd896fa070308a926ebf6792cc5fc672c0e8df69f6bea7975c59b5f7", + "second_sha256": "d2014d4edd896fa070308a926ebf6792cc5fc672c0e8df69f6bea7975c59b5f7" + }, + { + "file": "MXFP4.gguf", + "bytes": 943168, + "first_sha256": "e60f444269c54001579cb5bd2ad652b00ac6070984f62ca4a88450e36e8260de", + "second_sha256": "e60f444269c54001579cb5bd2ad652b00ac6070984f62ca4a88450e36e8260de" + }, + { + "file": "MXFP4.packed", + "bytes": 17408, + "first_sha256": "baa828a753cc6f496f356271dad644976cd57d1218517e3f641fe56b7ec8db4a", + "second_sha256": "baa828a753cc6f496f356271dad644976cd57d1218517e3f641fe56b7ec8db4a" + }, + { + "file": "Q2_K-dense.gguf", + "bytes": 991296, + "first_sha256": "17f23518c6149e753d0d3af3edd348084fb9b1ff6ba7f9d290843917a045f3c1", + "second_sha256": "17f23518c6149e753d0d3af3edd348084fb9b1ff6ba7f9d290843917a045f3c1" + }, + { + "file": "Q2_K.gguf", + "bytes": 936512, + "first_sha256": "05a27385779d3dc34a9b537b7228afc756ea5450cfc8b6051913caab3eaf8e48", + "second_sha256": "05a27385779d3dc34a9b537b7228afc756ea5450cfc8b6051913caab3eaf8e48" + }, + { + "file": "Q2_K.packed", + "bytes": 10752, + "first_sha256": "655f77ebb32d974da7907da4a217c45bbbcd0b127df73a731f85afe427bb7dcc", + "second_sha256": "655f77ebb32d974da7907da4a217c45bbbcd0b127df73a731f85afe427bb7dcc" + }, + { + "file": "Q3_K-dense.gguf", + "bytes": 991296, + "first_sha256": "05d3df4df491774148b9d0bab74c10d0b328c5515452641859b0fd5e1ca052d3", + "second_sha256": "05d3df4df491774148b9d0bab74c10d0b328c5515452641859b0fd5e1ca052d3" + }, + { + "file": "Q3_K.gguf", + "bytes": 939840, + "first_sha256": "dcd12ce3b94f0f35033a220fbc3b72836321382857f1e1ffe077ab24239a28c2", + "second_sha256": "dcd12ce3b94f0f35033a220fbc3b72836321382857f1e1ffe077ab24239a28c2" + }, + { + "file": "Q3_K.packed", + "bytes": 14080, + "first_sha256": "e4c3a11c1bbff806ae2729033910b7db606452d1b9aa499aa2ced46740ad85d4", + "second_sha256": "e4c3a11c1bbff806ae2729033910b7db606452d1b9aa499aa2ced46740ad85d4" + }, + { + "file": "Q4_0-dense.gguf", + "bytes": 991296, + "first_sha256": "0e6554ba521edfde00d4d025a3058eabaaacdce6b24344f959e83d8dde35df7f", + "second_sha256": "0e6554ba521edfde00d4d025a3058eabaaacdce6b24344f959e83d8dde35df7f" + }, + { + "file": "Q4_0.gguf", + "bytes": 944192, + "first_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af", + "second_sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + { + "file": "Q4_0.packed", + "bytes": 18432, + "first_sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19", + "second_sha256": "cd292582c61c4b4b89606c85a80a6bb6ed8af98503a4d2f3737e02ae4e711d19" + }, + { + "file": "Q4_K-dense.gguf", + "bytes": 991296, + "first_sha256": "735e81ccff33826d58db60d097e5ef1f37fe385178c680460c4d620d3b5363b0", + "second_sha256": "735e81ccff33826d58db60d097e5ef1f37fe385178c680460c4d620d3b5363b0" + }, + { + "file": "Q4_K.gguf", + "bytes": 944192, + "first_sha256": "fbf3a0d6132bdb891b7747f0867a7c4da7262bc10a7cd63ea98a62a2523aa479", + "second_sha256": "fbf3a0d6132bdb891b7747f0867a7c4da7262bc10a7cd63ea98a62a2523aa479" + }, + { + "file": "Q4_K.packed", + "bytes": 18432, + "first_sha256": "b0c107c4b353e2a169247e0406c1eb015d7a6d97edd5906f710868b8c2cecce1", + "second_sha256": "b0c107c4b353e2a169247e0406c1eb015d7a6d97edd5906f710868b8c2cecce1" + }, + { + "file": "Q5_0-dense.gguf", + "bytes": 991296, + "first_sha256": "65a79d19109a0c5d8f0387da349eeac75ab47913ecf7749ef2498056e57e943f", + "second_sha256": "65a79d19109a0c5d8f0387da349eeac75ab47913ecf7749ef2498056e57e943f" + }, + { + "file": "Q5_0.gguf", + "bytes": 948288, + "first_sha256": "8609984dbf7afeb73ca7f8f82029bac3da9195140888c37294056ceff0c22e37", + "second_sha256": "8609984dbf7afeb73ca7f8f82029bac3da9195140888c37294056ceff0c22e37" + }, + { + "file": "Q5_0.packed", + "bytes": 22528, + "first_sha256": "b3531b1abd263bc4c6a3174b22fa1fd54a4b89b66e737f2eb3ba36be771d7528", + "second_sha256": "b3531b1abd263bc4c6a3174b22fa1fd54a4b89b66e737f2eb3ba36be771d7528" + }, + { + "file": "Q5_K-dense.gguf", + "bytes": 991296, + "first_sha256": "21b1500a894f9c5ec5e3c48cf8968a652ba45b697e18ffdd7f3bb4fc05ef2c17", + "second_sha256": "21b1500a894f9c5ec5e3c48cf8968a652ba45b697e18ffdd7f3bb4fc05ef2c17" + }, + { + "file": "Q5_K.gguf", + "bytes": 948288, + "first_sha256": "e054b9109627c979fb24a0cdb3497068a0e26b49d16944cbf69741d5d1e1ca99", + "second_sha256": "e054b9109627c979fb24a0cdb3497068a0e26b49d16944cbf69741d5d1e1ca99" + }, + { + "file": "Q5_K.packed", + "bytes": 22528, + "first_sha256": "5e63d961f36486c9636dd2657e9069bd83ece65e466c56c657265fbd97353b07", + "second_sha256": "5e63d961f36486c9636dd2657e9069bd83ece65e466c56c657265fbd97353b07" + }, + { + "file": "Q6_K-dense.gguf", + "bytes": 991296, + "first_sha256": "64a8f5bd55792bb8635ebb46c81f10580e28a74fec18fe568a8da387375d21e1", + "second_sha256": "64a8f5bd55792bb8635ebb46c81f10580e28a74fec18fe568a8da387375d21e1" + }, + { + "file": "Q6_K.gguf", + "bytes": 952640, + "first_sha256": "7e2c9a7f3ef05fe47848ada7b7dfdb4d1fe38f9a7c84becda978030130490c45", + "second_sha256": "7e2c9a7f3ef05fe47848ada7b7dfdb4d1fe38f9a7c84becda978030130490c45" + }, + { + "file": "Q6_K.packed", + "bytes": 26880, + "first_sha256": "134bba0f8076d43d31fcbbaeb37c3d8191b0d383e6595f787d3387e2e016738a", + "second_sha256": "134bba0f8076d43d31fcbbaeb37c3d8191b0d383e6595f787d3387e2e016738a" + }, + { + "file": "Q8_0-dense.gguf", + "bytes": 991296, + "first_sha256": "068dcce514e4f6262bea8da54470b1295f046238f4e27cbe08679410ea72509d", + "second_sha256": "068dcce514e4f6262bea8da54470b1295f046238f4e27cbe08679410ea72509d" + }, + { + "file": "Q8_0.gguf", + "bytes": 960576, + "first_sha256": "23064f6eb0c979d64eec4ac86299756c416ea40565b92ff38d527a39964b4d9d", + "second_sha256": "23064f6eb0c979d64eec4ac86299756c416ea40565b92ff38d527a39964b4d9d" + }, + { + "file": "Q8_0.packed", + "bytes": 34816, + "first_sha256": "8be599e194405f9b5429e578a03a5717e4d78f53ab4c9c11a52659495ce805ee", + "second_sha256": "8be599e194405f9b5429e578a03a5717e4d78f53ab4c9c11a52659495ce805ee" + }, + { + "file": "Q8_K-dense.gguf", + "bytes": 991296, + "first_sha256": "89ae45d6d0247457067a9e9dd23fef45c3805a08ee7708655e4979a0192e4f96", + "second_sha256": "89ae45d6d0247457067a9e9dd23fef45c3805a08ee7708655e4979a0192e4f96" + }, + { + "file": "Q8_K.gguf", + "bytes": 963136, + "first_sha256": "11d9cfc51e3c8badb93a0ca48c20280c46794647cab65cb30aa0102c8818bfbc", + "second_sha256": "11d9cfc51e3c8badb93a0ca48c20280c46794647cab65cb30aa0102c8818bfbc" + }, + { + "file": "Q8_K.packed", + "bytes": 37376, + "first_sha256": "bcf6f6691a787fe3f2a9bcc3d7923eeb4953862c30f904a6b10fc07410f8d657", + "second_sha256": "bcf6f6691a787fe3f2a9bcc3d7923eeb4953862c30f904a6b10fc07410f8d657" + } + ], + "all_identical": true +} diff --git a/include/vllm/model_executor/model_loader/gguf_keep_quant.h b/include/vllm/model_executor/model_loader/gguf_keep_quant.h index bad0fc712b..b7b6e93c03 100644 --- a/include/vllm/model_executor/model_loader/gguf_keep_quant.h +++ b/include/vllm/model_executor/model_loader/gguf_keep_quant.h @@ -104,8 +104,9 @@ using GgufRoutingAudit = // writing its vt block dtype to `*out`. The rule is `HasQuantDotKernel`, not a // list: the encoding needs a keep-quant `vec_dot` and its activation encoding // needs a `from_float`. False for the unquantized types, for Q8_K -// (activation-only, never a file weight type, and upstream gives it no `vec_dot` -// row at all) and for every unported encoding. The list used to be spelled out +// (the K-quants' activation encoding: a file type only the decode-only gather +// arm takes, and one upstream gives no `vec_dot` row at all) and for every +// unported encoding. The list used to be spelled out // here and went stale six encodings ago, so it deliberately is not any more. bool KeepQuantDType(uint32_t ggml_type, vt::DType* out); diff --git a/include/vllm/model_executor/models/qwen3_5_gguf_weights.h b/include/vllm/model_executor/models/qwen3_5_gguf_weights.h index 961b1b83af..75ea1e06d7 100644 --- a/include/vllm/model_executor/models/qwen3_5_gguf_weights.h +++ b/include/vllm/model_executor/models/qwen3_5_gguf_weights.h @@ -94,11 +94,14 @@ namespace vllm { // artifact off disk to populate pages nothing will ever look at. Measured: the // GLM-5.3 `UD-IQ1_S` load's RSS grew past 48 GiB against a 18.99 GiB resident // class, linearly, at the filesystem's read rate, with no plateau. +// The default role requires a dot kernel. An embedding role requires only a +// row decoder and refuses matrix repacks. The caller sets embedding metadata. OwnedTensor OwnGgufQuantBlocks(const GgufTensorInfo& tensor, int64_t n, int64_t k, int64_t row_offset = 0, const GgufFile* mmap_src = nullptr, bool repack = false, bool cuda_align = false, - bool prefault = true); + bool prefault = true, + GgufTensorRole role = GgufTensorRole::kMatmulWeight); // L6 (keep-f16 residency). Take `n` rows of `k` F16 elements of `tensor`'s raw // bytes — starting at row `row_offset` (how a stacked [E, out, in] expert tensor diff --git a/include/vt/dtype.h b/include/vt/dtype.h index 00962501e4..475b5df190 100644 --- a/include/vt/dtype.h +++ b/include/vt/dtype.h @@ -38,8 +38,9 @@ namespace vt { // :305-310 (block_q3_K), :317-327 (block_q4_K), :334-345 (block_q5_K), // :352-357 (block_q6_K), :361-365 (block_q8_K), :371-374 (block_iq2_xxs), // :385-400 (block_iq3_xxs) -// kQ8_K is ACTIVATION-ONLY: it is the `vec_dot_type` of the K-quants and never -// appears as a weight/storage type in a GGUF file. +// kQ8_K is ACTIVATION-ONLY: it is the `vec_dot_type` of the K-quants and has no +// weight-side `vec_dot`, so it is never keep-quant capable — although a GGUF +// file can carry it, and only the decode-only embedding gather takes it packed. // // kQ2_K, kIQ2_XXS and kIQ3_XXS are the ~2-3-bit storage encodings the `unsloth/ // DeepSeek-V4-Flash-GGUF UD-IQ2_XXS/UD-Q2_K_XL` checkpoints use (the real diff --git a/scripts/agent-preflight.sh b/scripts/agent-preflight.sh index f8e6c0d5d3..cf0675091a 100755 --- a/scripts/agent-preflight.sh +++ b/scripts/agent-preflight.sh @@ -192,6 +192,7 @@ SUITES=( test_agent_issue_index test_check_symbol_anchors test_check_oracle_denominator_flags + test_rocm_quant_gather_compare test_rocm_strix_ourarm_staged # The TOKENGATE job's build stage (#2895). `--no-build-isolation` makes pip # skip the pin's [build-system] requires, so the harness has to install them diff --git a/src/vllm/model_executor/model_loader/gguf_keep_quant.cpp b/src/vllm/model_executor/model_loader/gguf_keep_quant.cpp index 3c93f22ea2..633b86ba9f 100644 --- a/src/vllm/model_executor/model_loader/gguf_keep_quant.cpp +++ b/src/vllm/model_executor/model_loader/gguf_keep_quant.cpp @@ -203,7 +203,7 @@ bool KeepQuantGatherDType(uint32_t ggml_type, vt::DType* out) { // CUDA backend (cuda_ops.cu, through cuda_quant_dequant.cuh) -- named in prose // rather than as the enumerator on purpose, because the leakage checker greps // the token in comments too, and rightly so: a prose mention is how the next -// hand-kept device list starts. METAL, VULKAN, ROCM and +// hand-kept device list starts. METAL, VULKAN and // TENSTORRENT register only `kEmbedding`, whose kernels each assert a float // table by name, so they answer false here and keep their pre-existing // expand-bf16 residency -- and their gather arms are owed. @@ -221,8 +221,8 @@ bool DeviceQuantGatherSupported(vt::DeviceType dev) { bool KeepQuantDType(uint32_t ggml_type, vt::DType* out) { vt::DType dt = vt::DType::kF32; if (!vt::BlockDTypeFromGgmlTypeId(ggml_type, &dt)) return false; - // Q8_K is the K-quants' ACTIVATION encoding; it never appears as a file - // weight type and has no vec_dot, so it is not keep-quant capable. + // Q8_K is the K-quants' ACTIVATION encoding and has no weight-side vec_dot, so + // it is not keep-quant capable, although a file CAN carry it (reader case 15). if (!vt::cpu::HasQuantDotKernel(dt)) return false; if (out != nullptr) *out = dt; return true; diff --git a/src/vllm/model_executor/model_loader/gguf_reader.cpp b/src/vllm/model_executor/model_loader/gguf_reader.cpp index 2cdc19ac04..aa1f372adc 100644 --- a/src/vllm/model_executor/model_loader/gguf_reader.cpp +++ b/src/vllm/model_executor/model_loader/gguf_reader.cpp @@ -244,6 +244,12 @@ const GgmlTypeTraits* FindGgmlTraits(uint32_t type) { static constexpr GgmlTypeTraits t{256, 210, "Q6_K"}; return &t; } + case 15: { + // Q8_K is decoder-only for embedding tables. Stock llama.cpp 10bf611e, + // ggml-common.h:370-376: f32 delta + 256 i8 values + 16 i16 sums. + static constexpr GgmlTypeTraits t{256, 292, "Q8_K"}; + return &t; + } case 16: { // block_iq2_xxs (ggml-common.h:371-374): f16 d + QK_K/8 u16 qs // = 2 + 32*2 = 66. The Unsloth-Dynamic `UD-IQ2_XXS` ~2-bit encoding diff --git a/src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp b/src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp index f341873d07..2eae71b739 100644 --- a/src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp +++ b/src/vllm/model_executor/models/qwen3_5_gguf_weights.cpp @@ -84,11 +84,15 @@ inline void PrefaultBorrowedSpan(const uint8_t* src, size_t bytes) { OwnedTensor OwnGgufQuantBlocks(const GgufTensorInfo& tensor, int64_t n, int64_t k, int64_t row_offset, const GgufFile* mmap_src, bool repack, - bool cuda_align, bool prefault) { + bool cuda_align, bool prefault, GgufTensorRole role) { vt::DType dt = vt::DType::kF32; - VT_CHECK(KeepQuantDType(tensor.ggml_type, &dt), + const bool gather = role == GgufTensorRole::kEmbeddingTable; + VT_CHECK(gather ? KeepQuantGatherDType(tensor.ggml_type, &dt) + : KeepQuantDType(tensor.ggml_type, &dt), "qwen3_5 gguf: keep-quant on a non-keep-quant encoding for " + tensor.name); + VT_CHECK(!gather || (!repack && !cuda_align), + "qwen3_5 gguf: embedding blocks cannot use a matrix repack"); VT_CHECK(n > 0 && k > 0 && row_offset >= 0, "qwen3_5 gguf: bad keep-quant slice for " + tensor.name); // Throws when k is not a whole number of blocks (ggml_row_size contract). @@ -810,7 +814,8 @@ void LoadEmbedAndHead(const GgufFile& g, const GgufLoadPolicy& pol, // it does on the f16 arm — this tensor is a table, not a [N,K] GEMM weight. *embed = OwnGgufQuantBlocks(et, et.shape[0], et.shape[1], /*row_offset=*/0, MmapSrc(g, pol), /*repack=*/false, - /*cuda_align=*/false); + /*cuda_align=*/false, /*prefault=*/true, + GgufTensorRole::kEmbeddingTable); embed->nk = false; } else if (embed_r == GgufResidency::kKeepF16) { VT_CHECK(et.shape.size() == 2, "qwen3_5 gguf: token_embd must be 2-D"); diff --git a/src/vt/cpu/cpu_quant_dequant.cpp b/src/vt/cpu/cpu_quant_dequant.cpp index c8c4a918d8..02f954ae54 100644 --- a/src/vt/cpu/cpu_quant_dequant.cpp +++ b/src/vt/cpu/cpu_quant_dequant.cpp @@ -265,9 +265,9 @@ void DequantQ6_K(const uint8_t* data, int64_t nb, float* y) { } // block_q8_K = { f32 d; i8 qs[256]; i16 bsums[16]; } (292 bytes) -// dequantize_row_q8_K (ggml-quants.c). Q8_K is the K-quant ACTIVATION type; it -// never appears in a GGUF file, but the decoder completes the table and lets -// the activation-quant round trip be unit-tested in G2. +// dequantize_row_q8_K (ggml-quants.c). Q8_K is the K-quant ACTIVATION type; a +// GGUF file can still carry it (the reader admits id 15) and the decode-only +// embedding gather takes it packed. The decoder completes the table for G2. void DequantQ8_K(const uint8_t* data, int64_t nb, float* y) { constexpr int qk = 256; for (int64_t i = 0; i < nb; ++i) { diff --git a/src/vt/cuda/cuda_quant_dequant.cuh b/src/vt/cuda/cuda_quant_dequant.cuh index 74cbee6606..d59e14947c 100644 --- a/src/vt/cuda/cuda_quant_dequant.cuh +++ b/src/vt/cuda/cuda_quant_dequant.cuh @@ -384,9 +384,9 @@ struct DqQ6_K { }; // block_q8_K = { f32 d; i8 qs[256]; i16 bsums[16] } — DequantQ8_K. -// Activation-only encoding; it never appears as a FILE weight, so it can never -// reach the gather from the loader. It is here because the CPU table carries it -// and an asymmetric capability list is a defect waiting to be found later. +// Activation-only encoding with no weight-side vec_dot, but a GGUF file can +// carry it: the reader admits id 15 and the decode-only embedding gather takes +// it packed, so this decoder belongs here beside the CPU table's entry. struct DqQ8_K { static constexpr int kBytes = 292; static constexpr int kElems = 256; diff --git a/src/vt/rocm/rocm_embedding_quant.h b/src/vt/rocm/rocm_embedding_quant.h new file mode 100644 index 0000000000..9decbb5053 --- /dev/null +++ b/src/vt/rocm/rocm_embedding_quant.h @@ -0,0 +1,22 @@ +// Native packed-row gather seam for BACKEND-ROCM-QUANT-GATHER (#3093). +#pragma once + +#include + +#include "vt/ops.h" + +namespace vt::rocm { + +// Constant-size scratch copied after the gather finishes on its stream. +struct EmbeddingQuantError { + int32_t status; + int32_t reserved; + int64_t id; +}; +static_assert(sizeof(EmbeddingQuantError) == 16); + +bool EmbeddingQuantSupported(DType dtype); +void EmbeddingQuantKernelRocm(Queue& queue, Tensor& out, const Tensor& table, + const Tensor& ids); + +} // namespace vt::rocm diff --git a/src/vt/rocm/rocm_embedding_quant.hip b/src/vt/rocm/rocm_embedding_quant.hip new file mode 100644 index 0000000000..5542f10bad --- /dev/null +++ b/src/vt/rocm/rocm_embedding_quant.hip @@ -0,0 +1,179 @@ +// Native selected-row decode for BACKEND-ROCM-QUANT-GATHER (#3093). +// Primary: plugin d4c1f0d vocal_embeds.py:79-98. The plugin selects packed rows +// then decodes them. This HIP adaptation fuses both operations into caller output. +#include + +#include +#include +#include +#include + +#include "vt/rocm/rocm_embedding_quant.h" +#include "vt/rocm/rocm_quant_dequant.h" + +namespace vt::rocm { +namespace { + +constexpr int kThreads = 128; +constexpr int64_t kMaxBlocks = 1024; + +void Check(hipError_t error, const char* operation) { + if (error != hipSuccess) { + throw std::runtime_error(std::string("vt rocm: embedding_quant: ") + operation + + ": " + hipGetErrorString(error)); + } +} + +class DeviceError { + public: + DeviceError() { + Check(hipMalloc(reinterpret_cast(&pointer_), sizeof(EmbeddingQuantError)), + "allocate bounds record"); + } + ~DeviceError() { + if (pointer_ != nullptr) (void)hipFree(pointer_); + } + DeviceError(const DeviceError&) = delete; + DeviceError& operator=(const DeviceError&) = delete; + EmbeddingQuantError* get() const { return pointer_; } + void Free() { + Check(hipFree(pointer_), "free bounds record"); + pointer_ = nullptr; + } + + private: + EmbeddingQuantError* pointer_ = nullptr; +}; + +// One thread decodes one selected block. The cap retains bounded launch size; +// the grid-stride loop covers every remaining block without truncating IDs. +template +__global__ void EmbeddingQuantGatherKernel(Tout* out, const uint8_t* table, + const Tid* ids, int64_t total, + int64_t blocks_per_row, int64_t width, + int64_t vocab, size_t row_bytes, + EmbeddingQuantError* error) { + const int64_t step = static_cast(gridDim.x) * blockDim.x; + for (int64_t index = static_cast(blockIdx.x) * blockDim.x + threadIdx.x; + index < total; index += step) { + const int64_t token = index / blocks_per_row; + const int64_t block = index - token * blocks_per_row; + const int64_t id = static_cast(ids[token]); + if (id < 0 || id >= vocab) { + if (atomicCAS(&error->status, 0, 1) == 0) error->id = id; + continue; // Reject before reading any packed bytes for this ID. + } + const auto* packed = table + static_cast(id) * row_bytes + + static_cast(block) * Codec::kBytes; + Codec::template Decode(packed, out + token * width + block * Codec::kElems); + } +} + +// The same complete list defines both capability and dispatch. Registration +// enables the existing loader policy, whose device gate has no dtype argument. +#define VT_ROCM_GATHER_TYPES(X) \ + X(kQ4_0, DqQ4_0) \ + X(kQ5_0, DqQ5_0) \ + X(kQ8_0, DqQ8_0) \ + X(kQ2_K, DqQ2_K) \ + X(kQ3_K, DqQ3_K) \ + X(kQ4_K, DqQ4_K) \ + X(kQ5_K, DqQ5_K) \ + X(kQ6_K, DqQ6_K) \ + X(kQ8_K, DqQ8_K) \ + X(kIQ2_XXS, DqIQ2_XXS) \ + X(kIQ2_XS, DqIQ2_XS) \ + X(kIQ3_XXS, DqIQ3_XXS) \ + X(kIQ1_S, DqIQ1_S) \ + X(kIQ4_NL, DqIQ4_NL) \ + X(kIQ3_S, DqIQ3_S) \ + X(kIQ2_S, DqIQ2_S) \ + X(kIQ4_XS, DqIQ4_XS) \ + X(kMXFP4, DqMXFP4) \ + X(kIQ1_XXXS, DqIQ1_XXXS) + +template +void Launch(hipStream_t stream, Tensor& out, const Tensor& table, const Tensor& ids, + int64_t total, int64_t blocks_per_row, size_t row_bytes, + EmbeddingQuantError* error) { + const auto grid = static_cast(std::min((total - 1) / kThreads + 1, kMaxBlocks)); + switch (table.dtype) { +#define VT_ROCM_GATHER_LAUNCH(DT, CODEC) \ + case DType::DT: \ + VT_CHECK(BlockBytes(table.dtype) == CODEC::kBytes && \ + BlockElems(table.dtype) == CODEC::kElems, \ + "rocm embedding_quant: codec geometry mismatch"); \ + EmbeddingQuantGatherKernel<<>>( \ + static_cast(out.data), static_cast(table.data), \ + static_cast(ids.data), total, blocks_per_row, table.shape[1], \ + table.shape[0], row_bytes, error); \ + break; + VT_ROCM_GATHER_TYPES(VT_ROCM_GATHER_LAUNCH) +#undef VT_ROCM_GATHER_LAUNCH + default: + throw std::runtime_error("rocm embedding_quant: unsupported table dtype"); + } + Check(hipGetLastError(), "launch"); +} + +} // namespace + +bool EmbeddingQuantSupported(DType dtype) { + switch (dtype) { +#define VT_ROCM_GATHER_SUPPORTED(DT, CODEC) case DType::DT: return true; + VT_ROCM_GATHER_TYPES(VT_ROCM_GATHER_SUPPORTED) +#undef VT_ROCM_GATHER_SUPPORTED + default: return false; + } +} + +void EmbeddingQuantKernelRocm(Queue& queue, Tensor& out, const Tensor& table, + const Tensor& ids) { + // vt::Embedding validates ranks, shapes, dtypes, layout, and device equality. + VT_CHECK(EmbeddingQuantSupported(table.dtype), "rocm embedding_quant: unsupported table dtype"); + const int64_t tokens = ids.shape[0]; + const int64_t width = table.shape[1]; + const int64_t vocab = table.shape[0]; + VT_CHECK(tokens >= 0 && width >= 0 && vocab >= 0, "rocm embedding_quant: negative shape"); + if (tokens == 0 || width == 0) return; + VT_CHECK(vocab > 0, "rocm embedding_quant: empty table (vocab 0)"); + VT_CHECK(tokens <= std::numeric_limits::max() / width, + "rocm embedding_quant: output element count overflow"); + const int64_t blocks_per_row = width / BlockElems(table.dtype); + const int64_t total = tokens * blocks_per_row; + const size_t row_bytes = RowSizeBytes(table.dtype, width); + VT_CHECK(row_bytes <= std::numeric_limits::max() / static_cast(vocab), + "rocm embedding_quant: table byte count overflow"); + + // Allocation, launch, and error cleanup belong to the queue's device even + // when the caller most recently selected another device on this host thread. + Check(hipSetDevice(queue.device.index), "bind queue device"); + const auto stream = static_cast(queue.handle); + DeviceError error; + Check(hipMemsetAsync(error.get(), 0, sizeof(EmbeddingQuantError), stream), "clear bounds record"); + // f32 is an explicit caller-selected primitive mode. Model gathers use bf16. + if (out.dtype == DType::kF32) { + if (ids.dtype == DType::kI32) + Launch(stream, out, table, ids, total, blocks_per_row, row_bytes, error.get()); + else + Launch(stream, out, table, ids, total, blocks_per_row, row_bytes, error.get()); + } else { + if (ids.dtype == DType::kI32) + Launch(stream, out, table, ids, total, blocks_per_row, row_bytes, error.get()); + else + Launch(stream, out, table, ids, total, blocks_per_row, row_bytes, error.get()); + } + EmbeddingQuantError host_error{}; + Check(hipMemcpyAsync(&host_error, error.get(), sizeof(host_error), hipMemcpyDeviceToHost, stream), + "copy bounds record"); + Check(hipStreamSynchronize(stream), "synchronize bounds record"); + error.Free(); + if (host_error.status != 0) { + throw std::runtime_error("vt rocm: embedding_quant: id " + std::to_string(host_error.id) + + " out of range [0, " + std::to_string(vocab) + ")"); + } +} + +#undef VT_ROCM_GATHER_TYPES + +} // namespace vt::rocm diff --git a/src/vt/rocm/rocm_ops.hip b/src/vt/rocm/rocm_ops.hip index 841f49bf3c..0edf0e276b 100644 --- a/src/vt/rocm/rocm_ops.hip +++ b/src/vt/rocm/rocm_ops.hip @@ -1,6 +1,7 @@ // ROCm op-table registration (BACKEND-ROCM). #include "vt/op_provider.h" #include "vt/ops.h" +#include "vt/rocm/rocm_embedding_quant.h" namespace vt::rocm { @@ -186,6 +187,8 @@ struct Registrar { reinterpret_cast(static_cast(&RmsNormKernelRocm))); RegisterOp(OpId::kEmbedding, DeviceType::kROCM, reinterpret_cast(static_cast(&EmbeddingKernelRocm))); + RegisterOp(OpId::kEmbeddingQuant, DeviceType::kROCM, + reinterpret_cast(static_cast(&EmbeddingQuantKernelRocm))); RegisterOp(OpId::kMulScalar, DeviceType::kROCM, reinterpret_cast(static_cast(&MulScalarKernelRocm))); RegisterOp(OpId::kSoftCap, DeviceType::kROCM, diff --git a/src/vt/rocm/rocm_quant_dequant.h b/src/vt/rocm/rocm_quant_dequant.h new file mode 100644 index 0000000000..c9add9f76f --- /dev/null +++ b/src/vt/rocm/rocm_quant_dequant.h @@ -0,0 +1,671 @@ +// Selected-block decoders for BACKEND-ROCM-QUANT-GATHER (#3093). +// Independently owned port of src/vt/cuda/cuda_quant_dequant.cuh at 6db4bef90685. +// Primary behavior: plugin d4c1f0d vocal_embeds.py:79-98 selects and decodes rows. +// This fused HIP implementation writes selected blocks directly to caller output. +// Scalar expressions preserve the pinned ggml-quants.c dequantize_row_* order. +// Stock pin: 10bf611e533d81f739128304991c5e133c6aebd8. Fork: 36fe8e1cc. +// Byte assembly permits every packed-table offset, including odd block strides. +#pragma once + +#include +#include + +#include "vt/rocm/rocm_quant_iq_tables.h" + +namespace vt::rocm { +namespace { + +// Numeric conversions mirror src/vt/dtype.cpp, including bf16 ties and NaNs. +__device__ inline float DF16ToF32(uint16_t h) { + uint32_t sign = static_cast(h & 0x8000) << 16; + uint32_t exp = (h >> 10) & 0x1F; + uint32_t mant = h & 0x3FF; + if (exp == 0x1F) return __int_as_float(sign | 0x7F800000 | (mant << 13)); + if (exp == 0) { + if (mant == 0) return __int_as_float(sign); + int shift = 0; + while ((mant & 0x400) == 0) { + mant <<= 1; + ++shift; + } + mant &= 0x3FF; + return __int_as_float(sign | ((113 - shift) << 23) | (mant << 13)); + } + return __int_as_float(sign | ((exp + 112) << 23) | (mant << 13)); +} + +__device__ inline uint16_t DF32ToBF16(float f) { + uint32_t u = __float_as_int(f); + if ((u & 0x7F800000) == 0x7F800000 && (u & 0x7FFFFF)) { + return static_cast((u >> 16) | 0x0040); + } + uint32_t rounding = 0x7FFF + ((u >> 16) & 1); + return static_cast((u + rounding) >> 16); +} + +__device__ inline float DE8M0ToF32Half(uint8_t x) { + const uint32_t bits = + x < 2 ? (0x00200000u << x) : (static_cast(x - 1) << 23); + return __int_as_float(static_cast(bits)); +} + +// --- unaligned little-endian loads ----------------------------------------- +__device__ inline uint16_t DqU16(const uint8_t* p) { + return static_cast(static_cast(p[0]) | + (static_cast(p[1]) << 8)); +} + +__device__ inline uint32_t DqU32(const uint8_t* p) { + return static_cast(p[0]) | (static_cast(p[1]) << 8) | + (static_cast(p[2]) << 16) | (static_cast(p[3]) << 24); +} + +// `ReadF16` of cpu_quant_dequant.cpp: little-endian ggml_half widened to f32. +__device__ inline float DqF16(const uint8_t* p) { return DF16ToF32(DqU16(p)); } + +// The f32 super-block delta Q8_K stores raw (cpu_quant_dequant.cpp DequantQ8_K). +__device__ inline float DqF32(const uint8_t* p) { + return __int_as_float(static_cast(DqU32(p))); +} + +// --- output store ------------------------------------------------------------ +// `StoreF32(out, i, v)` of the CPU arm, specialised on the out dtype. bf16 uses +// the SAME round-to-nearest-even + NaN-quieting `DF32ToBF16` the rest of this +// TU uses, which is the bit-exact port of `vt::F32ToBF16` (src/vt/dtype.cpp). +template +struct DqStore; +template <> +struct DqStore { + static __device__ inline void Set(float* y, int64_t i, float v) { y[i] = v; } +}; +template <> +struct DqStore { + static __device__ inline void Set(uint16_t* y, int64_t i, float v) { + y[i] = DF32ToBF16(v); + } +}; + +// HIP and the scalar control use -ffp-contract=off to preserve two operations. +__device__ inline float DqMulSub(float a, float b, float c) { + return a * b - c; +} + +// get_scale_min_k4 (ggml-quants.c:880); cpu_quant_dequant.cpp GetScaleMinK4. +__device__ inline void DqGetScaleMinK4(int j, const uint8_t* q, uint8_t* d, uint8_t* m) { + if (j < 4) { + *d = q[j] & 63; + *m = q[j + 4] & 63; + } else { + *d = static_cast((q[j + 4] & 0xF) | ((q[j - 4] >> 6) << 4)); + *m = static_cast((q[j + 4] >> 4) | ((q[j - 0] >> 6) << 4)); + } +} + +// --- one CODEC per block dtype ---------------------------------------------- +// Each `Decode` writes exactly `kElems` outputs starting at `y[0]`, in the same +// ORDER and with the same float expression as its `Dequant*` twin in +// src/vt/cpu/cpu_quant_dequant.cpp. `kBytes` is the block stride and is +// cross-checked against `vt::BlockBytes` by the host launcher below. + +// block_q4_0 = { f16 d; u8 qs[16] }. +// Stock ggml/src/ggml-quants.c:459. +struct DqQ4_0 { + static constexpr int kBytes = 18; + static constexpr int kElems = 32; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const uint8_t* qs = blk + 2; + for (int j = 0; j < 16; ++j) { + const int x0 = (qs[j] & 0x0F) - 8; + const int x1 = (qs[j] >> 4) - 8; + DqStore::Set(y, j, x0 * d); + DqStore::Set(y, j + 16, x1 * d); + } + } +}; + +// block_q5_0 = { f16 d; u8 qh[4]; u8 qs[16] }. +// Stock ggml/src/ggml-quants.c:500. +struct DqQ5_0 { + static constexpr int kBytes = 22; + static constexpr int kElems = 32; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const uint32_t qh = DqU32(blk + 2); + const uint8_t* qs = blk + 6; + for (int j = 0; j < 16; ++j) { + const uint8_t xh_0 = static_cast(((qh >> (j + 0)) << 4) & 0x10); + const uint8_t xh_1 = static_cast((qh >> (j + 12)) & 0x10); + const int32_t x0 = ((qs[j] & 0x0F) | xh_0) - 16; + const int32_t x1 = ((qs[j] >> 4) | xh_1) - 16; + DqStore::Set(y, j, x0 * d); + DqStore::Set(y, j + 16, x1 * d); + } + } +}; + +// block_iq4_nl = { f16 d; u8 qs[16] }. +// Stock ggml/src/ggml-quants.c:2725. +struct DqIQ4_NL { + static constexpr int kBytes = 18; + static constexpr int kElems = 32; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const uint8_t* qs = blk + 2; + for (int j = 0; j < 16; ++j) { + DqStore::Set(y, j, d * d_kvalues_iq4nl[qs[j] & 0x0F]); + DqStore::Set(y, j + 16, d * d_kvalues_iq4nl[qs[j] >> 4]); + } + } +}; + +// block_q8_0 = { f16 d; i8 qs[32] }. +// Stock ggml/src/ggml-quants.c:553. +struct DqQ8_0 { + static constexpr int kBytes = 34; + static constexpr int kElems = 32; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const int8_t* qs = reinterpret_cast(blk + 2); + for (int j = 0; j < 32; ++j) DqStore::Set(y, j, qs[j] * d); + } +}; + +// block_mxfp4 = { u8 e; u8 qs[16] }. +// Stock ggml/src/ggml-quants.c:569. +struct DqMXFP4 { + static constexpr int kBytes = 17; + static constexpr int kElems = 32; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DE8M0ToF32Half(blk[0]); + const uint8_t* qs = blk + 1; + for (int j = 0; j < 16; ++j) { + const int8_t x0 = d_kvalues_mxfp4[qs[j] & 0x0F]; + const int8_t x1 = d_kvalues_mxfp4[qs[j] >> 4]; + DqStore::Set(y, j, x0 * d); + DqStore::Set(y, j + 16, x1 * d); + } + } +}; + +// block_q2_K = { u8 scales[16]; u8 qs[64]; f16 d; f16 dmin }. +// Stock ggml/src/ggml-quants.c:961. +struct DqQ2_K { + static constexpr int kBytes = 84; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const uint8_t* scales = blk; + const uint8_t* q = blk + 16; + const float d = DqF16(blk + 80); + const float min = DqF16(blk + 82); + int o = 0; + int is = 0; + for (int n = 0; n < 256; n += 128) { + int shift = 0; + for (int j = 0; j < 4; ++j) { + uint8_t sc = scales[is++]; + float dl = d * (sc & 0xF); + float ml = min * (sc >> 4); + for (int l = 0; l < 16; ++l) + DqStore::Set( + y, o++, DqMulSub(dl, static_cast((q[l] >> shift) & 3), ml)); + sc = scales[is++]; + dl = d * (sc & 0xF); + ml = min * (sc >> 4); + for (int l = 0; l < 16; ++l) + DqStore::Set( + y, o++, DqMulSub(dl, static_cast((q[l + 16] >> shift) & 3), ml)); + shift += 2; + } + q += 32; + } + } +}; + +// block_q3_K = { u8 hmask[32]; u8 qs[64]; u8 scales[12]; f16 d }. +// Stock ggml/src/ggml-quants.c:1305. +struct DqQ3_K { + static constexpr int kBytes = 110; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const uint32_t kmask1 = 0x03030303; + const uint32_t kmask2 = 0x0f0f0f0f; + const uint8_t* hm = blk; + const uint8_t* q = blk + 32; + const uint8_t* sc_raw = blk + 96; + const float d_all = DqF16(blk + 108); + + uint32_t aux[4]; + aux[0] = DqU32(sc_raw + 0); + aux[1] = DqU32(sc_raw + 4); + aux[2] = DqU32(sc_raw + 8); + const uint32_t tmp = aux[2]; + aux[2] = ((aux[0] >> 4) & kmask2) | (((tmp >> 4) & kmask1) << 4); + aux[3] = ((aux[1] >> 4) & kmask2) | (((tmp >> 6) & kmask1) << 4); + aux[0] = (aux[0] & kmask2) | (((tmp >> 0) & kmask1) << 4); + aux[1] = (aux[1] & kmask2) | (((tmp >> 2) & kmask1) << 4); + const int8_t* scales = reinterpret_cast(aux); + + int o = 0; + int is = 0; + uint8_t m = 1; + for (int n = 0; n < 256; n += 128) { + int shift = 0; + for (int j = 0; j < 4; ++j) { + float dl = d_all * (scales[is++] - 32); + for (int l = 0; l < 16; ++l) + DqStore::Set(y, o++, + dl * (static_cast((q[l + 0] >> shift) & 3) - + ((hm[l + 0] & m) ? 0 : 4))); + dl = d_all * (scales[is++] - 32); + for (int l = 0; l < 16; ++l) + DqStore::Set(y, o++, + dl * (static_cast((q[l + 16] >> shift) & 3) - + ((hm[l + 16] & m) ? 0 : 4))); + shift += 2; + m = static_cast(m << 1); + } + q += 32; + } + } +}; + +// block_q4_K = { f16 d; f16 dmin; u8 scales[12]; u8 qs[128] }. +// Stock ggml/src/ggml-quants.c:1529. +struct DqQ4_K { + static constexpr int kBytes = 144; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const float min = DqF16(blk + 2); + const uint8_t* scales = blk + 4; + const uint8_t* q = blk + 16; + int o = 0; + int is = 0; + uint8_t sc = 0; + uint8_t mm = 0; + for (int j = 0; j < 256; j += 64) { + DqGetScaleMinK4(is + 0, scales, &sc, &mm); + const float d1 = d * sc; + const float m1 = min * mm; + DqGetScaleMinK4(is + 1, scales, &sc, &mm); + const float d2 = d * sc; + const float m2 = min * mm; + for (int l = 0; l < 32; ++l) + DqStore::Set(y, o++, DqMulSub(d1, q[l] & 0xF, m1)); + for (int l = 0; l < 32; ++l) + DqStore::Set(y, o++, DqMulSub(d2, q[l] >> 4, m2)); + q += 32; + is += 2; + } + } +}; + +// block_q5_K = { f16 d; f16 dmin; u8 scales[12]; u8 qh[32]; u8 qs[128] } +// Stock ggml/src/ggml-quants.c:1731. +struct DqQ5_K { + static constexpr int kBytes = 176; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const float min = DqF16(blk + 2); + const uint8_t* scales = blk + 4; + const uint8_t* qh = blk + 16; + const uint8_t* ql = blk + 48; + int o = 0; + int is = 0; + uint8_t sc = 0; + uint8_t mm = 0; + uint8_t u1 = 1; + uint8_t u2 = 2; + for (int j = 0; j < 256; j += 64) { + DqGetScaleMinK4(is + 0, scales, &sc, &mm); + const float d1 = d * sc; + const float m1 = min * mm; + DqGetScaleMinK4(is + 1, scales, &sc, &mm); + const float d2 = d * sc; + const float m2 = min * mm; + for (int l = 0; l < 32; ++l) + DqStore::Set( + y, o++, DqMulSub(d1, (ql[l] & 0xF) + ((qh[l] & u1) ? 16 : 0), m1)); + for (int l = 0; l < 32; ++l) + DqStore::Set( + y, o++, DqMulSub(d2, (ql[l] >> 4) + ((qh[l] & u2) ? 16 : 0), m2)); + ql += 32; + is += 2; + u1 = static_cast(u1 << 2); + u2 = static_cast(u2 << 2); + } + } +}; + +// block_q6_K = { u8 ql[128]; u8 qh[64]; i8 scales[16]; f16 d }. +// Stock ggml/src/ggml-quants.c:1939. +struct DqQ6_K { + static constexpr int kBytes = 210; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const uint8_t* ql = blk; + const uint8_t* qh = blk + 128; + const int8_t* sc = reinterpret_cast(blk + 192); + const float d = DqF16(blk + 208); + int o = 0; + for (int n = 0; n < 256; n += 128) { + for (int l = 0; l < 32; ++l) { + const int is = l / 16; + const int8_t q1 = static_cast((ql[l + 0] & 0xF) | + (((qh[l] >> 0) & 3) << 4)) - 32; + const int8_t q2 = static_cast((ql[l + 32] & 0xF) | + (((qh[l] >> 2) & 3) << 4)) - 32; + const int8_t q3 = static_cast((ql[l + 0] >> 4) | + (((qh[l] >> 4) & 3) << 4)) - 32; + const int8_t q4 = static_cast((ql[l + 32] >> 4) | + (((qh[l] >> 6) & 3) << 4)) - 32; + DqStore::Set(y, o + l + 0, d * sc[is + 0] * q1); + DqStore::Set(y, o + l + 32, d * sc[is + 2] * q2); + DqStore::Set(y, o + l + 64, d * sc[is + 4] * q3); + DqStore::Set(y, o + l + 96, d * sc[is + 6] * q4); + } + o += 128; + ql += 64; + qh += 32; + sc += 8; + } + } +}; + +// block_q8_K = { f32 d; i8 qs[256]; i16 bsums[16] }. +// The GGUF reader admits type 15 for embedding tables. Matrix weights still +// require a dot kernel; the embedding role keeps these original packed bytes. +// Stock ggml/src/ggml-quants.c:2807. +struct DqQ8_K { + static constexpr int kBytes = 292; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF32(blk); + const int8_t* qs = reinterpret_cast(blk + 4); + for (int j = 0; j < 256; ++j) DqStore::Set(y, j, d * qs[j]); + } +}; + +// block_iq2_xxs = { f16 d; u16 qs[32] }. +// `aux32` is a memcpy of the eight bytes at `qs + 8*ib32` on the CPU side, so +// `aux8` IS those bytes; the device reads them directly and takes only aux32[1] +// through the byte-assembly load. +// Stock ggml/src/ggml-quants.c:2488. +struct DqIQ2_XXS { + static constexpr int kBytes = 66; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const uint8_t* qs = blk + 2; + int o = 0; + for (int ib32 = 0; ib32 < 8; ++ib32) { + const uint8_t* aux8 = qs + 8 * ib32; + const uint32_t aux32_1 = DqU32(qs + 8 * ib32 + 4); + const float db = d * (0.5f + (aux32_1 >> 28)) * 0.25f; + for (int l = 0; l < 4; ++l) { + const uint8_t* grid = + reinterpret_cast(d_iq2xxs_grid + aux8[l]); + const uint8_t signs = d_ksigns_iq2xs[(aux32_1 >> (7 * l)) & 127]; + for (int j = 0; j < 8; ++j) + DqStore::Set(y, o + j, + db * grid[j] * ((signs & d_kmask_iq2xs[j]) ? -1.f : 1.f)); + o += 8; + } + } + } +}; + +// block_iq3_xxs = { f16 d; u8 qs[96] }. +// Stock ggml/src/ggml-quants.c:2575. +struct DqIQ3_XXS { + static constexpr int kBytes = 98; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const uint8_t* qs = blk + 2; + const uint8_t* scales_and_signs = qs + 64; + int o = 0; + for (int ib32 = 0; ib32 < 8; ++ib32) { + const uint32_t aux32 = DqU32(scales_and_signs + 4 * ib32); + const float db = d * (0.5f + (aux32 >> 28)) * 0.5f; + for (int l = 0; l < 4; ++l) { + const uint8_t signs = d_ksigns_iq2xs[(aux32 >> (7 * l)) & 127]; + // d_iq3xxs_grid is the one codebook held in `__constant__`; it is read + // Read the codebook word by value, then extract its bytes, + // rather than through a pointer into the constant window. + const uint32_t g1 = d_iq3xxs_grid[qs[2 * l + 0]]; + const uint32_t g2 = d_iq3xxs_grid[qs[2 * l + 1]]; + for (int j = 0; j < 4; ++j) { + const uint8_t b1 = static_cast((g1 >> (8 * j)) & 0xFFu); + const uint8_t b2 = static_cast((g2 >> (8 * j)) & 0xFFu); + DqStore::Set(y, o + j + 0, + db * b1 * ((signs & d_kmask_iq2xs[j + 0]) ? -1.f : 1.f)); + DqStore::Set(y, o + j + 4, + db * b2 * ((signs & d_kmask_iq2xs[j + 4]) ? -1.f : 1.f)); + } + o += 8; + } + qs += 8; + } + } +}; + +// block_iq3_s = { f16 d; u8 qs[64]; u8 qh[8]; u8 signs[32]; u8 scales[4] } +// The 512-entry codebook uses a ninth index bit from qh and direct sign bytes. +// Each odd-integer scale is shared by two sub-blocks, matching the upstream loop. +// This decoder serves HIP gather only. IQ3_S matrix compute is outside this row; +// its owning record is `.agents/specs/gguf-iq3s.md`. +// Stock ggml/src/ggml-quants.c:2607. +struct DqIQ3_S { + static constexpr int kBytes = 110; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const uint8_t* qs = blk + 2; + const uint8_t* qh = blk + 66; + const uint8_t* signs = blk + 74; + const uint8_t* scales = blk + 106; + int o = 0; + for (int ib32 = 0; ib32 < 8; ib32 += 2) { + const float db1 = d * (1 + 2 * (scales[ib32 / 2] & 0xf)); + const float db2 = d * (1 + 2 * (scales[ib32 / 2] >> 4)); + for (int l = 0; l < 4; ++l) { + const uint32_t g1 = d_iq3s_grid[qs[2 * l + 0] | ((qh[0] << (8 - 2 * l)) & 256)]; + const uint32_t g2 = d_iq3s_grid[qs[2 * l + 1] | ((qh[0] << (7 - 2 * l)) & 256)]; + for (int j = 0; j < 4; ++j) { + const uint8_t b1 = static_cast((g1 >> (8 * j)) & 0xFFu); + const uint8_t b2 = static_cast((g2 >> (8 * j)) & 0xFFu); + DqStore::Set(y, o + j + 0, + db1 * b1 * ((signs[l] & d_kmask_iq2xs[j + 0]) ? -1.f : 1.f)); + DqStore::Set(y, o + j + 4, + db1 * b2 * ((signs[l] & d_kmask_iq2xs[j + 4]) ? -1.f : 1.f)); + } + o += 8; + } + qs += 8; + signs += 4; + for (int l = 0; l < 4; ++l) { + const uint32_t g1 = d_iq3s_grid[qs[2 * l + 0] | ((qh[1] << (8 - 2 * l)) & 256)]; + const uint32_t g2 = d_iq3s_grid[qs[2 * l + 1] | ((qh[1] << (7 - 2 * l)) & 256)]; + for (int j = 0; j < 4; ++j) { + const uint8_t b1 = static_cast((g1 >> (8 * j)) & 0xFFu); + const uint8_t b2 = static_cast((g2 >> (8 * j)) & 0xFFu); + DqStore::Set(y, o + j + 0, + db2 * b1 * ((signs[l] & d_kmask_iq2xs[j + 0]) ? -1.f : 1.f)); + DqStore::Set(y, o + j + 4, + db2 * b2 * ((signs[l] & d_kmask_iq2xs[j + 4]) ? -1.f : 1.f)); + } + o += 8; + } + qh += 2; + qs += 8; + signs += 4; + } + } +}; + +// block_iq2_xs = { f16 d; u16 qs[32]; u8 scales[8] }. +// Stock ggml/src/ggml-quants.c:2516. +struct DqIQ2_XS { + static constexpr int kBytes = 74; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const uint8_t* qs = blk + 2; + const uint8_t* scales = blk + 66; + int o = 0; + float db[2]; + for (int ib32 = 0; ib32 < 8; ++ib32) { + db[0] = d * (0.5f + (scales[ib32] & 0xf)) * 0.25f; + db[1] = d * (0.5f + (scales[ib32] >> 4)) * 0.25f; + for (int l = 0; l < 4; ++l) { + const uint16_t q = DqU16(qs + 2 * (4 * ib32 + l)); + const uint8_t* grid = + reinterpret_cast(d_iq2xs_grid + (q & 511)); + const uint8_t signs = d_ksigns_iq2xs[q >> 9]; + for (int j = 0; j < 8; ++j) + DqStore::Set(y, o + j, + db[l / 2] * grid[j] * + ((signs & d_kmask_iq2xs[j]) ? -1.f : 1.f)); + o += 8; + } + } + } +}; + +// block_iq4_xs = { f16 d; u16 scales_h; u8 scales_l[4]; u8 qs[128] } +// Stock ggml/src/ggml-quants.c:2743. +struct DqIQ4_XS { + static constexpr int kBytes = 136; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const uint16_t scales_h = DqU16(blk + 2); + const uint8_t* scales_l = blk + 4; + const uint8_t* qs = blk + 8; + int o = 0; + for (int ib = 0; ib < 8; ++ib) { + const int ls = ((scales_l[ib / 2] >> (4 * (ib % 2))) & 0xf) | + (((scales_h >> (2 * ib)) & 3) << 4); + const float dl = d * (ls - 32); + for (int j = 0; j < 16; ++j) { + DqStore::Set(y, o + j + 0, dl * d_kvalues_iq4nl[qs[j] & 0xf]); + DqStore::Set(y, o + j + 16, dl * d_kvalues_iq4nl[qs[j] >> 4]); + } + o += 32; + qs += 16; + } + } +}; + +// block_iq2_s = { f16 d; u8 qs[64]; u8 qh[8]; u8 scales[8] }. +// Stock ggml/src/ggml-quants.c:2543. +struct DqIQ2_S { + static constexpr int kBytes = 82; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + const float d = DqF16(blk); + const uint8_t* qs = blk + 2; + const uint8_t* qh = blk + 66; + const uint8_t* scales = blk + 74; + const uint8_t* signs = qs + 32; + int o = 0; + float db[2]; + for (int ib32 = 0; ib32 < 8; ++ib32) { + db[0] = d * (0.5f + (scales[ib32] & 0xf)) * 0.25f; + db[1] = d * (0.5f + (scales[ib32] >> 4)) * 0.25f; + for (int l = 0; l < 4; ++l) { + const float dl = db[l / 2]; + const uint8_t* grid = reinterpret_cast( + d_iq2s_grid + (qs[l] | ((qh[ib32] << (8 - 2 * l)) & 0x300))); + for (int j = 0; j < 8; ++j) + DqStore::Set(y, o + j, + dl * grid[j] * ((signs[l] & d_kmask_iq2xs[j]) ? -1.f : 1.f)); + o += 8; + } + qs += 4; + signs += 4; + } + } +}; + +// block_iq1_s = { f16 d; u8 qs[32]; u16 qh[8] }. +// kIq1sDelta (0.125f, ggml-common.h:1132 IQ1S_DELTA) is stated here rather than +// included, because this header sees no host constants. +// Stock ggml/src/ggml-quants.c:2650. +struct DqIQ1_S { + static constexpr int kBytes = 50; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + constexpr float kDelta = 0.125f; + const float d = DqF16(blk); + const uint8_t* qs = blk + 2; + int o = 0; + for (int ib = 0; ib < 8; ++ib) { + const uint16_t qh = DqU16(blk + 34 + 2 * ib); + const float dl = d * static_cast(2 * ((qh >> 12) & 7) + 1); + const float delta = (qh & 0x8000) ? -kDelta : kDelta; + for (int l = 0; l < 4; ++l) { + const int8_t* grid = reinterpret_cast( + d_iq1s_grid + (qs[l] | (((qh >> (3 * l)) & 7) << 8))); + for (int j = 0; j < 8; ++j) + DqStore::Set(y, o + j, dl * (static_cast(grid[j]) + delta)); + o += 8; + } + qs += 4; + } + } +}; + +// block_iq1_xxxs = { f16 d; u8 qs[32]; u8 sc[4] }. +// Encoding from the registered llama-cpp-unsloth pin above. +// Fork ggml/src/ggml-quants.c:2727. +struct DqIQ1_XXXS { + static constexpr int kBytes = 38; + static constexpr int kElems = 256; + template + static __device__ void Decode(const uint8_t* blk, Tout* y) { + constexpr float kDelta = 0.125f; + const float d = DqF16(blk); + const uint8_t* qs = blk + 2; + const uint8_t* sc = blk + 34; + int o = 0; + for (int ib = 0; ib < 8; ++ib) { + const int nib = (sc[ib / 2] >> (4 * (ib & 1))) & 0xf; + const float dl = d * static_cast(2 * (nib & 7) + 1); + const float delta = (nib & 8) ? -kDelta : kDelta; + for (int l = 0; l < 4; ++l) { + const int8_t* grid = + reinterpret_cast(d_iq1xxxs_grid + qs[l]); + for (int j = 0; j < 8; ++j) + DqStore::Set(y, o + j, dl * (static_cast(grid[j]) + delta)); + o += 8; + } + qs += 4; + } + } +}; + + +} // namespace +} // namespace vt::rocm diff --git a/src/vt/rocm/rocm_quant_iq_tables.h b/src/vt/rocm/rocm_quant_iq_tables.h new file mode 100644 index 0000000000..b5d9b0e69e --- /dev/null +++ b/src/vt/rocm/rocm_quant_iq_tables.h @@ -0,0 +1,1208 @@ +// ROCm codebooks, independently owned by BACKEND-ROCM-QUANT-GATHER (#3093). +// Literal port of src/vt/cuda/cuda_quant_iq_tables.cuh at 6db4bef90685. +// Stock references use llama.cpp 10bf611e ggml/src/ggml-common.h. +// The IQ1_XXXS table uses fork 36fe8e1cc and its separate pinned definition. +#pragma once + +#include + +namespace vt::rocm { +namespace { + +// Stock ggml-common.h:509, kmask_iq2xs. +__device__ __constant__ const uint8_t d_kmask_iq2xs[8] = {1, 2, 4, 8, 16, 32, 64, 128}; + +// Stock ggml-common.h:513, ksigns_iq2xs. +__device__ const uint8_t d_ksigns_iq2xs[128] = { + 0, 129, 130, 3, 132, 5, 6, 135, 136, 9, 10, 139, 12, 141, 142, 15, + 144, 17, 18, 147, 20, 149, 150, 23, 24, 153, 154, 27, 156, 29, 30, 159, + 160, 33, 34, 163, 36, 165, 166, 39, 40, 169, 170, 43, 172, 45, 46, 175, + 48, 177, 178, 51, 180, 53, 54, 183, 184, 57, 58, 187, 60, 189, 190, 63, + 192, 65, 66, 195, 68, 197, 198, 71, 72, 201, 202, 75, 204, 77, 78, 207, + 80, 209, 210, 83, 212, 85, 86, 215, 216, 89, 90, 219, 92, 221, 222, 95, + 96, 225, 226, 99, 228, 101, 102, 231, 232, 105, 106, 235, 108, 237, 238, 111, + 240, 113, 114, 243, 116, 245, 246, 119, 120, 249, 250, 123, 252, 125, 126, 255 +}; + +// Stock ggml-common.h:1135, iq1s_grid. +__device__ const uint64_t d_iq1s_grid[2048] = { + 0xffffffffffffffffULL, 0xffffffffffffff01ULL, 0xffffffffffff0000ULL, 0xffffffffffff01ffULL, + 0xffffffffffff0101ULL, 0xffffffffff00ff00ULL, 0xffffffffff000000ULL, 0xffffffffff01ffffULL, + 0xffffffffff01ff01ULL, 0xffffffffff0101ffULL, 0xffffffffff010101ULL, 0xffffffff00ff0000ULL, + 0xffffffff0000ff00ULL, 0xffffffff000000ffULL, 0xffffffff00000001ULL, 0xffffffff00010000ULL, + 0xffffffff01ffffffULL, 0xffffffff01ffff01ULL, 0xffffffff01ff01ffULL, 0xffffffff01ff0101ULL, + 0xffffffff01000000ULL, 0xffffffff0101ffffULL, 0xffffffff0101ff01ULL, 0xffffffff010101ffULL, + 0xffffffff01010101ULL, 0xffffff00ffff00ffULL, 0xffffff00ffff0000ULL, 0xffffff00ff00ff00ULL, + 0xffffff00ff0000ffULL, 0xffffff00ff000001ULL, 0xffffff00ff000100ULL, 0xffffff00ff000101ULL, + 0xffffff00ff010000ULL, 0xffffff0000ffff00ULL, 0xffffff0000ff0001ULL, 0xffffff0000ff0100ULL, + 0xffffff000000ff01ULL, 0xffffff0000000000ULL, 0xffffff0000000101ULL, 0xffffff000001ff00ULL, + 0xffffff00000100ffULL, 0xffffff0000010001ULL, 0xffffff00000101ffULL, 0xffffff0001ff0000ULL, + 0xffffff000100ff00ULL, 0xffffff00010000ffULL, 0xffffff0001000001ULL, 0xffffff0001010000ULL, + 0xffffff01ffffffffULL, 0xffffff01ffffff01ULL, 0xffffff01ffff01ffULL, 0xffffff01ffff0101ULL, + 0xffffff01ff000000ULL, 0xffffff01ff01ffffULL, 0xffffff01ff01ff01ULL, 0xffffff01ff0101ffULL, + 0xffffff01ff010101ULL, 0xffffff0100ff0000ULL, 0xffffff010000ff00ULL, 0xffffff0100000100ULL, + 0xffffff01000100ffULL, 0xffffff0100010100ULL, 0xffffff0101ffffffULL, 0xffffff0101ffff01ULL, + 0xffffff0101ff01ffULL, 0xffffff0101ff0101ULL, 0xffffff010100ff00ULL, 0xffffff0101000000ULL, + 0xffffff0101000100ULL, 0xffffff010101ffffULL, 0xffffff010101ff01ULL, 0xffffff01010101ffULL, + 0xffffff0101010101ULL, 0xffff00ffff00ff00ULL, 0xffff00ffff0000ffULL, 0xffff00ffff000001ULL, + 0xffff00ffff010000ULL, 0xffff00ff00ffff00ULL, 0xffff00ff00ff0100ULL, 0xffff00ff00000000ULL, + 0xffff00ff00000101ULL, 0xffff00ff000100ffULL, 0xffff00ff00010000ULL, 0xffff00ff0100ff00ULL, + 0xffff00ff01000100ULL, 0xffff00ff01010000ULL, 0xffff0000ffffff00ULL, 0xffff0000ffff00ffULL, + 0xffff0000ffff0000ULL, 0xffff0000ffff0001ULL, 0xffff0000ff000000ULL, 0xffff0000ff0001ffULL, + 0xffff0000ff000101ULL, 0xffff0000ff010100ULL, 0xffff000000ffffffULL, 0xffff000000ff0000ULL, + 0xffff000000ff0101ULL, 0xffff00000000ffffULL, 0xffff00000000ff00ULL, 0xffff0000000000ffULL, + 0xffff000000000000ULL, 0xffff000000000001ULL, 0xffff000000000100ULL, 0xffff00000001ffffULL, + 0xffff00000001ff01ULL, 0xffff000000010000ULL, 0xffff0000000101ffULL, 0xffff000000010101ULL, + 0xffff000001ffff00ULL, 0xffff00000100ff00ULL, 0xffff000001000000ULL, 0xffff0000010001ffULL, + 0xffff000001000101ULL, 0xffff00000101ff00ULL, 0xffff0000010100ffULL, 0xffff000001010000ULL, + 0xffff000001010001ULL, 0xffff000001010100ULL, 0xffff0001ff0000ffULL, 0xffff0001ff000100ULL, + 0xffff000100ffff00ULL, 0xffff000100ff00ffULL, 0xffff00010000ffffULL, 0xffff00010000ff01ULL, + 0xffff000100000000ULL, 0xffff0001000001ffULL, 0xffff00010001ffffULL, 0xffff00010001ff00ULL, + 0xffff000100010001ULL, 0xffff000100010100ULL, 0xffff000101ff0000ULL, 0xffff00010100ff00ULL, + 0xffff0001010000ffULL, 0xffff000101000100ULL, 0xffff01ffffffffffULL, 0xffff01ffffffff01ULL, + 0xffff01ffffff01ffULL, 0xffff01ffffff0101ULL, 0xffff01ffff000000ULL, 0xffff01ffff01ffffULL, + 0xffff01ffff01ff01ULL, 0xffff01ffff0101ffULL, 0xffff01ffff010101ULL, 0xffff01ff00ff0000ULL, + 0xffff01ff0000ff00ULL, 0xffff01ff00000001ULL, 0xffff01ff00010000ULL, 0xffff01ff01ffffffULL, + 0xffff01ff01ffff01ULL, 0xffff01ff01ff01ffULL, 0xffff01ff01ff0101ULL, 0xffff01ff01000000ULL, + 0xffff01ff0101ffffULL, 0xffff01ff0101ff01ULL, 0xffff01ff010101ffULL, 0xffff01ff01010101ULL, + 0xffff0100ffff0000ULL, 0xffff0100ff00ff00ULL, 0xffff0100ff0000ffULL, 0xffff0100ff000100ULL, + 0xffff0100ff0100ffULL, 0xffff0100ff010000ULL, 0xffff010000ffff00ULL, 0xffff01000000ffffULL, + 0xffff01000000ff00ULL, 0xffff010000000000ULL, 0xffff01000001ff00ULL, 0xffff0100000100ffULL, + 0xffff010000010100ULL, 0xffff01000100ff00ULL, 0xffff0100010000ffULL, 0xffff010001000001ULL, + 0xffff010001000100ULL, 0xffff010001010000ULL, 0xffff0101ffffffffULL, 0xffff0101ffffff01ULL, + 0xffff0101ffff01ffULL, 0xffff0101ffff0101ULL, 0xffff0101ff000000ULL, 0xffff0101ff01ffffULL, + 0xffff0101ff01ff01ULL, 0xffff0101ff0101ffULL, 0xffff0101ff010101ULL, 0xffff010100ff0000ULL, + 0xffff01010000ff00ULL, 0xffff010100000100ULL, 0xffff01010001ff00ULL, 0xffff010100010000ULL, + 0xffff010101ffffffULL, 0xffff010101ffff01ULL, 0xffff010101ff0000ULL, 0xffff010101ff01ffULL, + 0xffff010101ff0101ULL, 0xffff010101000000ULL, 0xffff01010101ffffULL, 0xffff01010101ff01ULL, + 0xffff0101010101ffULL, 0xffff010101010101ULL, 0xff00ffffff00ffffULL, 0xff00ffffff00ff00ULL, + 0xff00ffffff0000ffULL, 0xff00ffffff000100ULL, 0xff00ffffff0100ffULL, 0xff00ffffff010000ULL, + 0xff00ffff00ffff00ULL, 0xff00ffff00ff00ffULL, 0xff00ffff0000ffffULL, 0xff00ffff00000000ULL, + 0xff00ffff000001ffULL, 0xff00ffff0001ff00ULL, 0xff00ffff000100ffULL, 0xff00ffff00010000ULL, + 0xff00ffff00010100ULL, 0xff00ffff0100ff00ULL, 0xff00ffff010000ffULL, 0xff00ffff01000001ULL, + 0xff00ffff0101ff00ULL, 0xff00ffff01010000ULL, 0xff00ff00ffffff00ULL, 0xff00ff00ffff00ffULL, + 0xff00ff00ffff0001ULL, 0xff00ff00ffff0100ULL, 0xff00ff00ff00ffffULL, 0xff00ff00ff00ff01ULL, + 0xff00ff00ff000000ULL, 0xff00ff00ff0001ffULL, 0xff00ff00ff01ff00ULL, 0xff00ff00ff0100ffULL, + 0xff00ff00ff010100ULL, 0xff00ff0000ff0000ULL, 0xff00ff0000ff0101ULL, 0xff00ff000000ffffULL, + 0xff00ff000000ff00ULL, 0xff00ff000000ff01ULL, 0xff00ff00000000ffULL, 0xff00ff0000000000ULL, + 0xff00ff0000000001ULL, 0xff00ff0000000100ULL, 0xff00ff000001ffffULL, 0xff00ff0000010000ULL, + 0xff00ff0001ff00ffULL, 0xff00ff000100ff01ULL, 0xff00ff0001000000ULL, 0xff00ff000101ff00ULL, + 0xff00ff00010100ffULL, 0xff00ff01ff00ff00ULL, 0xff00ff01ff0000ffULL, 0xff00ff01ff000001ULL, + 0xff00ff01ff010000ULL, 0xff00ff0100ffffffULL, 0xff00ff0100ff0001ULL, 0xff00ff0100ff0100ULL, + 0xff00ff010000ff01ULL, 0xff00ff0100000000ULL, 0xff00ff01000001ffULL, 0xff00ff0100000101ULL, + 0xff00ff01000100ffULL, 0xff00ff0100010001ULL, 0xff00ff0101ff0000ULL, 0xff00ff010100ff00ULL, + 0xff00ff01010000ffULL, 0xff00ff0101000001ULL, 0xff00ff0101010000ULL, 0xff0000ffffffff00ULL, + 0xff0000ffffff0001ULL, 0xff0000ffffff0100ULL, 0xff0000ffff0000ffULL, 0xff0000ffff000000ULL, + 0xff0000ffff0001ffULL, 0xff0000ffff000100ULL, 0xff0000ffff01ff00ULL, 0xff0000ffff010001ULL, + 0xff0000ff00ffff00ULL, 0xff0000ff00ff0000ULL, 0xff0000ff00ff0001ULL, 0xff0000ff00ff01ffULL, + 0xff0000ff00ff0101ULL, 0xff0000ff0000ff00ULL, 0xff0000ff000000ffULL, 0xff0000ff00000000ULL, + 0xff0000ff00000001ULL, 0xff0000ff00000100ULL, 0xff0000ff0001ff01ULL, 0xff0000ff00010000ULL, + 0xff0000ff000101ffULL, 0xff0000ff01ff00ffULL, 0xff0000ff01ff0100ULL, 0xff0000ff0100ffffULL, + 0xff0000ff010000ffULL, 0xff0000ff01000000ULL, 0xff0000ff010001ffULL, 0xff0000ff01000100ULL, + 0xff0000ff01000101ULL, 0xff0000ff0101ff00ULL, 0xff0000ff010100ffULL, 0xff0000ff01010000ULL, + 0xff0000ff01010100ULL, 0xff000000ffffff01ULL, 0xff000000ffff0000ULL, 0xff000000ffff0101ULL, + 0xff000000ff00ff00ULL, 0xff000000ff0000ffULL, 0xff000000ff000000ULL, 0xff000000ff000001ULL, + 0xff000000ff000100ULL, 0xff000000ff01ffffULL, 0xff000000ff01ff01ULL, 0xff000000ff010000ULL, + 0xff000000ff0101ffULL, 0xff000000ff010101ULL, 0xff00000000ffff00ULL, 0xff00000000ff00ffULL, + 0xff00000000ff0000ULL, 0xff00000000ff0001ULL, 0xff0000000000ff00ULL, 0xff0000000000ff01ULL, + 0xff000000000000ffULL, 0xff00000000000000ULL, 0xff00000000000001ULL, 0xff00000000000100ULL, + 0xff00000000000101ULL, 0xff0000000001ff00ULL, 0xff000000000100ffULL, 0xff00000000010000ULL, + 0xff00000000010001ULL, 0xff00000000010100ULL, 0xff00000001ffffffULL, 0xff00000001ffff01ULL, + 0xff00000001ff00ffULL, 0xff00000001ff0000ULL, 0xff00000001ff01ffULL, 0xff00000001ff0101ULL, + 0xff0000000100ffffULL, 0xff0000000100ff00ULL, 0xff000000010000ffULL, 0xff00000001000000ULL, + 0xff00000001000001ULL, 0xff00000001000100ULL, 0xff00000001000101ULL, 0xff0000000101ffffULL, + 0xff0000000101ff01ULL, 0xff00000001010000ULL, 0xff000001ffffff00ULL, 0xff000001ffff00ffULL, + 0xff000001ffff0000ULL, 0xff000001ffff0001ULL, 0xff000001ff000000ULL, 0xff000001ff000001ULL, + 0xff000001ff0001ffULL, 0xff000001ff000101ULL, 0xff000001ff01ff00ULL, 0xff000001ff010001ULL, + 0xff00000100ffffffULL, 0xff00000100ffff01ULL, 0xff00000100ff00ffULL, 0xff00000100ff0000ULL, + 0xff00000100ff01ffULL, 0xff00000100ff0101ULL, 0xff0000010000ff00ULL, 0xff00000100000000ULL, + 0xff00000100000001ULL, 0xff000001000001ffULL, 0xff00000100000100ULL, 0xff0000010001ff00ULL, + 0xff000001000100ffULL, 0xff00000100010000ULL, 0xff000001000101ffULL, 0xff00000100010100ULL, + 0xff00000100010101ULL, 0xff00000101ff0001ULL, 0xff00000101ff0101ULL, 0xff0000010100ff01ULL, + 0xff00000101000000ULL, 0xff000001010100ffULL, 0xff00000101010100ULL, 0xff0001ffff00ff00ULL, + 0xff0001ffff000001ULL, 0xff0001ffff010000ULL, 0xff0001ff00ffff00ULL, 0xff0001ff00ff00ffULL, + 0xff0001ff00ff0001ULL, 0xff0001ff00ff0100ULL, 0xff0001ff0000ffffULL, 0xff0001ff00000000ULL, + 0xff0001ff000001ffULL, 0xff0001ff00000101ULL, 0xff0001ff0001ffffULL, 0xff0001ff0001ff00ULL, + 0xff0001ff000100ffULL, 0xff0001ff00010001ULL, 0xff0001ff00010100ULL, 0xff0001ff01ff0000ULL, + 0xff0001ff0100ff00ULL, 0xff0001ff010000ffULL, 0xff0001ff01010000ULL, 0xff000100ff00ffffULL, + 0xff000100ff00ff01ULL, 0xff000100ff000000ULL, 0xff000100ff000101ULL, 0xff000100ff01ff00ULL, + 0xff000100ff010000ULL, 0xff00010000ffff01ULL, 0xff00010000ff00ffULL, 0xff00010000ff0000ULL, + 0xff00010000ff01ffULL, 0xff0001000000ff00ULL, 0xff000100000000ffULL, 0xff00010000000000ULL, + 0xff00010000000001ULL, 0xff00010000000100ULL, 0xff00010000000101ULL, 0xff0001000001ffffULL, + 0xff00010000010000ULL, 0xff00010000010101ULL, 0xff00010001ff0100ULL, 0xff0001000100ff00ULL, + 0xff0001000100ff01ULL, 0xff00010001000000ULL, 0xff000100010001ffULL, 0xff0001000101ff00ULL, + 0xff00010001010001ULL, 0xff00010001010100ULL, 0xff000101ffff0100ULL, 0xff000101ff000001ULL, + 0xff000101ff0100ffULL, 0xff000101ff010001ULL, 0xff00010100ff00ffULL, 0xff00010100ff0001ULL, + 0xff00010100ff0100ULL, 0xff0001010000ffffULL, 0xff0001010000ff01ULL, 0xff00010100000000ULL, + 0xff000101000001ffULL, 0xff0001010001ff00ULL, 0xff00010100010001ULL, 0xff00010100010100ULL, + 0xff00010101ff0000ULL, 0xff0001010100ff00ULL, 0xff00010101000001ULL, 0xff00010101000101ULL, + 0xff01ffffffffffffULL, 0xff01ffffffffff01ULL, 0xff01ffffffff01ffULL, 0xff01ffffffff0101ULL, + 0xff01ffffff000000ULL, 0xff01ffffff01ffffULL, 0xff01ffffff01ff01ULL, 0xff01ffffff010000ULL, + 0xff01ffffff0101ffULL, 0xff01ffffff010101ULL, 0xff01ffff00ff0000ULL, 0xff01ffff0000ff00ULL, + 0xff01ffff00000100ULL, 0xff01ffff0001ff00ULL, 0xff01ffff00010000ULL, 0xff01ffff01ffffffULL, + 0xff01ffff01ffff01ULL, 0xff01ffff01ff01ffULL, 0xff01ffff01ff0101ULL, 0xff01ffff01000000ULL, + 0xff01ffff0101ffffULL, 0xff01ffff0101ff01ULL, 0xff01ffff01010000ULL, 0xff01ffff010101ffULL, + 0xff01ffff01010101ULL, 0xff01ff00ffff0000ULL, 0xff01ff00ff00ff00ULL, 0xff01ff00ff0000ffULL, + 0xff01ff00ff000100ULL, 0xff01ff00ff010000ULL, 0xff01ff0000ffff01ULL, 0xff01ff0000ff00ffULL, + 0xff01ff0000ff0100ULL, 0xff01ff0000000000ULL, 0xff01ff00000001ffULL, 0xff01ff0000000101ULL, + 0xff01ff000001ff00ULL, 0xff01ff00000100ffULL, 0xff01ff0000010000ULL, 0xff01ff0000010001ULL, + 0xff01ff0001ff0000ULL, 0xff01ff000100ffffULL, 0xff01ff0001000001ULL, 0xff01ff0001000100ULL, + 0xff01ff0001010000ULL, 0xff01ff01ffffff00ULL, 0xff01ff01ffff01ffULL, 0xff01ff01ffff0101ULL, + 0xff01ff01ff00ff00ULL, 0xff01ff01ff000000ULL, 0xff01ff01ff01ffffULL, 0xff01ff01ff01ff01ULL, + 0xff01ff01ff0101ffULL, 0xff01ff01ff010101ULL, 0xff01ff0100ff0000ULL, 0xff01ff010000ff00ULL, + 0xff01ff0100000001ULL, 0xff01ff0100000100ULL, 0xff01ff0100010000ULL, 0xff01ff0101ffff00ULL, + 0xff01ff0101ff01ffULL, 0xff01ff0101ff0101ULL, 0xff01ff010100ff00ULL, 0xff01ff0101000000ULL, + 0xff01ff010101ffffULL, 0xff01ff010101ff01ULL, 0xff01ff01010101ffULL, 0xff01ff0101010101ULL, + 0xff0100ffffff0000ULL, 0xff0100ffff0000ffULL, 0xff0100ffff000001ULL, 0xff0100ffff000100ULL, + 0xff0100ffff010000ULL, 0xff0100ff00ff00ffULL, 0xff0100ff00ff0000ULL, 0xff0100ff00ff0001ULL, + 0xff0100ff00ff0100ULL, 0xff0100ff0000ff01ULL, 0xff0100ff00000000ULL, 0xff0100ff000001ffULL, + 0xff0100ff00000101ULL, 0xff0100ff00010001ULL, 0xff0100ff01ff0000ULL, 0xff0100ff0100ff00ULL, + 0xff0100ff010000ffULL, 0xff0100ff01000100ULL, 0xff0100ff0101ff00ULL, 0xff0100ff01010000ULL, + 0xff010000ffff0100ULL, 0xff010000ff000000ULL, 0xff010000ff01ff00ULL, 0xff010000ff010100ULL, + 0xff01000000ffffffULL, 0xff01000000ff0000ULL, 0xff01000000ff01ffULL, 0xff0100000000ff00ULL, + 0xff010000000000ffULL, 0xff01000000000000ULL, 0xff01000000000100ULL, 0xff0100000001ff01ULL, + 0xff01000000010000ULL, 0xff010000000101ffULL, 0xff01000001ff0100ULL, 0xff0100000100ffffULL, + 0xff010000010000ffULL, 0xff01000001000000ULL, 0xff010000010001ffULL, 0xff01000001000101ULL, + 0xff0100000101ff00ULL, 0xff010000010100ffULL, 0xff01000001010001ULL, 0xff01000001010100ULL, + 0xff010001ffff0000ULL, 0xff010001ff00ffffULL, 0xff010001ff00ff01ULL, 0xff010001ff000100ULL, + 0xff010001ff010000ULL, 0xff01000100ffff00ULL, 0xff01000100ff0100ULL, 0xff01000100000000ULL, + 0xff0100010001ffffULL, 0xff0100010001ff00ULL, 0xff01000100010100ULL, 0xff01000101ff00ffULL, + 0xff01000101ff0001ULL, 0xff0100010100ffffULL, 0xff01000101000101ULL, 0xff0101ffffffffffULL, + 0xff0101ffffffff01ULL, 0xff0101ffffff01ffULL, 0xff0101ffffff0101ULL, 0xff0101ffff000000ULL, + 0xff0101ffff01ffffULL, 0xff0101ffff01ff01ULL, 0xff0101ffff0101ffULL, 0xff0101ffff010101ULL, + 0xff0101ff00ff0000ULL, 0xff0101ff0000ff00ULL, 0xff0101ff000000ffULL, 0xff0101ff00010000ULL, + 0xff0101ff01ffffffULL, 0xff0101ff01ffff01ULL, 0xff0101ff01ff01ffULL, 0xff0101ff01ff0101ULL, + 0xff0101ff0101ffffULL, 0xff0101ff0101ff01ULL, 0xff0101ff010101ffULL, 0xff0101ff01010101ULL, + 0xff010100ffff0100ULL, 0xff010100ff00ff00ULL, 0xff010100ff0000ffULL, 0xff010100ff000100ULL, + 0xff010100ff010000ULL, 0xff01010000ff0001ULL, 0xff01010000ff0100ULL, 0xff0101000000ff01ULL, + 0xff01010000000000ULL, 0xff0101000001ff00ULL, 0xff010100000100ffULL, 0xff01010000010001ULL, + 0xff01010000010100ULL, 0xff01010001ff0000ULL, 0xff0101000100ffffULL, 0xff01010001000001ULL, + 0xff01010001000100ULL, 0xff010100010100ffULL, 0xff01010001010000ULL, 0xff010101ffffffffULL, + 0xff010101ffffff01ULL, 0xff010101ffff01ffULL, 0xff010101ffff0101ULL, 0xff010101ff01ffffULL, + 0xff010101ff01ff01ULL, 0xff010101ff0101ffULL, 0xff010101ff010101ULL, 0xff01010100ff0000ULL, + 0xff0101010000ff00ULL, 0xff01010100000001ULL, 0xff01010100000100ULL, 0xff01010100010000ULL, + 0xff01010101ffffffULL, 0xff01010101ffff01ULL, 0xff01010101ff01ffULL, 0xff01010101ff0101ULL, + 0xff01010101000000ULL, 0xff0101010101ffffULL, 0xff0101010101ff01ULL, 0xff010101010101ffULL, + 0xff01010101010101ULL, 0x00ffffffffff0000ULL, 0x00ffffffff00ff00ULL, 0x00ffffffff000001ULL, + 0x00ffffffff010000ULL, 0x00ffffff00ff0100ULL, 0x00ffffff0000ff01ULL, 0x00ffffff00000000ULL, + 0x00ffffff000001ffULL, 0x00ffffff00000101ULL, 0x00ffffff0001ff00ULL, 0x00ffffff000100ffULL, + 0x00ffffff00010001ULL, 0x00ffffff010000ffULL, 0x00ffffff01000100ULL, 0x00ffffff0101ff00ULL, + 0x00ffffff01010001ULL, 0x00ffff00ffffffffULL, 0x00ffff00ffffff00ULL, 0x00ffff00ffff00ffULL, + 0x00ffff00ffff0001ULL, 0x00ffff00ffff0100ULL, 0x00ffff00ff00ff01ULL, 0x00ffff00ff000000ULL, + 0x00ffff00ff000001ULL, 0x00ffff00ff0001ffULL, 0x00ffff00ff000101ULL, 0x00ffff00ff01ff00ULL, + 0x00ffff00ff010001ULL, 0x00ffff00ff010100ULL, 0x00ffff0000ff0000ULL, 0x00ffff0000ff01ffULL, + 0x00ffff0000ff0101ULL, 0x00ffff000000ff00ULL, 0x00ffff00000000ffULL, 0x00ffff0000000000ULL, + 0x00ffff0000000001ULL, 0x00ffff0000000100ULL, 0x00ffff0000000101ULL, 0x00ffff0000010000ULL, + 0x00ffff00000101ffULL, 0x00ffff0000010101ULL, 0x00ffff0001ffff00ULL, 0x00ffff0001ff00ffULL, + 0x00ffff0001ff0001ULL, 0x00ffff000100ffffULL, 0x00ffff000100ff01ULL, 0x00ffff0001000000ULL, + 0x00ffff000101ffffULL, 0x00ffff000101ff00ULL, 0x00ffff000101ff01ULL, 0x00ffff01ffff0000ULL, + 0x00ffff01ff00ff00ULL, 0x00ffff01ff0000ffULL, 0x00ffff01ff000001ULL, 0x00ffff01ff010000ULL, + 0x00ffff0100ffff00ULL, 0x00ffff010000ff01ULL, 0x00ffff0100000000ULL, 0x00ffff0100000101ULL, + 0x00ffff01000100ffULL, 0x00ffff0100010100ULL, 0x00ffff0101ff0100ULL, 0x00ffff01010000ffULL, + 0x00ffff0101010000ULL, 0x00ff00ffffffff00ULL, 0x00ff00ffff000000ULL, 0x00ff00ffff000100ULL, + 0x00ff00ffff010100ULL, 0x00ff00ff00ff0000ULL, 0x00ff00ff00ff01ffULL, 0x00ff00ff00ff0101ULL, + 0x00ff00ff0000ff00ULL, 0x00ff00ff000000ffULL, 0x00ff00ff00000000ULL, 0x00ff00ff00000001ULL, + 0x00ff00ff0001ff00ULL, 0x00ff00ff0001ff01ULL, 0x00ff00ff00010000ULL, 0x00ff00ff000101ffULL, + 0x00ff00ff00010101ULL, 0x00ff00ff01ffff00ULL, 0x00ff00ff01ff0001ULL, 0x00ff00ff01ff0100ULL, + 0x00ff00ff0100ffffULL, 0x00ff00ff0100ff01ULL, 0x00ff00ff01000000ULL, 0x00ff00ff0101ffffULL, + 0x00ff00ff0101ff00ULL, 0x00ff00ff01010100ULL, 0x00ff0000ffffff00ULL, 0x00ff0000ffffff01ULL, + 0x00ff0000ffff0000ULL, 0x00ff0000ffff0101ULL, 0x00ff0000ff00ff00ULL, 0x00ff0000ff0000ffULL, + 0x00ff0000ff000000ULL, 0x00ff0000ff000001ULL, 0x00ff0000ff000100ULL, 0x00ff0000ff01ffffULL, + 0x00ff0000ff010000ULL, 0x00ff0000ff010101ULL, 0x00ff000000ffff00ULL, 0x00ff000000ff00ffULL, + 0x00ff000000ff0000ULL, 0x00ff000000ff0001ULL, 0x00ff000000ff0100ULL, 0x00ff00000000ffffULL, + 0x00ff00000000ff00ULL, 0x00ff0000000000ffULL, 0x00ff000000000000ULL, 0x00ff000000000001ULL, + 0x00ff0000000001ffULL, 0x00ff000000000100ULL, 0x00ff00000001ff00ULL, 0x00ff0000000100ffULL, + 0x00ff000000010000ULL, 0x00ff000000010001ULL, 0x00ff000000010100ULL, 0x00ff000001ffff01ULL, + 0x00ff000001ff00ffULL, 0x00ff000001ff0000ULL, 0x00ff000001ff01ffULL, 0x00ff00000100ff00ULL, + 0x00ff0000010000ffULL, 0x00ff000001000000ULL, 0x00ff000001000001ULL, 0x00ff000001000100ULL, + 0x00ff000001000101ULL, 0x00ff000001010000ULL, 0x00ff0000010101ffULL, 0x00ff000001010101ULL, + 0x00ff0001ffffff00ULL, 0x00ff0001ffff0000ULL, 0x00ff0001ffff0100ULL, 0x00ff0001ff0000ffULL, + 0x00ff0001ff000000ULL, 0x00ff0001ff0001ffULL, 0x00ff0001ff000101ULL, 0x00ff0001ff01ff00ULL, + 0x00ff0001ff0100ffULL, 0x00ff0001ff010100ULL, 0x00ff000100ffffffULL, 0x00ff000100ffff01ULL, + 0x00ff000100ff0000ULL, 0x00ff000100ff01ffULL, 0x00ff00010000ffffULL, 0x00ff00010000ff00ULL, + 0x00ff00010000ff01ULL, 0x00ff000100000000ULL, 0x00ff000100000001ULL, 0x00ff000100000100ULL, + 0x00ff00010001ff01ULL, 0x00ff000100010000ULL, 0x00ff0001000101ffULL, 0x00ff000101ffff00ULL, + 0x00ff000101ff0000ULL, 0x00ff000101ff0101ULL, 0x00ff0001010000ffULL, 0x00ff000101000000ULL, + 0x00ff00010101ff00ULL, 0x00ff0001010100ffULL, 0x00ff000101010001ULL, 0x00ff01ffffff0000ULL, + 0x00ff01ffff00ff00ULL, 0x00ff01ffff000000ULL, 0x00ff01ffff000101ULL, 0x00ff01ffff010000ULL, + 0x00ff01ff00ffff01ULL, 0x00ff01ff00ff0100ULL, 0x00ff01ff0000ffffULL, 0x00ff01ff00000000ULL, + 0x00ff01ff000001ffULL, 0x00ff01ff0001ff00ULL, 0x00ff01ff000100ffULL, 0x00ff01ff00010001ULL, + 0x00ff01ff00010100ULL, 0x00ff01ff01ff0000ULL, 0x00ff01ff0100ff00ULL, 0x00ff01ff010000ffULL, + 0x00ff01ff01000001ULL, 0x00ff01ff01000100ULL, 0x00ff01ff01010000ULL, 0x00ff0100ffffff00ULL, + 0x00ff0100ffff0000ULL, 0x00ff0100ffff0001ULL, 0x00ff0100ffff0101ULL, 0x00ff0100ff00ffffULL, + 0x00ff0100ff0000ffULL, 0x00ff0100ff000000ULL, 0x00ff0100ff0001ffULL, 0x00ff0100ff01ff00ULL, + 0x00ff0100ff0100ffULL, 0x00ff0100ff010001ULL, 0x00ff010000ffffffULL, 0x00ff010000ff0000ULL, + 0x00ff010000ff0101ULL, 0x00ff01000000ff00ULL, 0x00ff01000000ff01ULL, 0x00ff0100000000ffULL, + 0x00ff010000000000ULL, 0x00ff010000000001ULL, 0x00ff010000000100ULL, 0x00ff01000001ffffULL, + 0x00ff01000001ff01ULL, 0x00ff010000010000ULL, 0x00ff010000010001ULL, 0x00ff010000010101ULL, + 0x00ff010001ff0001ULL, 0x00ff010001ff0100ULL, 0x00ff01000100ff01ULL, 0x00ff010001000000ULL, + 0x00ff010001000001ULL, 0x00ff0100010001ffULL, 0x00ff01000101ff00ULL, 0x00ff0100010100ffULL, + 0x00ff010001010001ULL, 0x00ff010001010100ULL, 0x00ff0101ff000001ULL, 0x00ff010100ff00ffULL, + 0x00ff010100ff0001ULL, 0x00ff010100ff0100ULL, 0x00ff010100000000ULL, 0x00ff0101000001ffULL, + 0x00ff010100000101ULL, 0x00ff0101000100ffULL, 0x00ff010100010100ULL, 0x00ff0101010000ffULL, + 0x00ff010101010000ULL, 0x0000ffffffffff00ULL, 0x0000ffffffff00ffULL, 0x0000ffffffff0000ULL, + 0x0000ffffffff0001ULL, 0x0000ffffffff0100ULL, 0x0000ffffff00ff01ULL, 0x0000ffffff000000ULL, + 0x0000ffffff000101ULL, 0x0000ffffff01ff00ULL, 0x0000ffffff0100ffULL, 0x0000ffffff010100ULL, + 0x0000ffff00ffffffULL, 0x0000ffff00ff0000ULL, 0x0000ffff00ff01ffULL, 0x0000ffff0000ff00ULL, + 0x0000ffff000000ffULL, 0x0000ffff00000000ULL, 0x0000ffff00000001ULL, 0x0000ffff00000100ULL, + 0x0000ffff00010000ULL, 0x0000ffff000101ffULL, 0x0000ffff01ff0001ULL, 0x0000ffff01ff0100ULL, + 0x0000ffff01000000ULL, 0x0000ffff010001ffULL, 0x0000ffff0101ffffULL, 0x0000ffff0101ff00ULL, + 0x0000ffff01010001ULL, 0x0000ffff01010100ULL, 0x0000ff00ffff0000ULL, 0x0000ff00ffff01ffULL, + 0x0000ff00ffff0100ULL, 0x0000ff00ffff0101ULL, 0x0000ff00ff00ff00ULL, 0x0000ff00ff0000ffULL, + 0x0000ff00ff000000ULL, 0x0000ff00ff000001ULL, 0x0000ff00ff0001ffULL, 0x0000ff00ff000100ULL, + 0x0000ff00ff01ffffULL, 0x0000ff00ff010000ULL, 0x0000ff00ff010001ULL, 0x0000ff00ff0101ffULL, + 0x0000ff00ff010101ULL, 0x0000ff0000ffff00ULL, 0x0000ff0000ff00ffULL, 0x0000ff0000ff0000ULL, + 0x0000ff0000ff0001ULL, 0x0000ff0000ff0100ULL, 0x0000ff000000ffffULL, 0x0000ff000000ff00ULL, + 0x0000ff000000ff01ULL, 0x0000ff00000000ffULL, 0x0000ff0000000000ULL, 0x0000ff0000000001ULL, + 0x0000ff00000001ffULL, 0x0000ff0000000100ULL, 0x0000ff0000000101ULL, 0x0000ff000001ff00ULL, + 0x0000ff00000100ffULL, 0x0000ff0000010000ULL, 0x0000ff0000010001ULL, 0x0000ff0000010100ULL, + 0x0000ff0001ffff01ULL, 0x0000ff0001ff0000ULL, 0x0000ff000100ff00ULL, 0x0000ff00010000ffULL, + 0x0000ff0001000000ULL, 0x0000ff0001000001ULL, 0x0000ff0001000100ULL, 0x0000ff000101ffffULL, + 0x0000ff0001010000ULL, 0x0000ff0001010101ULL, 0x0000ff01ffffff00ULL, 0x0000ff01ffff0001ULL, + 0x0000ff01ff00ff01ULL, 0x0000ff01ff000000ULL, 0x0000ff01ff000101ULL, 0x0000ff01ff01ff00ULL, + 0x0000ff01ff0100ffULL, 0x0000ff0100ffff01ULL, 0x0000ff0100ff0000ULL, 0x0000ff0100ff0101ULL, + 0x0000ff010000ff00ULL, 0x0000ff01000000ffULL, 0x0000ff0100000000ULL, 0x0000ff0100000001ULL, + 0x0000ff0100000100ULL, 0x0000ff010001ff01ULL, 0x0000ff0100010000ULL, 0x0000ff0101ff0000ULL, + 0x0000ff010100ffffULL, 0x0000ff010100ff01ULL, 0x0000ff0101000000ULL, 0x0000ff0101000100ULL, + 0x0000ff0101000101ULL, 0x0000ff01010100ffULL, 0x000000ffffff00ffULL, 0x000000ffffff0000ULL, + 0x000000ffff00ff00ULL, 0x000000ffff0000ffULL, 0x000000ffff000000ULL, 0x000000ffff000001ULL, + 0x000000ffff0001ffULL, 0x000000ffff000100ULL, 0x000000ffff01ff00ULL, 0x000000ffff010000ULL, + 0x000000ffff0101ffULL, 0x000000ffff010101ULL, 0x000000ff00ffff00ULL, 0x000000ff00ff00ffULL, + 0x000000ff00ff0000ULL, 0x000000ff00ff0001ULL, 0x000000ff00ff0100ULL, 0x000000ff00ff0101ULL, + 0x000000ff0000ffffULL, 0x000000ff0000ff00ULL, 0x000000ff000000ffULL, 0x000000ff00000000ULL, + 0x000000ff00000001ULL, 0x000000ff000001ffULL, 0x000000ff00000100ULL, 0x000000ff00000101ULL, + 0x000000ff0001ff00ULL, 0x000000ff0001ff01ULL, 0x000000ff000100ffULL, 0x000000ff00010000ULL, + 0x000000ff00010001ULL, 0x000000ff00010100ULL, 0x000000ff01ffffffULL, 0x000000ff01ff01ffULL, + 0x000000ff01ff0101ULL, 0x000000ff0100ff00ULL, 0x000000ff010000ffULL, 0x000000ff01000000ULL, + 0x000000ff01000001ULL, 0x000000ff01000100ULL, 0x000000ff0101ff00ULL, 0x000000ff010100ffULL, + 0x000000ff01010000ULL, 0x000000ff01010101ULL, 0x00000000ffffff00ULL, 0x00000000ffffff01ULL, + 0x00000000ffff00ffULL, 0x00000000ffff0000ULL, 0x00000000ffff0001ULL, 0x00000000ffff0100ULL, + 0x00000000ff00ffffULL, 0x00000000ff00ff00ULL, 0x00000000ff00ff01ULL, 0x00000000ff0000ffULL, + 0x00000000ff000000ULL, 0x00000000ff000001ULL, 0x00000000ff000100ULL, 0x00000000ff000101ULL, + 0x00000000ff01ff00ULL, 0x00000000ff0100ffULL, 0x00000000ff010000ULL, 0x00000000ff010001ULL, + 0x00000000ff010100ULL, 0x0000000000ffffffULL, 0x0000000000ffff00ULL, 0x0000000000ffff01ULL, + 0x0000000000ff00ffULL, 0x0000000000ff0000ULL, 0x0000000000ff0001ULL, 0x0000000000ff01ffULL, + 0x0000000000ff0100ULL, 0x000000000000ffffULL, 0x000000000000ff00ULL, 0x000000000000ff01ULL, + 0x00000000000000ffULL, 0x0000000000000000ULL, 0x0000000000000001ULL, 0x00000000000001ffULL, + 0x0000000000000100ULL, 0x0000000000000101ULL, 0x000000000001ffffULL, 0x000000000001ff00ULL, + 0x00000000000100ffULL, 0x0000000000010000ULL, 0x0000000000010001ULL, 0x00000000000101ffULL, + 0x0000000000010100ULL, 0x0000000000010101ULL, 0x0000000001ffff00ULL, 0x0000000001ff00ffULL, + 0x0000000001ff0000ULL, 0x0000000001ff0100ULL, 0x0000000001ff0101ULL, 0x000000000100ffffULL, + 0x000000000100ff00ULL, 0x00000000010000ffULL, 0x0000000001000000ULL, 0x0000000001000001ULL, + 0x00000000010001ffULL, 0x0000000001000100ULL, 0x000000000101ff00ULL, 0x00000000010100ffULL, + 0x0000000001010000ULL, 0x0000000001010001ULL, 0x0000000001010100ULL, 0x00000001ffffffffULL, + 0x00000001ffffff00ULL, 0x00000001ffffff01ULL, 0x00000001ffff00ffULL, 0x00000001ffff0001ULL, + 0x00000001ffff01ffULL, 0x00000001ffff0100ULL, 0x00000001ff00ff00ULL, 0x00000001ff0000ffULL, + 0x00000001ff000000ULL, 0x00000001ff0001ffULL, 0x00000001ff000100ULL, 0x00000001ff01ffffULL, + 0x00000001ff01ff00ULL, 0x00000001ff01ff01ULL, 0x00000001ff0100ffULL, 0x00000001ff010000ULL, + 0x00000001ff010001ULL, 0x00000001ff0101ffULL, 0x00000001ff010100ULL, 0x0000000100ffff00ULL, + 0x0000000100ff0000ULL, 0x0000000100ff0001ULL, 0x0000000100ff01ffULL, 0x0000000100ff0100ULL, + 0x0000000100ff0101ULL, 0x000000010000ffffULL, 0x000000010000ff00ULL, 0x000000010000ff01ULL, + 0x00000001000000ffULL, 0x0000000100000000ULL, 0x0000000100000001ULL, 0x00000001000001ffULL, + 0x0000000100000100ULL, 0x0000000100000101ULL, 0x000000010001ff00ULL, 0x00000001000100ffULL, + 0x0000000100010000ULL, 0x0000000100010100ULL, 0x0000000101ffff01ULL, 0x0000000101ff0000ULL, + 0x0000000101ff0001ULL, 0x0000000101ff01ffULL, 0x0000000101ff0100ULL, 0x0000000101ff0101ULL, + 0x000000010100ff00ULL, 0x0000000101000000ULL, 0x0000000101000101ULL, 0x000000010101ff01ULL, + 0x0000000101010000ULL, 0x0000000101010001ULL, 0x00000001010101ffULL, 0x0000000101010100ULL, + 0x000001ffffff00ffULL, 0x000001ffffff0000ULL, 0x000001ffffff0001ULL, 0x000001ffffff0100ULL, + 0x000001ffff00ffffULL, 0x000001ffff000000ULL, 0x000001ffff0001ffULL, 0x000001ffff01ff00ULL, + 0x000001ffff010101ULL, 0x000001ff00ff0000ULL, 0x000001ff00ff01ffULL, 0x000001ff00ff0101ULL, + 0x000001ff0000ff00ULL, 0x000001ff000000ffULL, 0x000001ff00000000ULL, 0x000001ff00000001ULL, + 0x000001ff000001ffULL, 0x000001ff00000100ULL, 0x000001ff0001ffffULL, 0x000001ff0001ff01ULL, + 0x000001ff000100ffULL, 0x000001ff00010000ULL, 0x000001ff01ffff01ULL, 0x000001ff01ff0100ULL, + 0x000001ff0100ffffULL, 0x000001ff0100ff01ULL, 0x000001ff01000000ULL, 0x000001ff010001ffULL, + 0x000001ff0101ff00ULL, 0x000001ff01010100ULL, 0x00000100ffffff00ULL, 0x00000100ffffff01ULL, + 0x00000100ffff0000ULL, 0x00000100ffff0101ULL, 0x00000100ff00ff00ULL, 0x00000100ff0000ffULL, + 0x00000100ff000000ULL, 0x00000100ff000001ULL, 0x00000100ff000100ULL, 0x00000100ff010000ULL, + 0x0000010000ffff00ULL, 0x0000010000ff00ffULL, 0x0000010000ff0000ULL, 0x0000010000ff0001ULL, + 0x0000010000ff0100ULL, 0x000001000000ffffULL, 0x000001000000ff00ULL, 0x000001000000ff01ULL, + 0x00000100000000ffULL, 0x0000010000000000ULL, 0x0000010000000001ULL, 0x00000100000001ffULL, + 0x0000010000000100ULL, 0x0000010000000101ULL, 0x000001000001ff00ULL, 0x00000100000100ffULL, + 0x0000010000010000ULL, 0x0000010000010001ULL, 0x0000010000010100ULL, 0x0000010001ffff00ULL, + 0x0000010001ff0000ULL, 0x0000010001ff0100ULL, 0x000001000100ff00ULL, 0x00000100010000ffULL, + 0x0000010001000000ULL, 0x0000010001000001ULL, 0x00000100010001ffULL, 0x0000010001000100ULL, + 0x0000010001010000ULL, 0x00000101ffff00ffULL, 0x00000101ffff01ffULL, 0x00000101ff000000ULL, + 0x00000101ff000101ULL, 0x00000101ff01ffffULL, 0x00000101ff010000ULL, 0x00000101ff010001ULL, + 0x00000101ff010100ULL, 0x0000010100ff0000ULL, 0x0000010100ff01ffULL, 0x0000010100ff0100ULL, + 0x000001010000ff00ULL, 0x0000010100000000ULL, 0x0000010100000001ULL, 0x00000101000001ffULL, + 0x0000010100000100ULL, 0x000001010001ff01ULL, 0x0000010100010000ULL, 0x00000101000101ffULL, + 0x0000010100010101ULL, 0x0000010101ffff00ULL, 0x0000010101ff0101ULL, 0x000001010100ff01ULL, + 0x0000010101000000ULL, 0x0000010101000001ULL, 0x00000101010001ffULL, 0x0000010101000101ULL, + 0x000001010101ff00ULL, 0x0001ffffffff0000ULL, 0x0001ffffff0000ffULL, 0x0001ffffff000001ULL, + 0x0001ffffff000100ULL, 0x0001ffffff010000ULL, 0x0001ffff00ff00ffULL, 0x0001ffff0000ffffULL, + 0x0001ffff00000000ULL, 0x0001ffff00000001ULL, 0x0001ffff000001ffULL, 0x0001ffff00000101ULL, + 0x0001ffff0001ff00ULL, 0x0001ffff000100ffULL, 0x0001ffff00010001ULL, 0x0001ffff00010100ULL, + 0x0001ffff01ffff00ULL, 0x0001ffff01000001ULL, 0x0001ffff01010000ULL, 0x0001ff00ffffff00ULL, + 0x0001ff00ffff00ffULL, 0x0001ff00ffff0001ULL, 0x0001ff00ffff0100ULL, 0x0001ff00ff00ff01ULL, + 0x0001ff00ff000000ULL, 0x0001ff00ff01ff00ULL, 0x0001ff00ff01ff01ULL, 0x0001ff00ff010001ULL, + 0x0001ff00ff010100ULL, 0x0001ff0000ff0000ULL, 0x0001ff0000ff0100ULL, 0x0001ff000000ff00ULL, + 0x0001ff0000000000ULL, 0x0001ff0000000001ULL, 0x0001ff0000000100ULL, 0x0001ff0000010000ULL, + 0x0001ff0000010001ULL, 0x0001ff0000010101ULL, 0x0001ff0001ff00ffULL, 0x0001ff0001ff0101ULL, + 0x0001ff000100ff01ULL, 0x0001ff0001000000ULL, 0x0001ff000101ff00ULL, 0x0001ff0001010001ULL, + 0x0001ff0001010100ULL, 0x0001ff01ff00ff00ULL, 0x0001ff01ff000001ULL, 0x0001ff01ff000100ULL, + 0x0001ff0100ffffffULL, 0x0001ff0100ffff00ULL, 0x0001ff0100ff0001ULL, 0x0001ff0100000000ULL, + 0x0001ff0100000001ULL, 0x0001ff01000001ffULL, 0x0001ff010001ffffULL, 0x0001ff0101ff0000ULL, + 0x0001ff010100ff00ULL, 0x0001ff0101000001ULL, 0x0001ff0101010000ULL, 0x000100ffff00ff00ULL, + 0x000100ffff00ff01ULL, 0x000100ffff000000ULL, 0x000100ffff000001ULL, 0x000100ffff000101ULL, + 0x000100ffff01ff00ULL, 0x000100ffff010001ULL, 0x000100ffff010100ULL, 0x000100ff00ffffffULL, + 0x000100ff00ffff01ULL, 0x000100ff00ff0000ULL, 0x000100ff00ff01ffULL, 0x000100ff00ff0101ULL, + 0x000100ff0000ff00ULL, 0x000100ff000000ffULL, 0x000100ff00000000ULL, 0x000100ff00000001ULL, + 0x000100ff00000100ULL, 0x000100ff00000101ULL, 0x000100ff0001ffffULL, 0x000100ff0001ff01ULL, + 0x000100ff00010000ULL, 0x000100ff01ff00ffULL, 0x000100ff01ff0000ULL, 0x000100ff01ff0100ULL, + 0x000100ff0100ffffULL, 0x000100ff0100ff01ULL, 0x000100ff010000ffULL, 0x000100ff01000000ULL, + 0x000100ff01000001ULL, 0x000100ff010001ffULL, 0x000100ff01000101ULL, 0x000100ff0101ff00ULL, + 0x000100ff010100ffULL, 0x000100ff01010100ULL, 0x00010000ffff0000ULL, 0x00010000ffff01ffULL, + 0x00010000ffff0101ULL, 0x00010000ff00ff00ULL, 0x00010000ff000000ULL, 0x00010000ff000001ULL, + 0x00010000ff000100ULL, 0x0001000000ff00ffULL, 0x0001000000ff0000ULL, 0x0001000000ff0001ULL, + 0x0001000000ff0100ULL, 0x000100000000ffffULL, 0x000100000000ff00ULL, 0x00010000000000ffULL, + 0x0001000000000000ULL, 0x0001000000000001ULL, 0x0001000000000100ULL, 0x000100000001ff00ULL, + 0x00010000000100ffULL, 0x0001000000010000ULL, 0x0001000000010001ULL, 0x0001000000010100ULL, + 0x0001000001ff0001ULL, 0x0001000001ff0100ULL, 0x0001000001ff0101ULL, 0x000100000100ff00ULL, + 0x0001000001000000ULL, 0x0001000001000001ULL, 0x0001000001000100ULL, 0x0001000001000101ULL, + 0x000100000101ff01ULL, 0x0001000001010000ULL, 0x0001000001010001ULL, 0x00010000010101ffULL, + 0x00010001ffffff01ULL, 0x00010001ffff0100ULL, 0x00010001ff000000ULL, 0x00010001ff01ffffULL, + 0x00010001ff010001ULL, 0x00010001ff0101ffULL, 0x00010001ff010100ULL, 0x0001000100ffffffULL, + 0x0001000100ff0000ULL, 0x0001000100ff01ffULL, 0x0001000100ff0101ULL, 0x000100010000ff00ULL, + 0x00010001000000ffULL, 0x0001000100000000ULL, 0x0001000100000001ULL, 0x00010001000001ffULL, + 0x0001000100000101ULL, 0x000100010001ffffULL, 0x0001000100010000ULL, 0x00010001000101ffULL, + 0x0001000101ffffffULL, 0x0001000101ffff01ULL, 0x0001000101ff0000ULL, 0x0001000101ff0101ULL, + 0x00010001010000ffULL, 0x0001000101000001ULL, 0x00010001010001ffULL, 0x0001000101000100ULL, + 0x000100010101ffffULL, 0x00010001010100ffULL, 0x0001000101010001ULL, 0x0001000101010101ULL, + 0x000101ffff000001ULL, 0x000101ffff000100ULL, 0x000101ffff010000ULL, 0x000101ff00ffff00ULL, + 0x000101ff0000ff01ULL, 0x000101ff00000000ULL, 0x000101ff00000101ULL, 0x000101ff0001ff00ULL, + 0x000101ff00010100ULL, 0x000101ff01ff0000ULL, 0x000101ff0100ff00ULL, 0x000101ff010001ffULL, + 0x000101ff01010001ULL, 0x00010100ffffff00ULL, 0x00010100ffff00ffULL, 0x00010100ff00ffffULL, + 0x00010100ff000000ULL, 0x00010100ff01ff00ULL, 0x00010100ff0100ffULL, 0x00010100ff010001ULL, + 0x00010100ff010100ULL, 0x0001010000ffffffULL, 0x0001010000ffff00ULL, 0x0001010000ff0000ULL, + 0x0001010000ff0001ULL, 0x0001010000ff01ffULL, 0x000101000000ff00ULL, 0x00010100000000ffULL, + 0x0001010000000000ULL, 0x0001010000000001ULL, 0x0001010000000100ULL, 0x000101000001ffffULL, + 0x0001010000010000ULL, 0x0001010000010101ULL, 0x0001010001ffff01ULL, 0x0001010001ff00ffULL, + 0x0001010001ff0101ULL, 0x0001010001000000ULL, 0x000101000101ff00ULL, 0x00010100010100ffULL, + 0x0001010001010000ULL, 0x0001010001010100ULL, 0x00010101ff00ff00ULL, 0x00010101ff000001ULL, + 0x00010101ff0001ffULL, 0x0001010100ffff00ULL, 0x0001010100ff00ffULL, 0x0001010100ff0100ULL, + 0x000101010000ffffULL, 0x0001010100000000ULL, 0x00010101000001ffULL, 0x0001010100000101ULL, + 0x00010101000100ffULL, 0x0001010100010000ULL, 0x0001010100010100ULL, 0x0001010101ff0001ULL, + 0x00010101010000ffULL, 0x00010101010001ffULL, 0x0001010101000101ULL, 0x0001010101010001ULL, + 0x01ffffffffffffffULL, 0x01ffffffffffff01ULL, 0x01ffffffffff01ffULL, 0x01ffffffffff0101ULL, + 0x01ffffffff01ffffULL, 0x01ffffffff01ff01ULL, 0x01ffffffff0101ffULL, 0x01ffffffff010101ULL, + 0x01ffffff00ff0000ULL, 0x01ffffff0000ffffULL, 0x01ffffff0000ff00ULL, 0x01ffffff000000ffULL, + 0x01ffffff00000001ULL, 0x01ffffff00000100ULL, 0x01ffffff00010000ULL, 0x01ffffff01ffffffULL, + 0x01ffffff01ffff01ULL, 0x01ffffff01ff01ffULL, 0x01ffffff01ff0101ULL, 0x01ffffff01000000ULL, + 0x01ffffff0101ffffULL, 0x01ffffff0101ff01ULL, 0x01ffffff010101ffULL, 0x01ffffff01010101ULL, + 0x01ffff00ffff0000ULL, 0x01ffff00ff00ff00ULL, 0x01ffff00ff0000ffULL, 0x01ffff00ff000001ULL, + 0x01ffff00ff000100ULL, 0x01ffff00ff010000ULL, 0x01ffff0000ffff00ULL, 0x01ffff0000ff00ffULL, + 0x01ffff0000ff0100ULL, 0x01ffff000000ffffULL, 0x01ffff000000ff01ULL, 0x01ffff0000000000ULL, + 0x01ffff0000000001ULL, 0x01ffff00000001ffULL, 0x01ffff0000000100ULL, 0x01ffff00000100ffULL, + 0x01ffff0000010001ULL, 0x01ffff0000010100ULL, 0x01ffff0001ff0000ULL, 0x01ffff0001ff0100ULL, + 0x01ffff00010000ffULL, 0x01ffff0001000001ULL, 0x01ffff0001000100ULL, 0x01ffff0001010000ULL, + 0x01ffff01ffffffffULL, 0x01ffff01ffffff01ULL, 0x01ffff01ffff01ffULL, 0x01ffff01ffff0101ULL, + 0x01ffff01ff000000ULL, 0x01ffff01ff01ffffULL, 0x01ffff01ff01ff01ULL, 0x01ffff01ff0101ffULL, + 0x01ffff01ff010101ULL, 0x01ffff010000ff00ULL, 0x01ffff01000000ffULL, 0x01ffff0100000100ULL, + 0x01ffff0100010000ULL, 0x01ffff0101ffffffULL, 0x01ffff0101ffff01ULL, 0x01ffff0101ff01ffULL, + 0x01ffff0101ff0101ULL, 0x01ffff0101000000ULL, 0x01ffff010101ffffULL, 0x01ffff010101ff01ULL, + 0x01ffff01010101ffULL, 0x01ffff0101010101ULL, 0x01ff00ffff0000ffULL, 0x01ff00ffff000100ULL, + 0x01ff00ff00ffff00ULL, 0x01ff00ff00ff00ffULL, 0x01ff00ff0000ff00ULL, 0x01ff00ff00000000ULL, + 0x01ff00ff00000101ULL, 0x01ff00ff0001ff00ULL, 0x01ff00ff000100ffULL, 0x01ff00ff00010100ULL, + 0x01ff00ff010000ffULL, 0x01ff00ff01000100ULL, 0x01ff0000ffffff00ULL, 0x01ff0000ffff0100ULL, + 0x01ff0000ff00ff01ULL, 0x01ff0000ff000000ULL, 0x01ff0000ff000101ULL, 0x01ff0000ff010001ULL, + 0x01ff0000ff010100ULL, 0x01ff000000ffffffULL, 0x01ff000000ffff00ULL, 0x01ff000000ff0000ULL, + 0x01ff000000ff01ffULL, 0x01ff00000000ff00ULL, 0x01ff0000000000ffULL, 0x01ff000000000000ULL, + 0x01ff000000000001ULL, 0x01ff000000000100ULL, 0x01ff000000000101ULL, 0x01ff000000010000ULL, + 0x01ff000000010001ULL, 0x01ff0000000101ffULL, 0x01ff000000010101ULL, 0x01ff000001ffff00ULL, + 0x01ff000001ff00ffULL, 0x01ff000001ff0001ULL, 0x01ff000001ff0100ULL, 0x01ff00000100ffffULL, + 0x01ff00000100ff01ULL, 0x01ff000001000000ULL, 0x01ff0000010001ffULL, 0x01ff000001010001ULL, + 0x01ff0001ff00ff00ULL, 0x01ff0001ff000001ULL, 0x01ff0001ff000100ULL, 0x01ff0001ff010000ULL, + 0x01ff000100ffff00ULL, 0x01ff000100ff00ffULL, 0x01ff000100ff0100ULL, 0x01ff000100ff0101ULL, + 0x01ff00010000ffffULL, 0x01ff000100000000ULL, 0x01ff000100000100ULL, 0x01ff000100000101ULL, + 0x01ff00010001ff00ULL, 0x01ff000100010001ULL, 0x01ff000100010101ULL, 0x01ff000101ff0000ULL, + 0x01ff00010100ff00ULL, 0x01ff000101000101ULL, 0x01ff0001010100ffULL, 0x01ff01ffffffffffULL, + 0x01ff01ffffffff01ULL, 0x01ff01ffffff01ffULL, 0x01ff01ffffff0101ULL, 0x01ff01ffff000000ULL, + 0x01ff01ffff01ffffULL, 0x01ff01ffff01ff01ULL, 0x01ff01ffff0101ffULL, 0x01ff01ffff010101ULL, + 0x01ff01ff00ffff00ULL, 0x01ff01ff00ff0000ULL, 0x01ff01ff0000ff00ULL, 0x01ff01ff000000ffULL, + 0x01ff01ff00000100ULL, 0x01ff01ff00010000ULL, 0x01ff01ff00010100ULL, 0x01ff01ff01ffffffULL, + 0x01ff01ff01ffff01ULL, 0x01ff01ff01ff01ffULL, 0x01ff01ff01ff0101ULL, 0x01ff01ff01000000ULL, + 0x01ff01ff0101ffffULL, 0x01ff01ff0101ff01ULL, 0x01ff01ff010101ffULL, 0x01ff01ff01010101ULL, + 0x01ff0100ffff0000ULL, 0x01ff0100ffff0001ULL, 0x01ff0100ff00ff00ULL, 0x01ff0100ff0000ffULL, + 0x01ff0100ff000001ULL, 0x01ff0100ff010000ULL, 0x01ff010000ffff00ULL, 0x01ff010000ff00ffULL, + 0x01ff010000ff0001ULL, 0x01ff010000ff0100ULL, 0x01ff01000000ffffULL, 0x01ff01000000ff01ULL, + 0x01ff010000000000ULL, 0x01ff010000000101ULL, 0x01ff01000001ff00ULL, 0x01ff0100000100ffULL, + 0x01ff010001ff0000ULL, 0x01ff010001000001ULL, 0x01ff010001000100ULL, 0x01ff010001010000ULL, + 0x01ff0101ffffffffULL, 0x01ff0101ffffff01ULL, 0x01ff0101ffff01ffULL, 0x01ff0101ffff0101ULL, + 0x01ff0101ff000000ULL, 0x01ff0101ff01ffffULL, 0x01ff0101ff01ff01ULL, 0x01ff0101ff0101ffULL, + 0x01ff0101ff010101ULL, 0x01ff010100ff0000ULL, 0x01ff01010000ff00ULL, 0x01ff0101000000ffULL, + 0x01ff010100000001ULL, 0x01ff010101ffffffULL, 0x01ff010101ffff01ULL, 0x01ff010101ff01ffULL, + 0x01ff010101ff0101ULL, 0x01ff010101000000ULL, 0x01ff01010101ffffULL, 0x01ff01010101ff01ULL, + 0x01ff0101010101ffULL, 0x01ff010101010101ULL, 0x0100ffffffff0000ULL, 0x0100ffffff00ff00ULL, + 0x0100ffffff000001ULL, 0x0100ffffff0001ffULL, 0x0100ffffff000100ULL, 0x0100ffffff010000ULL, + 0x0100ffff00ffff00ULL, 0x0100ffff00ff0001ULL, 0x0100ffff00ff0100ULL, 0x0100ffff00000000ULL, + 0x0100ffff000001ffULL, 0x0100ffff00000101ULL, 0x0100ffff00010100ULL, 0x0100ffff00010101ULL, + 0x0100ffff01ff0000ULL, 0x0100ffff0100ff00ULL, 0x0100ffff010000ffULL, 0x0100ffff01000001ULL, + 0x0100ffff01000100ULL, 0x0100ffff01010000ULL, 0x0100ff00ffffff00ULL, 0x0100ff00ffff00ffULL, + 0x0100ff00ffff0001ULL, 0x0100ff00ffff0100ULL, 0x0100ff00ff00ffffULL, 0x0100ff00ff000000ULL, + 0x0100ff00ff0001ffULL, 0x0100ff00ff000101ULL, 0x0100ff00ff01ff00ULL, 0x0100ff00ff0100ffULL, + 0x0100ff00ff010001ULL, 0x0100ff00ff010100ULL, 0x0100ff0000ffffffULL, 0x0100ff0000ff0000ULL, + 0x0100ff000000ffffULL, 0x0100ff000000ff00ULL, 0x0100ff00000000ffULL, 0x0100ff0000000000ULL, + 0x0100ff0000000001ULL, 0x0100ff0000000100ULL, 0x0100ff000001ff01ULL, 0x0100ff0000010000ULL, + 0x0100ff0001ff00ffULL, 0x0100ff0001ff0001ULL, 0x0100ff000100ff01ULL, 0x0100ff0001000000ULL, + 0x0100ff00010001ffULL, 0x0100ff000101ff00ULL, 0x0100ff00010100ffULL, 0x0100ff0001010001ULL, + 0x0100ff0001010100ULL, 0x0100ff01ffff0000ULL, 0x0100ff01ff00ff00ULL, 0x0100ff01ff0000ffULL, + 0x0100ff01ff000100ULL, 0x0100ff01ff010000ULL, 0x0100ff0100ff00ffULL, 0x0100ff0100ff0001ULL, + 0x0100ff0100ff0100ULL, 0x0100ff010000ffffULL, 0x0100ff010000ff01ULL, 0x0100ff0100000000ULL, + 0x0100ff01000001ffULL, 0x0100ff0100010001ULL, 0x0100ff0100010100ULL, 0x0100ff0101ff0000ULL, + 0x0100ff01010000ffULL, 0x0100ff0101000001ULL, 0x0100ff0101010100ULL, 0x010000ffffffff00ULL, + 0x010000ffffff00ffULL, 0x010000ffffff0001ULL, 0x010000ffff00ffffULL, 0x010000ffff000000ULL, + 0x010000ffff0001ffULL, 0x010000ffff010001ULL, 0x010000ff00ffffffULL, 0x010000ff00ff0101ULL, + 0x010000ff0000ff00ULL, 0x010000ff000000ffULL, 0x010000ff00000000ULL, 0x010000ff00000001ULL, + 0x010000ff000001ffULL, 0x010000ff00000100ULL, 0x010000ff0001ffffULL, 0x010000ff0001ff00ULL, + 0x010000ff0001ff01ULL, 0x010000ff00010000ULL, 0x010000ff01ff00ffULL, 0x010000ff01ff0001ULL, + 0x010000ff0100ff01ULL, 0x010000ff010000ffULL, 0x010000ff01000000ULL, 0x010000ff010001ffULL, + 0x010000ff0101ff00ULL, 0x010000ff01010100ULL, 0x01000000ffffffffULL, 0x01000000ffff0000ULL, + 0x01000000ffff01ffULL, 0x01000000ffff0101ULL, 0x01000000ff00ffffULL, 0x01000000ff00ff00ULL, + 0x01000000ff0000ffULL, 0x01000000ff000000ULL, 0x01000000ff000001ULL, 0x01000000ff000100ULL, + 0x01000000ff01ff00ULL, 0x01000000ff010000ULL, 0x01000000ff010100ULL, 0x01000000ff010101ULL, + 0x0100000000ffff00ULL, 0x0100000000ff00ffULL, 0x0100000000ff0000ULL, 0x0100000000ff0001ULL, + 0x0100000000ff0100ULL, 0x010000000000ffffULL, 0x010000000000ff00ULL, 0x010000000000ff01ULL, + 0x01000000000000ffULL, 0x0100000000000000ULL, 0x0100000000000001ULL, 0x01000000000001ffULL, + 0x0100000000000100ULL, 0x0100000000000101ULL, 0x010000000001ff00ULL, 0x01000000000100ffULL, + 0x0100000000010000ULL, 0x0100000000010001ULL, 0x0100000000010100ULL, 0x0100000001ffff00ULL, + 0x0100000001ff0000ULL, 0x0100000001ff01ffULL, 0x010000000100ff00ULL, 0x010000000100ff01ULL, + 0x01000000010000ffULL, 0x0100000001000000ULL, 0x0100000001000001ULL, 0x0100000001000100ULL, + 0x0100000001000101ULL, 0x010000000101ffffULL, 0x010000000101ff01ULL, 0x0100000001010000ULL, + 0x01000000010101ffULL, 0x0100000001010101ULL, 0x01000001ffffff00ULL, 0x01000001ffff00ffULL, + 0x01000001ff00ffffULL, 0x01000001ff000000ULL, 0x01000001ff000100ULL, 0x01000001ff01ffffULL, + 0x01000001ff010001ULL, 0x01000001ff010100ULL, 0x0100000100ff0000ULL, 0x0100000100ff01ffULL, + 0x0100000100ff0100ULL, 0x010000010000ff00ULL, 0x010000010000ff01ULL, 0x0100000100000000ULL, + 0x0100000100000001ULL, 0x0100000100000100ULL, 0x0100000100010000ULL, 0x01000001000101ffULL, + 0x0100000101ffff01ULL, 0x0100000101ff00ffULL, 0x0100000101ff0100ULL, 0x0100000101ff0101ULL, + 0x010000010100ff01ULL, 0x01000001010000ffULL, 0x0100000101000000ULL, 0x01000001010100ffULL, + 0x0100000101010001ULL, 0x0100000101010100ULL, 0x010001ffffff0000ULL, 0x010001ffff000001ULL, + 0x010001ffff000100ULL, 0x010001ffff010000ULL, 0x010001ff00ffff00ULL, 0x010001ff00ff0001ULL, + 0x010001ff0000ffffULL, 0x010001ff0000ff01ULL, 0x010001ff00000000ULL, 0x010001ff00000001ULL, + 0x010001ff00000101ULL, 0x010001ff000100ffULL, 0x010001ff00010000ULL, 0x010001ff01ff0000ULL, + 0x010001ff0100ff00ULL, 0x010001ff01000001ULL, 0x010001ff01000100ULL, 0x010001ff01010000ULL, + 0x01000100ffff00ffULL, 0x01000100ffff0001ULL, 0x01000100ffff0100ULL, 0x01000100ff00ffffULL, + 0x01000100ff00ff01ULL, 0x01000100ff000000ULL, 0x01000100ff0001ffULL, 0x01000100ff000101ULL, + 0x01000100ff01ffffULL, 0x01000100ff01ff00ULL, 0x01000100ff0100ffULL, 0x01000100ff010001ULL, + 0x0100010000ffffffULL, 0x0100010000ffff01ULL, 0x0100010000ff0000ULL, 0x0100010000ff01ffULL, + 0x0100010000ff0101ULL, 0x010001000000ff00ULL, 0x01000100000000ffULL, 0x0100010000000000ULL, + 0x0100010000000001ULL, 0x0100010000000100ULL, 0x010001000001ff01ULL, 0x0100010000010000ULL, + 0x0100010000010001ULL, 0x0100010000010101ULL, 0x0100010001ffff00ULL, 0x0100010001ff00ffULL, + 0x010001000100ffffULL, 0x010001000100ff01ULL, 0x0100010001000000ULL, 0x0100010001000101ULL, + 0x010001000101ff00ULL, 0x0100010001010001ULL, 0x01000101ffff0000ULL, 0x01000101ff000000ULL, + 0x01000101ff010000ULL, 0x0100010100ff00ffULL, 0x0100010100ff0001ULL, 0x0100010100ff0100ULL, + 0x010001010000ffffULL, 0x0100010100000000ULL, 0x01000101000001ffULL, 0x010001010001ff00ULL, + 0x0100010101ff0000ULL, 0x010001010100ff00ULL, 0x01000101010000ffULL, 0x0100010101000000ULL, + 0x0100010101000001ULL, 0x0101ffffffffffffULL, 0x0101ffffffffff01ULL, 0x0101ffffffff01ffULL, + 0x0101ffffffff0101ULL, 0x0101ffffff000000ULL, 0x0101ffffff01ffffULL, 0x0101ffffff01ff01ULL, + 0x0101ffffff0101ffULL, 0x0101ffffff010101ULL, 0x0101ffff00ff0000ULL, 0x0101ffff0000ff00ULL, + 0x0101ffff000000ffULL, 0x0101ffff00000001ULL, 0x0101ffff00000100ULL, 0x0101ffff01ffffffULL, + 0x0101ffff01ffff01ULL, 0x0101ffff01ff01ffULL, 0x0101ffff01ff0101ULL, 0x0101ffff01000000ULL, + 0x0101ffff0101ffffULL, 0x0101ffff0101ff01ULL, 0x0101ffff010101ffULL, 0x0101ffff01010101ULL, + 0x0101ff00ffff0000ULL, 0x0101ff00ffff0100ULL, 0x0101ff00ff00ff00ULL, 0x0101ff00ff0000ffULL, + 0x0101ff00ff000001ULL, 0x0101ff00ff000100ULL, 0x0101ff00ff000101ULL, 0x0101ff0000ff0001ULL, + 0x0101ff0000ff0100ULL, 0x0101ff000000ff00ULL, 0x0101ff0000000000ULL, 0x0101ff00000001ffULL, + 0x0101ff0000000101ULL, 0x0101ff000001ff00ULL, 0x0101ff00000100ffULL, 0x0101ff0001ff0000ULL, + 0x0101ff000100ffffULL, 0x0101ff000100ff01ULL, 0x0101ff0001000001ULL, 0x0101ff0001000100ULL, + 0x0101ff01ffffff01ULL, 0x0101ff01ffff01ffULL, 0x0101ff01ffff0101ULL, 0x0101ff01ff00ffffULL, + 0x0101ff01ff000100ULL, 0x0101ff01ff01ff01ULL, 0x0101ff01ff0101ffULL, 0x0101ff01ff010101ULL, + 0x0101ff0100ff0000ULL, 0x0101ff010000ff00ULL, 0x0101ff0100000001ULL, 0x0101ff0100000100ULL, + 0x0101ff0100010000ULL, 0x0101ff0101ffffffULL, 0x0101ff0101ffff01ULL, 0x0101ff0101ff01ffULL, + 0x0101ff0101ff0101ULL, 0x0101ff0101000000ULL, 0x0101ff010101ffffULL, 0x0101ff010101ff01ULL, + 0x0101ff01010101ffULL, 0x0101ff0101010101ULL, 0x010100ffff000100ULL, 0x010100ffff010000ULL, + 0x010100ff00ffff00ULL, 0x010100ff00ff00ffULL, 0x010100ff0000ffffULL, 0x010100ff000000ffULL, + 0x010100ff00000000ULL, 0x010100ff000001ffULL, 0x010100ff00000101ULL, 0x010100ff0001ff00ULL, + 0x010100ff00010000ULL, 0x010100ff00010001ULL, 0x010100ff000101ffULL, 0x010100ff00010100ULL, + 0x010100ff01ff0000ULL, 0x01010000ffff0001ULL, 0x01010000ffff0100ULL, 0x01010000ff00ffffULL, + 0x01010000ff00ff01ULL, 0x01010000ff000000ULL, 0x01010000ff0001ffULL, 0x01010000ff010001ULL, + 0x01010000ff010100ULL, 0x0101000000ffff01ULL, 0x0101000000ff0000ULL, 0x010100000000ff00ULL, + 0x01010000000000ffULL, 0x0101000000000000ULL, 0x0101000000000001ULL, 0x0101000000000100ULL, + 0x0101000000010000ULL, 0x0101000000010101ULL, 0x0101000001ffff00ULL, 0x0101000001ff00ffULL, + 0x0101000001ff0000ULL, 0x0101000001ff0001ULL, 0x0101000001ff0100ULL, 0x010100000100ff01ULL, + 0x0101000001000000ULL, 0x01010000010001ffULL, 0x01010001ffff0000ULL, 0x01010001ff00ff00ULL, + 0x01010001ff000001ULL, 0x01010001ff000101ULL, 0x01010001ff01ff00ULL, 0x01010001ff010000ULL, + 0x0101000100ff00ffULL, 0x0101000100ff0001ULL, 0x0101000100ff0101ULL, 0x010100010000ff01ULL, + 0x0101000100000000ULL, 0x0101000100000001ULL, 0x01010001000001ffULL, 0x010100010001ffffULL, + 0x010100010001ff01ULL, 0x0101000101ff0001ULL, 0x010100010100ffffULL, 0x0101000101000000ULL, + 0x0101000101000001ULL, 0x0101000101000100ULL, 0x010100010101ff00ULL, 0x01010001010100ffULL, + 0x0101000101010001ULL, 0x010101ffffffffffULL, 0x010101ffffffff01ULL, 0x010101ffffff01ffULL, + 0x010101ffffff0101ULL, 0x010101ffff01ffffULL, 0x010101ffff01ff01ULL, 0x010101ffff0101ffULL, + 0x010101ffff010101ULL, 0x010101ff0000ff00ULL, 0x010101ff000000ffULL, 0x010101ff00000001ULL, + 0x010101ff00000100ULL, 0x010101ff01ffffffULL, 0x010101ff01ffff01ULL, 0x010101ff01ff01ffULL, + 0x010101ff01ff0101ULL, 0x010101ff01000000ULL, 0x010101ff0101ffffULL, 0x010101ff0101ff01ULL, + 0x010101ff010101ffULL, 0x010101ff01010101ULL, 0x01010100ffff0000ULL, 0x01010100ff0000ffULL, + 0x01010100ff000100ULL, 0x01010100ff01ff00ULL, 0x01010100ff010000ULL, 0x0101010000ffff00ULL, + 0x010101000000ffffULL, 0x0101010000000000ULL, 0x0101010000000101ULL, 0x010101000001ff00ULL, + 0x0101010000010001ULL, 0x0101010000010100ULL, 0x010101000100ffffULL, 0x0101010001000001ULL, + 0x01010101ffffffffULL, 0x01010101ffffff01ULL, 0x01010101ffff01ffULL, 0x01010101ffff0101ULL, + 0x01010101ff01ffffULL, 0x01010101ff01ff01ULL, 0x01010101ff0101ffULL, 0x01010101ff010101ULL, + 0x010101010000ff00ULL, 0x01010101000000ffULL, 0x0101010100000001ULL, 0x0101010101ffffffULL, + 0x0101010101ffff01ULL, 0x0101010101ff01ffULL, 0x0101010101ff0101ULL, 0x0101010101000000ULL, + 0x010101010101ffffULL, 0x010101010101ff01ULL, 0x01010101010101ffULL, 0x0101010101010101ULL, +}; + +// Fork ggml-common.h:2095, iq1_xxxs_grid. +__device__ const uint64_t d_iq1xxxs_grid[256] = { + 0xffffffffffffffffULL, 0xffffffffff000000ULL, 0xffffffff01000000ULL, 0xffffff01ff000000ULL, + 0xffff00ff00010000ULL, 0xffff000000ff0000ULL, 0xffff0000000000ffULL, 0xffff000000000001ULL, + 0xffff000000000100ULL, 0xffff0100ff000100ULL, 0xffff01000000ff00ULL, 0xffff010101000000ULL, + 0xff00ff00ff0001ffULL, 0xff00ff0000ff0000ULL, 0xff00ff00000000ffULL, 0xff00ff0000000001ULL, + 0xff00ff0000010000ULL, 0xff00ff010100ff00ULL, 0xff0000ff00000000ULL, 0xff0000ff01ff0100ULL, + 0xff000000ffff0000ULL, 0xff000000ff00ff00ULL, 0xff000000ff000000ULL, 0xff0000000000ff00ULL, + 0xff0000000000ff01ULL, 0xff000000000000ffULL, 0xff00000000000000ULL, 0xff00000000000100ULL, + 0xff00000000000101ULL, 0xff0000000001ff00ULL, 0xff00000000010001ULL, 0xff00000001ff00ffULL, + 0xff00000001000000ULL, 0xff00000100ffffffULL, 0xff00000100000000ULL, 0xff000001000100ffULL, + 0xff00000100010100ULL, 0xff00000101ff0001ULL, 0xff0001ff00ffff00ULL, 0xff00010000ff01ffULL, + 0xff00010000000000ULL, 0xff00010000010000ULL, 0xff01ff0000000000ULL, 0xff0100ffff000001ULL, + 0xff0100ff000001ffULL, 0xff0100ff0100ff00ULL, 0xff01000000ff0000ULL, 0xff01000000010000ULL, + 0xff010001ff000100ULL, 0xff010100ff0000ffULL, 0xff01010001000001ULL, 0x00ffff00ff000001ULL, + 0x00ffff000000ff00ULL, 0x00ffff0000000000ULL, 0x00ffff0000000101ULL, 0x00ff00ff00ff01ffULL, + 0x00ff00ff00000000ULL, 0x00ff00ff00000001ULL, 0x00ff0000ffff0000ULL, 0x00ff0000ff00ff00ULL, + 0x00ff0000ff0000ffULL, 0x00ff0000ff010000ULL, 0x00ff00000000ffffULL, 0x00ff000000000000ULL, + 0x00ff000000000001ULL, 0x00ff0000000001ffULL, 0x00ff00000001ff00ULL, 0x00ff000000010100ULL, + 0x00ff000001ff0000ULL, 0x00ff00000100ff00ULL, 0x00ff0000010000ffULL, 0x00ff000001000100ULL, + 0x00ff000001010000ULL, 0x00ff000100ff0000ULL, 0x00ff00010000ff01ULL, 0x00ff000100000000ULL, + 0x00ff010000ffffffULL, 0x00ff010000000000ULL, 0x00ff010000010001ULL, 0x00ff010001000001ULL, + 0x00ff0101000100ffULL, 0x0000ffff000000ffULL, 0x0000ffff00000000ULL, 0x0000ffff00000001ULL, + 0x0000ff00ffff0100ULL, 0x0000ff00ff000000ULL, 0x0000ff00ff010000ULL, 0x0000ff0000ff00ffULL, + 0x0000ff0000ff0001ULL, 0x0000ff000000ff00ULL, 0x0000ff000000ff01ULL, 0x0000ff0000000000ULL, + 0x0000ff0000000100ULL, 0x0000ff00000100ffULL, 0x0000ff0001ff0000ULL, 0x0000ff0001000000ULL, + 0x0000ff0001010000ULL, 0x0000ff01000000ffULL, 0x0000ff0100000000ULL, 0x0000ff0100000001ULL, + 0x0000ff0101000100ULL, 0x000000ffffff0000ULL, 0x000000ffff000001ULL, 0x000000ffff0001ffULL, + 0x000000ffff010000ULL, 0x000000ff00ff0000ULL, 0x000000ff00ff0101ULL, 0x000000ff0000ff00ULL, + 0x000000ff000000ffULL, 0x000000ff00000100ULL, 0x000000ff0001ff01ULL, 0x000000ff00010000ULL, + 0x000000ff00010100ULL, 0x000000ff01000000ULL, 0x000000ff01000001ULL, 0x000000ff010100ffULL, + 0x00000000ff00ffffULL, 0x00000000ff00ff01ULL, 0x00000000ff000000ULL, 0x00000000ff000100ULL, + 0x00000000ff000101ULL, 0x00000000ff0100ffULL, 0x0000000000ffff00ULL, 0x0000000000ff00ffULL, + 0x0000000000ff0000ULL, 0x0000000000ff0001ULL, 0x0000000000ff0100ULL, 0x000000000000ff00ULL, + 0x00000000000000ffULL, 0x0000000000000000ULL, 0x0000000000000001ULL, 0x0000000000000100ULL, + 0x000000000001ffffULL, 0x00000000000100ffULL, 0x0000000000010000ULL, 0x0000000000010001ULL, + 0x00000000000101ffULL, 0x0000000001ffff00ULL, 0x000000000100ffffULL, 0x0000000001000000ULL, + 0x0000000001000001ULL, 0x00000000010001ffULL, 0x000000000101ff00ULL, 0x0000000001010100ULL, + 0x00000001ffffff00ULL, 0x00000001ffff0001ULL, 0x00000001ff0000ffULL, 0x00000001ff000000ULL, + 0x00000001ff01ff00ULL, 0x00000001ff010001ULL, 0x000000010000ff00ULL, 0x0000000100000000ULL, + 0x00000001000001ffULL, 0x0000000100000101ULL, 0x0000000100010000ULL, 0x0000000101ff0100ULL, + 0x0000000101000000ULL, 0x0000000101010001ULL, 0x000001ffff000000ULL, 0x000001ff000000ffULL, + 0x000001ff00000001ULL, 0x000001ff01000000ULL, 0x00000100ffff0000ULL, 0x00000100ff000001ULL, + 0x00000100ff010000ULL, 0x0000010000ff0001ULL, 0x000001000000ff00ULL, 0x000001000000ff01ULL, + 0x00000100000000ffULL, 0x0000010000000000ULL, 0x0000010000000100ULL, 0x0000010001ff0000ULL, + 0x00000100010000ffULL, 0x0000010001000100ULL, 0x0000010001010000ULL, 0x00000101ff000000ULL, + 0x0000010100ff0000ULL, 0x0000010100000001ULL, 0x0000010100000100ULL, 0x0001ff0000ff0000ULL, + 0x0001ff000000ff00ULL, 0x0001ff0000000100ULL, 0x0001ff0000010001ULL, 0x000100ffff00ff00ULL, + 0x000100ff00ffffffULL, 0x000100ff00ffff01ULL, 0x000100ff00000000ULL, 0x00010000ffff0000ULL, + 0x00010000ff000000ULL, 0x0001000000ff0100ULL, 0x00010000000000ffULL, 0x0001000000000000ULL, + 0x0001000000000001ULL, 0x000100000001ff00ULL, 0x0001000000010100ULL, 0x0001000001000000ULL, + 0x0001000001000101ULL, 0x0001000001010000ULL, 0x0001000100ff0000ULL, 0x000100010000ff00ULL, + 0x0001000100000000ULL, 0x00010001010000ffULL, 0x0001010000ff0000ULL, 0x000101000000ff00ULL, + 0x0001010000000100ULL, 0x0001010000010000ULL, 0x0001010100000000ULL, 0x01ffffff00ff0000ULL, + 0x01ffffff00010000ULL, 0x01ffff0101000000ULL, 0x01ff00ffff0000ffULL, 0x01ff000000ffff00ULL, + 0x01ff000000000000ULL, 0x01ff000000000101ULL, 0x01ff0001ff000100ULL, 0x01ff01ff00010000ULL, + 0x01ff010000ff0100ULL, 0x0100ff00ffffff00ULL, 0x0100ff00000000ffULL, 0x0100ff0000000000ULL, + 0x0100ff0000000001ULL, 0x0100ff0100ff0100ULL, 0x010000ff00000000ULL, 0x01000000ff000000ULL, + 0x01000000ff01ff00ULL, 0x0100000000ff0000ULL, 0x0100000000ff0001ULL, 0x010000000000ffffULL, + 0x010000000000ff00ULL, 0x0100000000000100ULL, 0x0100000000010000ULL, 0x0100000000010100ULL, + 0x010000000100ff01ULL, 0x01000000010000ffULL, 0x0100000001000000ULL, 0x0100000001000101ULL, + 0x0100000100000000ULL, 0x0100000100000001ULL, 0x01000001010100ffULL, 0x010001ff00ffff00ULL, + 0x01000100ff000000ULL, 0x01000100000000ffULL, 0x0100010000000001ULL, 0x0100010100ff00ffULL, + 0x010001010001ff00ULL, 0x0100010101000000ULL, 0x0101ffffff000000ULL, 0x0101ffff01000000ULL, + 0x0101ff0100010000ULL, 0x010100ff000000ffULL, 0x010100ff00010001ULL, 0x01010000ff0001ffULL, + 0x0101000000000000ULL, 0x0101000001ff0000ULL, 0x01010001ff000001ULL, 0x010101ff00000100ULL, +}; + +// Stock ggml-common.h:560, iq2xxs_grid. +__device__ const uint64_t d_iq2xxs_grid[256] = { + 0x0808080808080808ULL, 0x080808080808082bULL, 0x0808080808081919ULL, 0x0808080808082b08ULL, + 0x0808080808082b2bULL, 0x0808080808190819ULL, 0x0808080808191908ULL, 0x08080808082b0808ULL, + 0x08080808082b082bULL, 0x08080808082b2b08ULL, 0x08080808082b2b2bULL, 0x0808080819080819ULL, + 0x0808080819081908ULL, 0x0808080819190808ULL, 0x0808080819192b08ULL, 0x08080808192b0819ULL, + 0x08080808192b1908ULL, 0x080808082b080808ULL, 0x080808082b08082bULL, 0x080808082b082b2bULL, + 0x080808082b2b082bULL, 0x0808081908080819ULL, 0x0808081908081908ULL, 0x0808081908190808ULL, + 0x0808081908191919ULL, 0x0808081919080808ULL, 0x080808192b081908ULL, 0x080808192b192b08ULL, + 0x0808082b08080808ULL, 0x0808082b0808082bULL, 0x0808082b082b082bULL, 0x0808082b2b08082bULL, + 0x0808190808080819ULL, 0x0808190808081908ULL, 0x0808190808190808ULL, 0x08081908082b0819ULL, + 0x08081908082b1908ULL, 0x0808190819080808ULL, 0x080819081908082bULL, 0x0808190819082b08ULL, + 0x08081908192b0808ULL, 0x080819082b080819ULL, 0x080819082b081908ULL, 0x080819082b190808ULL, + 0x080819082b2b1908ULL, 0x0808191908080808ULL, 0x080819190808082bULL, 0x0808191908082b08ULL, + 0x08081919082b0808ULL, 0x080819191908192bULL, 0x08081919192b2b19ULL, 0x080819192b080808ULL, + 0x080819192b190819ULL, 0x0808192b08082b19ULL, 0x0808192b08190808ULL, 0x0808192b19080808ULL, + 0x0808192b2b081908ULL, 0x0808192b2b2b1908ULL, 0x08082b0808080808ULL, 0x08082b0808081919ULL, + 0x08082b0808082b08ULL, 0x08082b0808191908ULL, 0x08082b08082b2b08ULL, 0x08082b0819080819ULL, + 0x08082b0819081908ULL, 0x08082b0819190808ULL, 0x08082b081919082bULL, 0x08082b082b082b08ULL, + 0x08082b1908081908ULL, 0x08082b1919080808ULL, 0x08082b2b0808082bULL, 0x08082b2b08191908ULL, + 0x0819080808080819ULL, 0x0819080808081908ULL, 0x0819080808190808ULL, 0x08190808082b0819ULL, + 0x0819080819080808ULL, 0x08190808192b0808ULL, 0x081908082b081908ULL, 0x081908082b190808ULL, + 0x081908082b191919ULL, 0x0819081908080808ULL, 0x0819081908082b08ULL, 0x08190819082b0808ULL, + 0x0819081919190808ULL, 0x0819081919192b2bULL, 0x081908192b080808ULL, 0x0819082b082b1908ULL, + 0x0819082b19081919ULL, 0x0819190808080808ULL, 0x0819190808082b08ULL, 0x08191908082b0808ULL, + 0x08191908082b1919ULL, 0x0819190819082b19ULL, 0x081919082b080808ULL, 0x0819191908192b08ULL, + 0x08191919192b082bULL, 0x0819192b08080808ULL, 0x0819192b0819192bULL, 0x08192b0808080819ULL, + 0x08192b0808081908ULL, 0x08192b0808190808ULL, 0x08192b0819080808ULL, 0x08192b082b080819ULL, + 0x08192b1908080808ULL, 0x08192b1908081919ULL, 0x08192b192b2b0808ULL, 0x08192b2b19190819ULL, + 0x082b080808080808ULL, 0x082b08080808082bULL, 0x082b080808082b2bULL, 0x082b080819081908ULL, + 0x082b0808192b0819ULL, 0x082b08082b080808ULL, 0x082b08082b08082bULL, 0x082b0819082b2b19ULL, + 0x082b081919082b08ULL, 0x082b082b08080808ULL, 0x082b082b0808082bULL, 0x082b190808080819ULL, + 0x082b190808081908ULL, 0x082b190808190808ULL, 0x082b190819080808ULL, 0x082b19081919192bULL, + 0x082b191908080808ULL, 0x082b191919080819ULL, 0x082b1919192b1908ULL, 0x082b192b2b190808ULL, + 0x082b2b0808082b08ULL, 0x082b2b08082b0808ULL, 0x082b2b082b191908ULL, 0x082b2b2b19081908ULL, + 0x1908080808080819ULL, 0x1908080808081908ULL, 0x1908080808190808ULL, 0x1908080808192b08ULL, + 0x19080808082b0819ULL, 0x19080808082b1908ULL, 0x1908080819080808ULL, 0x1908080819082b08ULL, + 0x190808081919192bULL, 0x19080808192b0808ULL, 0x190808082b080819ULL, 0x190808082b081908ULL, + 0x190808082b190808ULL, 0x1908081908080808ULL, 0x19080819082b0808ULL, 0x19080819192b0819ULL, + 0x190808192b080808ULL, 0x190808192b081919ULL, 0x1908082b08080819ULL, 0x1908082b08190808ULL, + 0x1908082b19082b08ULL, 0x1908082b1919192bULL, 0x1908082b192b2b08ULL, 0x1908190808080808ULL, + 0x1908190808082b08ULL, 0x19081908082b0808ULL, 0x190819082b080808ULL, 0x190819082b192b19ULL, + 0x190819190819082bULL, 0x19081919082b1908ULL, 0x1908192b08080808ULL, 0x19082b0808080819ULL, + 0x19082b0808081908ULL, 0x19082b0808190808ULL, 0x19082b0819080808ULL, 0x19082b0819081919ULL, + 0x19082b1908080808ULL, 0x19082b1919192b08ULL, 0x19082b19192b0819ULL, 0x19082b192b08082bULL, + 0x19082b2b19081919ULL, 0x19082b2b2b190808ULL, 0x1919080808080808ULL, 0x1919080808082b08ULL, + 0x1919080808190819ULL, 0x1919080808192b19ULL, 0x19190808082b0808ULL, 0x191908082b080808ULL, + 0x191908082b082b08ULL, 0x1919081908081908ULL, 0x191908191908082bULL, 0x191908192b2b1908ULL, + 0x1919082b2b190819ULL, 0x191919082b190808ULL, 0x191919082b19082bULL, 0x1919191908082b2bULL, + 0x1919192b08080819ULL, 0x1919192b19191908ULL, 0x19192b0808080808ULL, 0x19192b0808190819ULL, + 0x19192b0808192b19ULL, 0x19192b08192b1908ULL, 0x19192b1919080808ULL, 0x19192b2b08082b08ULL, + 0x192b080808081908ULL, 0x192b080808190808ULL, 0x192b080819080808ULL, 0x192b0808192b2b08ULL, + 0x192b081908080808ULL, 0x192b081919191919ULL, 0x192b082b08192b08ULL, 0x192b082b192b0808ULL, + 0x192b190808080808ULL, 0x192b190808081919ULL, 0x192b191908190808ULL, 0x192b19190819082bULL, + 0x192b19192b081908ULL, 0x192b2b081908082bULL, 0x2b08080808080808ULL, 0x2b0808080808082bULL, + 0x2b08080808082b2bULL, 0x2b08080819080819ULL, 0x2b0808082b08082bULL, 0x2b08081908081908ULL, + 0x2b08081908192b08ULL, 0x2b08081919080808ULL, 0x2b08082b08190819ULL, 0x2b08190808080819ULL, + 0x2b08190808081908ULL, 0x2b08190808190808ULL, 0x2b08190808191919ULL, 0x2b08190819080808ULL, + 0x2b081908192b0808ULL, 0x2b08191908080808ULL, 0x2b0819191908192bULL, 0x2b0819192b191908ULL, + 0x2b08192b08082b19ULL, 0x2b08192b19080808ULL, 0x2b08192b192b0808ULL, 0x2b082b080808082bULL, + 0x2b082b1908081908ULL, 0x2b082b2b08190819ULL, 0x2b19080808081908ULL, 0x2b19080808190808ULL, + 0x2b190808082b1908ULL, 0x2b19080819080808ULL, 0x2b1908082b2b0819ULL, 0x2b1908190819192bULL, + 0x2b1908192b080808ULL, 0x2b19082b19081919ULL, 0x2b19190808080808ULL, 0x2b191908082b082bULL, + 0x2b19190819081908ULL, 0x2b19191919190819ULL, 0x2b192b082b080819ULL, 0x2b192b19082b0808ULL, + 0x2b2b08080808082bULL, 0x2b2b080819190808ULL, 0x2b2b08082b081919ULL, 0x2b2b081908082b19ULL, + 0x2b2b082b08080808ULL, 0x2b2b190808192b08ULL, 0x2b2b2b0819190808ULL, 0x2b2b2b1908081908ULL, +}; + +// Stock ggml-common.h:1017, iq3xxs_grid. +__device__ __constant__ const uint32_t d_iq3xxs_grid[256] = { + 0x04040404, 0x04040414, 0x04040424, 0x04040c0c, 0x04040c1c, 0x04040c3e, 0x04041404, 0x04041414, + 0x04041c0c, 0x04042414, 0x04043e1c, 0x04043e2c, 0x040c040c, 0x040c041c, 0x040c0c04, 0x040c0c14, + 0x040c140c, 0x040c142c, 0x040c1c04, 0x040c1c14, 0x040c240c, 0x040c2c24, 0x040c3e04, 0x04140404, + 0x04140414, 0x04140424, 0x04140c0c, 0x04141404, 0x04141414, 0x04141c0c, 0x04141c1c, 0x04141c3e, + 0x04142c0c, 0x04142c3e, 0x04143e2c, 0x041c040c, 0x041c043e, 0x041c0c04, 0x041c0c14, 0x041c142c, + 0x041c3e04, 0x04240c1c, 0x04241c3e, 0x04242424, 0x04242c3e, 0x04243e1c, 0x04243e2c, 0x042c040c, + 0x042c043e, 0x042c1c14, 0x042c2c14, 0x04341c2c, 0x04343424, 0x043e0c04, 0x043e0c24, 0x043e0c34, + 0x043e241c, 0x043e340c, 0x0c04040c, 0x0c04041c, 0x0c040c04, 0x0c040c14, 0x0c04140c, 0x0c04141c, + 0x0c041c04, 0x0c041c14, 0x0c041c24, 0x0c04243e, 0x0c042c04, 0x0c0c0404, 0x0c0c0414, 0x0c0c0c0c, + 0x0c0c1404, 0x0c0c1414, 0x0c14040c, 0x0c14041c, 0x0c140c04, 0x0c140c14, 0x0c14140c, 0x0c141c04, + 0x0c143e14, 0x0c1c0404, 0x0c1c0414, 0x0c1c1404, 0x0c1c1c0c, 0x0c1c2434, 0x0c1c3434, 0x0c24040c, + 0x0c24042c, 0x0c242c04, 0x0c2c1404, 0x0c2c1424, 0x0c2c2434, 0x0c2c3e0c, 0x0c34042c, 0x0c3e1414, + 0x0c3e2404, 0x14040404, 0x14040414, 0x14040c0c, 0x14040c1c, 0x14041404, 0x14041414, 0x14041434, + 0x14041c0c, 0x14042414, 0x140c040c, 0x140c041c, 0x140c042c, 0x140c0c04, 0x140c0c14, 0x140c140c, + 0x140c1c04, 0x140c341c, 0x140c343e, 0x140c3e04, 0x14140404, 0x14140414, 0x14140c0c, 0x14140c3e, + 0x14141404, 0x14141414, 0x14141c3e, 0x14142404, 0x14142c2c, 0x141c040c, 0x141c0c04, 0x141c0c24, + 0x141c3e04, 0x141c3e24, 0x14241c2c, 0x14242c1c, 0x142c041c, 0x142c143e, 0x142c240c, 0x142c3e24, + 0x143e040c, 0x143e041c, 0x143e0c34, 0x143e242c, 0x1c04040c, 0x1c040c04, 0x1c040c14, 0x1c04140c, + 0x1c04141c, 0x1c042c04, 0x1c04342c, 0x1c043e14, 0x1c0c0404, 0x1c0c0414, 0x1c0c1404, 0x1c0c1c0c, + 0x1c0c2424, 0x1c0c2434, 0x1c14040c, 0x1c14041c, 0x1c140c04, 0x1c14142c, 0x1c142c14, 0x1c143e14, + 0x1c1c0c0c, 0x1c1c1c1c, 0x1c241c04, 0x1c24243e, 0x1c243e14, 0x1c2c0404, 0x1c2c0434, 0x1c2c1414, + 0x1c2c2c2c, 0x1c340c24, 0x1c341c34, 0x1c34341c, 0x1c3e1c1c, 0x1c3e3404, 0x24040424, 0x24040c3e, + 0x24041c2c, 0x24041c3e, 0x24042c1c, 0x24042c3e, 0x240c3e24, 0x24141404, 0x24141c3e, 0x24142404, + 0x24143404, 0x24143434, 0x241c043e, 0x241c242c, 0x24240424, 0x24242c0c, 0x24243424, 0x242c142c, + 0x242c241c, 0x242c3e04, 0x243e042c, 0x243e0c04, 0x243e0c14, 0x243e1c04, 0x2c040c14, 0x2c04240c, + 0x2c043e04, 0x2c0c0404, 0x2c0c0434, 0x2c0c1434, 0x2c0c2c2c, 0x2c140c24, 0x2c141c14, 0x2c143e14, + 0x2c1c0414, 0x2c1c2c1c, 0x2c240c04, 0x2c24141c, 0x2c24143e, 0x2c243e14, 0x2c2c0414, 0x2c2c1c0c, + 0x2c342c04, 0x2c3e1424, 0x2c3e2414, 0x34041424, 0x34042424, 0x34042434, 0x34043424, 0x340c140c, + 0x340c340c, 0x34140c3e, 0x34143424, 0x341c1c04, 0x341c1c34, 0x34242424, 0x342c042c, 0x342c2c14, + 0x34341c1c, 0x343e041c, 0x343e140c, 0x3e04041c, 0x3e04042c, 0x3e04043e, 0x3e040c04, 0x3e041c14, + 0x3e042c14, 0x3e0c1434, 0x3e0c2404, 0x3e140c14, 0x3e14242c, 0x3e142c14, 0x3e1c0404, 0x3e1c0c2c, + 0x3e1c1c1c, 0x3e1c3404, 0x3e24140c, 0x3e24240c, 0x3e2c0404, 0x3e2c0414, 0x3e2c1424, 0x3e341c04, +}; + +// Stock ggml-common.h:758, iq2s_grid. +__device__ const uint64_t d_iq2s_grid[1024] = { + 0x0808080808080808ULL, 0x080808080808082bULL, 0x0808080808081919ULL, 0x0808080808082b08ULL, + 0x0808080808082b2bULL, 0x0808080808190819ULL, 0x0808080808191908ULL, 0x080808080819192bULL, + 0x0808080808192b19ULL, 0x08080808082b0808ULL, 0x08080808082b082bULL, 0x08080808082b1919ULL, + 0x08080808082b2b08ULL, 0x0808080819080819ULL, 0x0808080819081908ULL, 0x080808081908192bULL, + 0x0808080819082b19ULL, 0x0808080819190808ULL, 0x080808081919082bULL, 0x0808080819191919ULL, + 0x0808080819192b08ULL, 0x08080808192b0819ULL, 0x08080808192b1908ULL, 0x08080808192b192bULL, + 0x08080808192b2b19ULL, 0x080808082b080808ULL, 0x080808082b08082bULL, 0x080808082b081919ULL, + 0x080808082b082b08ULL, 0x080808082b190819ULL, 0x080808082b191908ULL, 0x080808082b2b0808ULL, + 0x080808082b2b1919ULL, 0x080808082b2b2b2bULL, 0x0808081908080819ULL, 0x0808081908081908ULL, + 0x080808190808192bULL, 0x0808081908082b19ULL, 0x0808081908190808ULL, 0x080808190819082bULL, + 0x0808081908191919ULL, 0x0808081908192b08ULL, 0x08080819082b0819ULL, 0x08080819082b1908ULL, + 0x0808081919080808ULL, 0x080808191908082bULL, 0x0808081919081919ULL, 0x0808081919082b08ULL, + 0x0808081919190819ULL, 0x0808081919191908ULL, 0x080808191919192bULL, 0x0808081919192b19ULL, + 0x08080819192b0808ULL, 0x08080819192b1919ULL, 0x08080819192b2b08ULL, 0x080808192b080819ULL, + 0x080808192b081908ULL, 0x080808192b190808ULL, 0x080808192b19082bULL, 0x080808192b191919ULL, + 0x080808192b2b0819ULL, 0x080808192b2b1908ULL, 0x0808082b08080808ULL, 0x0808082b0808082bULL, + 0x0808082b08081919ULL, 0x0808082b08082b08ULL, 0x0808082b08190819ULL, 0x0808082b08191908ULL, + 0x0808082b082b0808ULL, 0x0808082b082b2b2bULL, 0x0808082b19080819ULL, 0x0808082b19081908ULL, + 0x0808082b1908192bULL, 0x0808082b19082b19ULL, 0x0808082b19190808ULL, 0x0808082b19191919ULL, + 0x0808082b2b080808ULL, 0x0808082b2b081919ULL, 0x0808082b2b082b2bULL, 0x0808082b2b191908ULL, + 0x0808082b2b2b082bULL, 0x0808190808080819ULL, 0x0808190808081908ULL, 0x080819080808192bULL, + 0x0808190808082b19ULL, 0x0808190808190808ULL, 0x080819080819082bULL, 0x0808190808191919ULL, + 0x0808190808192b08ULL, 0x08081908082b0819ULL, 0x08081908082b1908ULL, 0x08081908082b192bULL, + 0x08081908082b2b19ULL, 0x0808190819080808ULL, 0x080819081908082bULL, 0x0808190819081919ULL, + 0x0808190819082b08ULL, 0x0808190819082b2bULL, 0x0808190819190819ULL, 0x0808190819191908ULL, + 0x080819081919192bULL, 0x0808190819192b19ULL, 0x08081908192b0808ULL, 0x08081908192b082bULL, + 0x08081908192b1919ULL, 0x080819082b080819ULL, 0x080819082b081908ULL, 0x080819082b08192bULL, + 0x080819082b082b19ULL, 0x080819082b190808ULL, 0x080819082b191919ULL, 0x080819082b192b08ULL, + 0x080819082b2b0819ULL, 0x080819082b2b1908ULL, 0x0808191908080808ULL, 0x080819190808082bULL, + 0x0808191908081919ULL, 0x0808191908082b08ULL, 0x0808191908082b2bULL, 0x0808191908190819ULL, + 0x0808191908191908ULL, 0x080819190819192bULL, 0x0808191908192b19ULL, 0x08081919082b0808ULL, + 0x08081919082b1919ULL, 0x08081919082b2b08ULL, 0x0808191919080819ULL, 0x0808191919081908ULL, + 0x080819191908192bULL, 0x0808191919082b19ULL, 0x0808191919190808ULL, 0x080819191919082bULL, + 0x0808191919191919ULL, 0x0808191919192b08ULL, 0x08081919192b0819ULL, 0x08081919192b1908ULL, + 0x080819192b080808ULL, 0x080819192b08082bULL, 0x080819192b081919ULL, 0x080819192b082b08ULL, + 0x080819192b190819ULL, 0x080819192b191908ULL, 0x080819192b2b0808ULL, 0x0808192b08080819ULL, + 0x0808192b08081908ULL, 0x0808192b0808192bULL, 0x0808192b08082b19ULL, 0x0808192b08190808ULL, + 0x0808192b08191919ULL, 0x0808192b19080808ULL, 0x0808192b19081919ULL, 0x0808192b19082b08ULL, + 0x0808192b19190819ULL, 0x0808192b19191908ULL, 0x0808192b192b0808ULL, 0x0808192b2b080819ULL, + 0x0808192b2b081908ULL, 0x0808192b2b190808ULL, 0x08082b0808080808ULL, 0x08082b080808082bULL, + 0x08082b0808081919ULL, 0x08082b0808082b08ULL, 0x08082b0808190819ULL, 0x08082b0808191908ULL, + 0x08082b080819192bULL, 0x08082b0808192b19ULL, 0x08082b08082b0808ULL, 0x08082b08082b1919ULL, + 0x08082b08082b2b2bULL, 0x08082b0819080819ULL, 0x08082b0819081908ULL, 0x08082b081908192bULL, + 0x08082b0819082b19ULL, 0x08082b0819190808ULL, 0x08082b081919082bULL, 0x08082b0819191919ULL, + 0x08082b0819192b08ULL, 0x08082b08192b0819ULL, 0x08082b08192b1908ULL, 0x08082b082b080808ULL, + 0x08082b082b081919ULL, 0x08082b082b191908ULL, 0x08082b082b2b2b2bULL, 0x08082b1908080819ULL, + 0x08082b1908081908ULL, 0x08082b1908190808ULL, 0x08082b190819082bULL, 0x08082b1908191919ULL, + 0x08082b1908192b08ULL, 0x08082b19082b0819ULL, 0x08082b1919080808ULL, 0x08082b1919081919ULL, + 0x08082b1919082b08ULL, 0x08082b1919190819ULL, 0x08082b1919191908ULL, 0x08082b19192b0808ULL, + 0x08082b192b080819ULL, 0x08082b192b190808ULL, 0x08082b2b08080808ULL, 0x08082b2b08190819ULL, + 0x08082b2b08191908ULL, 0x08082b2b082b082bULL, 0x08082b2b082b2b08ULL, 0x08082b2b082b2b2bULL, + 0x08082b2b19190808ULL, 0x08082b2b2b192b19ULL, 0x0819080808080819ULL, 0x0819080808081908ULL, + 0x081908080808192bULL, 0x0819080808082b19ULL, 0x0819080808190808ULL, 0x081908080819082bULL, + 0x0819080808191919ULL, 0x0819080808192b08ULL, 0x08190808082b0819ULL, 0x08190808082b1908ULL, + 0x08190808082b192bULL, 0x0819080819080808ULL, 0x081908081908082bULL, 0x0819080819081919ULL, + 0x0819080819082b08ULL, 0x0819080819190819ULL, 0x0819080819191908ULL, 0x081908081919192bULL, + 0x0819080819192b19ULL, 0x08190808192b0808ULL, 0x08190808192b082bULL, 0x08190808192b1919ULL, + 0x08190808192b2b08ULL, 0x081908082b080819ULL, 0x081908082b081908ULL, 0x081908082b08192bULL, + 0x081908082b190808ULL, 0x081908082b191919ULL, 0x081908082b192b08ULL, 0x081908082b2b0819ULL, + 0x081908082b2b1908ULL, 0x0819081908080808ULL, 0x081908190808082bULL, 0x0819081908081919ULL, + 0x0819081908082b08ULL, 0x0819081908082b2bULL, 0x0819081908190819ULL, 0x0819081908191908ULL, + 0x081908190819192bULL, 0x0819081908192b19ULL, 0x08190819082b0808ULL, 0x08190819082b082bULL, + 0x08190819082b1919ULL, 0x08190819082b2b08ULL, 0x0819081919080819ULL, 0x0819081919081908ULL, + 0x081908191908192bULL, 0x0819081919082b19ULL, 0x0819081919190808ULL, 0x081908191919082bULL, + 0x0819081919191919ULL, 0x0819081919192b08ULL, 0x08190819192b0819ULL, 0x08190819192b1908ULL, + 0x081908192b080808ULL, 0x081908192b08082bULL, 0x081908192b081919ULL, 0x081908192b082b08ULL, + 0x081908192b190819ULL, 0x081908192b191908ULL, 0x0819082b08080819ULL, 0x0819082b08081908ULL, + 0x0819082b08082b19ULL, 0x0819082b08190808ULL, 0x0819082b08191919ULL, 0x0819082b082b0819ULL, + 0x0819082b082b1908ULL, 0x0819082b19080808ULL, 0x0819082b19081919ULL, 0x0819082b19190819ULL, + 0x0819082b19191908ULL, 0x0819082b2b080819ULL, 0x0819082b2b081908ULL, 0x0819082b2b190808ULL, + 0x0819190808080808ULL, 0x081919080808082bULL, 0x0819190808081919ULL, 0x0819190808082b08ULL, + 0x0819190808190819ULL, 0x0819190808191908ULL, 0x081919080819192bULL, 0x0819190808192b19ULL, + 0x08191908082b0808ULL, 0x08191908082b1919ULL, 0x08191908082b2b08ULL, 0x0819190819080819ULL, + 0x0819190819081908ULL, 0x081919081908192bULL, 0x0819190819082b19ULL, 0x0819190819190808ULL, + 0x081919081919082bULL, 0x0819190819191919ULL, 0x0819190819192b08ULL, 0x08191908192b0819ULL, + 0x08191908192b1908ULL, 0x081919082b080808ULL, 0x081919082b08082bULL, 0x081919082b081919ULL, + 0x081919082b082b08ULL, 0x081919082b190819ULL, 0x081919082b191908ULL, 0x081919082b2b0808ULL, + 0x0819191908080819ULL, 0x0819191908081908ULL, 0x081919190808192bULL, 0x0819191908082b19ULL, + 0x0819191908190808ULL, 0x081919190819082bULL, 0x0819191908191919ULL, 0x0819191908192b08ULL, + 0x08191919082b0819ULL, 0x08191919082b1908ULL, 0x0819191919080808ULL, 0x081919191908082bULL, + 0x0819191919081919ULL, 0x0819191919082b08ULL, 0x0819191919190819ULL, 0x0819191919191908ULL, + 0x08191919192b0808ULL, 0x081919192b080819ULL, 0x081919192b081908ULL, 0x081919192b190808ULL, + 0x0819192b08080808ULL, 0x0819192b08081919ULL, 0x0819192b08082b08ULL, 0x0819192b08190819ULL, + 0x0819192b08191908ULL, 0x0819192b082b0808ULL, 0x0819192b19080819ULL, 0x0819192b19081908ULL, + 0x0819192b19190808ULL, 0x0819192b2b080808ULL, 0x0819192b2b2b2b2bULL, 0x08192b0808080819ULL, + 0x08192b0808081908ULL, 0x08192b080808192bULL, 0x08192b0808082b19ULL, 0x08192b0808190808ULL, + 0x08192b0808191919ULL, 0x08192b0808192b08ULL, 0x08192b08082b0819ULL, 0x08192b0819080808ULL, + 0x08192b081908082bULL, 0x08192b0819081919ULL, 0x08192b0819082b08ULL, 0x08192b0819190819ULL, + 0x08192b0819191908ULL, 0x08192b08192b0808ULL, 0x08192b082b080819ULL, 0x08192b082b081908ULL, + 0x08192b1908080808ULL, 0x08192b190808082bULL, 0x08192b1908081919ULL, 0x08192b1908082b08ULL, + 0x08192b1908190819ULL, 0x08192b1908191908ULL, 0x08192b19082b0808ULL, 0x08192b1919080819ULL, + 0x08192b1919081908ULL, 0x08192b1919190808ULL, 0x08192b19192b2b19ULL, 0x08192b192b2b082bULL, + 0x08192b2b08081908ULL, 0x08192b2b08190808ULL, 0x08192b2b19080808ULL, 0x08192b2b1919192bULL, + 0x082b080808080808ULL, 0x082b08080808082bULL, 0x082b080808081919ULL, 0x082b080808082b08ULL, + 0x082b080808190819ULL, 0x082b080808191908ULL, 0x082b08080819192bULL, 0x082b080808192b19ULL, + 0x082b0808082b0808ULL, 0x082b0808082b1919ULL, 0x082b0808082b2b2bULL, 0x082b080819080819ULL, + 0x082b080819081908ULL, 0x082b080819190808ULL, 0x082b08081919082bULL, 0x082b080819191919ULL, + 0x082b0808192b1908ULL, 0x082b08082b080808ULL, 0x082b08082b082b2bULL, 0x082b08082b191908ULL, + 0x082b08082b2b2b2bULL, 0x082b081908080819ULL, 0x082b081908081908ULL, 0x082b081908190808ULL, + 0x082b08190819082bULL, 0x082b081908191919ULL, 0x082b0819082b0819ULL, 0x082b081919080808ULL, + 0x082b08191908082bULL, 0x082b081919081919ULL, 0x082b081919190819ULL, 0x082b081919191908ULL, + 0x082b0819192b0808ULL, 0x082b08192b080819ULL, 0x082b08192b081908ULL, 0x082b08192b190808ULL, + 0x082b082b08080808ULL, 0x082b082b08082b2bULL, 0x082b082b082b082bULL, 0x082b082b082b2b08ULL, + 0x082b082b082b2b2bULL, 0x082b082b19081908ULL, 0x082b082b19190808ULL, 0x082b082b2b082b08ULL, + 0x082b082b2b082b2bULL, 0x082b082b2b2b2b08ULL, 0x082b190808080819ULL, 0x082b190808081908ULL, + 0x082b19080808192bULL, 0x082b190808082b19ULL, 0x082b190808190808ULL, 0x082b190808191919ULL, + 0x082b190808192b08ULL, 0x082b1908082b0819ULL, 0x082b1908082b1908ULL, 0x082b190819080808ULL, + 0x082b19081908082bULL, 0x082b190819081919ULL, 0x082b190819082b08ULL, 0x082b190819190819ULL, + 0x082b190819191908ULL, 0x082b1908192b0808ULL, 0x082b19082b080819ULL, 0x082b19082b081908ULL, + 0x082b19082b190808ULL, 0x082b191908080808ULL, 0x082b191908081919ULL, 0x082b191908082b08ULL, + 0x082b191908190819ULL, 0x082b191908191908ULL, 0x082b1919082b0808ULL, 0x082b191919080819ULL, + 0x082b191919081908ULL, 0x082b191919190808ULL, 0x082b1919192b192bULL, 0x082b19192b080808ULL, + 0x082b192b08080819ULL, 0x082b192b08081908ULL, 0x082b192b08190808ULL, 0x082b192b19080808ULL, + 0x082b192b19192b19ULL, 0x082b2b0808080808ULL, 0x082b2b0808081919ULL, 0x082b2b0808190819ULL, + 0x082b2b0808191908ULL, 0x082b2b0819080819ULL, 0x082b2b0819081908ULL, 0x082b2b0819190808ULL, + 0x082b2b082b082b2bULL, 0x082b2b082b2b2b2bULL, 0x082b2b1908080819ULL, 0x082b2b1908081908ULL, + 0x082b2b1908190808ULL, 0x082b2b192b191919ULL, 0x082b2b2b08082b2bULL, 0x082b2b2b082b082bULL, + 0x082b2b2b192b1908ULL, 0x082b2b2b2b082b08ULL, 0x082b2b2b2b082b2bULL, 0x1908080808080819ULL, + 0x1908080808081908ULL, 0x190808080808192bULL, 0x1908080808082b19ULL, 0x1908080808190808ULL, + 0x190808080819082bULL, 0x1908080808191919ULL, 0x1908080808192b08ULL, 0x1908080808192b2bULL, + 0x19080808082b0819ULL, 0x19080808082b1908ULL, 0x19080808082b192bULL, 0x1908080819080808ULL, + 0x190808081908082bULL, 0x1908080819081919ULL, 0x1908080819082b08ULL, 0x1908080819082b2bULL, + 0x1908080819190819ULL, 0x1908080819191908ULL, 0x190808081919192bULL, 0x1908080819192b19ULL, + 0x19080808192b0808ULL, 0x19080808192b082bULL, 0x19080808192b1919ULL, 0x190808082b080819ULL, + 0x190808082b081908ULL, 0x190808082b190808ULL, 0x190808082b191919ULL, 0x190808082b192b08ULL, + 0x190808082b2b0819ULL, 0x190808082b2b1908ULL, 0x1908081908080808ULL, 0x190808190808082bULL, + 0x1908081908081919ULL, 0x1908081908082b08ULL, 0x1908081908190819ULL, 0x1908081908191908ULL, + 0x190808190819192bULL, 0x1908081908192b19ULL, 0x19080819082b0808ULL, 0x19080819082b082bULL, + 0x19080819082b1919ULL, 0x1908081919080819ULL, 0x1908081919081908ULL, 0x190808191908192bULL, + 0x1908081919082b19ULL, 0x1908081919190808ULL, 0x190808191919082bULL, 0x1908081919191919ULL, + 0x1908081919192b08ULL, 0x19080819192b0819ULL, 0x19080819192b1908ULL, 0x190808192b080808ULL, + 0x190808192b08082bULL, 0x190808192b081919ULL, 0x190808192b082b08ULL, 0x190808192b190819ULL, + 0x190808192b191908ULL, 0x190808192b2b0808ULL, 0x1908082b08080819ULL, 0x1908082b08081908ULL, + 0x1908082b08190808ULL, 0x1908082b0819082bULL, 0x1908082b08191919ULL, 0x1908082b08192b08ULL, + 0x1908082b082b1908ULL, 0x1908082b19080808ULL, 0x1908082b19081919ULL, 0x1908082b19082b08ULL, + 0x1908082b19190819ULL, 0x1908082b19191908ULL, 0x1908082b192b0808ULL, 0x1908082b2b080819ULL, + 0x1908082b2b081908ULL, 0x1908190808080808ULL, 0x190819080808082bULL, 0x1908190808081919ULL, + 0x1908190808082b08ULL, 0x1908190808082b2bULL, 0x1908190808190819ULL, 0x1908190808191908ULL, + 0x190819080819192bULL, 0x1908190808192b19ULL, 0x19081908082b0808ULL, 0x19081908082b082bULL, + 0x19081908082b1919ULL, 0x19081908082b2b08ULL, 0x1908190819080819ULL, 0x1908190819081908ULL, + 0x190819081908192bULL, 0x1908190819082b19ULL, 0x1908190819190808ULL, 0x190819081919082bULL, + 0x1908190819191919ULL, 0x1908190819192b08ULL, 0x19081908192b0819ULL, 0x19081908192b1908ULL, + 0x190819082b080808ULL, 0x190819082b08082bULL, 0x190819082b081919ULL, 0x190819082b082b08ULL, + 0x190819082b190819ULL, 0x190819082b191908ULL, 0x190819082b2b0808ULL, 0x1908191908080819ULL, + 0x1908191908081908ULL, 0x190819190808192bULL, 0x1908191908082b19ULL, 0x1908191908190808ULL, + 0x190819190819082bULL, 0x1908191908191919ULL, 0x1908191908192b08ULL, 0x19081919082b0819ULL, + 0x19081919082b1908ULL, 0x1908191919080808ULL, 0x190819191908082bULL, 0x1908191919081919ULL, + 0x1908191919082b08ULL, 0x1908191919190819ULL, 0x1908191919191908ULL, 0x19081919192b0808ULL, + 0x19081919192b2b2bULL, 0x190819192b080819ULL, 0x190819192b081908ULL, 0x190819192b190808ULL, + 0x1908192b08080808ULL, 0x1908192b0808082bULL, 0x1908192b08081919ULL, 0x1908192b08082b08ULL, + 0x1908192b08190819ULL, 0x1908192b08191908ULL, 0x1908192b082b0808ULL, 0x1908192b19080819ULL, + 0x1908192b19081908ULL, 0x1908192b19190808ULL, 0x1908192b2b080808ULL, 0x1908192b2b2b1919ULL, + 0x19082b0808080819ULL, 0x19082b0808081908ULL, 0x19082b0808082b19ULL, 0x19082b0808190808ULL, + 0x19082b080819082bULL, 0x19082b0808191919ULL, 0x19082b0808192b08ULL, 0x19082b08082b0819ULL, + 0x19082b08082b1908ULL, 0x19082b0819080808ULL, 0x19082b081908082bULL, 0x19082b0819081919ULL, + 0x19082b0819082b08ULL, 0x19082b0819190819ULL, 0x19082b0819191908ULL, 0x19082b08192b0808ULL, + 0x19082b082b081908ULL, 0x19082b082b190808ULL, 0x19082b1908080808ULL, 0x19082b190808082bULL, + 0x19082b1908081919ULL, 0x19082b1908082b08ULL, 0x19082b1908190819ULL, 0x19082b1908191908ULL, + 0x19082b19082b0808ULL, 0x19082b1919080819ULL, 0x19082b1919081908ULL, 0x19082b1919190808ULL, + 0x19082b192b080808ULL, 0x19082b192b19192bULL, 0x19082b2b08080819ULL, 0x19082b2b08081908ULL, + 0x19082b2b08190808ULL, 0x19082b2b19080808ULL, 0x1919080808080808ULL, 0x191908080808082bULL, + 0x1919080808081919ULL, 0x1919080808082b08ULL, 0x1919080808190819ULL, 0x1919080808191908ULL, + 0x191908080819192bULL, 0x1919080808192b19ULL, 0x19190808082b0808ULL, 0x19190808082b082bULL, + 0x19190808082b1919ULL, 0x19190808082b2b08ULL, 0x1919080819080819ULL, 0x1919080819081908ULL, + 0x191908081908192bULL, 0x1919080819082b19ULL, 0x1919080819190808ULL, 0x191908081919082bULL, + 0x1919080819191919ULL, 0x1919080819192b08ULL, 0x19190808192b0819ULL, 0x19190808192b1908ULL, + 0x191908082b080808ULL, 0x191908082b08082bULL, 0x191908082b081919ULL, 0x191908082b082b08ULL, + 0x191908082b190819ULL, 0x191908082b191908ULL, 0x1919081908080819ULL, 0x1919081908081908ULL, + 0x191908190808192bULL, 0x1919081908082b19ULL, 0x1919081908190808ULL, 0x191908190819082bULL, + 0x1919081908191919ULL, 0x1919081908192b08ULL, 0x19190819082b0819ULL, 0x19190819082b1908ULL, + 0x1919081919080808ULL, 0x191908191908082bULL, 0x1919081919081919ULL, 0x1919081919082b08ULL, + 0x1919081919190819ULL, 0x1919081919191908ULL, 0x19190819192b0808ULL, 0x191908192b080819ULL, + 0x191908192b081908ULL, 0x191908192b190808ULL, 0x1919082b08080808ULL, 0x1919082b08081919ULL, + 0x1919082b08082b08ULL, 0x1919082b08190819ULL, 0x1919082b08191908ULL, 0x1919082b082b0808ULL, + 0x1919082b19080819ULL, 0x1919082b19081908ULL, 0x1919082b19190808ULL, 0x1919082b192b2b19ULL, + 0x1919082b2b080808ULL, 0x1919190808080819ULL, 0x1919190808081908ULL, 0x191919080808192bULL, + 0x1919190808082b19ULL, 0x1919190808190808ULL, 0x191919080819082bULL, 0x1919190808191919ULL, + 0x1919190808192b08ULL, 0x19191908082b0819ULL, 0x19191908082b1908ULL, 0x1919190819080808ULL, + 0x191919081908082bULL, 0x1919190819081919ULL, 0x1919190819082b08ULL, 0x1919190819190819ULL, + 0x1919190819191908ULL, 0x19191908192b0808ULL, 0x191919082b080819ULL, 0x191919082b081908ULL, + 0x191919082b190808ULL, 0x1919191908080808ULL, 0x191919190808082bULL, 0x1919191908081919ULL, + 0x1919191908082b08ULL, 0x1919191908190819ULL, 0x1919191908191908ULL, 0x19191919082b0808ULL, + 0x1919191919080819ULL, 0x1919191919081908ULL, 0x1919191919190808ULL, 0x191919192b080808ULL, + 0x1919192b08080819ULL, 0x1919192b08081908ULL, 0x1919192b08190808ULL, 0x1919192b082b192bULL, + 0x1919192b19080808ULL, 0x19192b0808080808ULL, 0x19192b080808082bULL, 0x19192b0808081919ULL, + 0x19192b0808082b08ULL, 0x19192b0808190819ULL, 0x19192b0808191908ULL, 0x19192b08082b0808ULL, + 0x19192b0819080819ULL, 0x19192b0819081908ULL, 0x19192b0819190808ULL, 0x19192b0819192b2bULL, + 0x19192b082b080808ULL, 0x19192b1908080819ULL, 0x19192b1908081908ULL, 0x19192b1908190808ULL, + 0x19192b1919080808ULL, 0x19192b2b08080808ULL, 0x19192b2b08192b19ULL, 0x19192b2b2b081919ULL, + 0x19192b2b2b2b2b08ULL, 0x192b080808080819ULL, 0x192b080808081908ULL, 0x192b08080808192bULL, + 0x192b080808190808ULL, 0x192b08080819082bULL, 0x192b080808191919ULL, 0x192b080808192b08ULL, + 0x192b0808082b0819ULL, 0x192b0808082b1908ULL, 0x192b080819080808ULL, 0x192b080819081919ULL, + 0x192b080819082b08ULL, 0x192b080819190819ULL, 0x192b080819191908ULL, 0x192b0808192b0808ULL, + 0x192b08082b081908ULL, 0x192b08082b190808ULL, 0x192b081908080808ULL, 0x192b08190808082bULL, + 0x192b081908081919ULL, 0x192b081908082b08ULL, 0x192b081908190819ULL, 0x192b081908191908ULL, + 0x192b0819082b0808ULL, 0x192b081919080819ULL, 0x192b081919081908ULL, 0x192b081919190808ULL, + 0x192b08192b080808ULL, 0x192b08192b192b19ULL, 0x192b082b08081908ULL, 0x192b082b08190808ULL, + 0x192b082b19080808ULL, 0x192b082b1919192bULL, 0x192b082b2b2b0819ULL, 0x192b190808080808ULL, + 0x192b190808081919ULL, 0x192b190808082b08ULL, 0x192b190808190819ULL, 0x192b190808191908ULL, + 0x192b1908082b0808ULL, 0x192b190819080819ULL, 0x192b190819081908ULL, 0x192b190819190808ULL, + 0x192b19082b080808ULL, 0x192b191908080819ULL, 0x192b191908081908ULL, 0x192b191908190808ULL, + 0x192b191919080808ULL, 0x192b191919082b2bULL, 0x192b1919192b2b08ULL, 0x192b19192b19082bULL, + 0x192b192b08080808ULL, 0x192b192b2b191908ULL, 0x192b2b0808080819ULL, 0x192b2b0808081908ULL, + 0x192b2b0808190808ULL, 0x192b2b08192b1919ULL, 0x192b2b082b192b08ULL, 0x192b2b1908080808ULL, + 0x192b2b19082b2b2bULL, 0x192b2b2b1908082bULL, 0x192b2b2b2b2b0819ULL, 0x2b08080808080808ULL, + 0x2b0808080808082bULL, 0x2b08080808081919ULL, 0x2b08080808082b08ULL, 0x2b08080808190819ULL, + 0x2b08080808191908ULL, 0x2b08080808192b19ULL, 0x2b080808082b0808ULL, 0x2b080808082b1919ULL, + 0x2b08080819080819ULL, 0x2b08080819081908ULL, 0x2b08080819190808ULL, 0x2b0808081919082bULL, + 0x2b08080819191919ULL, 0x2b08080819192b08ULL, 0x2b080808192b0819ULL, 0x2b0808082b080808ULL, + 0x2b0808082b081919ULL, 0x2b0808082b190819ULL, 0x2b0808082b191908ULL, 0x2b08081908080819ULL, + 0x2b08081908081908ULL, 0x2b08081908082b19ULL, 0x2b08081908190808ULL, 0x2b0808190819082bULL, + 0x2b08081908191919ULL, 0x2b08081908192b08ULL, 0x2b080819082b0819ULL, 0x2b080819082b1908ULL, + 0x2b08081919080808ULL, 0x2b0808191908082bULL, 0x2b08081919081919ULL, 0x2b08081919082b08ULL, + 0x2b08081919190819ULL, 0x2b08081919191908ULL, 0x2b0808192b080819ULL, 0x2b0808192b081908ULL, + 0x2b0808192b190808ULL, 0x2b0808192b2b2b19ULL, 0x2b08082b08080808ULL, 0x2b08082b08081919ULL, + 0x2b08082b08082b2bULL, 0x2b08082b08190819ULL, 0x2b08082b08191908ULL, 0x2b08082b19080819ULL, + 0x2b08082b19081908ULL, 0x2b08082b19190808ULL, 0x2b08190808080819ULL, 0x2b08190808081908ULL, + 0x2b0819080808192bULL, 0x2b08190808082b19ULL, 0x2b08190808190808ULL, 0x2b0819080819082bULL, + 0x2b08190808191919ULL, 0x2b08190808192b08ULL, 0x2b081908082b0819ULL, 0x2b08190819080808ULL, + 0x2b0819081908082bULL, 0x2b08190819081919ULL, 0x2b08190819082b08ULL, 0x2b08190819190819ULL, + 0x2b08190819191908ULL, 0x2b081908192b0808ULL, 0x2b0819082b080819ULL, 0x2b0819082b081908ULL, + 0x2b0819082b190808ULL, 0x2b08191908080808ULL, 0x2b0819190808082bULL, 0x2b08191908081919ULL, + 0x2b08191908082b08ULL, 0x2b08191908190819ULL, 0x2b08191908191908ULL, 0x2b081919082b0808ULL, + 0x2b08191919080819ULL, 0x2b08191919081908ULL, 0x2b08191919190808ULL, 0x2b0819192b080808ULL, + 0x2b0819192b082b2bULL, 0x2b08192b08080819ULL, 0x2b08192b08081908ULL, 0x2b08192b08190808ULL, + 0x2b08192b082b2b19ULL, 0x2b08192b19080808ULL, 0x2b082b0808080808ULL, 0x2b082b0808081919ULL, + 0x2b082b0808190819ULL, 0x2b082b0808191908ULL, 0x2b082b0819080819ULL, 0x2b082b0819081908ULL, + 0x2b082b0819190808ULL, 0x2b082b082b2b082bULL, 0x2b082b1908080819ULL, 0x2b082b1908081908ULL, + 0x2b082b1919080808ULL, 0x2b082b19192b1919ULL, 0x2b082b2b082b082bULL, 0x2b082b2b19192b08ULL, + 0x2b082b2b19192b2bULL, 0x2b082b2b2b08082bULL, 0x2b082b2b2b2b082bULL, 0x2b19080808080819ULL, + 0x2b19080808081908ULL, 0x2b19080808082b19ULL, 0x2b19080808190808ULL, 0x2b1908080819082bULL, + 0x2b19080808191919ULL, 0x2b19080808192b08ULL, 0x2b190808082b1908ULL, 0x2b19080819080808ULL, + 0x2b1908081908082bULL, 0x2b19080819081919ULL, 0x2b19080819082b08ULL, 0x2b19080819190819ULL, + 0x2b19080819191908ULL, 0x2b190808192b0808ULL, 0x2b1908082b080819ULL, 0x2b1908082b081908ULL, + 0x2b1908082b190808ULL, 0x2b19081908080808ULL, 0x2b19081908081919ULL, 0x2b19081908190819ULL, + 0x2b19081908191908ULL, 0x2b19081919080819ULL, 0x2b19081919081908ULL, 0x2b19081919190808ULL, + 0x2b19081919192b2bULL, 0x2b19082b08080819ULL, 0x2b19082b08081908ULL, 0x2b19082b08190808ULL, + 0x2b19082b19080808ULL, 0x2b19082b2b2b192bULL, 0x2b19190808080808ULL, 0x2b1919080808082bULL, + 0x2b19190808081919ULL, 0x2b19190808082b08ULL, 0x2b19190808190819ULL, 0x2b19190808191908ULL, + 0x2b191908082b0808ULL, 0x2b19190819080819ULL, 0x2b19190819081908ULL, 0x2b19190819190808ULL, + 0x2b1919082b080808ULL, 0x2b1919082b19192bULL, 0x2b19191908080819ULL, 0x2b19191908081908ULL, + 0x2b19191908190808ULL, 0x2b19191919080808ULL, 0x2b1919192b192b08ULL, 0x2b1919192b2b0819ULL, + 0x2b19192b08080808ULL, 0x2b19192b1908192bULL, 0x2b19192b192b1908ULL, 0x2b192b0808080819ULL, + 0x2b192b0808081908ULL, 0x2b192b0808190808ULL, 0x2b192b08082b192bULL, 0x2b192b0819080808ULL, + 0x2b192b082b2b2b19ULL, 0x2b192b1908080808ULL, 0x2b192b1919082b19ULL, 0x2b192b191919082bULL, + 0x2b192b2b2b190808ULL, 0x2b2b080808080808ULL, 0x2b2b080808081919ULL, 0x2b2b080808082b2bULL, + 0x2b2b080808191908ULL, 0x2b2b0808082b082bULL, 0x2b2b0808082b2b2bULL, 0x2b2b080819080819ULL, + 0x2b2b080819081908ULL, 0x2b2b080819190808ULL, 0x2b2b08082b2b082bULL, 0x2b2b08082b2b2b2bULL, + 0x2b2b081919080808ULL, 0x2b2b0819192b1919ULL, 0x2b2b082b0808082bULL, 0x2b2b082b08082b2bULL, + 0x2b2b082b082b082bULL, 0x2b2b082b082b2b08ULL, 0x2b2b082b082b2b2bULL, 0x2b2b082b2b08082bULL, + 0x2b2b082b2b082b08ULL, 0x2b2b082b2b082b2bULL, 0x2b2b082b2b2b2b08ULL, 0x2b2b190808080819ULL, + 0x2b2b190808081908ULL, 0x2b2b190808190808ULL, 0x2b2b190819080808ULL, 0x2b2b19082b082b19ULL, + 0x2b2b19082b2b1908ULL, 0x2b2b191908080808ULL, 0x2b2b191908192b19ULL, 0x2b2b192b19190819ULL, + 0x2b2b2b0808082b2bULL, 0x2b2b2b08082b2b08ULL, 0x2b2b2b082b2b082bULL, 0x2b2b2b1919191908ULL, + 0x2b2b2b192b08192bULL, 0x2b2b2b2b08082b08ULL, 0x2b2b2b2b08082b2bULL, 0x2b2b2b2b082b0808ULL, + 0x2b2b2b2b082b082bULL, 0x2b2b2b2b082b2b08ULL, 0x2b2b2b2b2b082b08ULL, 0x2b2b2b2b2b2b2b2bULL, +}; + +// Stock ggml-common.h:1052, iq3s_grid. +__device__ const uint32_t d_iq3s_grid[512] = { + 0x01010101u, 0x01010103u, 0x01010105u, 0x0101010bu, 0x0101010fu, 0x01010301u, + 0x01010303u, 0x01010305u, 0x01010309u, 0x0101030du, 0x01010501u, 0x01010503u, + 0x0101050bu, 0x01010707u, 0x01010901u, 0x01010905u, 0x0101090bu, 0x0101090fu, + 0x01010b03u, 0x01010b07u, 0x01010d01u, 0x01010d05u, 0x01010f03u, 0x01010f09u, + 0x01010f0fu, 0x01030101u, 0x01030103u, 0x01030105u, 0x01030109u, 0x01030301u, + 0x01030303u, 0x0103030bu, 0x01030501u, 0x01030507u, 0x0103050fu, 0x01030703u, + 0x0103070bu, 0x01030909u, 0x01030d03u, 0x01030d0bu, 0x01030f05u, 0x01050101u, + 0x01050103u, 0x0105010bu, 0x0105010fu, 0x01050301u, 0x01050307u, 0x0105030du, + 0x01050503u, 0x0105050bu, 0x01050701u, 0x01050709u, 0x01050905u, 0x0105090bu, + 0x0105090fu, 0x01050b03u, 0x01050b07u, 0x01050f01u, 0x01050f07u, 0x01070107u, + 0x01070303u, 0x0107030bu, 0x01070501u, 0x01070505u, 0x01070703u, 0x01070707u, + 0x0107070du, 0x01070909u, 0x01070b01u, 0x01070b05u, 0x01070d0fu, 0x01070f03u, + 0x01070f0bu, 0x01090101u, 0x01090307u, 0x0109030fu, 0x01090503u, 0x01090509u, + 0x01090705u, 0x01090901u, 0x01090907u, 0x01090b03u, 0x01090f01u, 0x010b0105u, + 0x010b0109u, 0x010b0501u, 0x010b0505u, 0x010b050du, 0x010b0707u, 0x010b0903u, + 0x010b090bu, 0x010b090fu, 0x010b0d0du, 0x010b0f07u, 0x010d010du, 0x010d0303u, + 0x010d0307u, 0x010d0703u, 0x010d0b05u, 0x010d0f03u, 0x010f0101u, 0x010f0105u, + 0x010f0109u, 0x010f0501u, 0x010f0505u, 0x010f050du, 0x010f0707u, 0x010f0b01u, + 0x010f0b09u, 0x03010101u, 0x03010103u, 0x03010105u, 0x03010109u, 0x03010301u, + 0x03010303u, 0x03010307u, 0x0301030bu, 0x0301030fu, 0x03010501u, 0x03010505u, + 0x03010703u, 0x03010709u, 0x0301070du, 0x03010b09u, 0x03010b0du, 0x03010d03u, + 0x03010f05u, 0x03030101u, 0x03030103u, 0x03030107u, 0x0303010du, 0x03030301u, + 0x03030309u, 0x03030503u, 0x03030701u, 0x03030707u, 0x03030903u, 0x03030b01u, + 0x03030b05u, 0x03030f01u, 0x03030f0du, 0x03050101u, 0x03050305u, 0x0305030bu, + 0x0305030fu, 0x03050501u, 0x03050509u, 0x03050705u, 0x03050901u, 0x03050907u, + 0x03050b0bu, 0x03050d01u, 0x03050f05u, 0x03070103u, 0x03070109u, 0x0307010fu, + 0x03070301u, 0x03070307u, 0x03070503u, 0x0307050fu, 0x03070701u, 0x03070709u, + 0x03070903u, 0x03070d05u, 0x03070f01u, 0x03090107u, 0x0309010bu, 0x03090305u, + 0x03090309u, 0x03090703u, 0x03090707u, 0x03090905u, 0x0309090du, 0x03090b01u, + 0x03090b09u, 0x030b0103u, 0x030b0301u, 0x030b0307u, 0x030b0503u, 0x030b0701u, + 0x030b0705u, 0x030b0b03u, 0x030d0501u, 0x030d0509u, 0x030d050fu, 0x030d0909u, + 0x030d090du, 0x030f0103u, 0x030f0107u, 0x030f0301u, 0x030f0305u, 0x030f0503u, + 0x030f070bu, 0x030f0903u, 0x030f0d05u, 0x030f0f01u, 0x05010101u, 0x05010103u, + 0x05010107u, 0x0501010bu, 0x0501010fu, 0x05010301u, 0x05010305u, 0x05010309u, + 0x0501030du, 0x05010503u, 0x05010507u, 0x0501050fu, 0x05010701u, 0x05010705u, + 0x05010903u, 0x05010907u, 0x0501090bu, 0x05010b01u, 0x05010b05u, 0x05010d0fu, + 0x05010f01u, 0x05010f07u, 0x05010f0bu, 0x05030101u, 0x05030105u, 0x05030301u, + 0x05030307u, 0x0503030fu, 0x05030505u, 0x0503050bu, 0x05030703u, 0x05030709u, + 0x05030905u, 0x05030b03u, 0x05050103u, 0x05050109u, 0x0505010fu, 0x05050503u, + 0x05050507u, 0x05050701u, 0x0505070fu, 0x05050903u, 0x05050b07u, 0x05050b0fu, + 0x05050f03u, 0x05050f09u, 0x05070101u, 0x05070105u, 0x0507010bu, 0x05070303u, + 0x05070505u, 0x05070509u, 0x05070703u, 0x05070707u, 0x05070905u, 0x05070b01u, + 0x05070d0du, 0x05090103u, 0x0509010fu, 0x05090501u, 0x05090507u, 0x05090705u, + 0x0509070bu, 0x05090903u, 0x05090f05u, 0x05090f0bu, 0x050b0109u, 0x050b0303u, + 0x050b0505u, 0x050b070fu, 0x050b0901u, 0x050b0b07u, 0x050b0f01u, 0x050d0101u, + 0x050d0105u, 0x050d010fu, 0x050d0503u, 0x050d0b0bu, 0x050d0d03u, 0x050f010bu, + 0x050f0303u, 0x050f050du, 0x050f0701u, 0x050f0907u, 0x050f0b01u, 0x07010105u, + 0x07010303u, 0x07010307u, 0x0701030bu, 0x0701030fu, 0x07010505u, 0x07010703u, + 0x07010707u, 0x0701070bu, 0x07010905u, 0x07010909u, 0x0701090fu, 0x07010b03u, + 0x07010d07u, 0x07010f03u, 0x07030103u, 0x07030107u, 0x0703010bu, 0x07030309u, + 0x07030503u, 0x07030507u, 0x07030901u, 0x07030d01u, 0x07030f05u, 0x07030f0du, + 0x07050101u, 0x07050305u, 0x07050501u, 0x07050705u, 0x07050709u, 0x07050b01u, + 0x07070103u, 0x07070301u, 0x07070309u, 0x07070503u, 0x07070507u, 0x0707050fu, + 0x07070701u, 0x07070903u, 0x07070907u, 0x0707090fu, 0x07070b0bu, 0x07070f07u, + 0x07090107u, 0x07090303u, 0x0709030du, 0x07090505u, 0x07090703u, 0x07090b05u, + 0x07090d01u, 0x07090d09u, 0x070b0103u, 0x070b0301u, 0x070b0305u, 0x070b050bu, + 0x070b0705u, 0x070b0909u, 0x070b0b0du, 0x070b0f07u, 0x070d030du, 0x070d0903u, + 0x070f0103u, 0x070f0107u, 0x070f0501u, 0x070f0505u, 0x070f070bu, 0x09010101u, + 0x09010109u, 0x09010305u, 0x09010501u, 0x09010509u, 0x0901050fu, 0x09010705u, + 0x09010903u, 0x09010b01u, 0x09010f01u, 0x09030105u, 0x0903010fu, 0x09030303u, + 0x09030307u, 0x09030505u, 0x09030701u, 0x0903070bu, 0x09030907u, 0x09030b03u, + 0x09030b0bu, 0x09050103u, 0x09050107u, 0x09050301u, 0x0905030bu, 0x09050503u, + 0x09050707u, 0x09050901u, 0x09050b0fu, 0x09050d05u, 0x09050f01u, 0x09070109u, + 0x09070303u, 0x09070307u, 0x09070501u, 0x09070505u, 0x09070703u, 0x0907070bu, + 0x09090101u, 0x09090105u, 0x09090509u, 0x0909070fu, 0x09090901u, 0x09090f03u, + 0x090b010bu, 0x090b010fu, 0x090b0503u, 0x090b0d05u, 0x090d0307u, 0x090d0709u, + 0x090d0d01u, 0x090f0301u, 0x090f030bu, 0x090f0701u, 0x090f0907u, 0x090f0b03u, + 0x0b010105u, 0x0b010301u, 0x0b010309u, 0x0b010505u, 0x0b010901u, 0x0b010909u, + 0x0b01090fu, 0x0b010b05u, 0x0b010d0du, 0x0b010f09u, 0x0b030103u, 0x0b030107u, + 0x0b03010bu, 0x0b030305u, 0x0b030503u, 0x0b030705u, 0x0b030f05u, 0x0b050101u, + 0x0b050303u, 0x0b050507u, 0x0b050701u, 0x0b05070du, 0x0b050b07u, 0x0b070105u, + 0x0b07010fu, 0x0b070301u, 0x0b07050fu, 0x0b070909u, 0x0b070b03u, 0x0b070d0bu, + 0x0b070f07u, 0x0b090103u, 0x0b090109u, 0x0b090501u, 0x0b090705u, 0x0b09090du, + 0x0b0b0305u, 0x0b0b050du, 0x0b0b0b03u, 0x0b0b0b07u, 0x0b0d0905u, 0x0b0f0105u, + 0x0b0f0109u, 0x0b0f0505u, 0x0d010303u, 0x0d010307u, 0x0d01030bu, 0x0d010703u, + 0x0d010707u, 0x0d010d01u, 0x0d030101u, 0x0d030501u, 0x0d03050fu, 0x0d030d09u, + 0x0d050305u, 0x0d050709u, 0x0d050905u, 0x0d050b0bu, 0x0d050d05u, 0x0d050f01u, + 0x0d070101u, 0x0d070309u, 0x0d070503u, 0x0d070901u, 0x0d09050bu, 0x0d090907u, + 0x0d090d05u, 0x0d0b0101u, 0x0d0b0107u, 0x0d0b0709u, 0x0d0b0d01u, 0x0d0d010bu, + 0x0d0d0901u, 0x0d0f0303u, 0x0d0f0307u, 0x0f010101u, 0x0f010109u, 0x0f01010fu, + 0x0f010501u, 0x0f010505u, 0x0f01070du, 0x0f010901u, 0x0f010b09u, 0x0f010d05u, + 0x0f030105u, 0x0f030303u, 0x0f030509u, 0x0f030907u, 0x0f03090bu, 0x0f050103u, + 0x0f050109u, 0x0f050301u, 0x0f05030du, 0x0f050503u, 0x0f050701u, 0x0f050b03u, + 0x0f070105u, 0x0f070705u, 0x0f07070bu, 0x0f070b07u, 0x0f090103u, 0x0f09010bu, + 0x0f090307u, 0x0f090501u, 0x0f090b01u, 0x0f0b0505u, 0x0f0b0905u, 0x0f0d0105u, + 0x0f0d0703u, 0x0f0f0101u, +}; + +// Stock ggml-common.h:627, iq2xs_grid. +__device__ const uint64_t d_iq2xs_grid[512] = { + 0x0808080808080808ULL, 0x080808080808082bULL, 0x0808080808081919ULL, 0x0808080808082b08ULL, + 0x0808080808082b2bULL, 0x0808080808190819ULL, 0x0808080808191908ULL, 0x080808080819192bULL, + 0x0808080808192b19ULL, 0x08080808082b0808ULL, 0x08080808082b082bULL, 0x08080808082b1919ULL, + 0x08080808082b2b08ULL, 0x0808080819080819ULL, 0x0808080819081908ULL, 0x080808081908192bULL, + 0x0808080819082b19ULL, 0x0808080819190808ULL, 0x080808081919082bULL, 0x0808080819191919ULL, + 0x0808080819192b08ULL, 0x08080808192b0819ULL, 0x08080808192b1908ULL, 0x080808082b080808ULL, + 0x080808082b08082bULL, 0x080808082b081919ULL, 0x080808082b082b08ULL, 0x080808082b190819ULL, + 0x080808082b191908ULL, 0x080808082b192b19ULL, 0x080808082b2b0808ULL, 0x0808081908080819ULL, + 0x0808081908081908ULL, 0x080808190808192bULL, 0x0808081908082b19ULL, 0x0808081908190808ULL, + 0x080808190819082bULL, 0x0808081908191919ULL, 0x0808081908192b08ULL, 0x0808081908192b2bULL, + 0x08080819082b0819ULL, 0x08080819082b1908ULL, 0x0808081919080808ULL, 0x080808191908082bULL, + 0x0808081919081919ULL, 0x0808081919082b08ULL, 0x0808081919190819ULL, 0x0808081919191908ULL, + 0x08080819192b0808ULL, 0x08080819192b2b08ULL, 0x080808192b080819ULL, 0x080808192b081908ULL, + 0x080808192b190808ULL, 0x0808082b08080808ULL, 0x0808082b0808082bULL, 0x0808082b08081919ULL, + 0x0808082b08082b08ULL, 0x0808082b08190819ULL, 0x0808082b08191908ULL, 0x0808082b082b0808ULL, + 0x0808082b19080819ULL, 0x0808082b19081908ULL, 0x0808082b19190808ULL, 0x0808082b19191919ULL, + 0x0808082b2b080808ULL, 0x0808082b2b082b2bULL, 0x0808190808080819ULL, 0x0808190808081908ULL, + 0x080819080808192bULL, 0x0808190808082b19ULL, 0x0808190808190808ULL, 0x080819080819082bULL, + 0x0808190808191919ULL, 0x0808190808192b08ULL, 0x08081908082b0819ULL, 0x08081908082b1908ULL, + 0x0808190819080808ULL, 0x080819081908082bULL, 0x0808190819081919ULL, 0x0808190819082b08ULL, + 0x0808190819190819ULL, 0x0808190819191908ULL, 0x080819081919192bULL, 0x08081908192b0808ULL, + 0x080819082b080819ULL, 0x080819082b081908ULL, 0x080819082b190808ULL, 0x0808191908080808ULL, + 0x080819190808082bULL, 0x0808191908081919ULL, 0x0808191908082b08ULL, 0x0808191908190819ULL, + 0x0808191908191908ULL, 0x08081919082b0808ULL, 0x0808191919080819ULL, 0x0808191919081908ULL, + 0x0808191919190808ULL, 0x08081919192b0819ULL, 0x080819192b080808ULL, 0x0808192b08080819ULL, + 0x0808192b08081908ULL, 0x0808192b08190808ULL, 0x0808192b082b192bULL, 0x0808192b19080808ULL, + 0x0808192b1908082bULL, 0x0808192b2b081908ULL, 0x08082b0808080808ULL, 0x08082b080808082bULL, + 0x08082b0808081919ULL, 0x08082b0808082b08ULL, 0x08082b0808082b2bULL, 0x08082b0808190819ULL, + 0x08082b0808191908ULL, 0x08082b08082b0808ULL, 0x08082b08082b1919ULL, 0x08082b0819080819ULL, + 0x08082b0819081908ULL, 0x08082b0819190808ULL, 0x08082b0819192b08ULL, 0x08082b082b080808ULL, + 0x08082b082b2b0808ULL, 0x08082b082b2b2b2bULL, 0x08082b1908080819ULL, 0x08082b1908081908ULL, + 0x08082b1908190808ULL, 0x08082b1919080808ULL, 0x08082b192b080819ULL, 0x08082b192b082b19ULL, + 0x08082b2b08080808ULL, 0x08082b2b082b0808ULL, 0x08082b2b082b2b08ULL, 0x08082b2b2b19192bULL, + 0x08082b2b2b2b0808ULL, 0x0819080808080819ULL, 0x0819080808081908ULL, 0x081908080808192bULL, + 0x0819080808082b19ULL, 0x0819080808190808ULL, 0x081908080819082bULL, 0x0819080808191919ULL, + 0x0819080808192b08ULL, 0x08190808082b0819ULL, 0x08190808082b1908ULL, 0x0819080819080808ULL, + 0x081908081908082bULL, 0x0819080819081919ULL, 0x0819080819082b08ULL, 0x0819080819190819ULL, + 0x0819080819191908ULL, 0x08190808192b0808ULL, 0x08190808192b2b2bULL, 0x081908082b080819ULL, + 0x081908082b081908ULL, 0x081908082b190808ULL, 0x0819081908080808ULL, 0x081908190808082bULL, + 0x0819081908081919ULL, 0x0819081908082b08ULL, 0x0819081908190819ULL, 0x0819081908191908ULL, + 0x08190819082b0808ULL, 0x0819081919080819ULL, 0x0819081919081908ULL, 0x0819081919190808ULL, + 0x081908192b080808ULL, 0x081908192b191908ULL, 0x081908192b19192bULL, 0x0819082b08080819ULL, + 0x0819082b08081908ULL, 0x0819082b0808192bULL, 0x0819082b08190808ULL, 0x0819082b19080808ULL, + 0x0819082b192b0808ULL, 0x0819190808080808ULL, 0x081919080808082bULL, 0x0819190808081919ULL, + 0x0819190808082b08ULL, 0x0819190808190819ULL, 0x0819190808191908ULL, 0x08191908082b0808ULL, + 0x0819190819080819ULL, 0x0819190819081908ULL, 0x0819190819082b19ULL, 0x0819190819190808ULL, + 0x08191908192b1908ULL, 0x081919082b080808ULL, 0x0819191908080819ULL, 0x0819191908081908ULL, + 0x0819191908190808ULL, 0x0819191919080808ULL, 0x0819192b08080808ULL, 0x0819192b08191908ULL, + 0x0819192b19082b19ULL, 0x08192b0808080819ULL, 0x08192b0808081908ULL, 0x08192b0808190808ULL, + 0x08192b080819082bULL, 0x08192b0819080808ULL, 0x08192b0819191908ULL, 0x08192b082b08192bULL, + 0x08192b1908080808ULL, 0x08192b1908081919ULL, 0x08192b19192b192bULL, 0x08192b2b19190819ULL, + 0x08192b2b2b2b2b19ULL, 0x082b080808080808ULL, 0x082b08080808082bULL, 0x082b080808081919ULL, + 0x082b080808082b08ULL, 0x082b080808082b2bULL, 0x082b080808190819ULL, 0x082b080808191908ULL, + 0x082b0808082b0808ULL, 0x082b080819080819ULL, 0x082b080819081908ULL, 0x082b080819190808ULL, + 0x082b08082b080808ULL, 0x082b08082b2b0808ULL, 0x082b081908080819ULL, 0x082b081908081908ULL, + 0x082b081908190808ULL, 0x082b081919080808ULL, 0x082b081919082b08ULL, 0x082b0819192b1919ULL, + 0x082b082b08080808ULL, 0x082b082b082b082bULL, 0x082b082b2b080808ULL, 0x082b082b2b2b2b08ULL, + 0x082b190808080819ULL, 0x082b190808081908ULL, 0x082b190808190808ULL, 0x082b1908082b2b19ULL, + 0x082b190819080808ULL, 0x082b191908080808ULL, 0x082b191919080819ULL, 0x082b19191919082bULL, + 0x082b19192b192b19ULL, 0x082b192b08080819ULL, 0x082b192b08192b2bULL, 0x082b192b2b2b192bULL, + 0x082b2b0808080808ULL, 0x082b2b0808082b08ULL, 0x082b2b0808082b2bULL, 0x082b2b08082b0808ULL, + 0x082b2b0819191919ULL, 0x082b2b082b082b08ULL, 0x082b2b082b2b082bULL, 0x082b2b19192b2b08ULL, + 0x082b2b192b190808ULL, 0x082b2b2b08082b08ULL, 0x082b2b2b082b0808ULL, 0x082b2b2b2b08082bULL, + 0x082b2b2b2b082b08ULL, 0x082b2b2b2b082b2bULL, 0x1908080808080819ULL, 0x1908080808081908ULL, + 0x190808080808192bULL, 0x1908080808082b19ULL, 0x1908080808190808ULL, 0x190808080819082bULL, + 0x1908080808191919ULL, 0x1908080808192b08ULL, 0x19080808082b0819ULL, 0x19080808082b1908ULL, + 0x1908080819080808ULL, 0x190808081908082bULL, 0x1908080819081919ULL, 0x1908080819082b08ULL, + 0x1908080819082b2bULL, 0x1908080819190819ULL, 0x1908080819191908ULL, 0x19080808192b0808ULL, + 0x19080808192b1919ULL, 0x190808082b080819ULL, 0x190808082b081908ULL, 0x190808082b190808ULL, + 0x1908081908080808ULL, 0x190808190808082bULL, 0x1908081908081919ULL, 0x1908081908082b08ULL, + 0x1908081908190819ULL, 0x1908081908191908ULL, 0x19080819082b0808ULL, 0x1908081919080819ULL, + 0x1908081919081908ULL, 0x1908081919190808ULL, 0x190808192b080808ULL, 0x190808192b081919ULL, + 0x190808192b2b082bULL, 0x1908082b08080819ULL, 0x1908082b08081908ULL, 0x1908082b08190808ULL, + 0x1908082b0819082bULL, 0x1908082b082b2b19ULL, 0x1908082b19080808ULL, 0x1908190808080808ULL, + 0x190819080808082bULL, 0x1908190808081919ULL, 0x1908190808082b08ULL, 0x1908190808190819ULL, + 0x1908190808191908ULL, 0x1908190808192b19ULL, 0x19081908082b0808ULL, 0x1908190819080819ULL, + 0x1908190819081908ULL, 0x1908190819190808ULL, 0x190819082b080808ULL, 0x190819082b191908ULL, + 0x1908191908080819ULL, 0x1908191908081908ULL, 0x1908191908190808ULL, 0x19081919082b1908ULL, + 0x1908191919080808ULL, 0x190819192b192b2bULL, 0x1908192b08080808ULL, 0x1908192b08082b2bULL, + 0x1908192b19081908ULL, 0x1908192b19190808ULL, 0x19082b0808080819ULL, 0x19082b0808081908ULL, + 0x19082b0808190808ULL, 0x19082b0819080808ULL, 0x19082b0819081919ULL, 0x19082b0819191908ULL, + 0x19082b08192b082bULL, 0x19082b1908080808ULL, 0x19082b1908190819ULL, 0x19082b1919081908ULL, + 0x19082b1919190808ULL, 0x19082b19192b2b19ULL, 0x19082b2b08081908ULL, 0x1919080808080808ULL, + 0x191908080808082bULL, 0x1919080808081919ULL, 0x1919080808082b08ULL, 0x1919080808190819ULL, + 0x1919080808191908ULL, 0x19190808082b0808ULL, 0x19190808082b2b08ULL, 0x1919080819080819ULL, + 0x1919080819081908ULL, 0x1919080819190808ULL, 0x191908082b080808ULL, 0x1919081908080819ULL, + 0x1919081908081908ULL, 0x1919081908190808ULL, 0x1919081908191919ULL, 0x1919081919080808ULL, + 0x191908191908082bULL, 0x1919082b08080808ULL, 0x1919082b19081908ULL, 0x1919082b2b2b2b2bULL, + 0x1919190808080819ULL, 0x1919190808081908ULL, 0x1919190808190808ULL, 0x19191908082b0819ULL, + 0x1919190819080808ULL, 0x19191908192b0808ULL, 0x191919082b080819ULL, 0x191919082b2b0819ULL, + 0x1919191908080808ULL, 0x1919191908082b08ULL, 0x191919192b080808ULL, 0x191919192b082b08ULL, + 0x1919192b082b0819ULL, 0x1919192b192b2b08ULL, 0x1919192b2b2b0819ULL, 0x19192b0808080808ULL, + 0x19192b0808191908ULL, 0x19192b0819080819ULL, 0x19192b0819190808ULL, 0x19192b082b192b19ULL, + 0x19192b1908192b2bULL, 0x19192b1919080808ULL, 0x19192b191908082bULL, 0x19192b2b2b081919ULL, + 0x192b080808080819ULL, 0x192b080808081908ULL, 0x192b080808190808ULL, 0x192b080819080808ULL, + 0x192b080819191908ULL, 0x192b0808192b082bULL, 0x192b08082b08192bULL, 0x192b08082b2b2b19ULL, + 0x192b081908080808ULL, 0x192b082b082b1908ULL, 0x192b082b19082b2bULL, 0x192b082b2b19082bULL, + 0x192b190808080808ULL, 0x192b19080819192bULL, 0x192b191908190808ULL, 0x192b191919080808ULL, + 0x192b191919081919ULL, 0x192b19192b2b1908ULL, 0x192b2b0808080819ULL, 0x192b2b08192b2b2bULL, + 0x192b2b19082b1919ULL, 0x192b2b2b0808192bULL, 0x192b2b2b19191908ULL, 0x192b2b2b192b082bULL, + 0x2b08080808080808ULL, 0x2b0808080808082bULL, 0x2b08080808081919ULL, 0x2b08080808082b08ULL, + 0x2b08080808190819ULL, 0x2b08080808191908ULL, 0x2b080808082b0808ULL, 0x2b080808082b2b2bULL, + 0x2b08080819080819ULL, 0x2b08080819081908ULL, 0x2b08080819190808ULL, 0x2b0808082b080808ULL, + 0x2b0808082b08082bULL, 0x2b0808082b2b2b08ULL, 0x2b0808082b2b2b2bULL, 0x2b08081908080819ULL, + 0x2b08081908081908ULL, 0x2b0808190808192bULL, 0x2b08081908190808ULL, 0x2b08081919080808ULL, + 0x2b08081919190819ULL, 0x2b08081919192b19ULL, 0x2b08082b08080808ULL, 0x2b08082b082b0808ULL, + 0x2b08082b2b080808ULL, 0x2b08082b2b08082bULL, 0x2b08082b2b2b0808ULL, 0x2b08082b2b2b2b08ULL, + 0x2b08190808080819ULL, 0x2b08190808081908ULL, 0x2b08190808190808ULL, 0x2b0819080819082bULL, + 0x2b08190808191919ULL, 0x2b08190819080808ULL, 0x2b081908192b0808ULL, 0x2b0819082b082b19ULL, + 0x2b08191908080808ULL, 0x2b08191919081908ULL, 0x2b0819192b2b1919ULL, 0x2b08192b08192b08ULL, + 0x2b08192b192b2b2bULL, 0x2b082b0808080808ULL, 0x2b082b0808082b08ULL, 0x2b082b08082b1919ULL, + 0x2b082b0819192b2bULL, 0x2b082b082b080808ULL, 0x2b082b082b08082bULL, 0x2b082b082b2b2b08ULL, + 0x2b082b190808192bULL, 0x2b082b2b082b082bULL, 0x2b082b2b2b080808ULL, 0x2b082b2b2b082b08ULL, + 0x2b082b2b2b19192bULL, 0x2b082b2b2b2b2b08ULL, 0x2b19080808080819ULL, 0x2b19080808081908ULL, + 0x2b19080808190808ULL, 0x2b19080819080808ULL, 0x2b1908081919192bULL, 0x2b1908082b081908ULL, + 0x2b19081908080808ULL, 0x2b190819082b082bULL, 0x2b190819192b1908ULL, 0x2b19082b1919192bULL, + 0x2b19082b2b082b19ULL, 0x2b19190808080808ULL, 0x2b19190808081919ULL, 0x2b19190819081908ULL, + 0x2b19190819190808ULL, 0x2b19190819192b08ULL, 0x2b191919082b2b19ULL, 0x2b1919192b190808ULL, + 0x2b1919192b19082bULL, 0x2b19192b19080819ULL, 0x2b192b0819190819ULL, 0x2b192b082b2b192bULL, + 0x2b192b1919082b19ULL, 0x2b192b2b08191919ULL, 0x2b192b2b192b0808ULL, 0x2b2b080808080808ULL, + 0x2b2b08080808082bULL, 0x2b2b080808082b08ULL, 0x2b2b080808082b2bULL, 0x2b2b0808082b0808ULL, + 0x2b2b0808082b2b2bULL, 0x2b2b08082b2b0808ULL, 0x2b2b081919190819ULL, 0x2b2b081919192b19ULL, + 0x2b2b08192b2b192bULL, 0x2b2b082b08080808ULL, 0x2b2b082b0808082bULL, 0x2b2b082b08082b08ULL, + 0x2b2b082b082b2b2bULL, 0x2b2b082b2b080808ULL, 0x2b2b082b2b2b0808ULL, 0x2b2b190819080808ULL, + 0x2b2b19082b191919ULL, 0x2b2b192b192b1919ULL, 0x2b2b192b2b192b08ULL, 0x2b2b2b0808082b2bULL, + 0x2b2b2b08082b0808ULL, 0x2b2b2b08082b082bULL, 0x2b2b2b08082b2b08ULL, 0x2b2b2b082b2b0808ULL, + 0x2b2b2b082b2b2b08ULL, 0x2b2b2b1908081908ULL, 0x2b2b2b192b081908ULL, 0x2b2b2b192b08192bULL, + 0x2b2b2b2b082b2b08ULL, 0x2b2b2b2b082b2b2bULL, 0x2b2b2b2b2b190819ULL, 0x2b2b2b2b2b2b2b2bULL, +}; + +// Stock ggml-common.h:1120, kvalues_iq4nl. +__device__ __constant__ const int8_t d_kvalues_iq4nl[16] = { + -127, -104, -83, -65, -49, -35, -22, -10, 1, 13, 25, 38, 53, 69, 89, 113, +}; + +// Stock ggml-common.h:1125, kvalues_mxfp4, stores twice the E2M1 value. +__device__ __constant__ const int8_t d_kvalues_mxfp4[16] = { + 0, 1, 2, 3, 4, 6, 8, 12, 0, -1, -2, -3, -4, -6, -8, -12, +}; + +} // namespace +} // namespace vt::rocm diff --git a/tests/CMakeLists.txt b/tests/CMakeLists.txt index c452837784..9e24215043 100644 --- a/tests/CMakeLists.txt +++ b/tests/CMakeLists.txt @@ -2894,6 +2894,19 @@ target_include_directories(test_cuda_quant_dot PRIVATE ${CMAKE_SOURCE_DIR}/src) # header and the vt/cpu quant headers. vllm_cpp_add_test(test_cuda_embedding_quant vt/test_cuda_embedding_quant.cpp) target_include_directories(test_cuda_embedding_quant PRIVATE ${CMAKE_SOURCE_DIR}/src) +# BACKEND-ROCM-QUANT-GATHER: the production gate starts at the public loader. +vllm_cpp_add_test(test_capi_rocm_embedding_quant capi/test_rocm_embedding_quant.cpp) +vllm_cpp_add_test(test_rocm_embedding_quant vt/test_rocm_embedding_quant.cpp) +target_include_directories(test_rocm_embedding_quant PRIVATE ${CMAKE_SOURCE_DIR}/src) +if(VLLM_CPP_HIP) + set_source_files_properties(vt/test_rocm_embedding_quant.cpp PROPERTIES LANGUAGE HIP) + if(VLLM_CPP_HIP_ARCHITECTURES) + set_property(TARGET test_rocm_embedding_quant PROPERTY HIP_ARCHITECTURES "${VLLM_CPP_HIP_ARCHITECTURES}") + endif() +endif() +add_executable(rocm_quant_gather_capture ../tools/rocm_quant_gather/capture.cpp) +target_link_libraries(rocm_quant_gather_capture PRIVATE vllm) +target_include_directories(rocm_quant_gather_capture PRIVATE ${CMAKE_SOURCE_DIR}/src ${CMAKE_SOURCE_DIR}/tests) # CPU-ELEM-GEMM: the specialized + SIMD elementwise (bf16/f16/f32) GEMM. The # gate is BYTE-IDENTITY against an independent scalar reference across every # dtype combination, both weight orientations, ragged K/N, row-strided diff --git a/tests/capi/rocm_quant_gather_fixture.h b/tests/capi/rocm_quant_gather_fixture.h new file mode 100644 index 0000000000..defabe86ab --- /dev/null +++ b/tests/capi/rocm_quant_gather_fixture.h @@ -0,0 +1,202 @@ +// Deterministic public-loader fixture for BACKEND-ROCM-QUANT-GATHER (#3093). +// The GGUF metadata follows the pinned llama.cpp qwen35 loader. This fixture +// is a synthetic test vehicle, not a qualification of a published checkpoint. +#pragma once + +#include +#include +#include +#include +#include +#include +#include + +#include "vllm/gguf_builder.h" +#include "vt/dtype.h" +#include "vt/quant.h" + +namespace rocm_gather_test { + +inline constexpr uint32_t kSeed = 0x524f434d; +inline constexpr int64_t kHidden = 256; +inline constexpr int64_t kVocab = 128; + +struct Format { + vt::DType dtype; + uint32_t ggml; + const char* name; +}; + +inline constexpr Format kFormats[] = { + {vt::DType::kQ4_0, 2, "Q4_0"}, + {vt::DType::kQ5_0, 6, "Q5_0"}, + {vt::DType::kQ8_0, 8, "Q8_0"}, + {vt::DType::kQ2_K, 10, "Q2_K"}, + {vt::DType::kQ3_K, 11, "Q3_K"}, + {vt::DType::kQ4_K, 12, "Q4_K"}, + {vt::DType::kQ5_K, 13, "Q5_K"}, + {vt::DType::kQ6_K, 14, "Q6_K"}, + {vt::DType::kQ8_K, 15, "Q8_K"}, + {vt::DType::kIQ2_XXS, 16, "IQ2_XXS"}, + {vt::DType::kIQ2_XS, 17, "IQ2_XS"}, + {vt::DType::kIQ3_XXS, 18, "IQ3_XXS"}, + {vt::DType::kIQ1_S, 19, "IQ1_S"}, + {vt::DType::kIQ4_NL, 20, "IQ4_NL"}, + {vt::DType::kIQ3_S, 21, "IQ3_S"}, + {vt::DType::kIQ2_S, 22, "IQ2_S"}, + {vt::DType::kIQ4_XS, 23, "IQ4_XS"}, + {vt::DType::kMXFP4, 39, "MXFP4"}, + {vt::DType::kIQ1_XXXS, 66, "IQ1_XXXS"}, +}; + +inline void PutHalf(std::string& b, size_t offset, uint16_t bits) { + b[offset] = static_cast(bits & 255); + b[offset + 1] = static_cast(bits >> 8); +} + +// Every block consumes exactly BlockBytes outputs of mt19937. Its raw payload +// uses each output's low byte. Scale fields are then set to finite powers of +// two, independently of the decoded values. Projection RNG is separate, so +// changing the embedding format cannot change a dense projection. +inline std::string PackedTable(const Format& format, int64_t rows = kVocab, + int64_t width = kHidden) { + const auto dtype = format.dtype; + const int64_t block_bytes = vt::BlockBytes(dtype); + const int64_t blocks = rows * width / vt::BlockElems(dtype); + std::string packed(static_cast(blocks * block_bytes), '\0'); + std::mt19937 rng(kSeed); + for (int64_t block = 0; block < blocks; ++block) { + const size_t base = static_cast(block * block_bytes); + for (int64_t byte = 0; byte < block_bytes; ++byte) { + packed[base + static_cast(byte)] = static_cast(rng() & 255); + } + switch (dtype) { + case vt::DType::kQ2_K: + PutHalf(packed, base + 80, 0x1800); // d = 1/512 + PutHalf(packed, base + 82, 0x1800); + break; + case vt::DType::kQ3_K: + PutHalf(packed, base + 108, 0x1800); + break; + case vt::DType::kQ4_K: + case vt::DType::kQ5_K: + PutHalf(packed, base, 0x1400); // d = 1/1024 + PutHalf(packed, base + 2, 0x1400); + break; + case vt::DType::kQ6_K: + PutHalf(packed, base + 208, 0x1000); // d = 1/2048 + break; + case vt::DType::kQ8_K: { + const std::string scale = gguf_test::U32Le(0x3b000000); // 1/512 + packed.replace(base, 4, scale); + break; + } + case vt::DType::kMXFP4: + packed[base] = static_cast(119); // e8m0 scale = 1/256 + break; + default: + PutHalf(packed, base, 0x2000); // d = 1/128 + break; + } + } + return packed; +} + +inline std::string DenseEmbedding(const Format& format, + const std::string& packed) { + std::vector decoded(static_cast(kVocab * kHidden)); + const auto decode = vt::cpu::BlockToFloat(format.dtype); + if (decode == nullptr) throw std::runtime_error("fixture decoder absent"); + decode(packed.data(), decoded.data(), kVocab * kHidden); + std::string dense; + dense.reserve(decoded.size() * 2); + for (float value : decoded) { + if (!std::isfinite(value)) throw std::runtime_error("nonfinite fixture block"); + const uint16_t word = vt::F32ToBF16(value); + dense.push_back(static_cast(word & 255)); + dense.push_back(static_cast(word >> 8)); + } + return dense; +} + +inline std::string BuildModel(const Format& format, bool dense_embedding) { + using namespace gguf_test; + GgufModelBuilder builder; + builder.AddKv(StrKv("general.architecture", "qwen35")); + builder.AddKv(StrKv("general.name", "rocm-quant-gather-synthetic")); + builder.AddKv(U32Kv("general.quantization_version", 2)); + builder.AddKv(U32Kv("qwen35.embedding_length", 256)); + builder.AddKv(U32Kv("qwen35.block_count", 1)); + builder.AddKv(U32Kv("qwen35.attention.head_count", 4)); + builder.AddKv(U32Kv("qwen35.attention.head_count_kv", 1)); + builder.AddKv(U32Kv("qwen35.attention.key_length", 64)); + builder.AddKv(U32Kv("qwen35.attention.value_length", 64)); + builder.AddKv(U32Kv("qwen35.feed_forward_length", 256)); + builder.AddKv(U32Kv("qwen35.full_attention_interval", 1)); + builder.AddKv(U32Kv("qwen35.context_length", 64)); + builder.AddKv(U32Kv("qwen35.vocab_size", 128)); + builder.AddKv(F32Kv("qwen35.attention.layer_norm_rms_epsilon", 1e-6F)); + builder.AddKv(F32Kv("qwen35.rope.freq_base", 1000000.0F)); + builder.AddKv(U32Kv("qwen35.rope.dimension_count", 64)); + // Stock qwen35.cpp::load_arch_hparams requires these even when every layer + // is full attention. The SSM geometry stays inactive: no SSM tensor exists. + builder.AddKv(I32ArrayKv("qwen35.rope.dimension_sections", {16, 8, 8, 0})); + builder.AddKv(U32Kv("qwen35.ssm.conv_kernel", 4)); + builder.AddKv(U32Kv("qwen35.ssm.inner_size", 64)); + builder.AddKv(U32Kv("qwen35.ssm.state_size", 64)); + builder.AddKv(U32Kv("qwen35.ssm.time_step_rank", 1)); + builder.AddKv(U32Kv("qwen35.ssm.group_count", 1)); + builder.AddKv(StrKv("tokenizer.ggml.model", "gpt2")); + builder.AddKv(StrKv("tokenizer.ggml.pre", "qwen2")); + builder.AddKv(U32Kv("tokenizer.ggml.unknown_token_id", 0)); + builder.AddKv(U32Kv("tokenizer.ggml.bos_token_id", 1)); + builder.AddKv(U32Kv("tokenizer.ggml.eos_token_id", 2)); + builder.AddKv(BoolKv("tokenizer.ggml.add_bos_token", false)); + std::vector vocabulary = {"", "", ""}; + for (int id = 3; id < kVocab; ++id) vocabulary.push_back("token" + std::to_string(id)); + builder.AddKv(StrArrayKv("tokenizer.ggml.tokens", vocabulary)); + builder.AddKv(StrArrayKv("tokenizer.ggml.merges", {})); + std::vector types(128, 1); + types[0] = 2; + types[1] = types[2] = 3; + builder.AddKv(I32ArrayKv("tokenizer.ggml.token_type", types)); + + const std::string packed = PackedTable(format); + builder.AddTensor("token_embd.weight", {256, 128}, + dense_embedding ? 30 : format.ggml, + dense_embedding ? DenseEmbedding(format, packed) : packed); + std::mt19937 rng(kSeed); + const auto projection = [&](const char* name, uint64_t input, uint64_t output) { + std::string bytes; + bytes.reserve(static_cast(input * output * 2)); + for (uint64_t i = 0; i < input * output; ++i) { + const int32_t integer = static_cast(rng() & 65535) - 32768; + const float value = static_cast(integer) / 2097152.0F; + const uint16_t word = vt::F32ToBF16(value); + bytes.push_back(static_cast(word & 255)); + bytes.push_back(static_cast(word >> 8)); + } + builder.AddTensor(name, {input, output}, 30, bytes); + }; + const auto norm = [&](const char* name, uint64_t width) { + std::string bytes; + for (uint64_t i = 0; i < width; ++i) bytes += U32Le(0x3f800000); + builder.AddTensor(name, {width}, 0, bytes); + }; + projection("output.weight", 256, 128); + projection("blk.0.attn_q.weight", 256, 512); + projection("blk.0.attn_k.weight", 256, 64); + projection("blk.0.attn_v.weight", 256, 64); + projection("blk.0.attn_output.weight", 256, 256); + projection("blk.0.ffn_gate.weight", 256, 256); + projection("blk.0.ffn_up.weight", 256, 256); + projection("blk.0.ffn_down.weight", 256, 256); + norm("output_norm.weight", 256); + norm("blk.0.attn_norm.weight", 256); + norm("blk.0.post_attention_norm.weight", 256); + norm("blk.0.attn_q_norm.weight", 64); + norm("blk.0.attn_k_norm.weight", 64); + return builder.Build(); +} + +} // namespace rocm_gather_test diff --git a/tests/capi/test_rocm_embedding_quant.cpp b/tests/capi/test_rocm_embedding_quant.cpp new file mode 100644 index 0000000000..85b83907c8 --- /dev/null +++ b/tests/capi/test_rocm_embedding_quant.cpp @@ -0,0 +1,193 @@ +// Public reachability gate for BACKEND-ROCM-QUANT-GATHER (#3093). +// Pinned plugin d4c1f0d: vocal_embeds.py::_apply_gguf_embedding, lines 79-98. +// Oracle model comparisons use the exact generated GGUFs exported by this test. +#define DOCTEST_CONFIG_IMPLEMENT +#include + +#include +#include +#include +#include +#include +#include +#include +#include + +#include "capi/rocm_quant_gather_fixture.h" +#include "vllm.h" +#include "vt/backend.h" +#include "vt/op_provider.h" +#include "vt/ops.h" + +namespace { + +struct Completion { + std::vector ids; + std::vector logits; + bool capture_overflow = false; + bool nonfinite_logits = false; + int callbacks = 0; +}; + +// This C callback cannot throw. Reserve the complete capture before entering +// the public API, then append within that bound. +void Capture(const int32_t*, int32_t, float* logits, int32_t vocab, void* data) { + auto& result = *static_cast(data); + if (vocab != 128 || result.callbacks >= 4) { + result.capture_overflow = true; + return; + } + for (int32_t i = 0; i < vocab; ++i) { + if (!std::isfinite(logits[i])) result.nonfinite_logits = true; + result.logits.push_back(logits[i]); + } + ++result.callbacks; +} + +Completion Run(const std::string& path, const std::vector& prompt, + bool expect_quant) { + auto model = vllm_model_params_default(); + model.model_path = path.c_str(); + model.device = 0; // Public AUTO in a HIP-only build, measured by provider stats. + model.block_size = 16; + model.num_blocks = 16; + model.max_model_len = 64; + model.max_num_seqs = 1; + model.kv_cache_dtype = "auto"; + vllm_engine* raw = nullptr; + const vllm_status loaded = vllm_engine_load(&model, &raw); + INFO("public load: " << std::string(vllm_last_error())); + REQUIRE(loaded == VLLM_OK); + std::unique_ptr engine(raw, &vllm_engine_free); + + Completion result; + result.ids.resize(4); + result.logits.reserve(4 * 128); + auto sampling = vllm_sampling_params_default(); + sampling.temperature = 0.0F; + sampling.max_tokens = 4; + sampling.ignore_eos = 1; + sampling.has_seed = 1; + sampling.seed = rocm_gather_test::kSeed; + sampling.logits_processor = Capture; + sampling.logits_processor_user_data = &result; + int32_t count = 0; + const auto hits_before = vt::GetReferenceTierHits(); + vt::ResetOpProviderStats(vt::OpId::kEmbeddingQuant, vt::DeviceType::kROCM); + vt::ResetOpProviderStats(vt::OpId::kEmbedding, vt::DeviceType::kROCM); + vt::EnableOpProviderCallStats(true); + const vllm_status completed = vllm_complete_tokens( + engine.get(), prompt.data(), static_cast(prompt.size()), &sampling, + result.ids.data(), 4, &count, nullptr); + vt::EnableOpProviderCallStats(false); + INFO("public complete: " << std::string(vllm_last_error())); + REQUIRE(completed == VLLM_OK); + REQUIRE(count == 4); + REQUIRE_FALSE(result.capture_overflow); + REQUIRE_FALSE(result.nonfinite_logits); + REQUIRE(result.callbacks == 4); + CHECK(vt::GetReferenceTierHits() == hits_before); + const auto op = expect_quant ? vt::OpId::kEmbeddingQuant : vt::OpId::kEmbedding; + const auto stats = vt::GetOpProviderStats(op, vt::DeviceType::kROCM); + INFO("public load and complete must select the ROCm embedding provider"); + CHECK(stats.selections > 0); + CHECK(stats.declines == 0); + CHECK(stats.fallbacks == 0); + std::printf("PUBLIC prompt_tokens=%zu embedding=%s selections=%llu provider=%s tokens=", + prompt.size(), expect_quant ? "quant" : "bf16", stats.selections, + stats.last_selected == nullptr ? "none" : stats.last_selected); + for (int32_t id : result.ids) std::printf(" %d", id); + std::printf("\n"); + return result; +} + +void Write(const std::filesystem::path& path, const std::string& data) { + std::ofstream file(path, std::ios::binary); + file.write(data.data(), static_cast(data.size())); + if (!file) throw std::runtime_error("cannot write gather fixture " + path.string()); +} + +void Export(const std::filesystem::path& directory) { + std::filesystem::create_directories(directory); + for (const auto& format : rocm_gather_test::kFormats) { + Write(directory / (std::string(format.name) + ".gguf"), + rocm_gather_test::BuildModel(format, false)); + Write(directory / (std::string(format.name) + "-dense.gguf"), + rocm_gather_test::BuildModel(format, true)); + Write(directory / (std::string(format.name) + ".packed"), + rocm_gather_test::PackedTable(format)); + } +} + +void RecordCompletion(const char* format, int repeat, size_t prompt, + const char* arm, const Completion& result) { + const char* directory = std::getenv("VT_ROCM_GATHER_PUBLIC_EVIDENCE"); + if (directory == nullptr) return; + std::filesystem::create_directories(directory); + const auto prefix = std::filesystem::path(directory) / + (std::string(format) + "-r" + std::to_string(repeat) + "-p" + + std::to_string(prompt) + "-" + arm); + Write(prefix.string() + "-logits-f32.bin", + std::string(reinterpret_cast(result.logits.data()), + result.logits.size() * sizeof(float))); + std::string ids; + for (int32_t id : result.ids) ids += std::to_string(id) + "\n"; + Write(prefix.string() + "-tokens.txt", ids); +} + +} // namespace + +TEST_CASE("ROCm compressed embeddings enter through public load and completion") { + const std::vector> prompts = {{1, 0, 63, 127, 63}, {1, 127, 0, 127}}; + for (const auto& format : rocm_gather_test::kFormats) { + CAPTURE(std::string(format.name)); + const auto packed = rocm_gather_test::BuildModel(format, false); + const auto dense = rocm_gather_test::BuildModel(format, true); + REQUIRE(packed.size() < 8 * 1024 * 1024); + REQUIRE(dense.size() < 8 * 1024 * 1024); + CHECK(packed == rocm_gather_test::BuildModel(format, false)); + CHECK(dense == rocm_gather_test::BuildModel(format, true)); + gguf_test::TempFile packed_file(packed), dense_file(dense); + std::vector first; + for (int repeat = 0; repeat < 3; ++repeat) { + CAPTURE(repeat); + for (size_t p = 0; p < prompts.size(); ++p) { + CAPTURE(p); + const auto quant_result = Run(packed_file.path(), prompts[p], true); + const auto dense_result = Run(dense_file.path(), prompts[p], false); + RecordCompletion(format.name, repeat, p, "quant", quant_result); + RecordCompletion(format.name, repeat, p, "dense", dense_result); + CHECK(quant_result.ids == dense_result.ids); + REQUIRE(quant_result.logits.size() == dense_result.logits.size()); + CHECK(std::memcmp(quant_result.logits.data(), dense_result.logits.data(), + quant_result.logits.size() * sizeof(float)) == 0); + if (repeat == 0) first.push_back(quant_result); + else { + CHECK(quant_result.ids == first[p].ids); + CHECK(std::memcmp(quant_result.logits.data(), first[p].logits.data(), + quant_result.logits.size() * sizeof(float)) == 0); + } + } + } + } +} + +int main(int argc, char** argv) { + if (argc == 3 && std::string(argv[1]) == "--write-fixtures") { + Export(argv[2]); + return 0; + } +#if defined(VLLM_CPP_HIP) && !defined(VLLM_CPP_CUDA) + try { + vt::GetBackend(vt::DeviceType::kROCM); + } catch (const std::runtime_error& error) { + std::printf("SKIPPED: ROCm device unavailable: %s\n", error.what()); + return 77; + } + doctest::Context context(argc, argv); + return context.run(); +#else + std::puts("SKIPPED: the public AUTO selection gate requires a HIP-only build"); + return 77; +#endif +} diff --git a/tests/scripts/fixtures/rocm_quant_gather/primary-memory.json b/tests/scripts/fixtures/rocm_quant_gather/primary-memory.json new file mode 100644 index 0000000000..52d8963b48 --- /dev/null +++ b/tests/scripts/fixtures/rocm_quant_gather/primary-memory.json @@ -0,0 +1,177 @@ +{ + "model": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "config": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "primary_pin": "e126687a9a828d513c01a07cd69f025f27d63280", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "prompt": [ + 1, + 0, + 63, + 127, + 63 + ], + "repeat": 0, + "tokens": [ + 47, + 19, + 4, + 20 + ], + "requested": { + "model_dtype": "auto from bfloat16 config", + "block_size": 16, + "num_gpu_blocks_override": 16, + "max_model_len": 64, + "max_num_seqs": 1, + "kv_cache_dtype": "auto", + "seed": 1380926285, + "greedy": true, + "ignore_eos": true, + "max_tokens": 4, + "stop": [], + "stop_token_ids": [] + }, + "harness_adaptations": [ + "pre-tokenized IDs with skip_tokenizer_init=True", + "explicit local HF text config mirrors GGUF geometry", + "development model_class_overrides selects the pinned text class for the plugin architecture key", + "top-level partial_rotary_factor=1.0 preserves GGUF rotary width 64 during HF normalization", + "16 physical KV blocks match secondary 256-cell allocation; one primary null block is reserved; logical limit remains 64", + "no oracle patch or eager-mode override" + ], + "status": "EXECUTED; token comparison remains required", + "torch": "2.12.0+git6bbd260", + "torch_git": "6bbd26020da1c6dc198625dfcdd968b1e4e6b1c5", + "hip": "7.2.53211", + "vllm": "0.28.1rc1.dev132+ge126687a9", + "plugin": "0.0.5+d4c1f0d.gfx1100", + "resolved_model_dtype": "torch.bfloat16", + "resolved_cache_config": "CacheConfig(block_size=16, user_specified_block_size=True, user_specified_mamba_block_size=False, kv_cache_layout=None, prefix_match_unit=None, gpu_memory_utilization=0.92, cache_dtype='auto', is_attention_free=False, num_gpu_blocks_override=16, sliding_window=None, enable_prefix_caching=True, prefix_caching_hash_algo='sha256', prefix_cache_retention_interval=0, kv_cache_dtype_skip_layers=[], mamba_page_size_padded=None, skip_page_size_padded=None, mamba_block_size=None, mamba_cache_dtype='auto', mamba_ssm_cache_dtype='auto', mamba_cache_mode='none', replayssm_buffer_len=16, use_replayssm=False, use_kda_recoverssm=False, num_gpu_blocks=16, num_cpu_blocks=None, kv_cache_size_tokens=256, kv_cache_max_concurrency=4.0, kv_sharing_fast_prefill=False, kv_cache_memory_bytes=None, kv_offloading_size=None, kv_offloading_backend='native', _block_size_resolved=True)", + "memory_before_generation": [ + { + "cache_tensors": [ + { + "allocator": { + "block_address": 128225247247872, + "block_bytes": 65536, + "requested_bytes": 65536, + "segment_active_bytes": 1076736, + "segment_address": 128225246183424, + "segment_allocated_bytes": 1076736, + "segment_bytes": 2097152, + "state": "active_allocated" + }, + "contiguous": false, + "device": "cuda:0", + "dtype": "torch.bfloat16", + "element_bytes": 2, + "index": 0, + "numel": 32768, + "payload_bytes": 65536, + "shape": [ + 16, + 2, + 16, + 64 + ], + "storage_address": 128225247247872, + "storage_bytes": 65536, + "storage_offset_elements": 0, + "stride_elements": [ + 1024, + 16384, + 64, + 1 + ] + } + ], + "layout_stride_order": [ + 0, + 2, + 1, + 3, + 4 + ], + "measurement": "live tensor views and PyTorch allocator snapshot; backing allocations require rocprofv3", + "physical_blocks": 16, + "process_allocated_bytes": 34632192, + "process_peak_allocated_bytes": 34787328, + "process_reserved_bytes": 39845888, + "resolved_layout": "LHBNC", + "resolved_model_dtype": "torch.bfloat16", + "runner_class": "vllm.v1.worker.gpu.model_runner.GPUModelRunner" + } + ], + "cache_capacity_contract": { + "block_size": 16, + "physical_blocks": 16, + "physical_cells": 256, + "reserved_null_blocks": 1, + "usable_blocks": 15, + "usable_cells": 240, + "logical_max_model_len": 64, + "bf16_kv_payload_bytes": 65536, + "source": "vllm/v1/core/kv_cache_utils.py:2304-2309 at e126687a9a" + }, + "memory_after_generation": [ + { + "cache_tensors": [ + { + "allocator": { + "block_address": 128225247247872, + "block_bytes": 65536, + "requested_bytes": 65536, + "segment_active_bytes": 1076736, + "segment_address": 128225246183424, + "segment_allocated_bytes": 1076736, + "segment_bytes": 2097152, + "state": "active_allocated" + }, + "contiguous": false, + "device": "cuda:0", + "dtype": "torch.bfloat16", + "element_bytes": 2, + "index": 0, + "numel": 32768, + "payload_bytes": 65536, + "shape": [ + 16, + 2, + 16, + 64 + ], + "storage_address": 128225247247872, + "storage_bytes": 65536, + "storage_offset_elements": 0, + "stride_elements": [ + 1024, + 16384, + 64, + 1 + ] + } + ], + "layout_stride_order": [ + 0, + 2, + 1, + 3, + 4 + ], + "measurement": "live tensor views and PyTorch allocator snapshot; backing allocations require rocprofv3", + "physical_blocks": 16, + "process_allocated_bytes": 34632192, + "process_peak_allocated_bytes": 34787328, + "process_reserved_bytes": 39845888, + "resolved_layout": "LHBNC", + "resolved_model_dtype": "torch.bfloat16", + "runner_class": "vllm.v1.worker.gpu.model_runner.GPUModelRunner" + } + ] +} diff --git a/tests/scripts/fixtures/rocm_quant_gather/primary-token.json b/tests/scripts/fixtures/rocm_quant_gather/primary-token.json new file mode 100644 index 0000000000..e1359be036 --- /dev/null +++ b/tests/scripts/fixtures/rocm_quant_gather/primary-token.json @@ -0,0 +1,67 @@ +{ + "model": { + "bytes": 944192, + "sha256": "663597e28852d64097f7a67542675e4575ee158d3c5e6e2143aadd1b4188f4af" + }, + "config": { + "bytes": 974, + "sha256": "919812c5a2a3cfae586873cc965f0f4b33179a7ca96b8d8b1afcdac88afca71e" + }, + "primary_pin": "e126687a9a828d513c01a07cd69f025f27d63280", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "prompt": [ + 1, + 0, + 63, + 127, + 63 + ], + "repeat": 0, + "tokens": [ + 47, + 19, + 4, + 20 + ], + "requested": { + "model_dtype": "auto from bfloat16 config", + "block_size": 16, + "num_gpu_blocks_override": 16, + "max_model_len": 64, + "max_num_seqs": 1, + "kv_cache_dtype": "auto", + "seed": 1380926285, + "greedy": true, + "ignore_eos": true, + "max_tokens": 4, + "stop": [], + "stop_token_ids": [] + }, + "harness_adaptations": [ + "pre-tokenized IDs with skip_tokenizer_init=True", + "explicit local HF text config mirrors GGUF geometry", + "pinned model_class_overrides selects the registered text class for the plugin architecture key", + "top-level partial_rotary_factor=1.0 preserves the GGUF rotary width64 during HF normalization", + "16 physical KV blocks match the secondary256-cell allocation; the primary reserves one null block and retains logical max_model_len64", + "no oracle patch or eager-mode override" + ], + "status": "EXECUTED; token comparison remains required", + "torch": "2.12.0+git6bbd260", + "torch_git": "6bbd26020da1c6dc198625dfcdd968b1e4e6b1c5", + "hip": "7.2.53211", + "vllm": "0.28.1rc1.dev132+ge126687a9", + "plugin": "0.0.5+d4c1f0d.gfx1100", + "resolved_model_dtype": "torch.bfloat16", + "resolved_cache_config": "CacheConfig(block_size=16, user_specified_block_size=True, user_specified_mamba_block_size=False, kv_cache_layout=None, prefix_match_unit=None, gpu_memory_utilization=0.92, cache_dtype='auto', is_attention_free=False, num_gpu_blocks_override=16, sliding_window=None, enable_prefix_caching=True, prefix_caching_hash_algo='sha256', prefix_cache_retention_interval=0, kv_cache_dtype_skip_layers=[], mamba_page_size_padded=None, skip_page_size_padded=None, mamba_block_size=None, mamba_cache_dtype='auto', mamba_ssm_cache_dtype='auto', mamba_cache_mode='none', replayssm_buffer_len=16, use_replayssm=False, use_kda_recoverssm=False, num_gpu_blocks=16, num_cpu_blocks=None, kv_cache_size_tokens=256, kv_cache_max_concurrency=4.0, kv_sharing_fast_prefill=False, kv_cache_memory_bytes=None, kv_offloading_size=None, kv_offloading_backend='native', _block_size_resolved=True)", + "cache_capacity_contract": { + "block_size": 16, + "physical_blocks": 16, + "physical_cells": 256, + "reserved_null_blocks": 1, + "usable_blocks": 15, + "usable_cells": 240, + "logical_max_model_len": 64, + "bf16_kv_payload_bytes": 65536, + "source": "vllm/v1/core/kv_cache_utils.py:2304-2309 at e126687a9a" + } +} diff --git a/tests/scripts/test_rocm_quant_gather_compare.py b/tests/scripts/test_rocm_quant_gather_compare.py new file mode 100644 index 0000000000..f882cf2559 --- /dev/null +++ b/tests/scripts/test_rocm_quant_gather_compare.py @@ -0,0 +1,386 @@ +"""Exercise the row comparator with explicitly synthetic validator inputs. + +The two fixture reports preserve actual successful Q4_0 captures. The temporary +matrix uses their metadata with synthetic model bytes and repeated test tokens. +It is a validator test, not an oracle measurement or a model parity result. + +The last class covers `compare.py` instead. It runs the whole tool over one +synthesized single-case matrix so the report's `scope` line is measured through +the command line and through the module API, including the default that issue +#3126 is about. +""" + +from contextlib import redirect_stdout +from copy import deepcopy +import hashlib +import importlib.util +import io +import json +from pathlib import Path +import struct +import sys +import tempfile +from types import SimpleNamespace +import unittest + + +ROOT = Path(__file__).resolve().parents[2] +TOOLS = ROOT / "tools" / "rocm_quant_gather" +FIXTURES = Path(__file__).parent / "fixtures" / "rocm_quant_gather" +sys.path.insert(0, str(TOOLS)) +try: + spec = importlib.util.spec_from_file_location("gather_compare", TOOLS / "compare_models.py") + comparator = importlib.util.module_from_spec(spec) + spec.loader.exec_module(comparator) + tool_spec = importlib.util.spec_from_file_location("gather_compare_tool", TOOLS / "compare.py") + tool = importlib.util.module_from_spec(tool_spec) + tool_spec.loader.exec_module(tool) +finally: + sys.path.pop(0) + + +class CompareModelsTest(unittest.TestCase): + @classmethod + def setUpClass(cls): + cls.directory = tempfile.TemporaryDirectory(prefix="gather-validator-") + base = Path(cls.directory.name) + cls.args = SimpleNamespace(**{name: base / name for name in + ("native", "primary", "secondary", "fixtures", "config")}) + cls.args.native_log = base / "native.log" + cls.args.primary_memory = base / "memory.json" + for name in ("native", "primary", "secondary", "fixtures", "config"): + getattr(cls.args, name).mkdir() + (cls.args.secondary / "controls").mkdir() + (cls.args.config / "config.json").write_text('{"synthetic_validator": true}\n') + cls.token_control = json.loads((FIXTURES / "primary-token.json").read_text()) + cls.memory_control = json.loads((FIXTURES / "primary-memory.json").read_text()) + cls.primary_path = cls.args.primary / "Q4_0-p0-r0.json" + cls.args.native_log.write_text( + "1: Test command: /synthetic/test_capi_rocm_embedding_quant\n" + + "1: [kv-alloc] block_size=16 num_kv_heads=1 head_size=64 dtype=2 " + "page_size_bytes=4096 num_blocks=16\n" * 228) + + def capture(prefix): + Path(str(prefix) + "-tokens.txt").write_text("47 19 4 20\n") + Path(str(prefix) + "-logits-f32.bin").write_bytes(struct.pack("<512f", *([0.0] * 512))) + + for name in comparator.FORMATS: + model = cls.args.fixtures / (name + ".gguf") + model.write_text("synthetic validator model " + name) + for repeat in range(3): + for p, prompt in enumerate(comparator.PROMPTS): + stem = f"{name}-r{repeat}-p{p}" + for arm in ("quant", "dense"): + capture(cls.args.native / (stem + "-" + arm)) + if name in comparator.SECONDARY: + prefix = cls.args.secondary / (stem + "-oracle") + capture(prefix) + Path(str(prefix) + ".log").write_text("physical_context=256\n") + elif name != "Q8_K": + report = deepcopy(cls.token_control) + report.update(model=comparator.seal(model), + config=comparator.seal(cls.args.config / "config.json"), + prompt=prompt, repeat=repeat) + (cls.args.primary / f"{name}-p{p}-r{repeat}.json").write_text(json.dumps(report)) + for oracle in ("stock", "fork"): + for repeat in range(3): + for p in range(2): + for arm in ("pristine", "overlay"): + capture(cls.args.secondary / "controls" / f"{oracle}-{arm}-r{repeat}-p{p}") + cls.memory = deepcopy(cls.memory_control) + cls.memory.update(model=comparator.seal(cls.args.fixtures / "Q4_0.gguf"), + config=comparator.seal(cls.args.config / "config.json")) + cls.args.primary_memory.write_text(json.dumps(cls.memory)) + cls.primary = json.loads(cls.primary_path.read_text()) + + @classmethod + def tearDownClass(cls): + cls.directory.cleanup() + + def reject(self, path, report): + original = path.read_bytes() + try: + path.write_text(json.dumps(report)) + with self.assertRaises((ValueError, KeyError)): + comparator.compare(self.args) + finally: + path.write_bytes(original) + + def test_valid_synthetic_matrix(self): + report = comparator.compare(self.args) + self.assertEqual(report["status"], "PASS") + self.assertEqual(len(report["cases"]), 114) + + def reject_many(self, path, base, mutations): + for label, change in mutations: + with self.subTest(label=label): + report = deepcopy(base) + change(report) + self.reject(path, report) + + def test_primary_exception_after_generation_is_rejected(self): + report = deepcopy(self.primary) + report.update(status="PENDING: primary model refused before qualification", + exception_type="RuntimeError", exception="observer failed after tokens") + self.reject(self.primary_path, report) + + def test_primary_status_only_refusal_is_rejected(self): + # A PENDING status with no exception fields must still not qualify. + report = deepcopy(self.primary) + report.update(status="PENDING: primary model refused before qualification") + self.reject(self.primary_path, report) + + def test_primary_traceback_only_is_rejected(self): + report = deepcopy(self.primary) + report.update(traceback="Traceback (most recent call last):\n observer") + self.reject(self.primary_path, report) + + def test_primary_executed_status_with_exception_is_rejected(self): + # EXECUTED status with an exception field preserves tokens and must be + # rejected on the exception alone (issue #3113's core scenario). + report = deepcopy(self.primary) + report.update(exception="observer raised after token write") + self.reject(self.primary_path, report) + + def test_primary_pinned_identities_are_rejected_when_wrong(self): + def wrong_primary_pin(report): + report["primary_pin"] = "0" * 40 + + def wrong_plugin_pin(report): + report["plugin_pin"] = "0" * 40 + + self.reject_many(self.primary_path, self.primary, [ + ("primary_pin", wrong_primary_pin), ("plugin_pin", wrong_plugin_pin)]) + + def test_primary_runtime_identities_are_rejected_when_wrong(self): + def wrong_vllm(report): + report["vllm"] = "0.28.0" + + def wrong_plugin(report): + report["plugin"] = "0.0.4" + + def wrong_torch(report): + report["torch"] = "2.11.0" + + def wrong_torch_git(report): + report["torch_git"] = "0" * 40 + + def wrong_hip(report): + report["hip"] = "6.0.0" + + self.reject_many(self.primary_path, self.primary, [ + ("vllm", wrong_vllm), ("plugin", wrong_plugin), ("torch", wrong_torch), + ("torch_git", wrong_torch_git), ("hip", wrong_hip)]) + + def test_primary_workload_identity_is_rejected_when_wrong(self): + def wrong_model(report): + report["model"] = {"sha256": "0" * 64, "bytes": 1} + + def wrong_config(report): + report["config"] = {"sha256": "0" * 64, "bytes": 1} + + def wrong_prompt(report): + report["prompt"] = [1, 0, 63, 127] + + def wrong_repeat(report): + report["repeat"] = 1 + + self.reject_many(self.primary_path, self.primary, [ + ("model", wrong_model), ("config", wrong_config), + ("prompt", wrong_prompt), ("repeat", wrong_repeat)]) + + def test_primary_dtype_and_request_are_rejected_when_wrong(self): + def fp32_dtype(report): + report["resolved_model_dtype"] = "torch.float32" + + def fp8_kv(report): + report["requested"]["kv_cache_dtype"] = "fp8" + + def non_greedy(report): + report["requested"]["greedy"] = False + + def different_capacity(report): + report["cache_capacity_contract"]["bf16_kv_payload_bytes"] = 16384 + + self.reject_many(self.primary_path, self.primary, [ + ("fp32_dtype", fp32_dtype), ("fp8_kv", fp8_kv), + ("non_greedy", non_greedy), ("different_capacity", different_capacity)]) + + def test_memory_report_path_rejects_unqualified_records(self): + def exception_after_generation(report): + report.update(exception_type="RuntimeError", exception="observer failed after tokens") + + def status_refusal(report): + report["status"] = "PENDING: primary model refused before qualification" + + def wrong_primary_pin(report): + report["primary_pin"] = "0" * 40 + + def wrong_plugin_pin(report): + report["plugin_pin"] = "0" * 40 + + def non_greedy(report): + report["requested"]["greedy"] = False + + def fp32_dtype(report): + report["resolved_model_dtype"] = "torch.float32" + + self.reject_many(self.args.primary_memory, self.memory, [ + ("exception_after_generation", exception_after_generation), + ("status_refusal", status_refusal), ("wrong_primary_pin", wrong_primary_pin), + ("wrong_plugin_pin", wrong_plugin_pin), ("non_greedy", non_greedy), + ("fp32_dtype", fp32_dtype)]) + + def test_memory_storage_geometry_is_rejected_when_invalid(self): + def offset_five(report): + for phase in ("memory_before_generation", "memory_after_generation"): + report[phase][0]["cache_tensors"][0]["storage_offset_elements"] = 5 + + def offset_negative(report): + for phase in ("memory_before_generation", "memory_after_generation"): + report[phase][0]["cache_tensors"][0]["storage_offset_elements"] = -1 + + def offset_float(report): + for phase in ("memory_before_generation", "memory_after_generation"): + report[phase][0]["cache_tensors"][0]["storage_offset_elements"] = 0.0 + + def offset_bool(report): + for phase in ("memory_before_generation", "memory_after_generation"): + report[phase][0]["cache_tensors"][0]["storage_offset_elements"] = True + + def shrunk_storage(report): + for phase in ("memory_before_generation", "memory_after_generation"): + tensor = report[phase][0]["cache_tensors"][0] + tensor["storage_bytes"] = 32768 + tensor["allocator"]["block_bytes"] = 32768 + + def grown_shape(report): + for phase in ("memory_before_generation", "memory_after_generation"): + tensor = report[phase][0]["cache_tensors"][0] + tensor["shape"] = [16, 2, 16, 128] + tensor["numel"] = 65536 + tensor["payload_bytes"] = 131072 + tensor["storage_bytes"] = 131072 + + def inactive_allocator(report): + for phase in ("memory_before_generation", "memory_after_generation"): + report[phase][0]["cache_tensors"][0]["allocator"]["state"] = "inactive" + + def wrong_layout(report): + for phase in ("memory_before_generation", "memory_after_generation"): + report[phase][0]["resolved_layout"] = "BNLH" + + def changed_during_generation(report): + tensor = report["memory_after_generation"][0]["cache_tensors"][0] + tensor["storage_address"] = tensor["storage_address"] - 4096 + + self.reject_many(self.args.primary_memory, self.memory, [ + ("offset_five", offset_five), ("offset_negative", offset_negative), + ("offset_float", offset_float), ("offset_bool", offset_bool), + ("shrunk_storage", shrunk_storage), ("grown_shape", grown_shape), + ("inactive_allocator", inactive_allocator), ("wrong_layout", wrong_layout), + ("changed_during_generation", changed_during_generation)]) + + +class CompareReportScopeTest(unittest.TestCase): + """Measure the comparison report's scope line over one synthesized case. + + `compare.py` reads a manifest, a native report, and a primary report, so the + smallest end-to-end input is one primary codec case carrying both output + dtypes. Zeros on both sides keep the comparison itself trivial: this class + asks what the report says its scope is, not what the run compared. + """ + + # Issue #3126 is about this label, so the literal is pinned here rather than + # imported from the tool under test. + DEFAULT_SCOPE = ("synthetic operation parity; original fixture and " + "model gates remain separate") + NAME = "Q4_0-256" + WIDTH = 256 + ROWS = 128 + TOKENS = 4 + + @staticmethod + def seal(path): + data = Path(path).read_bytes() + return {"bytes": len(data), "sha256": hashlib.sha256(data).hexdigest()} + + def setUp(self): + self.directory = tempfile.TemporaryDirectory(prefix="gather-scope-") + self.base = Path(self.directory.name) + row_bytes = self.WIDTH // 32 * 18 # Q4_0 stores 18 bytes per 32 values. + self.packed_bytes = self.ROWS * row_bytes + packed = self.base / (self.NAME + ".packed") + packed.write_bytes(bytes(self.packed_bytes)) + ids = self.base / (self.NAME + ".ids") + ids.write_bytes(struct.pack("<4i", 0, 127, 64, 0)) + self.manifest = self.base / "manifest.json" + self.manifest.write_text(json.dumps( + {"version": 1, "source": "synthetic-scope-case", "cases": [ + {"name": self.NAME, "type": 2, "rows": self.ROWS, "width": self.WIDTH, + "packed": packed.name, "ids": ids.name}]}) + "\n") + self.native = self.base / "native" + self.primary = self.base / "primary" + self.secondary = self.base / "secondary" + for directory in (self.native, self.primary, self.secondary): + directory.mkdir() + elements = self.TOKENS * self.WIDTH + native_cases, primary_cases = [], [] + for dtype, size in (("f32", elements * 4), ("bf16", elements * 2)): + native_result = self.native / f"{self.NAME}-{dtype}.bin" + native_result.write_bytes(bytes(size)) + native_cases.append({"name": self.NAME, "type": 2, "dtype": dtype, + "table_bytes": self.packed_bytes, "id_dtype": "i32", + "id_bytes": self.TOKENS * 4, "output_bytes": size, + "scratch_bytes": 16, "selected_packed_bytes": 0, + "selections": 1, "provider": "synthetic-scope-case", + "result": native_result.name}) + primary_result = self.primary / f"{self.NAME}-{dtype}.bin" + primary_result.write_bytes(bytes(size)) + primary_cases.append({"name": self.NAME, "type": 2, "dtype": dtype, + "packed": self.seal(packed), "ids": self.seal(ids), + "table_bytes": self.packed_bytes, + "selected_packed_bytes": row_bytes, + "result": primary_result.name, + "seal": self.seal(primary_result)}) + (self.native / "report.json").write_text( + json.dumps({"version": 1, "cases": native_cases}) + "\n") + (self.primary / "report.json").write_text( + json.dumps({"version": 1, "cases": primary_cases}) + "\n") + + def tearDown(self): + self.directory.cleanup() + + def run_cli(self, output, *extra): + argv = ["compare.py", str(self.manifest), str(self.native), str(self.primary), + str(self.secondary), str(output), *extra] + original = sys.argv + sys.argv = argv + try: + with redirect_stdout(io.StringIO()) as printed: + tool.main() + finally: + sys.argv = original + return json.loads(Path(output).read_text()), printed.getvalue() + + def test_cli_default_scope_is_the_previous_literal(self): + report, printed = self.run_cli(self.base / "default-report.json") + self.assertTrue(report["all_pass"]) + self.assertEqual(report["scope"], self.DEFAULT_SCOPE) + self.assertEqual(printed.strip(), "PASS: 2 outputs, 2 byte-exact") + + def test_cli_scope_value_is_carried_into_the_report(self): + label = "original 32-fixture coverage; model and synthetic gates remain separate" + report, _ = self.run_cli(self.base / "explicit-report.json", "--scope", label) + self.assertTrue(report["all_pass"]) + self.assertEqual(report["scope"], label) + + def test_api_default_scope_is_the_previous_literal(self): + output = self.base / "api-report.json" + with redirect_stdout(io.StringIO()): + tool.compare(self.manifest, self.native, self.primary, self.secondary, output) + self.assertEqual(json.loads(output.read_text())["scope"], self.DEFAULT_SCOPE) + + +if __name__ == "__main__": + unittest.main() diff --git a/tests/vllm/test_gguf_keep_quant.cpp b/tests/vllm/test_gguf_keep_quant.cpp index c1fe283c63..1ab5b47e02 100644 --- a/tests/vllm/test_gguf_keep_quant.cpp +++ b/tests/vllm/test_gguf_keep_quant.cpp @@ -34,6 +34,7 @@ #include #include "gguf_builder.h" +#include "capi/rocm_quant_gather_fixture.h" #include "vllm/config/weight_residency.h" #include "vllm/model_executor/device_placement.h" #include "vllm/model_executor/model_loader/gguf_dequant.h" @@ -904,6 +905,52 @@ TEST_CASE("the gather table's admission is the DECODER, not the vec_dot") { } } +TEST_CASE("Q8_K GGUF reader preserves block geometry and packed bytes") { + const auto& format = rocm_gather_test::kFormats[8]; + const TempFile file(rocm_gather_test::BuildModel(format, false)); + const auto gguf = vllm::GgufFile::Open(file.path()); + const auto& traits = vllm::GgmlTraits(15); + CHECK(traits.block_elems == 256); + CHECK(traits.block_bytes == 292); + const auto& tensor = gguf.Get("token_embd.weight"); + CHECK(tensor.ggml_type == 15); + CHECK(tensor.shape == std::vector{128, 256}); + REQUIRE(tensor.nbytes == 128 * 292); + const auto packed = rocm_gather_test::PackedTable(format); + REQUIRE(packed.size() == tensor.nbytes); + CHECK(std::memcmp(tensor.data, packed.data(), tensor.nbytes) == 0); +} + +TEST_CASE("Q8_K embedding loader retains decoder-only blocks and preserves matrix refusal") { + // BACKEND-ROCM-QUANT-GATHER (#3093): the ordinary materializer must follow + // the tensor role already resolved by the loader, including decoder-only Q8_K. + const auto& format = rocm_gather_test::kFormats[8]; + REQUIRE(format.dtype == vt::DType::kQ8_K); + const TempFile file(rocm_gather_test::BuildModel(format, false)); + const auto gguf = vllm::GgufFile::Open(file.path()); + const auto config = vllm::HfConfigFromGguf(gguf); + auto policy = KeepQuantOn(); + policy.device = vt::DeviceType::kCPU; + const auto& packed = gguf.Get("token_embd.weight"); + CHECK_FALSE(vllm::KeepQuantDType(packed.ggml_type, nullptr)); + CHECK(vllm::KeepQuantGatherDType(packed.ggml_type, nullptr)); + CHECK(vllm::RouteGgufTensor(true, false, false, false, + vllm::GgufTensorRole::kMatmulWeight, packed.ggml_type, packed.shape, + vt::DeviceType::kCPU) == vllm::GgufResidency::kExpandBf16); + CHECK_THROWS_WITH_AS(vllm::OwnGgufQuantBlocks(packed, 128, 256), + doctest::Contains("non-keep-quant encoding"), std::runtime_error); + for (const bool cuda_align : {false, true}) { + CHECK_THROWS_WITH_AS(vllm::OwnGgufQuantBlocks(packed, 128, 256, 0, nullptr, + !cuda_align, cuda_align, true, vllm::GgufTensorRole::kEmbeddingTable), + doctest::Contains("cannot use a matrix repack"), std::runtime_error); + } + const auto weights = vllm::LoadQwen3_5DenseFromGguf(gguf, config, &policy); + CHECK(weights.embed_tokens.dtype == vt::DType::kQ8_K); + CHECK_FALSE(weights.embed_tokens.nk); + REQUIRE(weights.embed_tokens.bytes.size() == packed.nbytes); + CHECK(std::memcmp(weights.embed_tokens.bytes.data(), packed.data, packed.nbytes) == 0); +} + TEST_CASE("the gather's DEVICE gate is the OP TABLE, not a hand-kept device list") { // KGATHER. `DeviceQuantGatherSupported` is `OpRegistered(kEmbeddingQuant, dev)` // and names no device. That is not a style preference: the GGUF loader is the @@ -931,7 +978,7 @@ TEST_CASE("the gather's DEVICE gate is the OP TABLE, not a hand-kept device list vt::OpRegistered(vt::OpId::kEmbeddingQuant, d)); } - // The four that have no block-decoding gather in ANY build: each of their + // The three that have no block-decoding gather in ANY build: each of their // `kEmbedding` kernels asserts a float table by name (e.g. tenstorrent_ops.cpp // "tenstorrent kEmbedding: float table, f32/bf16 out"), none registers // `kEmbeddingQuant`, and their arms are owed. A backend that registered the @@ -939,11 +986,16 @@ TEST_CASE("the gather's DEVICE gate is the OP TABLE, not a hand-kept device list // into a forward-time throw with the whole model resident — the #523 failure — // so this half stays an absolute assertion and not a registry echo. for (vt::DeviceType d : - {vt::DeviceType::kMETAL, vt::DeviceType::kVULKAN, vt::DeviceType::kROCM, + {vt::DeviceType::kMETAL, vt::DeviceType::kVULKAN, vt::DeviceType::kTENSTORRENT}) { CAPTURE(vt::DeviceTypeName(d)); CHECK_FALSE(vllm::DeviceQuantGatherSupported(d)); } +#ifdef VLLM_CPP_HIP + CHECK(vllm::DeviceQuantGatherSupported(vt::DeviceType::kROCM)); +#else + CHECK_FALSE(vllm::DeviceQuantGatherSupported(vt::DeviceType::kROCM)); +#endif } TEST_CASE("GgufLoadPolicy::FromEnv reads VT_CPU_REF and VT_GGUF_KEEP_QUANT") { diff --git a/tests/vt/test_cuda_embedding_quant.cpp b/tests/vt/test_cuda_embedding_quant.cpp index 2ffbdbbabf..3d8abdcf29 100644 --- a/tests/vt/test_cuda_embedding_quant.cpp +++ b/tests/vt/test_cuda_embedding_quant.cpp @@ -264,14 +264,19 @@ TEST_CASE("the CUDA block gather is REGISTERED, so the residency policy can find // itself would be measuring its own setup rather than the product. CHECK(vt::OpRegistered(vt::OpId::kEmbeddingQuant, DeviceType::kCUDA)); - // The four with no decoder must NOT be registered in any build: registering + // The three with no decoder must NOT be registered in any build: registering // the id without writing the decoder converts a clean load-time refusal into a // forward-time throw with the whole model resident. - for (DeviceType d : {DeviceType::kMETAL, DeviceType::kVULKAN, DeviceType::kROCM, + for (DeviceType d : {DeviceType::kMETAL, DeviceType::kVULKAN, DeviceType::kTENSTORRENT}) { CAPTURE(vt::DeviceTypeName(d)); CHECK_FALSE(vt::OpRegistered(vt::OpId::kEmbeddingQuant, d)); } +#ifdef VLLM_CPP_HIP + CHECK(vt::OpRegistered(vt::OpId::kEmbeddingQuant, DeviceType::kROCM)); +#else + CHECK_FALSE(vt::OpRegistered(vt::OpId::kEmbeddingQuant, DeviceType::kROCM)); +#endif #endif } diff --git a/tests/vt/test_ops_quant_traits.cpp b/tests/vt/test_ops_quant_traits.cpp index 3feda074df..17b4b00d44 100644 --- a/tests/vt/test_ops_quant_traits.cpp +++ b/tests/vt/test_ops_quant_traits.cpp @@ -21,6 +21,7 @@ #include #include "vllm/model_executor/model_loader/gguf_dequant.h" +#include "vllm/model_executor/model_loader/gguf_keep_quant.h" #include "vllm/model_executor/model_loader/gguf_reader.h" #include "vt/device.h" #include "vt/dtype.h" @@ -130,12 +131,15 @@ TEST_CASE("vt block geometry agrees with the GGUF reader's GgmlTraits") { CHECK(vt::GgmlTypeId(c.dtype) == c.ggml_type); CHECK(std::string(vt::Name(c.dtype)) == c.name); - // Q8_K (id 15) is activation-only: it never appears in a GGUF file, so the - // reader's table deliberately does not carry it. Every FILE type must. - if (c.dtype == vt::DType::kQ8_K) { - CHECK_THROWS(vllm::GgmlTraits(c.ggml_type)); - continue; - } + // Q8_K (id 15) is a FILE type the reader carries like any other, so the + // cross-check below runs for it too. Q8_K is the K-quants' ACTIVATION + // encoding, and llama.cpp still declares its file geometry: a real traits + // row with a nonzero `type_size` and deliberately with NO `.vec_dot` and NO + // `.to_float` (ggml/src/ggml.c:879-884 at the pinned llama.cpp `10bf611e`, + // tag `b10451`), whose row data the file loader validates as it reads the + // tensor (src/llama-model-loader.cpp:1416 into + // ggml/src/ggml-quants.c:5590-5598). This case asserted the OPPOSITE -- an + // absence -- until #3097, and no test pins that absence any more. const vllm::GgmlTypeTraits& g = vllm::GgmlTraits(c.ggml_type); CHECK(g.block_elems == vt::BlockElems(c.dtype)); CHECK(g.block_bytes == vt::BlockBytes(c.dtype)); @@ -144,6 +148,18 @@ TEST_CASE("vt block geometry agrees with the GGUF reader's GgmlTraits") { vt::DType back = vt::DType::kF32; REQUIRE(vt::BlockDTypeFromGgmlTypeId(c.ggml_type, &back)); CHECK(back == c.dtype); + + if (c.dtype == vt::DType::kQ8_K) { + // FILE GEOMETRY IS NOT KEEP-QUANT CAPABILITY. The reader names the block + // size, and the loader still refuses to keep Q8_K resident through a GEMM: + // the encoding has no weight-side dot kernel, so the public predicate the + // production loader itself calls answers false, and the embedding role + // preserves the blocks packed through the decode-only gather arm. + CHECK(vt::cpu::BlockVecDot(vt::DType::kQ8_K) == nullptr); + CHECK_FALSE(vt::cpu::HasQuantDotKernel(vt::DType::kQ8_K)); + vt::DType kept = vt::DType::kF32; + CHECK_FALSE(vllm::KeepQuantDType(c.ggml_type, &kept)); + } } } @@ -401,7 +417,9 @@ TEST_CASE("quant traits mirror type_traits_cpu (ggml-cpu.c:211-406)") { TEST_CASE("BlockToFloat matches the GGUF loader's dequant byte-for-byte") { for (const BlockCase& c : kBlockCases) { CAPTURE(c.name); - if (c.dtype == vt::DType::kQ8_K) continue; // not a file type + // Q8_K has no arm in the loader's expansion path: `DequantGgufRowToF32` + // refuses it there, while the decode-only embedding gather takes it packed. + if (c.dtype == vt::DType::kQ8_K) continue; constexpr int64_t kBlocks = 5; const std::vector bytes = RandomBlocks(c, kBlocks, 1234U); const int64_t numel = kBlocks * c.block_elems; @@ -471,8 +489,8 @@ TEST_CASE("MatmulBTQuant generic-composite fallback == dequant-then-matmul") { for (float& v : a) v = dist(rng); // The reference decodes the SAME bytes through the traits `to_float`. - // (Q8_K never appears in a GGUF file, so the loader's DequantGgufRowToF32 - // correctly refuses it — see the GgmlTraits cross-check above.) + // (The loader's DequantGgufRowToF32 has no Q8_K arm and refuses it there, + // while the reader carries its file geometry — see the cross-check above.) std::vector w(static_cast(n * k)); vt::cpu::BlockToFloat(c.dtype)(wq.data(), w.data(), n * k); const std::vector expected = ReferenceMatmul(a, w, m, k, n); diff --git a/tests/vt/test_rocm_embedding_quant.cpp b/tests/vt/test_rocm_embedding_quant.cpp new file mode 100644 index 0000000000..dd14843f19 --- /dev/null +++ b/tests/vt/test_rocm_embedding_quant.cpp @@ -0,0 +1,388 @@ +// BACKEND-ROCM-QUANT-GATHER (#3093): selected-row decoding and error contracts. +// Primary: plugin d4c1f0d, tests/test_kernels.py::test_gguf_embedding, 102-124. +// Its 2-D IDs flatten to the shared Embedding ABI's contiguous 1-D ID vector. +// Runtime primary and secondary captures remain separate from scalar controls. +#define DOCTEST_CONFIG_IMPLEMENT +#include + +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include "capi/rocm_quant_gather_fixture.h" +#include "vt/backend.h" +#include "vt/op_provider.h" +#include "vt/ops.h" +#include "vt/quant.h" + +#include "iq1_golden_vectors.h" +#include "iq2xs_iq4xs_golden_vectors.h" +#include "iq3s_golden_vectors.h" +#include "iq4nl_q5_0_golden_vectors.h" + +#ifdef VLLM_CPP_HIP +#include +#include "vt/rocm/rocm_embedding_quant.h" +#endif + +using vt::DType; +using vt::Tensor; + +namespace { + +constexpr vt::Device kDevice{vt::DeviceType::kROCM, 0}; + +class DeviceCase { + public: + DeviceCase() : backend(vt::GetBackend(kDevice.type)), queue(backend.CreateQueue()) {} + ~DeviceCase() { + for (void* allocation : allocations) backend.Free(allocation); + backend.DestroyQueue(queue); + } + DeviceCase(const DeviceCase&) = delete; + DeviceCase& operator=(const DeviceCase&) = delete; + + void* Allocate(size_t size) { + void* pointer = backend.Alloc(std::max(size, size_t{1})); + allocations.push_back(pointer); + return pointer; + } + Tensor Upload(const void* source, size_t bytes, DType dtype, + std::initializer_list shape, size_t offset = 0) { + auto* pointer = static_cast(Allocate(bytes + offset)) + offset; + if (bytes != 0) backend.Copy(queue, pointer, source, bytes); + return Tensor::Contiguous(pointer, dtype, kDevice, shape); + } + Tensor Output(DType dtype, int64_t tokens, int64_t width) { + const size_t bytes = static_cast(tokens * width) * vt::SizeOf(dtype); + void* pointer = Allocate(bytes); + backend.Memset(queue, pointer, 0xa5, std::max(bytes, size_t{1})); + return Tensor::Contiguous(pointer, dtype, kDevice, {tokens, width}); + } + std::vector Read(const Tensor& tensor) { + std::vector bytes(tensor.Bytes()); + if (!bytes.empty()) backend.Copy(queue, bytes.data(), tensor.data, bytes.size()); + backend.Synchronize(queue); + return bytes; + } + + vt::Backend& backend; + vt::Queue queue; + std::vector allocations; +}; + +std::vector RandomFiniteBlocks(DType dtype, int64_t rows, int64_t width, + uint32_t seed) { + const auto decode = vt::cpu::BlockToFloat(dtype); + REQUIRE(decode != nullptr); + const auto elements = vt::BlockElems(dtype); + const auto block_bytes = vt::BlockBytes(dtype); + std::vector bytes(static_cast(rows) * vt::RowSizeBytes(dtype, width)); + std::vector decoded(static_cast(elements)); + std::mt19937 rng(seed); + for (size_t block = 0; block < bytes.size(); block += static_cast(block_bytes)) { + int attempts = 0; + do { + REQUIRE(attempts++ < 200); + for (int64_t i = 0; i < block_bytes; ++i) { + bytes[block + static_cast(i)] = static_cast(rng() & 255); + } + decode(bytes.data() + block, decoded.data(), elements); + } while (!std::all_of(decoded.begin(), decoded.end(), [](float v) { return std::isfinite(v); })); + } + if (const char* directory = std::getenv("VT_ROCM_GATHER_TEST_CORPUS")) { + std::filesystem::create_directories(directory); + const auto name = std::string(vt::Name(dtype)) + "-" + std::to_string(width) + + "-seed-" + std::to_string(seed) + ".packed"; + std::ofstream file(std::filesystem::path(directory) / name, std::ios::binary); + file.write(reinterpret_cast(bytes.data()), static_cast(bytes.size())); + REQUIRE(file.good()); + } + return bytes; +} + +std::vector Expected(DType dtype, const uint8_t* packed, int64_t width, + const std::vector& ids, DType output_dtype, + const uint32_t* golden = nullptr) { + const size_t row_bytes = vt::RowSizeBytes(dtype, width); + const auto decode = vt::cpu::BlockToFloat(dtype); + REQUIRE(decode != nullptr); + std::vector expected(ids.size() * static_cast(width) * vt::SizeOf(output_dtype)); + std::vector row(static_cast(width)); + for (size_t token = 0; token < ids.size(); ++token) { + if (golden == nullptr) decode(packed + static_cast(ids[token]) * row_bytes, row.data(), width); + else std::memcpy(row.data(), golden + ids[token] * width, row.size() * sizeof(float)); + for (size_t element = 0; element < row.size(); ++element) { + const size_t index = token * row.size() + element; + if (output_dtype == DType::kF32) { + std::memcpy(expected.data() + index * sizeof(float), &row[element], sizeof(float)); + } else { + const uint16_t word = vt::F32ToBF16(row[element]); + std::memcpy(expected.data() + index * sizeof(uint16_t), &word, sizeof(uint16_t)); + } + } + } + return expected; +} + +void CheckGather(DType dtype, const uint8_t* bytes, int64_t rows, int64_t width, + const std::vector& ids, DType id_dtype, + DType output_dtype, size_t offset, const uint32_t* golden = nullptr) { + CAPTURE(std::string(vt::Name(dtype))); + CAPTURE(width); + CAPTURE(offset); + CAPTURE(std::string(vt::Name(id_dtype))); + CAPTURE(std::string(vt::Name(output_dtype))); + DeviceCase device; + auto table = device.Upload(bytes, static_cast(rows) * vt::RowSizeBytes(dtype, width), + dtype, {rows, width}, offset); + const std::vector ids32(ids.begin(), ids.end()); + auto indices = id_dtype == DType::kI32 + ? device.Upload(ids32.data(), ids32.size() * sizeof(int32_t), id_dtype, + {static_cast(ids.size())}) + : device.Upload(ids.data(), ids.size() * sizeof(int64_t), id_dtype, + {static_cast(ids.size())}); + auto output = device.Output(output_dtype, static_cast(ids.size()), width); + vt::Embedding(device.queue, output, table, indices); + const auto actual = device.Read(output); + const auto expected = Expected(dtype, bytes, width, ids, output_dtype, golden); + REQUIRE(actual.size() == expected.size()); + const auto mismatch = std::mismatch(actual.begin(), actual.end(), expected.begin()); + CAPTURE(static_cast(mismatch.first - actual.begin())); + CHECK(mismatch.first == actual.end()); +} + +} // namespace + +TEST_CASE("ROCm gather decoder set covers every admitted dtype through kIQ3_S") { + std::set named; + for (const auto& format : rocm_gather_test::kFormats) named.insert(format.dtype); + CHECK(named.size() == 19); + for (int value = 0; value <= static_cast(DType::kIQ3_S); ++value) { + const auto dtype = static_cast(value); + CAPTURE(value); + const bool decodable = vt::cpu::BlockToFloat(dtype) != nullptr; + CHECK((named.count(dtype) != 0) == decodable); +#ifdef VLLM_CPP_HIP + CHECK(vt::rocm::EmbeddingQuantSupported(dtype) == decodable); +#endif + } +} + +TEST_CASE("ROCm gather matches scalar bytes for every codec shape dtype and byte offset") { + uint32_t seed = 0x524f434d; + for (const auto& format : rocm_gather_test::kFormats) { + const int64_t block = vt::BlockElems(format.dtype); + const std::set widths = {block, 3 * block, 5 * block, 256, 1024}; + for (int64_t width : widths) { + const uint32_t case_seed = seed++; + CAPTURE(case_seed); + const auto packed = RandomFiniteBlocks(format.dtype, 17, width, case_seed); + const std::vector ids = {16, 8, 0, 8, 16, 4, 3, 2, 1, 0}; + for (auto id_dtype : {DType::kI32, DType::kI64}) { + for (auto output : {DType::kF32, DType::kBF16}) { + for (size_t offset = 0; offset < 4; ++offset) { + CheckGather(format.dtype, packed.data(), 17, width, ids, id_dtype, output, offset); + } + } + } + } + } +} + +TEST_CASE("ROCm gather matches committed independent decoder vectors") { + using namespace vllm_test; + struct Golden { DType dtype; const uint8_t* packed; int64_t rows; int64_t width; const uint32_t* bits; }; + const Golden cases[] = { + {DType::kIQ4_NL, kIq4nlGoldenBlocks, 2, 160, kIq4nlGoldenBits}, + {DType::kQ5_0, kQ50GoldenBlocks, 2, 64, kQ50GoldenBits}, + {DType::kIQ1_S, kIq1sGoldenBlocks, 4, 256, kIq1sGoldenBits}, + {DType::kIQ1_XXXS, kIq1xxxsGoldenBlocks, 4, 256, kIq1xxxsGoldenBits}, + {DType::kIQ2_XS, kIq2xsGoldenBlocks, 4, 256, kIq2xsGoldenBits}, + {DType::kIQ4_XS, kIq4xsGoldenBlocks, 4, 256, kIq4xsGoldenBits}, + {DType::kIQ3_S, kIq3sGoldenBlocks, 4, 256, kIq3sGoldenBits}, + }; + for (const auto& golden : cases) { + for (auto output : {DType::kF32, DType::kBF16}) { + for (auto id_dtype : {DType::kI32, DType::kI64}) { + CheckGather(golden.dtype, golden.packed, golden.rows, golden.width, + {golden.rows - 1, 0, golden.rows / 2, 0}, id_dtype, output, 3, golden.bits); + } + } + } +} + +TEST_CASE("ROCm gather crosses launch blocks and the capped grid") { + for (const auto& format : rocm_gather_test::kFormats) { + const auto packed = rocm_gather_test::PackedTable(format, 3, 256); + std::vector ids(257); + for (size_t i = 0; i < ids.size(); ++i) ids[i] = static_cast(i % 3); + CheckGather(format.dtype, reinterpret_cast(packed.data()), 3, 256, + ids, DType::kI64, DType::kBF16, 1); + } + const auto format = rocm_gather_test::kFormats[0]; + const auto packed = rocm_gather_test::PackedTable(format, 3, 32); + std::vector ids(1024 * 128 + 3); + for (size_t i = 0; i < ids.size(); ++i) ids[i] = static_cast(i % 3); + for (auto output : {DType::kF32, DType::kBF16}) { + CheckGather(format.dtype, reinterpret_cast(packed.data()), 3, 32, + ids, DType::kI32, output, 3); + } +} + +TEST_CASE("ROCm gather rounds bf16 ties to even including signed ties") { + // Q8_K stores an f32 scale. A payload of +1 or -1 reproduces these exact + // halfway values, independently of half-scale precision in other codecs. + const uint32_t ties[] = {0x3f808000, 0x3f818000, 0xbf808000, 0xbf818000}; + std::vector packed(4 * 292, 1); + for (size_t i = 0; i < 4; ++i) std::memcpy(packed.data() + i * 292, &ties[i], 4); + for (auto id_dtype : {DType::kI32, DType::kI64}) { + CheckGather(DType::kQ8_K, packed.data(), 4, 256, {0, 1, 2, 3}, id_dtype, DType::kBF16, 1); + } +} + +TEST_CASE("ROCm gather rejects full-width invalid IDs and recovers immediately") { + for (auto id_dtype : {DType::kI32, DType::kI64}) { + for (int64_t invalid : {int64_t{-1}, int64_t{3}, int64_t{4294967297}, + std::numeric_limits::min(), std::numeric_limits::max()}) { + if (id_dtype == DType::kI32 && (invalid < INT32_MIN || invalid > INT32_MAX)) continue; + CAPTURE(invalid); + DeviceCase device; + const auto format = rocm_gather_test::kFormats[0]; + const auto packed = rocm_gather_test::PackedTable(format, 3, 32); + auto table = device.Upload(packed.data(), packed.size(), format.dtype, {3, 32}, 3); + const int32_t narrow = static_cast(invalid); + auto ids = device.Upload(id_dtype == DType::kI32 ? static_cast(&narrow) : &invalid, + vt::SizeOf(id_dtype), id_dtype, {1}); + auto output = device.Output(DType::kBF16, 1, 32); + CHECK_THROWS_WITH_AS(vt::Embedding(device.queue, output, table, ids), + doctest::Contains("id " + std::to_string(invalid) + " out of range"), + std::runtime_error); + const int64_t valid = 1; + device.backend.Copy(device.queue, ids.data, &valid, vt::SizeOf(id_dtype)); + CHECK_NOTHROW(vt::Embedding(device.queue, output, table, ids)); + CHECK(device.Read(output) == Expected(format.dtype, + reinterpret_cast(packed.data()), 32, {1}, DType::kBF16)); + } + } +} + +TEST_CASE("ROCm gather accepts empty requests and rejects nonempty empty vocabulary") { + DeviceCase device; + // Contiguous constructs positive shapes. Empty non-owning views use the + // public tensor metadata, as the existing shared Embedding tests do. + auto table = Tensor::Contiguous(nullptr, DType::kQ4_0, kDevice, {1, 32}); + auto ids = Tensor::Contiguous(nullptr, DType::kI32, kDevice, {1}); + auto output = Tensor::Contiguous(nullptr, DType::kBF16, kDevice, {1, 32}); + table.shape[0] = 0; + ids.shape[0] = 0; + output.shape[0] = 0; + CHECK_NOTHROW(vt::Embedding(device.queue, output, table, ids)); + const int32_t id = 0; + ids = device.Upload(&id, sizeof(id), DType::kI32, {1}); + output = device.Output(DType::kBF16, 1, 32); + CHECK_THROWS_WITH_AS(vt::Embedding(device.queue, output, table, ids), + doctest::Contains("empty table"), std::runtime_error); +} + +TEST_CASE("ROCm gather preserves shared rank shape dtype contiguity and device checks") { + DeviceCase device; + const int32_t id = 0; + auto table = Tensor::Contiguous(nullptr, DType::kQ4_0, kDevice, {3, 32}); + auto ids = device.Upload(&id, sizeof(id), DType::kI32, {1}); + auto output = device.Output(DType::kBF16, 1, 32); + const auto refused = [&](const Tensor& t, const Tensor& i, Tensor o, const char* message) { + CHECK_THROWS_WITH_AS(vt::Embedding(device.queue, o, t, i), doctest::Contains(message), std::runtime_error); + }; + auto bad = table; bad.rank = 1; refused(bad, ids, output, "bad ranks"); + // Stock GET_ROWS batched tables exceed the shared rank-two table ABI. + bad = table; bad.rank = 3; refused(bad, ids, output, "bad ranks"); + bad = table; bad.rank = 4; refused(bad, ids, output, "bad ranks"); + bad = ids; bad.rank = 2; refused(table, bad, output, "bad ranks"); + bad = output; bad.rank = 1; refused(table, ids, bad, "bad ranks"); + bad = output; bad.shape[1] = 31; refused(table, ids, bad, "output shape mismatch"); + bad = output; bad.shape[0] = 2; refused(table, ids, bad, "output shape mismatch"); + bad = ids; bad.dtype = DType::kF32; refused(table, bad, output, "ids i32/i64"); + bad = table; bad.dtype = DType::kI8; refused(bad, ids, output, "table must be float or block-quantized"); + bad = output; bad.dtype = DType::kF16; refused(table, ids, bad, "f32/bf16 out"); + bad = table; bad.shape[1] = 31; + auto ragged = output; ragged.shape[1] = 31; + refused(bad, ids, ragged, "block table K must be a whole number of blocks"); + bad = table; bad.stride[1] = 2; refused(bad, ids, output, "contiguous required"); + bad = ids; bad.shape[0] = 2; bad.stride[0] = 2; + auto two = output; two.shape[0] = 2; + refused(table, bad, two, "contiguous required"); + bad = output; bad.stride[1] = 2; refused(table, ids, bad, "contiguous required"); + bad = table; bad.device.type = vt::DeviceType::kCPU; refused(bad, ids, output, "device mismatch"); + bad = ids; bad.device.type = vt::DeviceType::kCPU; refused(table, bad, output, "device mismatch"); + bad = output; bad.device.type = vt::DeviceType::kCPU; refused(table, ids, bad, "device mismatch"); +} + +TEST_CASE("ROCm gather follows the queue device when the ambient device differs") { +#ifdef VLLM_CPP_HIP + int count = 0; + REQUIRE(hipGetDeviceCount(&count) == hipSuccess); + if (std::getenv("VT_ROCM_GATHER_REQUIRE_TWO_DEVICES") != nullptr) REQUIRE(count >= 2); + if (count < 2) { + MESSAGE("The separate ambient-device gate requires two visible ROCm devices"); + return; + } + REQUIRE(hipSetDevice(0) == hipSuccess); + DeviceCase device; + struct CleanupOnQueueDevice { + ~CleanupOnQueueDevice() { (void)hipSetDevice(0); } + } cleanup; + const auto format = rocm_gather_test::kFormats[0]; + const auto packed = rocm_gather_test::PackedTable(format, 3, 32); + auto table = device.Upload(packed.data(), packed.size(), format.dtype, {3, 32}); + int64_t id = 2; + auto ids = device.Upload(&id, sizeof(id), DType::kI64, {1}); + auto output = device.Output(DType::kBF16, 1, 32); + device.backend.Synchronize(device.queue); + REQUIRE(hipSetDevice(1) == hipSuccess); + CHECK_NOTHROW(vt::Embedding(device.queue, output, table, ids)); + REQUIRE(hipSetDevice(0) == hipSuccess); + CHECK(device.Read(output) == Expected(format.dtype, + reinterpret_cast(packed.data()), 32, {2}, DType::kBF16)); + id = -1; + device.backend.Copy(device.queue, ids.data, &id, sizeof(id)); + device.backend.Synchronize(device.queue); + REQUIRE(hipSetDevice(1) == hipSuccess); + CHECK_THROWS_WITH_AS(vt::Embedding(device.queue, output, table, ids), + doctest::Contains("id -1 out of range"), std::runtime_error); + REQUIRE(hipSetDevice(0) == hipSuccess); + id = 1; + device.backend.Copy(device.queue, ids.data, &id, sizeof(id)); + device.backend.Synchronize(device.queue); + REQUIRE(hipSetDevice(1) == hipSuccess); + CHECK_NOTHROW(vt::Embedding(device.queue, output, table, ids)); + REQUIRE(hipSetDevice(0) == hipSuccess); + CHECK(device.Read(output) == Expected(format.dtype, + reinterpret_cast(packed.data()), 32, {1}, DType::kBF16)); +#endif +} + +int main(int argc, char** argv) { +#ifdef VLLM_CPP_HIP + try { vt::GetBackend(kDevice.type); } + catch (const std::runtime_error& error) { + std::printf("SKIPPED: ROCm device unavailable: %s\n", error.what()); + return 77; + } + doctest::Context context(argc, argv); + return context.run(); +#else + (void)argc; + (void)argv; + std::puts("SKIPPED: native ROCm gather requires a HIP build"); + return 77; +#endif +} diff --git a/tools/rocm_quant_gather/CMakeLists.txt b/tools/rocm_quant_gather/CMakeLists.txt new file mode 100644 index 0000000000..78f992daf1 --- /dev/null +++ b/tools/rocm_quant_gather/CMakeLists.txt @@ -0,0 +1,26 @@ +cmake_minimum_required(VERSION 3.25) +project(rocm_gather_secondary LANGUAGES C CXX) + +# Build an external, exact oracle source tree without editing that source. +if(NOT IS_DIRECTORY "${ORACLE_SOURCE}") + message(FATAL_ERROR "Set ORACLE_SOURCE to the pinned llama.cpp source tree") +endif() +set(BUILD_SHARED_LIBS OFF CACHE BOOL "" FORCE) +set(LLAMA_BUILD_COMMON OFF CACHE BOOL "" FORCE) +set(LLAMA_BUILD_TESTS OFF CACHE BOOL "" FORCE) +set(LLAMA_BUILD_EXAMPLES OFF CACHE BOOL "" FORCE) +set(LLAMA_BUILD_TOOLS OFF CACHE BOOL "" FORCE) +set(LLAMA_BUILD_SERVER OFF CACHE BOOL "" FORCE) +set(GGML_CUDA OFF CACHE BOOL "" FORCE) +set(GGML_HIP OFF CACHE BOOL "" FORCE) +set(GGML_VULKAN OFF CACHE BOOL "" FORCE) +set(GGML_NATIVE OFF CACHE BOOL "" FORCE) +set(GGML_AVX2 ON CACHE BOOL "" FORCE) +set(GGML_OPENMP OFF CACHE BOOL "" FORCE) +set(GGML_BACKEND_DL OFF CACHE BOOL "" FORCE) +add_subdirectory("${ORACLE_SOURCE}" oracle) + +add_executable(rocm_gather_secondary secondary.cpp) +target_compile_features(rocm_gather_secondary PRIVATE cxx_std_17) +target_include_directories(rocm_gather_secondary PRIVATE "${ORACLE_SOURCE}/ggml/src") +target_link_libraries(rocm_gather_secondary PRIVATE llama ggml ggml-cpu) diff --git a/tools/rocm_quant_gather/audit_trace.py b/tools/rocm_quant_gather/audit_trace.py new file mode 100644 index 0000000000..0283364afb --- /dev/null +++ b/tools/rocm_quant_gather/audit_trace.py @@ -0,0 +1,125 @@ +#!/usr/bin/env python3 +"""Audit matching production gather traces and each arm's untraced outputs. + +Native allocation sizes are rocprofv3 events correlated with hipMalloc. +Primary tensor sizes are PyTorch allocator telemetry; its caching allocator +appears as a backing pool in the profiler. These are separate evidence types. +""" + +from __future__ import annotations + +import argparse +from collections import Counter +import csv +import json +from pathlib import Path + +from primary import seal + + +def require(condition: bool, message: str) -> None: + if not condition: + raise ValueError(message) + + +def read_csv(directory: Path, suffix: str) -> tuple[Path, list[dict]]: + paths = list(directory.glob(f"*_{suffix}.csv")) + require(len(paths) == 1, f"expected one complete process trace for {suffix}") + with paths[0].open() as file: + return paths[0], list(csv.DictReader(file)) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("manifest", type=Path) + parser.add_argument("traces", type=Path) + parser.add_argument("untraced_native", type=Path) + parser.add_argument("untraced_primary", type=Path) + parser.add_argument("output", type=Path) + args = parser.parse_args() + inputs = json.loads(args.manifest.read_text())["cases"] + expected_outputs = 2 * len(inputs) + summary = {"manifest": seal(args.manifest), "outputs_per_arm": expected_outputs, + "scope": "same-byte operation dispatch and memory; no model or performance claim", + "arms": {}, "files": {}} + for arm in ("native", "primary"): + directory = args.traces / arm + report = json.loads((directory / "report.json").read_text()) + previous_dir = getattr(args, "untraced_" + arm) + previous = json.loads((previous_dir / "report.json").read_text()) + previous_entries = {(c["name"], c["dtype"]): c for c in previous["cases"]} + require(len(report["cases"]) == expected_outputs, "missing traced output cases") + expected_keys = [(c["name"], dtype) for c in inputs for dtype in ("f32", "bf16")] + require([(c["name"], c["dtype"]) for c in report["cases"]] == expected_keys, + "trace case sequence differs from the common manifest") + for entry in report["cases"]: + prior = previous_entries[(entry["name"], entry["dtype"])] + actual_path = directory / entry["result"] + require(actual_path.read_bytes() == (previous_dir / prior["result"]).read_bytes(), + "profiling or I64 ID adaptation changed an output byte") + require(entry["id_dtype"] == "i64" and entry["id_bytes"] == 32, + "both arms must consume the same four I64 IDs") + summary["files"][str(actual_path)] = seal(actual_path) + profiler = args.traces / (arm + "-profiler") + kernel_file, kernels = read_csv(profiler, "kernel_trace") + api_file, api = read_csv(profiler, "hip_api_trace") + allocation_file, allocations = read_csv(profiler, "memory_allocation_trace") + api_by_id = {r["Correlation_Id"]: r["Function"] for r in api} + marker = "EmbeddingQuantGatherKernel<" if arm == "native" else "void dequantize_block" + decode = sorted((r for r in kernels if marker in r["Kernel_Name"]), + key=lambda r: int(r["Start_Timestamp"])) + require(len(decode) == expected_outputs, "each output must have exactly one GPU decoder") + for entry, event in zip(report["cases"], decode): + name = event["Kernel_Name"] + require(event["Correlation_Id"] in api_by_id, "decoder lacks its HIP API event") + if arm == "native": + required_type = "" + require(required_type in name, "executed decoder output or ID template dtype differs") + if arm == "native": + require("Dq" + entry["name"].rsplit("-", 1)[0] + ">" in name, + "native decoder format differs from the selected packed table") + result = {"unchanged_output_bytes": expected_outputs, "gpu_decoders": len(decode), + "decoder_template_dtypes": {"f32": expected_outputs // 2, "bf16": expected_outputs // 2}, + "kernel_histogram": dict(Counter(r["Kernel_Name"] for r in kernels))} + if arm == "native": + ordered_allocations = sorted((r for r in allocations + if r["Operation"] == "MEMORY_ALLOCATION_ALLOCATE" + and api_by_id.get(r["Correlation_Id"]) == "hipMalloc"), + key=lambda r: int(r["Start_Timestamp"])) + expected_sizes = [] + for case in inputs: + table_bytes = (args.manifest.parent / case["packed"]).stat().st_size + expected_sizes.extend([table_bytes, 32, 4 * case["width"] * 4, 16, + 4 * case["width"] * 2, 16]) + require([int(r["Allocation_Size"]) for r in ordered_allocations] == expected_sizes, + "native allocation sequence has missing or extra table/ID/output/scratch storage") + result["profiler_hipmalloc_allocations"] = len(ordered_allocations) + result["scratch_allocations_16_bytes"] = expected_outputs + result["extra_device_allocations"] = 0 + result["allocation_evidence"] = "rocprofv3 sizes correlated with hipMalloc; runtime host allocations excluded" + else: + selected = [r for r in kernels if "indexSelectSmallIndex +#include +#include +#include +#include +#include +#include +#include +#include +#include + +#include + +#include "capi/rocm_quant_gather_fixture.h" +#include "vt/backend.h" +#include "vt/op_provider.h" +#include "vt/ops.h" +#ifdef VLLM_CPP_HIP +#include "vt/rocm/rocm_embedding_quant.h" +#endif + +namespace { + +using Json = nlohmann::json; +using Path = std::filesystem::path; +constexpr vt::Device kDevice{vt::DeviceType::kROCM, 0}; + +std::string Read(const Path& path) { + std::ifstream file(path, std::ios::binary); + if (!file) throw std::runtime_error("cannot read " + path.string()); + return std::string(std::istreambuf_iterator(file), std::istreambuf_iterator()); +} + +void Write(const Path& path, const std::string& bytes) { + std::ofstream file(path, std::ios::binary); + file.write(bytes.data(), static_cast(bytes.size())); + if (!file) throw std::runtime_error("cannot write " + path.string()); +} + +void Generate(const Path& directory) { + std::filesystem::create_directories(directory); + Json manifest = {{"version", 1}, {"source", "deterministic-synthetic"}, + {"seed", rocm_gather_test::kSeed}, {"cases", Json::array()}}; + constexpr int32_t ids[] = {0, 127, 64, 0}; + for (const auto& format : rocm_gather_test::kFormats) { + for (int width : {256, 1024}) { + const auto name = std::string(format.name) + "-" + std::to_string(width); + Write(directory / (name + ".packed"), rocm_gather_test::PackedTable(format, 128, width)); + Write(directory / (name + ".ids"), std::string(reinterpret_cast(ids), sizeof(ids))); + manifest["cases"].push_back({{"name", name}, {"type", format.ggml}, + {"rows", 128}, {"width", width}, + {"packed", name + ".packed"}, {"ids", name + ".ids"}}); + } + } + Write(directory / "manifest.json", manifest.dump(2) + "\n"); +} + +vt::DType FindType(uint32_t type) { + for (const auto& format : rocm_gather_test::kFormats) { + if (format.ggml == type) return format.dtype; + } + throw std::runtime_error("input type exceeds the shared gather decoder set"); +} + +class DeviceStorage { + public: + DeviceStorage() : backend(vt::GetBackend(kDevice.type)), queue(backend.CreateQueue()) {} + ~DeviceStorage() { + for (auto* allocation : allocations) backend.Free(allocation); + backend.DestroyQueue(queue); + } + DeviceStorage(const DeviceStorage&) = delete; + DeviceStorage& operator=(const DeviceStorage&) = delete; + void* Allocate(size_t size) { + auto* pointer = backend.Alloc(size); + allocations.push_back(pointer); + return pointer; + } + vt::Tensor Upload(const std::string& bytes, vt::DType dtype, + std::initializer_list shape) { + auto* pointer = Allocate(bytes.size()); + backend.Copy(queue, pointer, bytes.data(), bytes.size()); + return vt::Tensor::Contiguous(pointer, dtype, kDevice, shape); + } + vt::Backend& backend; + vt::Queue queue; + std::vector allocations; +}; + +void Capture(const Path& input, const Path& output, const std::string& selected) { +#ifdef VLLM_CPP_HIP + const auto manifest = Json::parse(Read(input)); + std::filesystem::create_directories(output); + Json report = {{"version", 1}, {"cases", Json::array()}}; + bool found = false; + for (const auto& entry : manifest.at("cases")) { + const auto name = entry.at("name").get(); + if (!selected.empty() && selected != name) continue; + found = true; + const auto dtype = FindType(entry.at("type").get()); + const int64_t rows = entry.at("rows").get(); + const int64_t width = entry.at("width").get(); + const auto packed = Read(input.parent_path() / entry.at("packed").get()); + const auto ids = Read(input.parent_path() / entry.at("ids").get()); + if (rows <= 0 || width <= 0 || width % vt::BlockElems(dtype) != 0 || + packed.size() != static_cast(rows) * vt::RowSizeBytes(dtype, width) || + ids.empty() || ids.size() % sizeof(int32_t) != 0) { + throw std::runtime_error("input geometry or byte count mismatch"); + } + const auto tokens = static_cast(ids.size() / sizeof(int32_t)); + DeviceStorage device; + auto table = device.Upload(packed, dtype, {rows, width}); + const bool use_i64 = std::getenv("VT_ROCM_GATHER_TRACE_I64_IDS") != nullptr; + std::string index_bytes = ids; + if (use_i64) { + index_bytes.resize(static_cast(tokens) * sizeof(int64_t)); + for (int64_t token = 0; token < tokens; ++token) { + int32_t narrow; + std::memcpy(&narrow, ids.data() + token * sizeof(narrow), sizeof(narrow)); + const int64_t wide = narrow; + std::memcpy(index_bytes.data() + token * sizeof(wide), &wide, sizeof(wide)); + } + } + const auto index_dtype = use_i64 ? vt::DType::kI64 : vt::DType::kI32; + auto indices = device.Upload(index_bytes, index_dtype, {tokens}); + // f32 is the explicit primitive oracle mode. Model gathers emit bf16. + for (auto out_dtype : {vt::DType::kF32, vt::DType::kBF16}) { + const size_t output_bytes = static_cast(tokens * width) * vt::SizeOf(out_dtype); + auto result = vt::Tensor::Contiguous(device.Allocate(output_bytes), out_dtype, + kDevice, {tokens, width}); + const auto hits = vt::GetReferenceTierHits(); + vt::ResetOpProviderStats(vt::OpId::kEmbeddingQuant, kDevice.type); + vt::EnableOpProviderCallStats(true); + vt::Embedding(device.queue, result, table, indices); + vt::EnableOpProviderCallStats(false); + const auto stats = vt::GetOpProviderStats(vt::OpId::kEmbeddingQuant, kDevice.type); + if (stats.selections != 1 || stats.fallbacks != 0 || stats.declines != 0 || + vt::GetReferenceTierHits() != hits) { + throw std::runtime_error("capture did not execute exactly one native gather"); + } + std::string bytes(output_bytes, '\0'); + device.backend.Copy(device.queue, bytes.data(), result.data, bytes.size()); + device.backend.Synchronize(device.queue); + const auto result_name = name + "-" + vt::Name(out_dtype) + ".bin"; + Write(output / result_name, bytes); + report["cases"].push_back({{"name", name}, {"type", entry.at("type")}, + {"dtype", vt::Name(out_dtype)}, {"table_bytes", packed.size()}, + {"id_dtype", vt::Name(index_dtype)}, {"id_bytes", index_bytes.size()}, + {"output_bytes", output_bytes}, + {"scratch_bytes", sizeof(vt::rocm::EmbeddingQuantError)}, + {"selected_packed_bytes", 0}, {"selections", stats.selections}, + {"provider", stats.last_selected == nullptr ? "" : stats.last_selected}, + {"result", result_name}}); + std::printf("NATIVE %s %s table=%zu output=%zu scratch=%zu\n", name.c_str(), + vt::Name(out_dtype), packed.size(), output_bytes, + sizeof(vt::rocm::EmbeddingQuantError)); + } + } + if (!found) throw std::runtime_error("case selection matched no manifest entries"); + Write(output / "report.json", report.dump(2) + "\n"); +#else + (void)input; + (void)output; + (void)selected; + throw std::runtime_error("native capture requires a HIP build"); +#endif +} + +} // namespace + +int main(int argc, char** argv) { + try { + if (argc == 3 && std::string(argv[1]) == "generate") Generate(argv[2]); + else if ((argc == 4 || argc == 5) && std::string(argv[1]) == "capture") + Capture(argv[2], argv[3], argc == 5 ? argv[4] : ""); + else throw std::runtime_error("use generate DIRECTORY or capture MANIFEST OUTPUT [CASE]"); + return 0; + } catch (const std::exception& error) { + std::fprintf(stderr, "CAPTURE FAILED: %s\n", error.what()); + return 1; + } +} diff --git a/tools/rocm_quant_gather/compare.py b/tools/rocm_quant_gather/compare.py new file mode 100644 index 0000000000..03c2171a16 --- /dev/null +++ b/tools/rocm_quant_gather/compare.py @@ -0,0 +1,117 @@ +#!/usr/bin/env python3 +"""Compare every native gather capture with its independently executed oracle.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import math +from pathlib import Path +import struct + + +def seal(path: Path) -> dict: + data = path.read_bytes() + return {"bytes": len(data), "sha256": hashlib.sha256(data).hexdigest()} + + +def values(data: bytes, dtype: str) -> list[float]: + if dtype == "f32": + return [item[0] for item in struct.iter_unpack(" bytes: + result = bytearray() + for (bits,) in struct.iter_unpack("> 16) | 0x40 + else: + word = ((bits + 0x7FFF + ((bits >> 16) & 1)) >> 16) & 0xFFFF + result.extend(struct.pack(" None: + cases = json.loads(manifest.read_text())["cases"] + native_report = json.loads((native / "report.json").read_text()) + needs_primary = any(case["type"] not in (15, 39, 66) for case in cases) + primary_report = json.loads((primary / "report.json").read_text()) if needs_primary else {"cases": []} + native_entries = {(entry["name"], entry["dtype"]): entry for entry in native_report["cases"]} + primary_entries = {(entry["name"], entry["dtype"]): entry for entry in primary_report["cases"]} + if len(native_entries) != 2 * len(cases) or len(native_report["cases"]) != len(native_entries): + raise ValueError("native report must contain exactly two outputs per input case") + results = [] + for case in cases: + count = len((manifest.parent / case["ids"]).read_bytes()) // 4 * case["width"] + is_secondary = case["type"] in (15, 39, 66) + for dtype, element_bytes in (("f32", 4), ("bf16", 2)): + entry = native_entries[(case["name"], dtype)] + actual_path = native / entry["result"] + actual = actual_path.read_bytes() + if len(actual) != count * element_bytes or entry["output_bytes"] != len(actual): + raise ValueError("native output byte count mismatch") + if is_secondary: + expected_path = secondary / f"{case['name']}-f32.bin" + expected = expected_path.read_bytes() + if dtype == "bf16": + expected = bf16_from_f32(expected) + oracle = "fork" if case["type"] == 66 else "stock" + tolerance = "byte-exact, with explicit f32-to-bf16 RNE for bf16 output" + else: + oracle_entry = primary_entries[(case["name"], dtype)] + if oracle_entry["packed"] != seal(manifest.parent / case["packed"]): + raise ValueError("primary capture used different packed bytes") + if oracle_entry["ids"] != seal(manifest.parent / case["ids"]): + raise ValueError("primary capture used different selected IDs") + expected_path = primary / oracle_entry["result"] + if oracle_entry["seal"] != seal(expected_path): + raise ValueError("primary output changed after capture") + expected = expected_path.read_bytes() + oracle = "primary-plugin" + tolerance = "atol=0.01 rtol=0.04, pinned upstream embedding test" + if len(expected) != len(actual): + raise ValueError("oracle output byte count mismatch") + a_values = values(actual, dtype) + b_values = values(expected, dtype) + finite = all(math.isfinite(a) and math.isfinite(b) for a, b in zip(a_values, b_values)) + mismatches = sum(not (abs(a - b) <= 0.01 + 0.04 * abs(b)) + for a, b in zip(a_values, b_values)) + passed = finite and (actual == expected if is_secondary else mismatches == 0) + results.append({"name": case["name"], "dtype": dtype, "oracle": oracle, + "elements": count, "table_bytes": entry["table_bytes"], + "output_bytes": len(actual), "scratch_bytes": entry["scratch_bytes"], + "tolerance": tolerance, "byte_exact": actual == expected, + "max_absolute_error": max(abs(a - b) for a, b in zip(a_values, b_values)), + "finite": finite, "pass": passed, + "native": {"file": str(actual_path), **seal(actual_path)}, + "oracle_output": {"file": str(expected_path), **seal(expected_path)}}) + passed = all(result["pass"] for result in results) + output.write_text(json.dumps({"manifest": seal(manifest), "cases": results, + "outputs": len(results), "all_pass": passed, + "scope": scope}, + indent=2) + "\n") + if not passed: + raise SystemExit("one or more native outputs differ from the pinned oracle") + print(f"PASS: {len(results)} outputs, {sum(result['byte_exact'] for result in results)} byte-exact") + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("manifest", type=Path) + parser.add_argument("native", type=Path) + parser.add_argument("primary", type=Path) + parser.add_argument("secondary", type=Path) + parser.add_argument("output", type=Path) + parser.add_argument("--scope", default="synthetic operation parity; original fixture and " + "model gates remain separate", + help="Report scope line describing the inputs this run compared.") + args = parser.parse_args() + compare(args.manifest, args.native, args.primary, args.secondary, args.output, args.scope) + + +if __name__ == "__main__": + main() diff --git a/tools/rocm_quant_gather/compare_models.py b/tools/rocm_quant_gather/compare_models.py new file mode 100644 index 0000000000..540c69c401 --- /dev/null +++ b/tools/rocm_quant_gather/compare_models.py @@ -0,0 +1,264 @@ +#!/usr/bin/env python3 +"""Compare the complete bounded model matrix and its physical cache contract. + +Native captures enter through the public C ABI. Every primary case is a fresh +pinned engine. Secondary model captures name their bounded allocation overlay. +Q8_K remains a decoder-only secondary qualification and is never counted as a +secondary model execution. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path +import re +import struct +import math + +from primary import seal + + +FORMATS = ("Q4_0", "Q5_0", "Q8_0", "Q2_K", "Q3_K", "Q4_K", "Q5_K", "Q6_K", + "Q8_K", "IQ2_XXS", "IQ2_XS", "IQ3_XXS", "IQ1_S", "IQ4_NL", "IQ3_S", + "IQ2_S", "IQ4_XS", "MXFP4", "IQ1_XXXS") +SECONDARY = {"MXFP4": "stock", "IQ1_XXXS": "fork"} +PROMPTS = ([1, 0, 63, 127, 63], [1, 127, 0, 127]) + + +def qualify_primary(report: dict, model: dict, config: dict, + prompt: list[int], repeat: int) -> None: + """Qualify either capture against the sealed runtime and bounded workload.""" + if any(key in report for key in ("exception_type", "exception", "traceback")): + raise ValueError("primary capture contains an exception") + # Runtime identities come from the row's successful sealed Q4_0 captures. + # These are local qualification requirements, not a global oracle pin change. + expected = { + "status": "EXECUTED; token comparison remains required", + "primary_pin": "e126687a9a828d513c01a07cd69f025f27d63280", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "vllm": "0.28.1rc1.dev132+ge126687a9", + "plugin": "0.0.5+d4c1f0d.gfx1100", + "torch": "2.12.0+git6bbd260", + "torch_git": "6bbd26020da1c6dc198625dfcdd968b1e4e6b1c5", + "hip": "7.2.53211", + "model": model, "config": config, "prompt": prompt, "repeat": repeat, + "resolved_model_dtype": "torch.bfloat16", + "requested": { + "model_dtype": "auto from bfloat16 config", "kv_cache_dtype": "auto", + "block_size": 16, "num_gpu_blocks_override": 16, + "max_model_len": 64, "max_num_seqs": 1, "seed": 0x524F434D, + "greedy": True, "ignore_eos": True, "max_tokens": 4, + "stop": [], "stop_token_ids": []}, + "cache_capacity_contract": { + "block_size": 16, "physical_blocks": 16, "physical_cells": 256, + "reserved_null_blocks": 1, "usable_blocks": 15, "usable_cells": 240, + "logical_max_model_len": 64, "bf16_kv_payload_bytes": 65536, + "source": "vllm/v1/core/kv_cache_utils.py:2304-2309 at e126687a9a"}, + } + for key, value in expected.items(): + # JSON comparison preserves the distinction between booleans and numbers, + # including prompt IDs and nested sampling fields. + if key not in report or json.dumps(report[key], sort_keys=True) != json.dumps(value, sort_keys=True): + raise ValueError(f"primary qualification field differs or is missing: {key}") + + +def tokens(path: Path) -> list[int]: + result = [int(value) for value in path.read_text().split()] + if len(result) != 4 or any(token < 0 or token >= 128 for token in result): + raise ValueError(f"invalid bounded completion: {path}") + return result + + +def logits(path: Path) -> bytes: + data = path.read_bytes() + if len(data) != 4 * 128 * 4: + raise ValueError(f"incomplete logits: {path}") + if not all(math.isfinite(value) for (value,) in struct.iter_unpack(" dict: + result = {} + for phase in ("memory_before_generation", "memory_after_generation"): + workers = report[phase] + if len(workers) != 1: + raise ValueError("bounded primary must have exactly one worker") + worker = workers[0] + if worker["physical_blocks"] != 16 or worker["resolved_model_dtype"] != "torch.bfloat16": + raise ValueError("primary physical block count or model dtype differs") + caches = worker["cache_tensors"] + if (worker["resolved_layout"] != "LHBNC" or + worker["layout_stride_order"] != [0, 2, 1, 3, 4]): + raise ValueError("primary executing cache layout differs from the pinned full-attention path") + if len(caches) != 1: + raise ValueError("bounded primary must expose one full-attention cache tensor") + unique = {tensor["storage_address"]: tensor for tensor in caches} + if not caches or sum(tensor["storage_bytes"] for tensor in unique.values()) != 65536: + raise ValueError("primary physical cache storage must contain exactly 65536 bytes") + if sum(tensor["payload_bytes"] for tensor in caches) != 65536: + raise ValueError("primary cache views do not describe the complete physical payload") + for tensor in caches: + if tensor["dtype"] != "torch.bfloat16" or tensor["element_bytes"] != 2: + raise ValueError("primary actual cache tensor is not BF16") + if math.prod(tensor["shape"]) != tensor["numel"]: + raise ValueError("primary cache shape and element count differ") + if tensor["shape"] != [16, 2, 16, 64] or tensor["stride_elements"] != [1024, 16384, 64, 1]: + raise ValueError("primary actual cache shape or stride differs") + storage_offset = tensor["storage_offset_elements"] + if type(storage_offset) is not int: + raise ValueError("primary cache storage offset must be an integer") + end_element = storage_offset + 1 + sum( + (size - 1) * stride for size, stride in zip(tensor["shape"], tensor["stride_elements"])) + if storage_offset < 0 or end_element * tensor["element_bytes"] > tensor["storage_bytes"]: + raise ValueError("primary strided cache view is outside its storage") + if storage_offset != 0: + raise ValueError("primary cache view must start at its storage origin") + allocation = tensor["allocator"] + if allocation is None or allocation["state"] != "active_allocated": + raise ValueError("primary cache storage has no live allocator block") + if allocation["block_bytes"] < tensor["storage_bytes"]: + raise ValueError("primary cache exceeds its allocator block") + offset = tensor["storage_address"] - allocation["block_address"] + if offset < 0 or offset + tensor["storage_bytes"] > allocation["block_bytes"]: + raise ValueError("primary cache storage is outside its allocator block") + result[phase] = worker + before = result["memory_before_generation"]["cache_tensors"] + after = result["memory_after_generation"]["cache_tensors"] + for left, right in zip(before, after): + for key in ("storage_address", "storage_bytes", "storage_offset_elements", "shape", + "stride_elements", "dtype", "element_bytes", "numel", "payload_bytes"): + if left[key] != right[key]: + raise ValueError("primary physical cache changed during generation") + return result + + +def compare(args) -> dict: + log = args.native_log.read_text() + tests = re.findall(r"^(\d+): Test command: [^\n]*/test_capi_rocm_embedding_quant$", log, re.MULTILINE) + if len(tests) != 1: + raise ValueError("native log must identify exactly one public CTest execution") + allocation_lines = re.findall(r"^" + tests[0] + r": \[kv-alloc\] ([^\n]+)", log, re.MULTILINE) + if len(allocation_lines) != 228: + raise ValueError("native public gate must capture all 228 engine cache allocations") + for line in allocation_lines: + values = dict(re.findall(r"(\w+)=(\w+)", line)) + expected = {"block_size": "16", "num_kv_heads": "1", "head_size": "64", + "dtype": "2", "page_size_bytes": "4096", "num_blocks": "16"} + if any(values.get(key) != value for key, value in expected.items()): + raise ValueError(f"native runtime cache differs from matched BF16 physical capacity: {line}") + cases = [] + first = {} + secondary_first = {} + files = {str(args.native_log): seal(args.native_log)} + for name in FORMATS: + for repeat in range(3): + for p, prompt in enumerate(PROMPTS): + stem = f"{name}-r{repeat}-p{p}" + captures = {} + for arm in ("quant", "dense"): + prefix = args.native / (stem + "-" + arm) + ids_path = Path(str(prefix) + "-tokens.txt") + logits_path = Path(str(prefix) + "-logits-f32.bin") + captures[arm] = (tokens(ids_path), logits(logits_path)) + files[str(ids_path)] = seal(ids_path) + files[str(logits_path)] = seal(logits_path) + if captures["quant"] != captures["dense"]: + raise ValueError(f"native quantized completion differs from dense control: {stem}") + key = (name, p) + if repeat == 0: + first[key] = captures["quant"] + elif first[key] != captures["quant"]: + raise ValueError(f"native repeated engine differs: {stem}") + actual = captures["quant"][0] + case = {"format": name, "repeat": repeat, "prompt_index": p, + "prompt": prompt, "native_tokens": actual, "native_dense_exact": True, + "native_logits_sha256": files[str(args.native / (stem + "-quant-logits-f32.bin"))]["sha256"]} + if name == "Q8_K": + case["oracle"] = "decoder-only stock adaptation; no stock GET_ROWS or model claim" + elif name in SECONDARY: + prefix = args.secondary / (stem + "-oracle") + path = Path(str(prefix) + "-tokens.txt") + expected_ids = tokens(path) + scores_path = Path(str(prefix) + "-logits-f32.bin") + oracle_scores = logits(scores_path) + if repeat == 0: + secondary_first[key] = (expected_ids, oracle_scores) + elif secondary_first[key] != (expected_ids, oracle_scores): + raise ValueError(f"secondary repeated engine differs: {stem}") + log_path = Path(str(prefix) + ".log") + if "physical_context=256" not in log_path.read_text(): + raise ValueError(f"secondary physical capacity is unmeasured: {stem}") + files[str(path)] = seal(path) + files[str(scores_path)] = seal(scores_path) + files[str(log_path)] = seal(log_path) + case.update(oracle=SECONDARY[name] + " pin plus bounded retain-hybrid-copy overlay", + oracle_tokens=expected_ids) + if actual != expected_ids: + raise ValueError(f"native {actual} differs from secondary {expected_ids}: {stem}") + else: + path = args.primary / f"{name}-p{p}-r{repeat}.json" + report = json.loads(path.read_text()) + files[str(path)] = seal(path) + qualify_primary(report, seal(args.fixtures / (name + ".gguf")), + seal(args.config / "config.json"), prompt, repeat) + case.update(oracle="primary pinned vLLM and GGUF plugin", oracle_tokens=report["tokens"]) + if actual != report["tokens"]: + raise ValueError(f"native {actual} differs from primary {report['tokens']}: {stem}") + cases.append(case) + recurrent = [] + for oracle in ("stock", "fork"): + for p in range(2): + baseline = None + for repeat in range(3): + for arm in ("pristine", "overlay"): + prefix = args.secondary / "controls" / f"{oracle}-{arm}-r{repeat}-p{p}" + ids_path = Path(str(prefix) + "-tokens.txt") + scores_path = Path(str(prefix) + "-logits-f32.bin") + actual = (tokens(ids_path), logits(scores_path)) + if baseline is None: + baseline = actual + elif actual != baseline: + raise ValueError(f"recurrent model changed across repeats or overlay: {prefix}") + files[str(ids_path)] = seal(ids_path) + files[str(scores_path)] = seal(scores_path) + recurrent.append({"oracle": oracle, "arm": arm, "repeat": repeat, + "prompt_index": p, "tokens": actual[0], + "logits_sha256": files[str(scores_path)]["sha256"]}) + # Token qualification and memory instrumentation are independent executions. + # Both gates remain required. The memory capture uses the identical Q4_0 + # workload and must preserve its completion, model, and configuration bytes. + memory_report = json.loads(args.primary_memory.read_text()) + qualify_primary(memory_report, seal(args.fixtures / "Q4_0.gguf"), + seal(args.config / "config.json"), PROMPTS[0], 0) + if memory_report["tokens"] != first[("Q4_0", 0)][0]: + raise ValueError("primary memory capture changed the qualified Q4_0 workload") + measured_memory = primary_memory(memory_report) + files[str(args.primary_memory)] = seal(args.primary_memory) + return {"status": "PASS", "comparison": "all native quantized and dense completions, followed by identical primary or named secondary model tokens", + "native_engines": 228, "primary_engines": 96, "secondary_overlay_engines": 12, + "q8_k_decoder_only_native_engines": 12, "physical_kv_payload_bytes": 65536, + "logical_native_primary_limit": 64, "primary_reserved_null_bytes": 4096, + "native_reserved_null_bytes": 4096, "primary_actual_memory": measured_memory, + "identical_tool_backing_allocations": "separate model trace audit is required", + "recurrent_pristine_overlay_engines": recurrent, + "cases": cases, "sealed_outputs": files} + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + for name in ("native", "native-log", "primary", "primary-memory", "secondary", "fixtures", "config", "output"): + parser.add_argument("--" + name, type=Path, required=True) + args = parser.parse_args() + try: + report = compare(args) + except (ValueError, KeyError, FileNotFoundError) as error: + args.output.write_text(json.dumps({"status": "FAIL", "error": str(error)}, indent=2) + "\n") + raise + args.output.write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps({key: value for key, value in report.items() if key not in ("cases", "sealed_outputs")})) + + +if __name__ == "__main__": + main() diff --git a/tools/rocm_quant_gather/native_model.cpp b/tools/rocm_quant_gather/native_model.cpp new file mode 100644 index 0000000000..2b66721908 --- /dev/null +++ b/tools/rocm_quant_gather/native_model.cpp @@ -0,0 +1,86 @@ +// Public-ABI capture for BACKEND-ROCM-QUANT-GATHER (#3093). +// The workload matches test_capi_rocm_embedding_quant. This runner accepts the +// original exported GGUF so native and oracle traces load identical bytes. +#include "vllm.h" + +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { +struct Capture { + std::vector logits; + int calls = 0; + bool invalid = false; +}; + +void Logits(const int32_t*, int32_t, float* values, int32_t count, void* opaque) { + auto& capture = *static_cast(opaque); + if (count != 128 || capture.calls >= 4) { + capture.invalid = true; + return; + } + for (int32_t i = 0; i < count; ++i) { + if (!std::isfinite(values[i])) capture.invalid = true; + capture.logits.push_back(values[i]); + } + ++capture.calls; +} +} // namespace + +int main(int argc, char** argv) { + try { + if (argc != 4) throw std::runtime_error("MODEL_GGUF PROMPT_INDEX OUTPUT_PREFIX"); + const int index = std::stoi(argv[2]); + if (index != 0 && index != 1) throw std::runtime_error("prompt index must be 0 or 1"); + const std::vector prompt = index == 0 ? std::vector{1, 0, 63, 127, 63} + : std::vector{1, 127, 0, 127}; + auto params = vllm_model_params_default(); + params.model_path = argv[1]; + params.device = 0; + params.block_size = 16; + params.num_blocks = 16; + params.max_model_len = 64; + params.max_num_seqs = 1; + params.kv_cache_dtype = "auto"; + vllm_engine* raw = nullptr; + if (vllm_engine_load(¶ms, &raw) != VLLM_OK) throw std::runtime_error(vllm_last_error()); + std::unique_ptr engine(raw, &vllm_engine_free); + Capture capture; + capture.logits.reserve(512); + auto sampling = vllm_sampling_params_default(); + sampling.temperature = 0.0F; + sampling.max_tokens = 4; + sampling.ignore_eos = 1; + sampling.has_seed = 1; + sampling.seed = 0x524F434D; + sampling.logits_processor = Logits; + sampling.logits_processor_user_data = &capture; + int32_t tokens[4]{}; + int32_t count = 0; + if (vllm_complete_tokens(engine.get(), prompt.data(), static_cast(prompt.size()), + &sampling, tokens, 4, &count, nullptr) != VLLM_OK) + throw std::runtime_error(vllm_last_error()); + if (count != 4 || capture.calls != 4 || capture.invalid) + throw std::runtime_error("incomplete or nonfinite public completion"); + std::ofstream ids(std::string(argv[3]) + "-tokens.txt"); + for (const int32_t token : tokens) ids << token << '\n'; + std::ofstream logits(std::string(argv[3]) + "-logits-f32.bin", std::ios::binary); + logits.write(reinterpret_cast(capture.logits.data()), + static_cast(capture.logits.size() * sizeof(float))); + if (!ids || !logits) throw std::runtime_error("capture write failed"); + std::printf("PUBLIC model=%s prompt=%d physical_blocks=16 block_size=16 logical_max=64 tokens=", + argv[1], index); + for (const int32_t token : tokens) std::printf(" %d", token); + std::puts(""); + return 0; + } catch (const std::exception& error) { + std::fprintf(stderr, "PUBLIC FAILED: %s\n", error.what()); + return 1; + } +} diff --git a/tools/rocm_quant_gather/primary.py b/tools/rocm_quant_gather/primary.py new file mode 100644 index 0000000000..560f7bd16b --- /dev/null +++ b/tools/rocm_quant_gather/primary.py @@ -0,0 +1,136 @@ +#!/usr/bin/env python3 +"""Capture the pinned plugin embedding method without downloading fixtures. + +The embedding parameters come from plugin d4c1f0d tests/test_kernels.py:102-124. +The shared local ABI flattens the original 2-D IDs. This harness restores that +shape for the plugin and keeps the upstream atol=0.01, rtol=0.04 comparison. +IQ1_M and f16 output exceed the shared loader and Embedding contracts. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +from pathlib import Path +import re +import sys + + +def seal(path: Path) -> dict: + return {"bytes": path.stat().st_size, "sha256": hashlib.sha256(path.read_bytes()).hexdigest()} + + +def export_upstream(directory: Path, output: Path, spec: Path) -> None: + """Keep every tensor from each complete, hash-verified upstream fixture.""" + import gguf + import numpy as np + + files = re.findall(r"\| `(Quant_[^`]+\.gguf)` \| (\d+) \| `([0-9a-f]{64})` \|", spec.read_text()) + if len(files) != 32: + raise ValueError("the committed spec must name all 32 applicable upstream fixtures") + # Verify the complete corpus before exporting any selected test input. + for filename, size, digest in files: + if seal(directory / filename) != {"bytes": int(size), "sha256": digest}: + raise ValueError(f"upstream fixture hash or size mismatch: {filename}") + output.mkdir(parents=True, exist_ok=True) + cases = [] + sources = {} + for filename, _, _ in files: + path = directory / filename + sources[filename] = seal(path) + hidden = int(path.stem.rsplit("_", 1)[1]) + for index, tensor in enumerate(gguf.GGUFReader(path).tensors): + packed = np.asarray(tensor.data, dtype=np.uint8) + rows = int(tensor.shape[1]) + if len(tensor.shape) != 2 or int(tensor.shape[0]) != hidden: + raise ValueError(f"unexpected upstream tensor geometry: {filename}:{tensor.name}") + name = f"{path.stem}-{index}" + (output / f"{name}.packed").write_bytes(packed.tobytes()) + (output / f"{name}.ids").write_bytes(np.array([0, rows - 1, rows // 2, 0], dtype=" None: + import gguf + import numpy as np + import torch + import vllm_gguf_plugin.ops as ops + from vllm_gguf_plugin.quantization.vocal_embeds import apply_gguf_embedding + + output.mkdir(parents=True, exist_ok=True) + manifest = json.loads(manifest_path.read_text()) + cases = manifest["cases"] + if selected is not None: + cases = [case for case in cases if case["name"] == selected] + if len(cases) != 1: + raise ValueError("--case must name exactly one manifest entry") + report = {"version": 1, "manifest": seal(manifest_path), "torch": torch.__version__, + "torch_git": torch.version.git_version, "hip": torch.version.hip, + "device": torch.cuda.get_device_name(0), + "architecture": torch.cuda.get_device_properties(0).gcnArchName, + "method": "vllm_gguf_plugin.quantization.vocal_embeds.apply_gguf_embedding", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "cases": [], "exclusions": {"types": ["IQ1_M", "Q8_K", "MXFP4", "IQ1_XXXS"], + "output_dtypes": ["f16"]}} + try: + for case in cases: + if case["type"] in (15, 39, 66): + continue # These three codecs have the separately pinned secondary oracle. + packed_path = manifest_path.parent / case["packed"] + ids_path = manifest_path.parent / case["ids"] + dtype = gguf.GGMLQuantizationType(case["type"]) + row_bytes = case["width"] // gguf.GGML_QUANT_SIZES[dtype][0] * gguf.GGML_QUANT_SIZES[dtype][1] + packed = np.frombuffer(packed_path.read_bytes(), dtype=np.uint8).copy().reshape(case["rows"], row_bytes) + host_ids = np.frombuffer(ids_path.read_bytes(), dtype=" None: + parser = argparse.ArgumentParser(description=__doc__) + sub = parser.add_subparsers(dest="command", required=True) + export = sub.add_parser("export-upstream") + export.add_argument("directory", type=Path) + export.add_argument("output", type=Path) + export.add_argument("--spec", type=Path, default=Path(__file__).resolve().parents[2] / ".agents/specs/rocm-quant-gather.md") + run = sub.add_parser("capture") + run.add_argument("manifest", type=Path) + run.add_argument("output", type=Path) + run.add_argument("--case") + args = parser.parse_args() + if args.command == "export-upstream": + export_upstream(args.directory, args.output, args.spec) + else: + capture(args.manifest, args.output, args.case) + + +if __name__ == "__main__": + main() diff --git a/tools/rocm_quant_gather/primary_model.py b/tools/rocm_quant_gather/primary_model.py new file mode 100644 index 0000000000..76e4adb989 --- /dev/null +++ b/tools/rocm_quant_gather/primary_model.py @@ -0,0 +1,159 @@ +#!/usr/bin/env python3 +"""Run one bounded model request through the pinned plugin production path. + +Each process makes one fresh engine. Repeat for both prompt indices and all +three repeats only after the first model load succeeds. A refusal is recorded +with its traceback and a nonzero exit; operation parity cannot replace it. +""" + +from __future__ import annotations + +import argparse +import importlib.metadata +import json +from pathlib import Path +import traceback + +from primary import seal + + +def capture_cache_memory(worker): + """Read the executing worker's cache views and allocator blocks. + + This callback observes both runner versions at e126687a9a. It does not + replace an allocator or change a tensor. rocprofv3 records the backing + allocations independently on the native and primary processes. + """ + import torch + import json + + torch.cuda.synchronize() + runner = worker.model_runner + snapshot = torch.cuda.memory_snapshot() + tensors = [] + for index, tensor in enumerate(runner.kv_caches): + if not isinstance(tensor, torch.Tensor): + raise RuntimeError("bounded full-attention cache is not a tensor") + storage = tensor.untyped_storage() + address = storage.data_ptr() + allocation = None + for segment in snapshot: + offset = segment["address"] + for block in segment["blocks"]: + start = block.get("address", offset) + if start <= address < start + block["size"]: + allocation = { + "block_address": start, "block_bytes": block["size"], + "requested_bytes": block.get("requested_size"), + "state": block["state"], + "segment_address": segment["address"], + "segment_bytes": segment["total_size"], + "segment_allocated_bytes": segment["allocated_size"], + "segment_active_bytes": segment["active_size"], + } + offset = start + block["size"] + tensors.append({ + "index": index, "dtype": str(tensor.dtype), + "shape": list(tensor.shape), "stride_elements": list(tensor.stride()), + "element_bytes": tensor.element_size(), "numel": tensor.numel(), + "payload_bytes": tensor.numel() * tensor.element_size(), + "storage_bytes": storage.nbytes(), "storage_address": address, + "storage_offset_elements": tensor.storage_offset(), + "contiguous": tensor.is_contiguous(), "device": str(tensor.device), + "allocator": allocation, + }) + if not tensors: + raise RuntimeError("executing worker exposes no full-attention cache") + layout = worker.vllm_config.cache_config.get_resolved_kv_cache_layout() + report = { + "runner_class": type(runner).__module__ + "." + type(runner).__name__, + "physical_blocks": runner.kv_cache_config.num_blocks, + "resolved_layout": layout.name, + "layout_stride_order": list(layout.stride_order), + "resolved_model_dtype": str(runner.dtype), + "cache_tensors": tensors, + "process_allocated_bytes": torch.cuda.memory_allocated(), + "process_reserved_bytes": torch.cuda.memory_reserved(), + "process_peak_allocated_bytes": torch.cuda.max_memory_allocated(), + "measurement": "live tensor views and PyTorch allocator snapshot; backing allocations require rocprofv3", + } + # The resolved layout is an Enum with a tuple value at this pin. Serialize + # a complete JSON value in the worker, so RPC sees only a bounded string. + return json.dumps(report, sort_keys=True) + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("model", type=Path) + parser.add_argument("config", type=Path) + parser.add_argument("output", type=Path) + parser.add_argument("--prompt-index", type=int, choices=(0, 1), default=0) + parser.add_argument("--repeat", type=int, choices=(0, 1, 2), default=0) + args = parser.parse_args() + prompt = ([1, 0, 63, 127, 63], [1, 127, 0, 127])[args.prompt_index] + report = {"model": seal(args.model), "config": seal(args.config / "config.json"), + "primary_pin": "e126687a9a828d513c01a07cd69f025f27d63280", + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "prompt": prompt, "repeat": args.repeat, "tokens": [], + "requested": {"model_dtype": "auto from bfloat16 config", "block_size": 16, + "num_gpu_blocks_override": 16, "max_model_len": 64, + "max_num_seqs": 1, "kv_cache_dtype": "auto", + "seed": 0x524F434D, "greedy": True, "ignore_eos": True, + "max_tokens": 4, "stop": [], "stop_token_ids": []}, + "harness_adaptations": ["pre-tokenized IDs with skip_tokenizer_init=True", + "explicit local HF text config mirrors GGUF geometry", + "development model_class_overrides selects the pinned text class for the plugin architecture key", + "top-level partial_rotary_factor=1.0 preserves GGUF rotary width 64 during HF normalization", + "16 physical KV blocks match secondary 256-cell allocation; one primary null block is reserved; logical limit remains 64", + "no oracle patch or eager-mode override"], + "status": "PENDING"} + args.output.parent.mkdir(parents=True, exist_ok=True) + try: + import torch + import vllm + from vllm import LLM, SamplingParams + from vllm.inputs import TokensPrompt + + report.update(torch=torch.__version__, torch_git=torch.version.git_version, + hip=torch.version.hip, vllm=vllm.__version__, + plugin=importlib.metadata.version("vllm-gguf-plugin")) + model = LLM(model=str(args.model), hf_config_path=str(args.config), + model_class_overrides={"Qwen3_5ForConditionalGeneration": + "vllm.model_executor.models.qwen3_5:Qwen3_5ForCausalLM"}, + hf_overrides={"partial_rotary_factor": 1.0}, + skip_tokenizer_init=True, dtype="auto", block_size=16, + num_gpu_blocks_override=16, max_model_len=64, max_num_seqs=1, + kv_cache_dtype="auto", seed=0x524F434D) + config = model.llm_engine.vllm_config + report["resolved_model_dtype"] = str(config.model_config.dtype) + report["resolved_cache_config"] = str(config.cache_config) + report["memory_before_generation"] = [json.loads(item) for item in + model.collective_rpc(capture_cache_memory, timeout=30)] + report["cache_capacity_contract"] = {"block_size": 16, "physical_blocks": 16, + "physical_cells": 256, "reserved_null_blocks": 1, "usable_blocks": 15, + "usable_cells": 240, "logical_max_model_len": 64, "bf16_kv_payload_bytes": 65536, + "source": "vllm/v1/core/kv_cache_utils.py:2304-2309 at e126687a9a"} + params = SamplingParams(temperature=0.0, max_tokens=4, ignore_eos=True, + seed=0x524F434D, stop=[], stop_token_ids=[]) + output = model.generate([TokensPrompt(prompt_token_ids=prompt)], params, use_tqdm=False) + if len(output) != 1 or list(output[0].prompt_token_ids) != prompt: + raise RuntimeError("the oracle changed the request or prompt IDs") + report["tokens"] = list(output[0].outputs[0].token_ids) + report["memory_after_generation"] = [json.loads(item) for item in + model.collective_rpc(capture_cache_memory, timeout=30)] + if len(report["tokens"]) != 4: + raise RuntimeError("the oracle did not generate exactly four tokens") + report["status"] = "EXECUTED; token comparison remains required" + print(json.dumps(report, sort_keys=True), flush=True) + except Exception as error: + report["status"] = "PENDING: primary model refused before qualification" + report["exception_type"] = type(error).__name__ + report["exception"] = str(error) + report["traceback"] = traceback.format_exc() + raise + finally: + args.output.write_text(json.dumps(report, indent=2) + "\n") + + +if __name__ == "__main__": + main() diff --git a/tools/rocm_quant_gather/primary_trace.py b/tools/rocm_quant_gather/primary_trace.py new file mode 100644 index 0000000000..399a7ec11d --- /dev/null +++ b/tools/rocm_quant_gather/primary_trace.py @@ -0,0 +1,79 @@ +#!/usr/bin/env python3 +"""Trace only the pinned production gather; dense controls run in primary.py. + +Capture both output dtypes on the same packed table and IDs as the native tool. +Allocator counters are PyTorch telemetry, separate from rocprofv3 trace evidence. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path + +from primary import seal + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("manifest", type=Path) + parser.add_argument("output", type=Path) + args = parser.parse_args() + + import gguf + import numpy as np + import torch + import vllm_gguf_plugin.ops as ops + from vllm_gguf_plugin.quantization.vocal_embeds import apply_gguf_embedding + + args.output.mkdir(parents=True, exist_ok=True) + manifest = json.loads(args.manifest.read_text()) + report = {"manifest": seal(args.manifest), "torch": torch.__version__, + "torch_git": torch.version.git_version, "hip": torch.version.hip, + "plugin_pin": "d4c1f0d082fc7cd4350da56689109a01c1f29d6c", + "method": "vllm_gguf_plugin.quantization.vocal_embeds.apply_gguf_embedding", + "scope": "production method only; no full-table dense control in this process", + "allocator_counters": "PyTorch telemetry, not profiler allocation events", + "cases": []} + for case in manifest["cases"]: + quant_type = int(case["type"]) + if quant_type in (15, 39, 66): + raise ValueError("the common trace manifest must contain only primary codecs") + block_elements, block_bytes = gguf.GGML_QUANT_SIZES[gguf.GGMLQuantizationType(quant_type)] + row_bytes = case["width"] // block_elements * block_bytes + packed_path = args.manifest.parent / case["packed"] + ids_path = args.manifest.parent / case["ids"] + packed = np.frombuffer(packed_path.read_bytes(), dtype=np.uint8).copy().reshape(case["rows"], row_bytes) + host_ids = np.frombuffer(ids_path.read_bytes(), dtype=" +#include +#include +#include +#include + +int main(int argc, char** argv) { + try { + if (argc != 2) throw std::runtime_error("OUTPUT_F32"); + ggml_init_params params{4 * 1024 * 1024, nullptr, true}; + std::unique_ptr ctx(ggml_init(params), ggml_free); + if (!ctx) throw std::runtime_error("ggml_init failed"); + auto* state = ggml_new_tensor_2d(ctx.get(), GGML_TYPE_F32, 4, 3); + auto* copy = ggml_new_tensor_1d(ctx.get(), GGML_TYPE_I32, 3); + auto* delta = ggml_new_tensor_2d(ctx.get(), GGML_TYPE_F32, 4, 3); + ggml_set_name(copy, "s_copy"); + ggml_set_input(state); + ggml_set_input(copy); + ggml_set_input(delta); + auto* next = ggml_add(ctx.get(), ggml_get_rows(ctx.get(), state, copy), delta); + ggml_set_output(next); + auto* graph = ggml_new_graph(ctx.get()); + ggml_build_forward_expand(graph, next); + std::unique_ptr backend( + ggml_backend_cpu_init(), ggml_backend_free); + if (!backend) throw std::runtime_error("CPU backend initialization failed"); + ggml_backend_t backends[] = {backend.get()}; + std::unique_ptr sched( + ggml_backend_sched_new(backends, nullptr, 1, GGML_DEFAULT_GRAPH_SIZE, false, true), + ggml_backend_sched_free); + if (!sched || !ggml_backend_sched_alloc_graph(sched.get(), graph)) + throw std::runtime_error("recurrent graph allocation failed"); + if (!copy->buffer || !copy->data) throw std::runtime_error("consumed recurrent copy has no storage"); + const void* copy_storage = copy->data; + std::array values{0,1,2,3, 10,11,12,13, 20,21,22,23}; + const std::array increment{1,2,3,4, 1,2,3,4, 1,2,3,4}; + const std::array, 3> indices{{{2,0,1}, {1,2,0}, {2,2,0}}}; + std::ofstream output(argv[1], std::ios::binary); + for (size_t step = 0; step < indices.size(); ++step) { + std::array expected{}; + for (size_t row = 0; row < 3; ++row) + for (size_t col = 0; col < 4; ++col) + expected[row * 4 + col] = values[static_cast(indices[step][row]) * 4 + col] + increment[row * 4 + col]; + ggml_backend_tensor_set(state, values.data(), 0, sizeof(values)); + ggml_backend_tensor_set(copy, indices[step].data(), 0, sizeof(indices[step])); + ggml_backend_tensor_set(delta, increment.data(), 0, sizeof(increment)); + if (ggml_backend_sched_graph_compute(sched.get(), graph) != GGML_STATUS_SUCCESS) + throw std::runtime_error("reused recurrent graph failed"); + ggml_backend_tensor_get(next, values.data(), 0, sizeof(values)); + if (values != expected || copy->data != copy_storage) + throw std::runtime_error("recurrent state or reused copy allocation changed"); + output.write(reinterpret_cast(values.data()), sizeof(values)); + std::printf("RECURRENT step=%zu consumed_s_copy=true reused_graph=true values=", step); + for (float value : values) std::printf(" %.0f", value); + std::puts(""); + } + if (!output) throw std::runtime_error("cannot write recurrent state evidence"); + return 0; + } catch (const std::exception& error) { + std::fprintf(stderr, "RECURRENT FAILED: %s\n", error.what()); + return 1; + } +} diff --git a/tools/rocm_quant_gather/recurrent_model.py b/tools/rocm_quant_gather/recurrent_model.py new file mode 100644 index 0000000000..d6e4a3f6ca --- /dev/null +++ b/tools/rocm_quant_gather/recurrent_model.py @@ -0,0 +1,108 @@ +#!/usr/bin/env python3 +"""Create a separate two-layer recurrent control without changing gate fixtures. + +The existing dense fixture supplies the vocabulary, embedding, output, and +full-attention layer. A deterministic recurrent layer precedes that layer. +Shapes and names follow both pinned secondary models/qwen35.cpp loaders. +This model is a control for the allocation overlay, not a native token gate. +""" + +from __future__ import annotations + +import argparse +import hashlib +import json +from pathlib import Path +import struct + +import gguf +import numpy as np + + +class ControlWriter(gguf.GGUFWriter): + def _pack_val(self, value, value_type, add_vtype, sub_type=None): + # Preserve the original zero-length BPE merge array. The pinned Python + # writer refuses empty arrays although both pinned C++ readers accept it. + if value_type == gguf.GGUFValueType.ARRAY and len(value) == 0: + if sub_type is None: + raise RuntimeError("empty control metadata needs its original element type") + prefix = struct.pack(" None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("original_dense", type=Path) + parser.add_argument("output", type=Path) + args = parser.parse_args() + if args.output.exists(): + raise RuntimeError("control output must be new") + source_hash = hashlib.sha256(args.original_dense.read_bytes()).hexdigest() + reader = gguf.GGUFReader(args.original_dense) + writer = ControlWriter(str(args.output), "qwen35") + for field in reader.fields.values(): + if field.name.startswith("GGUF.") or field.name == "general.architecture": + continue + value = field.contents() + if field.name == "general.name": + value = "rocm-gather-recurrent-allocation-control" + if field.name in ("qwen35.block_count", "qwen35.full_attention_interval"): + value = 2 + subtype = field.types[-1] if len(field.types) > 1 else None + if field.types[0] == gguf.GGUFValueType.ARRAY and len(value) == 0: + subtype = gguf.GGUFValueType(int(field.parts[3][0])) + writer.add_key_value(field.name, value, field.types[0], sub_type=subtype) + copied = [] + for tensor in reader.tensors: + name = tensor.name.replace("blk.0.", "blk.1.") + writer.add_tensor(name, tensor.data, raw_dtype=tensor.tensor_type) + copied.append(name) + + # The control stays deterministic across NumPy versions with integer data. + counter = 0 + added = {} + + def add(name, shape, *, constant=None): + nonlocal counter + count = int(np.prod(shape)) + if constant is None: + integers = ((np.arange(count, dtype=np.int64) + counter) * 1103515245 + 12345) % 65536 + data = (integers.astype(np.float32) - 32768) / np.float32(2097152) + counter += count + else: + data = np.full(count, constant, dtype=np.float32) + data = data.reshape(shape) + writer.add_tensor(name, data) + added[name] = {"shape": list(shape), "dtype": "float32", + "sha256": hashlib.sha256(data.tobytes()).hexdigest()} + + add("blk.0.attn_norm.weight", (256,), constant=1) + add("blk.0.post_attention_norm.weight", (256,), constant=1) + add("blk.0.attn_qkv.weight", (192, 256)) + add("blk.0.attn_gate.weight", (64, 256)) + add("blk.0.ssm_conv1d.weight", (192, 4)) + add("blk.0.ssm_dt.bias", (1,), constant=1) + add("blk.0.ssm_a", (1,), constant=-1) + add("blk.0.ssm_beta.weight", (1, 256)) + add("blk.0.ssm_alpha.weight", (1, 256)) + add("blk.0.ssm_norm.weight", (64,), constant=1) + add("blk.0.ssm_out.weight", (256, 64)) + for name in ("ffn_gate", "ffn_up", "ffn_down"): + add("blk.0." + name + ".weight", (256, 256)) + writer.write_header_to_file() + writer.write_kv_data_to_file() + writer.write_tensors_to_file() + writer.close() + if hashlib.sha256(args.original_dense.read_bytes()).hexdigest() != source_hash: + raise RuntimeError("original gate fixture changed") + report = {"purpose": "separate actual recurrent-layer control", "layers": ["recurrent", "full_attention"], + "original_sha256": source_hash, "control_sha256": hashlib.sha256(args.output.read_bytes()).hexdigest(), + "copied_tensors": copied, "added_tensors": added, + "source": "stock 10bf611e and fork 36fe8e1c models/qwen35.cpp::load_arch_tensors"} + args.output.with_suffix(".json").write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps({"control_sha256": report["control_sha256"], "bytes": args.output.stat().st_size})) + + +if __name__ == "__main__": + main() diff --git a/tools/rocm_quant_gather/secondary.cpp b/tools/rocm_quant_gather/secondary.cpp new file mode 100644 index 0000000000..9e72d85e91 --- /dev/null +++ b/tools/rocm_quant_gather/secondary.cpp @@ -0,0 +1,159 @@ +// BACKEND-ROCM-QUANT-GATHER (#3093), external oracle harness. +// Stock pin 10bf611e533d81f739128304991c5e133c6aebd8 and IQ1_XXXS fork pin +// 36fe8e1cc7f2b3b8c92fdda0ab07600141921786. No decoder is transcribed here. +// GET_ROWS enters the upstream CPU graph. Q8_K calls its upstream decoder +// directly because the stock GET_ROWS dispatch explicitly excludes that type. +#include "ggml.h" +#include "ggml-cpu.h" +#include "ggml-quants.h" +#include "llama.h" + +#include +#include +#include +#include +#include +#include +#include +#include +#include + +namespace { + +std::vector Read(const char* path) { + std::ifstream file(path, std::ios::binary | std::ios::ate); + if (!file) throw std::runtime_error(std::string("cannot read ") + path); + const auto size = file.tellg(); + if (size < 0) throw std::runtime_error("cannot determine file size"); + std::vector bytes(static_cast(size)); + file.seekg(0); + file.read(bytes.data(), static_cast(bytes.size())); + if (!file) throw std::runtime_error("incomplete input read"); + return bytes; +} + +void Write(const char* path, const void* data, size_t bytes) { + std::ofstream file(path, std::ios::binary); + file.write(static_cast(data), static_cast(bytes)); + if (!file) throw std::runtime_error(std::string("cannot write ") + path); +} + +void Gather(int argc, char** argv) { + if (argc != 8) throw std::runtime_error("gather TYPE ROWS WIDTH PACKED IDS_I32 OUT_F32"); + const auto type = static_cast(std::stoi(argv[2])); + const int64_t rows = std::stoll(argv[3]); + const int64_t width = std::stoll(argv[4]); + const auto packed = Read(argv[5]); + const auto ids_bytes = Read(argv[6]); + if (rows <= 0 || width <= 0 || width % ggml_blck_size(type) != 0 || ids_bytes.size() % 4 != 0) + throw std::runtime_error("invalid gather geometry"); + const size_t row_bytes = ggml_row_size(type, width); + if (packed.size() != static_cast(rows) * row_bytes) throw std::runtime_error("packed byte count mismatch"); + std::vector ids(ids_bytes.size() / 4); + std::memcpy(ids.data(), ids_bytes.data(), ids_bytes.size()); + for (auto id : ids) if (id < 0 || id >= rows) throw std::runtime_error("oracle input ID out of range"); + std::vector output(ids.size() * static_cast(width)); + if (type == GGML_TYPE_Q8_K) { + for (size_t token = 0; token < ids.size(); ++token) { + dequantize_row_q8_K(reinterpret_cast( + packed.data() + static_cast(ids[token]) * row_bytes), + output.data() + token * static_cast(width), width); + } + std::puts("ORACLE operation=dequantize_row_q8_K adaptation=stock_GET_ROWS_excludes_Q8_K"); + } else { + ggml_init_params params{}; + params.mem_size = packed.size() + output.size() * sizeof(float) + ids_bytes.size() + 8 * 1024 * 1024; + params.no_alloc = false; + std::unique_ptr context(ggml_init(params), &ggml_free); + if (!context) throw std::runtime_error("ggml_init failed"); + ggml_tensor* table = ggml_new_tensor_2d(context.get(), type, width, rows); + ggml_tensor* indices = ggml_new_tensor_1d(context.get(), GGML_TYPE_I32, static_cast(ids.size())); + std::memcpy(table->data, packed.data(), packed.size()); + std::memcpy(indices->data, ids.data(), ids_bytes.size()); + ggml_tensor* selected = ggml_get_rows(context.get(), table, indices); + ggml_cgraph* graph = ggml_new_graph(context.get()); + ggml_build_forward_expand(graph, selected); + if (ggml_graph_compute_with_ctx(context.get(), graph, 4) != GGML_STATUS_SUCCESS) + throw std::runtime_error("upstream GET_ROWS graph failed"); + if (selected->type != GGML_TYPE_F32 || ggml_nbytes(selected) != output.size() * sizeof(float)) + throw std::runtime_error("unexpected GET_ROWS output format"); + std::memcpy(output.data(), selected->data, output.size() * sizeof(float)); + std::puts("ORACLE operation=ggml_get_rows graph=ggml_graph_compute_with_ctx threads=4 output=f32"); + } + Write(argv[7], output.data(), output.size() * sizeof(float)); + std::printf("ORACLE type=%s rows=%lld width=%lld selected=%zu table_bytes=%zu output_bytes=%zu\n", + ggml_type_name(type), static_cast(rows), static_cast(width), + ids.size(), packed.size(), output.size() * sizeof(float)); +} + +void Model(int argc, char** argv, bool capture) { + const int first_token = capture ? 4 : 3; + if (argc <= first_token) throw std::runtime_error("model[-capture] MODEL_GGUF [OUTPUT_PREFIX] TOKEN_ID..."); + std::vector prompt; + for (int i = first_token; i < argc; ++i) prompt.push_back(std::stoi(argv[i])); + llama_backend_init(); + auto model_params = llama_model_default_params(); + model_params.n_gpu_layers = 0; + std::unique_ptr model( + llama_model_load_from_file(argv[2], model_params), &llama_model_free); + if (!model) throw std::runtime_error("pinned oracle refused bounded GGUF model"); + auto context_params = llama_context_default_params(); + context_params.n_ctx = 64; + context_params.n_batch = 64; + context_params.n_ubatch = 64; + context_params.n_seq_max = 1; + context_params.n_threads = 4; + context_params.n_threads_batch = 4; + context_params.type_k = GGML_TYPE_BF16; + context_params.type_v = GGML_TYPE_BF16; + std::unique_ptr context( + llama_init_from_model(model.get(), context_params), &llama_free); + if (!context) throw std::runtime_error("pinned oracle refused bf16 KV context"); + std::unique_ptr sampler(llama_sampler_init_greedy(), &llama_sampler_free); + auto batch = llama_batch_get_one(prompt.data(), static_cast(prompt.size())); + std::vector generated; + std::vector logits; + const int32_t vocab = llama_vocab_n_tokens(llama_model_get_vocab(model.get())); + if (vocab != 128) throw std::runtime_error("bounded model vocabulary changed"); + generated.reserve(4); + logits.reserve(4 * static_cast(vocab)); + for (int step = 0; step < 4; ++step) { + if (llama_decode(context.get(), batch) != 0) throw std::runtime_error("pinned oracle decode failed"); + const float* values = llama_get_logits_ith(context.get(), -1); + if (!values) throw std::runtime_error("oracle returned no logits"); + for (int32_t i = 0; i < vocab; ++i) { + if (!std::isfinite(values[i])) throw std::runtime_error("oracle returned nonfinite logits"); + logits.push_back(values[i]); + } + const llama_token token = llama_sampler_sample(sampler.get(), context.get(), -1); + generated.push_back(token); + batch = llama_batch_get_one(&generated.back(), 1); + } + std::printf("ORACLE model=%s requested_context=64 physical_context=%u kv_k=bf16 kv_v=bf16 concurrency=1 greedy=true ignore_eos=true tokens=", + argv[2], llama_n_ctx(context.get())); + for (auto token : generated) std::printf(" %d", token); + std::printf("\n"); + std::printf("ORACLE system=%s\n", llama_print_system_info()); + if (capture) { + const std::string prefix(argv[3]); + Write((prefix + "-logits-f32.bin").c_str(), logits.data(), logits.size() * sizeof(float)); + std::string ids; + for (auto token : generated) ids += std::to_string(token) + "\n"; + Write((prefix + "-tokens.txt").c_str(), ids.data(), ids.size()); + } +} + +} // namespace + +int main(int argc, char** argv) { + try { + if (argc > 1 && std::string(argv[1]) == "gather") Gather(argc, argv); + else if (argc > 1 && std::string(argv[1]) == "model") Model(argc, argv, false); + else if (argc > 1 && std::string(argv[1]) == "model-capture") Model(argc, argv, true); + else throw std::runtime_error("choose gather, model, or model-capture"); + return 0; + } catch (const std::exception& error) { + std::fprintf(stderr, "ORACLE FAILED: %s\n", error.what()); + return 1; + } +} diff --git a/tools/rocm_quant_gather/secondary_overlay.py b/tools/rocm_quant_gather/secondary_overlay.py new file mode 100644 index 0000000000..39c0bca77f --- /dev/null +++ b/tools/rocm_quant_gather/secondary_overlay.py @@ -0,0 +1,84 @@ +#!/usr/bin/env python3 +"""Prepare a row-owned secondary source copy with one retained input leaf. + +The pristine archive remains unchanged. This overlay qualifies only this row's +bounded synthetic model and never advances a global oracle pin. +""" + +from __future__ import annotations + +import argparse +import difflib +import hashlib +import json +from pathlib import Path +import shutil + + +PINS = { + "stock": ("10bf611e533d81f739128304991c5e133c6aebd8", + "bc1c497c4da0951a4f98de50e397bcb632b2e40d4f24b8c58203c964d4d6183b"), + "fork": ("36fe8e1cc7f2b3b8c92fdda0ab07600141921786", + "e7d4915cc66bc6187f42ad016f9b1288f8e1702e15da4e63d9862cf8afcb8267"), +} + + +def digest(path: Path) -> str: + return hashlib.sha256(path.read_bytes()).hexdigest() + + +def inventory(root: Path) -> dict[str, str]: + return {str(path.relative_to(root)): digest(path) + for path in sorted(root.rglob("*")) + if path.is_file() and ".git" not in path.relative_to(root).parts} + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("oracle", choices=PINS) + parser.add_argument("pristine", type=Path) + parser.add_argument("overlay", type=Path) + parser.add_argument("evidence", type=Path) + args = parser.parse_args() + pin, graph_sha = PINS[args.oracle] + relative = "src/llama-graph.cpp" + original = args.pristine / relative + if digest(original) != graph_sha: + raise RuntimeError("pristine graph source differs from the recorded pin") + if args.overlay.exists() or args.evidence.exists(): + raise RuntimeError("overlay and evidence destinations must be new") + before = inventory(args.pristine) + old = original.read_text() + anchor = "llm_graph_input_mem_hybrid * llm_graph_context::build_inp_mem_hybrid() const {" + if old.count(anchor) != 1: + raise RuntimeError("hybrid builder anchor is not unique") + start = old.index(anchor) + end = old.index("\n}", start) + function = old[start:end] + line = " auto inp_rs = build_rs_inp_impl (ctx0, ubatch, mctx_cur->get_recr());\n" + if function.count(line) != 1: + raise RuntimeError("recurrent copy construction is not unique") + replacement = function.replace(line, line + " ggml_build_forward_expand(gf, inp_rs->s_copy);\n") + new = old[:start] + replacement + old[end:] + shutil.copytree(args.pristine, args.overlay, symlinks=True, + ignore=shutil.ignore_patterns(".git")) + (args.overlay / relative).write_text(new) + after = inventory(args.overlay) + changed = [path for path in sorted(set(before) | set(after)) + if before.get(path) != after.get(path)] + if changed != [relative] or inventory(args.pristine) != before: + raise RuntimeError("overlay changed an unrelated file or the pristine source") + args.evidence.mkdir(parents=True) + patch = args.evidence / "retain-hybrid-copy.patch" + patch.write_text("".join(difflib.unified_diff(old.splitlines(True), new.splitlines(True), + fromfile="a/" + relative, tofile="b/" + relative))) + report = {"oracle": args.oracle, "pristine_pin": pin, "scope": "bounded synthetic model only", + "pristine": str(args.pristine.resolve()), "overlay": str(args.overlay.resolve()), + "pristine_files": before, "overlay_files": after, "changed_files": changed, + "patch_sha256": digest(patch), "pristine_restoration_verified": True} + (args.evidence / "source-manifest.json").write_text(json.dumps(report, indent=2) + "\n") + print(json.dumps({"pin": pin, "changed": changed, "patch_sha256": digest(patch)})) + + +if __name__ == "__main__": + main() diff --git a/tools/rocm_quant_gather/stock_shapes.py b/tools/rocm_quant_gather/stock_shapes.py new file mode 100644 index 0000000000..62d4421fbc --- /dev/null +++ b/tools/rocm_quant_gather/stock_shapes.py @@ -0,0 +1,47 @@ +#!/usr/bin/env python3 +"""Export applicable pinned stock GET_ROWS geometry from the frozen corpus. + +Stock b10451 tests/test-backend-ops.cpp:8371-8376 uses width256, rows5, +four IDs, batch1 and a contiguous view. Keep those dimensions for the three +secondary formats. Fix IDs to first/last/middle/repeated and reuse recorded +packed bytes, adapting upstream's unrecorded random values for reproduction. +Batched tables and strided IDs are refused through the shared local ABI. +""" + +from __future__ import annotations + +import argparse +import json +from pathlib import Path +import struct + +from primary import seal + + +def main() -> None: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("manifest", type=Path) + parser.add_argument("output", type=Path) + args = parser.parse_args() + args.output.mkdir(parents=True, exist_ok=True) + original = json.loads(args.manifest.read_text()) + result = {"version": 1, "source": "stock-GET_ROWS-applicable-shapes", + "source_manifest": seal(args.manifest), "cases": []} + for case in original["cases"]: + if case["type"] not in (15, 39, 66) or case["width"] != 256: + continue + packed_path = args.manifest.parent / case["packed"] + packed = packed_path.read_bytes() + if case["rows"] < 5 or len(packed) % case["rows"] != 0: + raise ValueError("source table cannot supply five complete rows") + exported = dict(case, rows=5, source_packed=seal(packed_path)) + (args.output / exported["packed"]).write_bytes(packed[:5 * (len(packed) // case["rows"])]) + (args.output / exported["ids"]).write_bytes(struct.pack("<4i", 0, 4, 2, 0)) + result["cases"].append(exported) + if len(result["cases"]) != 3: + raise ValueError("source manifest must supply all three secondary codecs") + (args.output / "manifest.json").write_text(json.dumps(result, indent=2) + "\n") + + +if __name__ == "__main__": + main()