Repository navigation
engine#2138 -- AMD pin 9bf5871 merged into 1bit/hrx-vulkan-patched - #327
Conversation
…org#31239) * vulkan: allow dim 3 broadcast in MUL_MAT supports_op rejected every MUL_MAT whose src0 and src1 differ in ne3, although mul_mm, mul_mmq and mul_mat_vec already broadcast src0 over dim 3 through broadcast3. Such a matmul then ran on the CPU with a copy of the weight, and the split layout changed with the number of streams in the ubatch. Keep the strict check for MUL_MAT_ID only, and route the broadcast cases away from mul_mat_vec_nc, the one path that requires equal ne3. * tests : harden MUL_MAT with permuted dim 3 broadcast cases Harden the MUL_MAT coverage with a weight broadcast over dim 3 against a permuted per-stream activation, the shape of an MLA output projection with a non unified KV cache, for a single token, several heads and several tokens per stream.
) * tests : speed up tensor data generation and model saving Refactor save_models to create only the model, not a llama_context, which is not needed for saving. Replace the slow std::normal_distribution draw with a splitmix64 + box-muller normal generator. Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * tests : use a shared pool of normal draws for tensor data Generate a pool of normal draws once with std::mt19937 and std::normal_distribution, then have every tensor copy a cyclic slice of it instead of running a per-tensor box-muller. The name-keyed slice selection keeps the draw deterministic. Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * tests : log the size of saved models Report the produced file size in bytes and MB after each save. Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB) * tests : simplify the model save log Put the model name, target and size on one line per saved model. Assisted-by: dsh:llama.cpp/ggml-org/DeepSeek-V4-Flash-Vision-Exp-GGUF (Mac Studio, M2 Ultra, 192GB)
…k shape (ggml-org#31243) * llama : build the k-pool sparse selection independently of the kq_mask shape qwen4exp (QSA) and glm5-next pick cache cells by index and no longer read the kq_mask values, but they still shaped the selection like the dense kq_mask. Build the selection as [n_kv, n_tokens] over the cache cells instead, and split it into the streams of the K view right before the attention. This lets the dense mask change its layout (e.g. per-sequence row lists) without touching the sparse layers. * improve comment Co-authored-by: Georgi Gerganov <ggerganov@gmail.com> * llama : drop the unused build_fused_top_k_mask args, review nits --------- Co-authored-by: Georgi Gerganov <ggerganov@gmail.com>
* model : add LiquidAI/LFM3-1.7B and LiquidAI/LFM3-VL-2B Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> Co-authored-by: Xuan-Son Nguyen <son@huggingface.co> Assisted-by: Claude Opus 5.5 * tests : set the SWA rope base of the LFM3 test model Assisted-by: Claude Opus 5.5 * llama : enable recurrent state rollback for LFM3 Assisted-by: Claude Opus 5.5 * Apply batched suggestions from code review Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> Co-authored-by: Tarek Dakhran <github@dakhran.com> * gguf-py : align the lfm3vl mapping comments Assisted-by: Claude Opus 5.5 * llama : fix CRLF line endings from the review suggestions Assisted-by: Claude Opus 5.5 * tests : add the LFM3 Metal fusion baseline Assisted-by: Claude Opus 5.5 * llama : keep the LFM3-VL depthwise conv kernels in F32 when quantizing Assisted-by: Claude Opus 5.5 --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> Co-authored-by: Xuan-Son Nguyen <son@huggingface.co>
…ml-org#31240) Signed-off-by: Mengsheng Wu <mengshengwu@outlook.com>
…l-org#31256) * common : support deferred tools and tool_reference content parts Assisted-by: Claude Opus 5.5 * common : credit the deferred tools co-author Co-authored-by: Aldehir Rojas <765227+aldehir@users.noreply.github.com> Assisted-by: Claude Opus 5.5 * tests : fix the type check of the deferred tools test Assisted-by: Claude Opus 5.5
…g#31257) * model : add LiquidAI/LFM3-Audio-2B and LiquidAI/LFM3-Omni-2B Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Assisted-by: Claude Opus 5.5 * convert : read the LFM3-Audio config.json as released A transformers that knows lfm3_audio rewrites the config into another layout (no lfm/encoder keys). Assisted-by: Claude Opus 5.5 * mtmd : disable flash attention for the LFM3-Audio depthformer Assisted-by: Claude Opus 5.5 * tts : add a voice option for LFM3-Audio and LFM3-Omni The model card offers a female and a male voice; --tts-voice for llama-tts, "voice" for the server /tts. Assisted-by: Claude Opus 5.5 * clean up template handling * convert : detect LFM3-Audio and LFM3-Omni by model_type Assisted-by: Claude Opus 5.5 * move banned tokens --> logits bias --> mtmd_helper * leave mtmd-cli unchanged * no more template_asr_system * left over * llama : keep the LFM3-Audio encoder convs and codec unquantized Assisted-by: Claude Opus 5.5 * common, mtmd : shorten comments Assisted-by: Claude Opus 5.5 --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
…count cant be infered at compile time (ggml-org#31255)
In --models mode the thread count per model is computed and printed, but never set in the params, so every context runs on the common default, the number of physical cores. With several models in parallel this oversubscribes the CPU, and the busy-wait barriers of the CPU backend make it collapse. Set the count in cpuparams and cpuparams_batch.
…-org#31271) * server: expose the reasoning effort levels of the chat template The capability probe renders a user turn for each candidate level (minimal, low, medium, high, xhigh, max) and for an unknown level. A level is honored when it renders without raising and differently from the unknown level, which shows the template fallback. Levels sharing a render are aliases and the highest one is kept. The caps serialize to JSON, and chat_template_caps in /props reports the list as reasoning_effort_levels, from lowest to highest. * jinja: keep the caps free of the json dependency The caps keep reporting their flags as a map, and the effort levels stay a plain member. common_chat_templates_get_caps() builds the JSON the server reports from both, so jinja depends on json only for the probe inputs.
* model : add LiquidAI/d1-2B and LiquidAI/d1-omni-2B Assisted-by: Claude Opus 5.5 * convert : keep the speech task templates out of the d1 GGUFs Assisted-by: Claude Opus 5.5 * small refactoring: no_batching --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
…gml-org#31269) * server : add realtime conversation and transcription on /v1/realtime Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Assisted-by: Claude Opus 5.5 * examples : default the realtime client to the server port 9931 Assisted-by: Claude Opus 5.5 * server : sample realtime responses right after their own decode Slots respond in turns on one context: sampling at the start of the next step read the logits of whichever slot decoded last, so concurrent sessions mixed their outputs. Assisted-by: Claude Opus 5.5 * move more stuff to server-realtime, making it self-contained * examples : fix the type check of the realtime client Assisted-by: Claude Opus 5.5 * examples : use the model card prompts in the realtime client Assisted-by: Claude Opus 5.5 --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* ui : replace the sidebar search mode with a search dialog Search opens a spotlight dialog (Cmd+K or the Search entry under New chat in the sidebar) with keyboard navigation over recent chats and sidebar actions. Phones keep the search page. Sidebar actions show their shortcuts, with new ones for Settings (Cmd+,) and Models (Shift+Cmd+M). Assisted-by: Claude (Opus 5.5 Medium) * ui : ignore search dialog keys while an IME is composing Enter confirms an IME candidate and the arrow keys move through the candidates, so the search field leaves them to the IME instead of opening the highlighted result or moving the highlight. Assisted-by: Claude (Opus 5.5 Medium)
…ssage" action (ggml-org#31214) * ui: keep new chat draft when adding a system prompt Fixes ggml-org#24338 Assisted-by: Claude (Opus 5.5 Medium) * ui: remove unused initialMessage from chat form Assisted-by: Claude (Opus 5.5 Medium)
…#31237) * ui: support OGG audio attachments by converting them to WAV Fixes ggml-org#24164 Assisted-by: Claude (Opus 5.5 Medium) * ui: convert all non WAV/MP3 audio uploads to WAV Assisted-by: Claude (Opus 5.5 Medium)
…g#31248) * ui : show save and reset settings buttons only when relevant Assisted-by: Claude (Opus 5.5 Medium) * ui : move reset settings button next to reload app, pad settings caption Assisted-by: Claude (Opus 5.5 Medium)
…ors (ggml-org#30557) * chat : strip trailing whitespace from parsed reasoning and content Assisted-by: Claude Fable 5.1 * chat : stop qwen3-coder reasoning and content before template separators Assisted-by: Claude * tests : remove the Qwen parser case from the chat session tests Assisted-by: Claude * common : handle partial prefix until match at end of generation --------- Co-authored-by: Zeshawn Zahid <zzshawnn@gmail.com> Co-authored-by: Alde Rojas <hello@alde.dev>
…0283) * fix: clamp the A prefetch to end_k in the Vulkan coopmat1 int8 matmul, like the B prefetch * fix: zero the A scale for k steps past end_k in the Vulkan coopmat1 int8 matmul instead of relying on the clamped block
Assisted-by: Codex
…g#31273) * ui: build the reasoning menu from the template effort levels When the chat template reports effort levels, the menu offers Default, Off and those levels, and a pick is sent as reasoning_effort. A template that reports none keeps the token budget menu. Picking a model other than the one that produced the last answer resets the conversation effort to Default, and an effort the current menu lacks resolves to Default. * ui: wait for a cold router model before building the request A router serves the props of a loaded model only, so a request sent to a cold model was built without its template effort levels and fell back to a token budget. The request now waits for the model to load, joining a load already in flight, and a stop during the wait sends nothing. Status waiters hold a list per model so a load and the requests that wait on it each get theirs, and a props request in flight is shared with a second caller instead of answering null.
* ggml : bump version to 0.28.0 * authors : update
* server : add tool calls to /v1/realtime Assisted-by: Claude Opus 5.5 * refactor --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* mtmd : add LFM3-VL and LFM3-Omni video input Co-authored-by: Xuan Son Nguyen <son@huggingface.co> Assisted-by: Claude Opus 5.5 * mtmd : reserve the soundtrack command to avoid a GCC 11 warning --------- Co-authored-by: Xuan Son Nguyen <son@huggingface.co>
* llama.cpp : bump version to 0.8.0 * authors : update
* gguf-py: fix np_roundf dropping the sign * Update requirements/requirements-convert_legacy_llama.txt Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> * requirements.txt * Update tools/server/tests/requirements.txt Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co> --------- Co-authored-by: Sigbjørn Skjæret <sigbjorn.skjaeret@huggingface.co>
|
cl-hrxkern: the CHANGES at 1. 2. HRX ctest rows (device, gfx1151, gpu-hold, corrected libhsa path, release add-on):
3. Still open, not mine: |
Three decisions the merge cannot make for itself — with the evidence and the cost of each optionEverything mechanical is done: 0 markers, the tree builds, both HRX suites run on the device, numerics pass ( 1. MoE descriptor layout — selector keys on expert count, decoders are keyed by kernel family. 2. Standalone RoPE 3. Still owed by cl-hrxkern (neither has appeared in their four pushes):
My honest summary of the two suites: |
Correction of my correction: the
|
All three "decisions" retire to one sentence: the merged tree carries OUR kernels, and these assertions encode AMD's kernel setI asked for decisions on three divergences. Checking each against the kernel that actually consumes the parameter, none of them needs a decision: Item 2 — rope So our dispatch must pass it as a compile parameter — making it dynamic would require editing the kernel. And this kernel is not in AMD's pin (the file does not exist at Item 3 — plus distinct storage roots and a qualifying consumer. That is a deliberate narrowing (the same style as every other gate in this backend); AMD's case expects the k16 producer more broadly. Item 1 — moe descriptor layout: already withdrawn last turn, same reasoning (the 7-bit kernel exists only in our tree, and the only selector call site is the dispatch that pairs with it). So the honest rule for this merge: a failing assertion whose expected value is produced by our kernel is not a merge defect, and "fix" here means "port AMD's kernel", which is kernel work with its own device risk — not merge work. Where that leaves #327: merge mechanics complete (0 markers, builds, both suites run on the device, numerics 1717/1717). The only open work is cl-hrxkern's: #330 merged ( |
… the #327 squash missed (#337) * 2138 merge: answer the CHANGES at ed02b25 (parser tools, two HRX ctest rows, working files) - tools/parser: AMD moved these into tests (ggml-org#27548); the restored sources predate the new chat/JSON API and do not compile. Drop the subdirectory and its sources, as AMD's pin does. - hrx-backend-test: the merged binary is our line's, whose main() takes case names only; the ctest row passed AMD's test_runner flags, which it reads as unknown cases. Register it with no arguments, as the base does. - hrx-loom-jit-test: drop the compile check for qwen3_moe_routed_down_weighted_reduce_next_rmsnorm_f32_publish_q8. The corpus carries our line's routed-down file, which has no such export and no dispatch that selects it; the base's test has no reference to it. - Remove the working files HRX-COMPILE-ERRORS.txt and merge-lost/. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * 2138 merge: restore our line's test-hrx-loom-jit (the base's passing ctest row) The base registers test-hrx-loom-jit (our file); the merge dropped it and kept only AMD's hrx-loom-jit-test, which compiles AMD's kernel set. Restore our file and its row, next to AMD's, as test-hrx-ops was. The merged GDN head128 kernel gained AMD's llm.gated_delta_net.additive_epsilon config (our dispatch already sets it, dispatch-gated-delta-net.cpp:784); the test's compile config now passes it too, 0 = the clamp path it tested before. Device (gfx1151, gpu-hold): test-hrx-loom-jit rc=0, test-hrx-buffer passed. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
VERDICT: READY Measured, not readA standalone configure and a real build of this head, no GPU needed:
The one wording fix leftThe body still says:
A reader cannot tell from that whether this branch drops AMD's two tools or ports them, and the difference matters at the next pin: if the files are absent, the next merge will re-offer them and somebody will have to decide again — with no record of why they were left out this time. Say which it is, in the row, one sentence: "AMD's two parser tools are not carried over from the pin; the The rest of this head
Nothing I looked at has been dropped by a re-squash or a merge. The device table in the body remains yours — what I have checked independently is the marker count, the three deltas, and now the standalone configure and build. |
Instrument note for anyone else reading these test results: per-case isolation is a different measurementLearned while wiring up the HRX dispatch profile ( Two consequences, both of which can waste somebody's afternoon:
This is also why I would rather trust |
The HRX dispatch profile works, and it answers "which kernel actually ran" without consulting anyone's expectationsFollowing export HSA_USE_SVM=0 HSA_USERPTR_FOR_PAGED_MEM=0 HRX_ALLOW_NO_SVM=1
export IREE_HAL_AMDGPU_LIBHSA_PATH=/opt/rocm-therock/lib/python3.14/site-packages/_rocm_sdk_core/lib/libhsa-runtime64.so.1
export HRX_PROFILE_MODE=dispatch
export HRX_PROFILE_FILE=/path/prof.json
./bin/hrx-ops-test --hrx-test-case glu_packed_f32.reglu.normal # a PASSING, EXECUTING caseResult: Three things that make it work or not, all learned by trying:
So: profile passing, executing cases; triage failures from the harness's own What this changes for ggml-org#2138: the claim "the tree schedules our kernels, not AMD's" no longer needs to be argued from failing expectations — it can be read off a profile for any |
…ired compact mask (engine#2374 item 3) (#343) Since the engine#2138 merge (#327) the postprocess match no longer publishes a compact [tokens x tokens] F16 mask through a qwen_attention_metadata initialisation dispatch; AMD removed both the code and this case at 9bf5871. The flash attention reads the graph's own mask over the whole key/value view, one query chunk at a time. The case now checks that path: no metadata initialisation, every FA chunk at key_value_token_count = the view, binding 3 = the graph mask at the chunk's own row offset and length, and the chunks covering every query token. A real graph's view is get_n_kv() (used cells padded to 256), so the full-capacity view only occurs in the reserve graph or a full cache. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…line's format codes (engine#2138) (#345) * hrx: define the 16 dequant motifs the #327 merge left declared, in this line's format codes (engine#2138) The #327 merge kept our motifs/dequant.loom and took AMD's callers, so eight functions were func.decl'd and called with no definition anywhere in the corpus (ggml_codebook_f16/f32_vector4, ggml_{iq4xs,q5k,q1_0}_load_decode_state and _preloaded_f32_vector4). Every kernel that reaches them fails at JIT link with "unresolved exact declaration": the routed SwiGLU pair kernels (ggml_mul_mat_id_{skinny,tiled}_pair_input_f32_swiglu_publish_f32), i.e. every MoE model, plus the codebook/IQ4_XS/Q5_K tiled preload and Q1_0 decode-pair kernels. Main's pin (e6c8001) declares none of them. This adds AMD's definitions (9bf5871 motifs/dequant.loom) for those eight and the eight helpers they call that our file lacks (seven *_global_f32_vector4 IQ decoders and ggml_iq2xs_sign_mask), 16 in all, in AMD's order. The five shared helpers they call (iq2s_value_f32, iq3s_signed_value_f32, iq4nl_lookup_i8_vector4, iq4xs_table_codes4, q5k_high_bit_f32) have identical signatures here. Format codes: this line numbers IQ1_S 26, IQ1_M 27, IQ3_XXS 28 (common_mul_mat_format_config_value); AMD's file uses 19, 29, 18. The copied ggml_codebook_f32_vector4 and the two AMD-sourced routed callers (motifs/matmul_id/pair_input_f32_dequant_accumulate_f32.loom, ops/mul_mat_id_swiglu_f32_f32_wmma.loom, both comparing IQ3_XXS to 18) take this line's codes, so an IQ3_XXS routed weight is decoded as IQ3_XXS rather than falling through to the IQ2_S branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tests: test-hrx-loom-jit links the routed gate/up pair kernels (engine#2138) Compiles ggml_mul_mat_id_{skinny,tiled}_pair_input_f32_swiglu_publish_f32 with the dispatch's own compile parameters, for Q4_K, Q6_K and IQ3_XXS (28) weights. On the previous commit's parent this aborts with "unresolved exact declaration '@ggml_codebook_f16_vector4'", the failure every MoE model hit at serve; the rest of this suite passed there, so nothing caught it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…tops buying a compiled program (#350) The standalone and chunked rope dispatches set ggml.rope_f32.input_span from the tensor's or chunk's EXACT row count. That parameter is compile-time config (rope_f32.loom:18 declares it, :51 reads it for the index.assume bounds), so every distinct token count compiled its own program -- and the capacity class computed one line above it (token_capacity_class, next power of two) had no effect on compilation. The file already says so: // AMD's contract for standalone RoPE: one compiled program per capacity CLASS, shape dynamic. Without this the // capacity classing above has no effect on compilation, because the specialization pins the exact token count. The code contradicted its own comment: line 806 set token_capacity_class(count), line 807 pinned (count - 1) * stride2 + head_span. Now both sites size the span from the class. capacity >= token_count by construction, so the bound stays valid for every shape inside the class. Test side, and why the assertion changes shape: the #327 pin brought REQUIRE(compile_parameters.count("ggml.rope_f32.input_span") == 0); REQUIRE(integer_parameters.at("input_span") == hidden_size * token_count); i.e. AMD's runtime-parameter design. Our line keeps it compile-time because the kernel's index.assume proofs need it at compile time, and no dispatch in this tree feeds a config from an integer parameter (0 sites of a dotted name in integer_parameters), so adopting AMD's shape is a kernel change, not a dispatch one. The assertion now states our contract AND the property both designs want: with 3 tokens the class is 4, so the span must exceed this tensor's own span. Nothing tested that before. Measured, gfx1151, one gpu-hold ticket per configuration, hrx-backend-test at the current fork head: loom path (no HIP add-on) host 77 passed/10 failed -> 78/9 device 14/14 unchanged with HIP add-on host 60 passed/27 failed -> 61/26 device 14/14 unchanged rope_set_rows_dispatch [PASS] in both; no case regressed. Refs engine#2388 (host tier red on main, this is the first of the ten). Co-authored-by: bcloud <agent@local>
…rms (engine#2388) (#352) * tests: four host-tier checks the #327 merge left asserting retired forms (engine#2388) The engine#2138 merge (#327) left four hrx-backend-test host cases failing on the loom path against correct plans. Each one checked against the code and against AMD's 9bf5871 suite: - common_mul_mat_id_swiglu_dispatch, qwen_routed_gate_up_dispatch: named the monolithic ggml_mul_mat_id_swiglu_f32_f32_wmma and 6 bindings. The selector now picks the skinny (token_count <= 5) or tiled pair-input kernel, which has 7 bindings (the IQ3_XXS grid slot). This is the same shape #341 fixed for routed_moe_decode. The checks use the existing helper and 7, as AMD's own suite does. - qwen_router_top8_dispatch: expected AMD's fixed 511/9/15 descriptor layout with workgroup = expert_count. Our router keeps the compact 127/7/13/128 layout up to 128 experts (96049a2). The values are written out per range, not read from moe_router_descriptor_layout(), so a wrong layout cannot pass by agreeing with itself. - qwen_attention_postprocess_dispatch: the merge kept both bodies, AMD's "no initialization, one dispatch" directly above ours "two initialization dispatches", so it could never pass. Both initialization kernels are gone from both dispatch trees; the check is now AMD's postprocess bindings with our engine ggml-org#533 setup (three projection dispatches first, postprocess last of four). Test-only, one file. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tests: pin the postprocess dispatch by name and slot 6 by what it binds (engine#2388) Review of #352: plan.dispatches.back() was an assumed position, and bindings.size() == 7 a bare count. postprocess is now checked to be qwen3_moe_attention_postprocess_f32_f16, and at the three gate/up sites slot 6 (the IQ3_XXS grid) is checked to be the input buffer as the read-only placeholder for these formats, as the common-path case already does. A mutant binding the gate weight there fails the new check. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…rms the #327 merge replaced (engine#2388) (#351) * tests: five publication checks keep our engine#1962 / ggml-org#449 forms the #327 merge replaced (engine#2388) The engine#2138 merge (#327) kept our dispatch code for the activation publications but took AMD's 9bf5871 bodies of five hrx-backend-test cases. Those name fused publish kernels our corpus no longer has (engine#449) and look the f16 alternate up unscaled, while ours stores it at 2^-6 (engine#1962). So on the loom path they failed against code that is right by our rules. Restored to our pre-merge bodies (2f7f04f^): packed_f16_producer_consumer, rmsnorm_binary_k16_output, rmsnorm_binary_f16_output, generic_swiglu_k16_publish, tiled_matmul_alternate_publish. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * tests: packed_f16_producer_consumer checks each published form per row (engine#2388 review) The restored check compared a union on both sides, so a row could pass with its output published by the wrong producer. The case table already says which form each row publishes: check the k16-major pack and the plain f16 alternate separately. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…oes (engine#2393) (#356) clear() zeroed every stat. AMD's clear() (9bf5871) zeroes only allocation_count and resident_bytes, which follow the entries it drops. Hits, misses, evictions, materialized bytes and duplicate warnings count over the cache's life. hrx-buffer-test's host_weight_cache case asserts exactly that (hits == 1 after clear()). The #327 merge kept this branch's older body, so the case failed at tests/hrx-buffer-test.cpp:569. The only production caller is the backend release, which logs the stats before it clears, so nothing else reads them after a clear. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…d (engine ggml-org#2433) (#376) The AMD-pin merge (#327) resolved common/speculative.cpp's draft-mtp conflicts onto upstream's restructured batch API and lost two onebit lines from common_speculative_impl_draft_mtp: - draft() no longer called budget.begin(n_seq). MtpBudgetTrace's per-sequence vectors stayed empty, keep()/drafted() wrote past them on the first draft, and llama-server died with SIGSEGV in draft() on the first request for any GGUF with its own MTP head (MTP is on by default). Restored, and MtpBudgetTrace now grows its state for any sequence it is handed, so a missing begin() can only reset the rule, not corrupt the heap. - process() seeded the first token of every batch with pending_h, dropping the engine #290 continuity guard: a fresh request at pos 0 drafted from the previous request's last h-row. Restored: pending_h only when pending_pos + 1 == the batch's first pos, else a zero row. tests/test-onebit-mtp-budget: drafts on sequence 3 of 4 without begin(), begin() resetting the product, and the rule. With the old indexing the first case aborts (operator[] bounds assertion). Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…#381) Both removals are deletions of pre-#327 fork code, not restores of it: AMD's design (9bf5871) replaced each one, and #327 kept the old next to the new. A comment at each site says so, so the next merge does not re-add them as an omission. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…could not load); blackmamba loader; test-llama-archs runs every architecture (engine#2444) (#381) * qwen4exp, blackmamba: undo two #327 merge slips; test-llama-archs runs every architecture (engine#2444) qwen4exp (Qwen3.8-Flash-Next) could not load at all on this line since #327: - build_attn_qsa kept the pre-merge index scatter (ggml_set_rows of a top-k index list into the kq mask) next to AMD's version, where `sel` is already the mask and goes to build_attn_mha as is. Its result was never used, but building it asserts: set_rows needs I32/I64 indices and `sel` is the F16/F32 mask. Removed; the function is now identical to 9bf5871's. - build_hc_combine kept the fork's sigmoid(inject / hc) where AMD's tree has sigmoid(inject). With the old block removed the model loads and produces "\n\nThe\n\n\n\n##"; with AMD's line it produces the same text as the pre-merge fork and as 9bf5871. blackmamba read expert_feed_forward_length with get_key into element 0, so the model saver's per-layer array ([n_ff, 0]) did not load back. Read it with get_key_or_arr like every other MoE architecture; a real BlackMamba-2.8B GGUF generates byte-identical text before and after. test-llama-archs: BlackMamba is MoE-only and alternates Mamba and MoE layers, so the fixture writes no dense variant and a per-layer n_ff of [0, n_ff]. zamba2 and zaya need hparams the generic fixture does not write; they are skipped with a FIXME instead of aborting every later architecture. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test-llama-archs: zaya and zamba2 run instead of being skipped (engine#2444) - zaya puts the CCA attention and the MoE in every layer, so it is MoE-only like BlackMamba (with no expert_feed_forward_length its n_ff_exp is 0 and a 0-wide tensor fails the load), and its CCA graph is 2-tap: the fixture writes ssm.conv_kernel = 2 for it. - zamba2's shared attention runs on [hidden, embedding] concatenated, so q/k/v are 2*n_embd wide: the fixture writes key/value lengths of 2*n_embd_head, where the default n_embd/n_head did not divide them. All 174 architectures in llama-arch.cpp now run in test-llama-archs on the CPU with no failure; test-generate-models and test-llama-context pass. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * qwen4exp: say at both sites why the pre-#327 code is absent (review on #381) Both removals are deletions of pre-#327 fork code, not restores of it: AMD's design (9bf5871) replaced each one, and #327 kept the old next to the new. A comment at each site says so, so the next merge does not re-add them as an omission. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: agent <agent@local> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
engine#2138 — AMD pin
9bf5871merged into1bit/hrx-vulkan-patched, ready for review ated02b25.State at the head (measured at
ed02b25)git grep -lE '^<{7} ')b6e1167)ggml-hrx/dispatch-mul-mat-id.cpp4934172); AMD's version returns inf onactivation_scalerowstests/test-hrx-ops.cpp(targettest-hrx-ops)1d56dca)MERGE-2138-NOTE.txt(WIP working note)ed02b25)The single-squash-commit description that used to be here described the earlier WIP tree (23 files with markers). It no longer applies: this branch carries the full resolution history on top of
b5ebe8eb0.Device results (gfx1151, under gpu-hold)
test-backend-ops -b HRX0: MUL_MAT 1717/1717, MUL_MAT_ID 925/925, ROPE 170/170, FLASH_ATTN_EXT 538/538, GET_ROWS 135/135.test-hrx-ops(our suite): 224/270. The fork-tip build alone fails 53, and the 46 failures left are the same cases there. The two failures new on this branch (hand-built graph nodes withoutGGML_TENSOR_FLAG_COMPUTE) are fixed in616c7bf.AMD's
hrx-ops-test(390 cases) is a different file and a different questiontests/hrx-ops-test.cppis AMD's suite. It asserts which kernel is scheduled and checks AMD's packed-layout contract. This tree carries our dispatch, so most of its failures are by construction. Of the 293 failures atc06e741, 226 say "missing expected kernel" or report a specialization mismatch, and 12 fail on theircpu_referencevariant. It stays registered so the gap stays visible. To make any of AMD's selections pass, those paths have to be ported onto our dispatch files, not expected from AMD's.Known exclusions
tools/parser/template-analysis.cppanddebug-template-parser.cppdo not compile: anordered_jsonvscommon_jsonclash afterchat-auto-parser.h. That needs a port, not a rename, and the HRX verification build excludes both files.Method note kept from the WIP record
The base-comparison check is a text-containment test, so a file with 0 markers can still be wrong.
dispatch-registry.hhad 0 markers but still containedconst DispatchRegistry * registry, which the base had and our line removed. A marked conflict is honest; a guessed resolution is not.🤖 Generated with Claude Code