Skip to content

metal : fix MiniCPM3 crash with automatic flash attention - #28599

Open
wyanzhao wants to merge 2 commits into
ggml-org:masterfrom
wyanzhao:codex/fa-minicpm-submit
Open

metal : fix MiniCPM3 crash with automatic flash attention#28599
wyanzhao wants to merge 2 commits into
ggml-org:masterfrom
wyanzhao:codex/fa-minicpm-submit

Conversation

@wyanzhao

@wyanzhao wyanzhao commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

Overview

MiniCPM3-4B uses head dimensions (dk, dv) = (96, 64). Metal's supports_op only checks dk, so -fa auto selects the missing kernel_flash_attn_ext_vec_f16_dk96_dv64 kernel and crashes.

Fixes #28602.

Check the full head-size pair so automatic FA selection falls back to ordinary attention. Both vec dispatch paths use the same helper to check their supported pairs.

The regression tests query supports_op without executing the graph: a normal (96, 64) test would dispatch a missing kernel on the baseline and crash the test process. The positive cases also check that supported pairs remain available.

Tested on M5 Max: baseline 64e9bceb2c3a856efed96feda784a50947049feb crashes with SIGSEGV; the patch generates normally. With a 32-token budget and the prompt below, patched FA auto and both FA off controls produce identical text. All 4832 FA numerical tests, 5 admission checks and the default CPU ci/run.sh passed. The new (96, 64) admission test fails with the baseline library.

Reproduce with MiniCPM3-4B

Use minicpm3-4b-q4_k_m.gguf from openbmb/MiniCPM3-4B-GGUF:

./build/bin/llama-cli -m minicpm3-4b-q4_k_m.gguf \
  -ngl 99 -fa auto -ctk f16 -ctv f16 -c 512 -b 128 -ub 128 \
  -n 32 --seed 1234 --temp 0 -st -p '请用一句话介绍北京。' --offline

Repeat with -fa off for the control.

Run the FA tests with:

cmake --build build --target test-backend-ops -j 8
./build/bin/test-backend-ops test -b MTL0 -o FLASH_ATTN_EXT

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: YES. AI assisted with code and tests.

@github-actions github-actions Bot added testing Everything test related ggml changes relating to the ggml tensor library for machine learning Apple Metal https://en.wikipedia.org/wiki/Metal_(API) labels Sep 8, 2026
@wyanzhao
wyanzhao marked this pull request as ready for review September 8, 2026 10:09
@wyanzhao
wyanzhao requested review from a team and ggerganov as code owners September 8, 2026 10:09
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Apple Metal https://en.wikipedia.org/wiki/Metal_(API) ggml changes relating to the ggml tensor library for machine learning testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Misc. bug: MiniCPM3-4B crashes on Metal with automatic flash attention

1 participant