Skip to content

Misc. bug: ggml_cuda_compute_forward: SOFT_MAX failed / invalid argument on GLM-5.3-Flash (Turing, glm5next) #28144

Description

@Junesgone

Name and Version

llama-server --version
version: 0.3.0-dev (build 10687, commit 92b19177a)
built with GNU 11.4.0 for Linux x86_64 (Compiled by the Unsloth team)

Note: unsloth fork prebuilt (app-b10687-mix-67dfc8b-linux-x64-cuda13-older.tar.gz). It includes the glm5next support from PR #27754, which is not merged on master, so I could not test this on upstream.

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-server

Command line

llama-server -m GLM-5.3-Flash-UD-IQ2_XXS-00001-of-00004.gguf --mmproj mmproj-F16.gguf -ngl 99 --cpu-moe --load-mode none -c 524288 -fa on -ctk q4_0 -ctv q4_0 --kv-offload --reasoning-preserve --spec-type ngram-map-k4v --chat-template-kwargs "{"reasoning_effort":"max"}" --temp 1.0 --top-p 0.95 --timeout 86400 -t 6 --host 0.0.0.0 --port 8092 --parallel 1

Problem description & steps to reproduce

Model: unsloth GLM-5.3-Flash UD-IQ2_XXS (320B MoE, glm5next), with mmproj.
Hardware: i5-12500T + RTX 2080 Ti 22GB (Turing, sm_75), 125GB RAM, driver 580.142.

Server ran fine for about 10h20m and roughly 25,000 requests (agent chat with tool calls, single slot). It crashed during one long generation (665 tokens generated, ~1.8 t/s). dmesg has no Xid or GPU errors, nvidia-smi stayed healthy, and systemd restarted it cleanly (model reloaded fine afterwards).

The error matches #25095 and #25060 verbatim, just a different arch (glm5next) and a Turing card.

First Bad Commit

No response

Relevant log output

Logs
618.57.749.164 I slot print_timing: id  0 | task 25655 | n_gen =    617, tg =   1.82 t/s, tg_3s =   2.36 t/s
619.18.203.059 I slot print_timing: id  0 | task 25655 | n_gen =    665, tg =   1.85 t/s, tg_3s =   2.35 t/s
619.19.966.851 E ggml_cuda_compute_forward: SOFT_MAX failed
/home/runner/work/llama.cpp/llama.cpp/src/ggml/src/ggml-cuda/ggml-cuda.cu:109: CUDA error
619.19.966.974 E CUDA error: invalid argument
619.19.966.978 E   current device: 0, in function ggml_cuda_compute_forward at /home/runner/work/llama.cpp/llama.cpp/src/ggml/src/ggml-cuda/ggml-cuda.cu:2455
619.19.966.979 E   err

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions