Skip to content

ggml-cpu(s390x): add Q1_0 vector intrinsic support - #28606

Open
taronaeo wants to merge 3 commits into
ggml-org:masterfrom
taronaeo:feat/s390x-q1_0
Open

ggml-cpu(s390x): add Q1_0 vector intrinsic support#28606
taronaeo wants to merge 3 commits into
ggml-org:masterfrom
taronaeo:feat/s390x-q1_0

Conversation

@taronaeo

@taronaeo taronaeo commented Sep 8, 2026

Copy link
Copy Markdown
Member

Overview

This PR introduces vector intrinsic support for Q1_0 data type. We are looking at performance improvements of approximately 139.53% for Prompt Processing and 131.59% for Token Generation.

Additional information

Users of the Bonsai 1-bit models are recommended to compile with -DGGML_BLAS=ON to achieve higher performance.

Performance Benchmark

$ build/bin/llama-bench -hf taronaeo/Bonsai-8B-BE-GGUF:Q1_0 -lm mlock

Before

model size params backend threads lm test t/s
qwen3 8B Q1_0 1.07 GiB 8.19 B CPU 2 mlock pp512 0.52 ± 0.00
qwen3 8B Q1_0 1.07 GiB 8.19 B CPU 2 mlock tg128 0.46 ± 0.00

build: ca86fb2 (10859)

After

model size params backend threads lm test t/s
qwen3 8B Q1_0 1.07 GiB 8.19 B CPU 2 mlock pp512 2.92 ± 0.01
qwen3 8B Q1_0 1.07 GiB 8.19 B CPU 2 mlock tg128 2.23 ± 0.01

build: 6577365 (10861)

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: Yes, Opus 5 because my initial idea was poor for performance

Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
Signed-off-by: Aaron Teo <aaron.teo1@ibm.com>
@taronaeo
taronaeo requested a review from ggerganov as a code owner September 8, 2026 13:59
@github-actions github-actions Bot added documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning labels Sep 8, 2026
@taronaeo

taronaeo commented Sep 8, 2026

Copy link
Copy Markdown
Member Author

cc: @AlekseiNikiforovIBM @Andreas-Krebbel for review if any

@AlekseiNikiforovIBM

Copy link
Copy Markdown
Contributor

LGTM

@taronaeo taronaeo added the merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge. label Sep 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation ggml changes relating to the ggml tensor library for machine learning merge ready A maintainer can use this label to indicate that they consider the changes final and ready to merge.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants