Skip to content

sycl: add Prism Q1_0 and Q2_0 g128 support#94

Draft
hancris wants to merge 4 commits into
PrismML-Eng:prismfrom
hancris:sycl-q2-g128-b580
Draft

sycl: add Prism Q1_0 and Q2_0 g128 support#94
hancris wants to merge 4 commits into
PrismML-Eng:prismfrom
hancris:sycl-q2-g128-b580

Conversation

@hancris

@hancris hancris commented Jul 19, 2026

Copy link
Copy Markdown

Summary

Adds Intel SYCL support for the Prism Q1_0 and Q2_0 group-128 formats.

  • Enables Prism Q1_0 group-128 SYCL kernels
  • Adds Q2_0 dequantization to FP16 and FP32
  • Adds the optimized Q2_0 x Q8_1 MMVQ kernel
  • Adds Q2_0 GET_ROWS support

Validation

  • Q2_0 MUL_MAT backend tests: 37/37 passed
  • Q2_0 GET_ROWS backend tests: 4/4 passed
  • Intel Arc B580 through Level Zero SYCL

Ternary-Bonsai 1.7B Q2_0:

  • pp512: 3038.61 tok/s
  • tg128: 110.94 tok/s

Ternary-Bonsai 27B Q2_0:

  • pp512: 207.38 tok/s
  • tg128: 9.97 tok/s
  • 26.90B parameters, 6.66 GiB, ngl 99

Environment

  • Intel Arc B580 12 GB
  • Intel oneAPI 2026.1
  • Level Zero SYCL backend
  • Windows

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant