Skip to content

hexagon: rope updates - #28628

Merged
max-krasnyansky merged 8 commits into
ggml-org:masterfrom
qualcomm:tb/hexagon-rope-update
Sep 9, 2026
Merged

hexagon: rope updates#28628
max-krasnyansky merged 8 commits into
ggml-org:masterfrom
qualcomm:tb/hexagon-rope-update

Conversation

@tboinovski1

@tboinovski1 tboinovski1 commented Sep 9, 2026

Copy link
Copy Markdown
Contributor

Overview

MROPE/IMROPE/VISION updates.

Additional information

  • MROPE/IMROPE theta-pick vectorization
  • NEOX rotate + aligned tail copy tightening and theta cache vectorization on v75
  • In-place RoPE for all scenarios and removal of ctx->spad/legacy timers.
  • Kernel params + host-side VTCM requirement enforcement, mode-check tightening/param cleanup
  ┌────────┬────────────────────────────────┬────────────┬────────────┬─────────────────────┐
  │  Mode  │             Shape              │   Before   │   After    │       Speedup       │
  ├────────┼────────────────────────────────┼────────────┼────────────┼─────────────────────┤
  │ MROPE  │ [128,12,512,1]                 │ 586–596 us │ 100–102 us │ ~5.6–5.9×           │
  ├────────┼────────────────────────────────┼────────────┼────────────┼─────────────────────┤
  │ IMROPE │ [128,12,512,1]                 │ 615–639 us │ 99–101 us  │ ~6.1–6.4×           │
  ├────────┼────────────────────────────────┼────────────┼────────────┼─────────────────────┤
  │ VISION │ [80,16,2,1]                    │ 8.5–8.6 us │ 5.6–5.7 us │ ~1.5-1.7×               │
  ├────────┼────────────────────────────────┼────────────┼────────────┼─────────────────────┤
  │ NEOX   │ [64,8,512,1], [80,32,512,1]    │ 54–242 us  │ 52–230 us  │ ~1.05–1.11×         │
  ├────────┼────────────────────────────────┼────────────┼────────────┼─────────────────────┤
  │ NORMAL │ [128,32,512,1], [128,64,512,1] │ 259–517 us │ 259–536 us │ ~0.98–1.06× │
  └────────┴────────────────────────────────┴────────────┴────────────┴─────────────────────┘

Testing: 288/288 tests passed (all f32 tests still passing)

Requirements

  • I have read and agree with the contributing guidelines
  • AI usage disclosure: Yes to reason about and test the code refactoring.

@github-actions github-actions Bot added testing Everything test related ggml changes relating to the ggml tensor library for machine learning Hexagon labels Sep 9, 2026
@tboinovski1
tboinovski1 force-pushed the tb/hexagon-rope-update branch from ae1c68c to 7c56eff Compare September 9, 2026 06:34
@tboinovski1
tboinovski1 marked this pull request as ready for review September 9, 2026 06:35
@tboinovski1
tboinovski1 requested review from a team and ggerganov as code owners September 9, 2026 06:35
@max-krasnyansky

Copy link
Copy Markdown
Member

@lhez for review & second ack

@max-krasnyansky
max-krasnyansky merged commit 6d9c82e into ggml-org:master Sep 9, 2026
25 of 31 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ggml changes relating to the ggml tensor library for machine learning Hexagon testing Everything test related

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants