Vulkan: Arc Pro B70 (BMG G31/xe) resets CCS engine during Ternary-Bonsai-27B-Q2_0 generation

[penica]

Summary

llama-server built from the Prism branch runs Ternary-Bonsai-27B-Q2_0.gguf on an Intel Arc Pro B70 through Vulkan at about 27.5 tok/s, then reliably terminates with vk::DeviceLostError. The Linux xe driver reports a timed-out CCS job in llama-server and resets the engine.

This reproduced with both the default automatic configuration and a reduced single-slot configuration, so it does not appear to be caused by prompt cache, four parallel slots, or a large context.

Build

version: 9596 (9fcaed763)
built with GNU 11.4.0 for Linux x86_64

Environment

OS: Linux 7.0.0-28-generic x86_64
GPU: Intel Arc Pro B70 / Battlemage G31 (PCI 8086:e223)
Kernel driver: xe
Vulkan driver: Intel open-source Mesa driver
Mesa: 26.0.3-1ubuntu1
Vulkan API: 1.4.335
VRAM reported free at startup: ~29.35 GiB / 32.66 GiB

Model

Ternary-Bonsai-27B-Q2_0.gguf

Reproduction

./llama-server \
  -m ../Ternary-Bonsai-27B-Q2_0.gguf \
  --device Vulkan0 \
  --gpu-layers all \
  --split-mode none \
  --main-gpu 0 \
  --parallel 1 \
  -c 16384 \
  --cache-ram 0 \
  --host 0.0.0.0 \
  --port 9999

Send a normal chat-completions request. Generation runs normally for several hundred tokens, then the process aborts. In the latest reproduction it reached 595 decoded tokens at 27.53 tok/s before crashing.

Application error

terminate called after throwing an instance of 'vk::DeviceLostError'
  what():  vk::Device::waitForFences: ErrorDeviceLost
Aborted (core dumped)

Kernel evidence

xe 0000:23:00.0: [drm] Tile0: GT0: Engine reset: engine_class=ccs, logical_mask: 0x1, guc_id=2, state=0x209
xe 0000:23:00.0: [drm] Tile0: GT0: Timedout job: seqno=14893, lrc_seqno=14893, guc_id=2, flags=0x0 in llama-server [3529467]

An earlier reproduction with the default --parallel auto / -c 65536 settings produced the same error and an xe device coredump.

What was tried

• Re-downloaded and used the fork-compatible Ternary-Bonsai-27B-Q2_0.gguf (not PQ2_0).
• Explicitly selected Vulkan0 and requested all GPU layers.
• Reduced from four 65K slots to one 16K slot.
• Disabled the llama-server prompt cache with --cache-ram 0.

All GPU configurations tested eventually hit the same CCS engine timeout and VK_ERROR_DEVICE_LOST. CPU-only mode avoids the Vulkan crash.

[penica]

Confirmed workaround on Arc Pro B70

Disabling both Vulkan cooperative-matrix paths avoids the CCS timeout in this environment:

GGML_VK_DISABLE_COOPMAT=1 GGML_VK_DISABLE_COOPMAT2=1 \
./llama-server \
  -m ../Ternary-Bonsai-27B-Q2_0.gguf \
  --device Vulkan0 --gpu-layers all --split-mode none --main-gpu 0 \
  --parallel 1 -c 16384 --cache-ram 0 \
  --host 127.0.0.1 --port 9999

I sent a local /v1/chat/completions request with max_tokens: 700 and temperature: 0. It completed successfully with HTTP 200 in 26.382 seconds. The server remained healthy afterwards, and no new xe Timedout job or Engine reset messages appeared.

This is a functional workaround, though it may reduce performance. It points to a BMG G31/Mesa cooperative-matrix path rather than VRAM pressure, server prompt caching, context size, or parallel-slot count.

[penica]

Follow-up: workaround is not stable across sequential requests

The cooperative-matrix disable flags delay the failure but do not eliminate it. With the exact command from my previous comment:

1. First request:
   • prompt: 919 tokens
   • generated: 1,569 tokens
   • completed normally at 26.83 tok/s (truncated = 0)
2. Second sequential request on the same server/slot:
   • server restored a context checkpoint: pos_min = 898, pos_max = 898, n_tokens = 899, n_past = 899
   • prompt processing completed (1,601 tokens)
   • generation reached 333 tokens at 25.77 tok/s
   • then aborted with vk::DeviceLostError / vk::Device::waitForFences: ErrorDeviceLost

This suggests the problem is not a simple generation-length or VRAM threshold. The failure appears after reuse/restoration of state between requests, which may implicate recurrent/Gated Delta Net state or the checkpoint restore path.

GGML_VK_DISABLE_COOPMAT=1 and GGML_VK_DISABLE_COOPMAT2=1 should therefore be treated only as a temporary diagnostic workaround, not a stable serving configuration.