ggml-meta, sched : allocate a meta transport ring at each device's share

[Piggidragon]

Overview

Follow-up to #8: under -sm tensor, each device now allocates only its share of the transport ring instead of the whole ring.

Stacked on #8 (and through it GenerelSchwerz#39 and #5). This PR targets that branch,
so the diff below is only its own commit, ggml-meta, sched : allocate a meta transport ring at each device's share. Merge #8 first.

A meta buffer sizes every simple buffer for the whole tensor and places a tensor at the same offset on each device. A device that holds half the heads therefore allocated the full ring.

• ggml_backend_meta_get_shares(buft, tensors, n, shares): for each device, the largest fraction (in 1/65536) of the given compute leaves it holds. The split state is taken through an empty compute meta buffer.
• ggml_backend_meta_alloc_buffer_shares(buft, size, shares): allocates simple buffer j at size * share_j. A tensor at meta offset X lands at X * share_j, rounded up to the alignment. The share includes one alignment per entry, so rounded neighbours cannot overlap, and binding asserts that the tensor fits. The meta size stays size, so the scheduler's layout and ggml_backend_tensor_alloc checks are unchanged.
• Scheduler:
  • The budget applies to the largest share.
  • The headroom check applies to each device's own share.
  • A plan that needs a larger share than the ring holds grows the ring, like a larger slot does.
  • A ring with a mirrored entry gets share 1, which is the previous behaviour.
• The meta compute buffers are not changed.

Testing

RTX 4070 + RTX 3060, CUDA + NCCL, -sm tensor -nkvo --kv-cpu-pinned, q8_0 KV.

model	split	slot	device 0	device 1
Qwen3.8-27B-UD-IQ2_M	50,50	8,192 KiB	4,097 KiB	4,097 KiB
gemma-4-26B-A4B	55,45	8,192 KiB	8,192 KiB	4,097 KiB

The share is the largest over the entries, so a layer that puts all of its KV heads on one device costs that device the whole slot, as on gemma-4's first device here.

This moves where the default 128 MiB budget declines. At 32k context on Qwen3.8 a slot is 70.7 MiB:

• sched, ggml-meta : pipeline the host KV delivery under split mode tensor #8 needs 212 MiB per device, over the budget, so it stays on the ordered path. That is derived from the slot size, not measured on sched, ggml-meta : pipeline the host KV delivery under split mode tensor #8.
• This PR needs 3 x 34.5 MiB per device and pipelines: tg257 at 32k goes from 4.07 to 4.88 t/s at the default budget.

Output is unchanged against #8, with the same hashes:

• greedy completion: Qwen3.8 at depths 0/1/4 64e86551f7ef1638; gemma-4 5525e3f5ac7337d7 (50,50) and 4f8986fb3655a567 (55,45)
• server exactness gate (LLAMA_KV_SM=tensor, depths 0 and 1): all 8 tasks identical to sched, ggml-meta : pipeline the host KV delivery under split mode tensor #8
• parallel gate: 125cb9c2082d36cf (-sm tensor, depths 0/1/4), db661b7a08686b97 (-sm layer), 17f946c340db110b (-sm none)
• throughput unchanged: Qwen3.8 22.97 t/s at depth 1 against 22.96 on sched, ggml-meta : pipeline the host KV delivery under split mode tensor #8
• test-llama-archs -s 1 on 2, 3 and 4 CUDA devices, and test-alloc, pass

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES - implemented by an agent on my instruction, see the Assisted-by: commit trailer.

🤖 Generated with Claude Code

https://claude.ai/code/session_0145V8xvxxUQMUVbg6ZiCZMg

🤖 Generated with Claude Code