perf(cpu): balanced work partitioning across compute threads (#402)

[jamesburton]

Adds ComputeThreadPool.PartitionRange and routes all 12 worker sites through it, replacing the
ceiling-division split each had copied.

Measurement

Zen 5 (Ryzen AI Max+ 395, 16C/32T), Llama-3.2-1B-Instruct Q8_0 (32 heads), 12 KB prompt,
prefill tok/s. A/B within one build via a temporary env override — comparing two builds would
confound the partitioning with everything else that differs.

configuration	balanced	ceiling	delta	paired t
--threads 24 (the awkward case)	121.44	119.73	+1.43%	3.86 (n=10)
--threads 32 (default, no imbalance)	119.06	118.55	+0.43%	0.48 (n=8)

Both acceptance criteria: a measured improvement where the counts do not divide, and no regression
on the default configuration — which is what you would expect, since 32 heads over 32 threads was
never imbalanced.

Ordering matters more than the effect here, so the design controls for it. Runs were ABBA-ordered
so each arm sits at equal mean position, and compared pairwise. Without that the result is not
meaningful: the raw series falls from 136 to 113 tok/s across a session regardless of arm (clock
drift on this part), which is ~15% — an order of magnitude larger than the effect being measured.
A naive alternating A/B returns whichever answer its ordering implies. I hit exactly that on #415
earlier tonight, in the opposite direction.

The +1.43% is larger than attention's ~1% share of compute would allow on its own, which fits:
--threads 24 also leaves the matmul tile/group partitions non-dividing, and those are the ~53%.

Correctness

Results are unchanged, not merely close. Ranges stay contiguous, disjoint and in thread order, so
each thread owns the same kind of output slice as before — this redistributes work without
reassociating any arithmetic.

PartitionRangeTests (21 cases) covers N < T, N == T, N == T + 1, N a multiple of T, and
N just under a multiple, asserting:

• no thread receives an empty range while N >= T
• per-thread counts differ by at most 1
• the ranges tile [0, N) exactly once, contiguously and in order
• with N < T, exactly N threads get one item each and none reads past the end (the granularity
  limit you noted, kept distinct from the defect)

Kernel numerics tests pass unchanged (112 in the touched areas).

Note on the helper

PartitionRange is public static on ComputeThreadPool rather than a private helper, since the
callers live in DotLLM.Cpu.Kernels. Happy to make it internal instead if you would rather it not
be surface area — it is only meaningful to kernel authors.

Closes #402

[Copilot]

Pull request overview

Introduces a shared balanced range-partitioning helper for CPU compute threads and applies it across kernel worker sites to reduce idle threads when the work item count doesn’t divide thread count (notably for attention head partitioning).

Changes:

• Added ComputeThreadPool.PartitionRange(...) implementing a contiguous, balanced floor/ceil split across threads.
• Updated CPU kernel workers (attention + multiple matmul variants) to use the shared partition helper instead of per-site ceiling-division logic.
• Added unit tests (PartitionRangeTests) covering tiling, balance, and N<T granularity behavior.

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tests/DotLLM.Tests.Unit/Threading/PartitionRangeTests.cs	Adds coverage for the new balanced partition helper across edge cases.
src/DotLLM.Cpu/Threading/ComputeThreadPool.cs	Adds PartitionRange helper for balanced contiguous work distribution.
src/DotLLM.Cpu/Kernels/MatMulQ5_0.cs	Routes Q5_0 workers’ tile/group partitioning through PartitionRange.
src/DotLLM.Cpu/Kernels/MatMulKQuants.cs	Routes K-quant workers’ tile/token partitioning through PartitionRange.
src/DotLLM.Cpu/Kernels/MatMul.cs	Routes multiple matmul worker partition sites through PartitionRange.
src/DotLLM.Cpu/Kernels/Attention.cs	Routes attention head partitioning through PartitionRange to reduce idle threads on awkward --threads values.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[jamesburton]

Both addressed in 27d4e5b — both agreed.

PartitionRange visibility (now internal). Right, and it answers the open question I left in
the PR description. Its only callers are the kernels in this assembly, DotLLM.Cpu already grants
InternalsVisibleTo to the test project, and a work-partitioning helper means nothing outside
kernel code — so nothing is lost and the shipped API surface does not grow.

Test location (moved). Also right. It is now at tests/DotLLM.Tests.Unit/Cpu/Threading/ in the
DotLLM.Tests.Unit.Cpu.Threading namespace, alongside ComputeThreadPoolTests and
NumaTopologyTests. It was the only thing under a top-level Threading namespace, which is exactly
the discoverability problem you describe — the duplication this PR removes spread the same way.

No behaviour change; the 21 partition cases pass unchanged.