Rank-local TP2 is structurally two-rank and requires two NUMA nodes, excluding 4-GPU and single-socket machines

[ro99]

Context

Rank-local decode is fixed at a world size of two, as a compile-time constant backing fixed-size arrays (include/strata/dsv4_rank_local_topology.hpp:23):

inline constexpr std::size_t kDsv4RankLocalWorld = 2U;

It is referenced 73 times across 7 files:

src/deepseek_runtime.cpp                      38
src/dsv4_rank_local_topology.cpp               8
include/strata/dsv4_rank_local_topology.hpp    6
src/dsv4_rank_local_kv.cpp                     6
src/dsv4_rank_local_weights.cpp                6
include/strata/dsv4_rank_local_weights.hpp     5
include/strata/dsv4_rank_local_kv.hpp          4

mostly as std::array<T, kDsv4RankLocalWorld> for per-rank device accounts, expert cache capacities, KV leases, patch spans, and CPU sets.

There is a second, independent constraint. src/dsv4_rank_local_topology.cpp:61 requires at least as many NUMA nodes as ranks:

if (topology.nodes < static_cast<int>(kDsv4RankLocalWorld)) {

Many single-socket consumer boards report one NUMA node, so those machines are excluded from rank-local decode regardless of how many GPUs they have.

Current behaviour is correct, and narrow

Both constraints fail closed with accurate messages:

rank-local decode requires exactly 2 CUDA devices, got 4
rank-local decode requires at least 2 NUMA nodes, found 1

Nothing is silently wrong. But a 4 x RTX 3090 machine — strictly more capable than the validated 2 x 3090 configuration — gets no rank-local decode, and neither does a 3-GPU machine or a single-socket 2-GPU machine.

The feature is named TP2 and two ranks is its declared design, so this is a scope limit rather than a defect. Filing it so the limit is tracked rather than assumed, and so the generalisation is costed honestly rather than discovered later.

What generalisation would actually cost

The fixed-size arrays are the cheap part; most loops are already written as for (rank = 0; rank < kDsv4RankLocalWorld; ++rank) and would follow a runtime bound. The parts that need real work and real re-validation:

• Collectives. The chain is built around a two-rank exchange with one completion. N ranks change the collective structure and the completion accounting, which is where the measured win lives.
• KV replication. Full context is replicated per device today. At N ranks this is either N copies (VRAM cost grows linearly and the admission ceiling moves) or it becomes sharded, which is a different design.
• Tensor sharding. Head and expert splits are derived for two ways; N-way changes the shard math and its alignment constraints.
• NUMA mapping. Rank-to-node assignment assumes ranks do not exceed nodes. Decoupling rank count from node count needs a real policy for oversubscribed nodes.
• Re-validation. Every claim in docs/dsv4-rank-local-architecture.md was measured at two ranks on two RTX 3090s. None of it transfers: per the charter, costs are functions of the operating point, so an N-rank configuration needs its own measurement, not an extrapolation.

Proposal

Not a single change. Suggested order, each independently useful:

1. Document the two-rank and two-NUMA-node limits where users meet them, not only in the architecture record.
2. Decide whether >2 GPUs should select a two-rank subset for rank-local rather than refusing outright. This alone would serve the 4 x 3090 case with no new topology work, and depends on No capability-aware GPU selection: mixed-hardware machines require the user to know CUDA ordinals #27.
3. Separate the NUMA-node requirement from the world size, so a single-socket 2-GPU machine is evaluated on its own merits.
4. Only then consider a true N-rank world, with its own measured cost model.

Step 2 is likely the highest value per unit of work and does not require touching the topology at all.