Add experimental non-coherent DMA-BUF GPUDirect RDMA path

[ducphuc]

Add experimental non-coherent DMA-BUF GPUDirect RDMA path

Important

This is a draft/RFC and depends on #35, which in turn depends on #33. Until
#35 merges, this branch also contains its BAR1 runtime-coverage commits
through 26e1ff53. Review the DMA-BUF GDR commits after that point. The
branch will be rebased onto the updated upstream base after #35 is resolved.

Summary

Add a default-off experimental DMA-BUF GPUDirect RDMA path for non-coherent GPUs using runtime capabilities rather than a GPU, architecture, or device-ID allowlist.

The feature is enabled explicitly with:

NVreg_ExperimentalDmaBufP2P=1

When enabled, a non-coherent FORCE_PCIE DMA-BUF export is admitted only when:

• static BAR1 is active and usable;
• MIG is disabled;
• the exported range is nonzero and wholly contained in static BAR1;
• the importer uses an identity IOMMU domain;
• the importer DMA mask covers the complete BAR1 resource; and
• Linux pci_p2pdma_distance() accepts the GPU-to-importer path.

The experimental option broadens only non-coherent FORCE_PCIE exports. It
does not change the admission or mapping behavior of other mapping types. The
existing coherent DMA-BUF path and its topology behavior remain unchanged.

This implements an experimental kernel-side path relevant to #20 — GDR support for RTX 5090, but does not claim to close it because current consumer CUDA userspace applies a separate capability gate.

Why this is not a simple gate bypass

The change does not unconditionally remove the existing non-coherent
FORCE_PCIE rejection.

Non-coherent FORCE_PCIE exports remain rejected unless every runtime
predicate succeeds.

Instead, it introduces explicit, fail-closed policy checks around:

• feature opt-in;
• mapping type;
• static BAR1 availability;
• MIG exclusion;
• overflow-safe aperture containment;
• importer DMA-mask coverage;
• IOMMU-domain type; and
• Linux PCI P2PDMA topology acceptance.

BAR1 is mapped through the importer DMA API for the experimental path rather than using the coherent path’s IOMMU bypass.

The existing RmGpuDirectRdmaForceSPA override remains restricted to coherent GPUs. Enabling the experimental non-coherent path does not make the coherent-platform SPA translation override reachable for non-coherent FORCE_PCIE exports.

The branch also serializes FORCE_PCIE physical-address operations. These operations map or unmap BAR1 and update per-GPU RUSD statistics, so treating them like immutable default static-address configurations can race concurrent registration and deregistration.

CUDA userspace prerequisite

This kernel change is necessary on the validated GB206 system, but it is not sufficient by itself.

Stock CUDA userspace does not advertise the required DMA-BUF capability on the validated GeForce configuration, so stock userspace does not exercise the kernel path introduced by this PR. Investigation or modification of CUDA userspace is outside the scope of this change.

Harry Chen independently reported analogous userspace gating on RTX 5090 while investigating GPUDirect RDMA support:

• Is there a GDR solution/workaround for RTX 5090? #20 (comment)
• https://harrychen.xyz/2026/05/20/enable-gpudirect-rdma-on-rtx-5090/

The functional validation below used a userspace configuration that allowed the kernel path to be exercised. This PR does not document or provide a userspace modification procedure.

Default behavior

The registry option defaults to disabled.

With NVreg_ExperimentalDmaBufP2P=0:

• non-coherent FORCE_PCIE exports retain their existing rejection;
• the stock coherent path is unchanged; and
• no device gains the experimental behavior automatically.

No chip-specific exception or consumer-product allowlist is introduced. Every driver-supported GPU is evaluated using the same runtime predicates.

Functional validation completed

Validated with:

• two NVIDIA GeForce RTX 5060 Ti 16 GB GPUs;
• NVIDIA ConnectX-6 Lx dual-port 25 GbE RoCE;
• NODE GPU-to-NIC topology;
• separate GPU and NIC IOMMU groups using identity domains;
• driver 610.43.03 and kernel 7.0.0-29;
• signed modules with verified hashes and srcversions;
• CUDA 13.3 with a userspace configuration that allowed the kernel path to be exercised;
• NCCL 2.30.7; and
• nccl-tests 2.19.6 built with MPI.

Observed results:

• CUDA device memory exported successfully through DMA-BUF.
• mlx5 successfully registered the exported DMA-BUF memory; no legacy pointer-based MR fallback was observed.
• ib_write_bw --use_cuda_dmabuf completed successfully on both GPUs.
• Two-rank NCCL formed a single communicator using separate GPUs.
• NCCL established connectors using NET/IB/.../GDRDMA.
• Collectives completed successfully from 64 MiB through 1 GiB.
• RDMA activity increased at workload scale.
• ConnectX-6 Lx IEEE 802.3 MAC counters read through MFT PPCNT grp=0 increased by only four frames and 576 octets per port during the NCCL workload.
• The workload-scale RDMA activity combined with negligible external MAC activity is consistent with ConnectX internal forwarding. The required physical-wire counter calibration remains follow-up validation, so this is not presented as proof of the adapter-internal route.
• No new Xid, assertion, IOMMU fault, AER error, BAR1 failure, or mlx5 reset was observed during the validated run.

This establishes functional operation of the experimental path on the recorded GB206/ConnectX-6 Lx configuration. Concurrency stress, broader hardware coverage, translated-IOMMU rejection testing, physical-wire counter calibration, and extended lifecycle validation remain outstanding.

The executed evidence is documented in:

• validation/nccl-same-host-gdr-transport-2026-08-06.md

A positive, negative, stress, teardown, and physical-wire calibration
procedure for systems already provisioned with compatible CUDA userspace is
provided in:

• validation/nccl-same-host-gdr-reproduction-plan.md

Source-level policy tests cover:

• feature disabled;
• coherent-path exclusion;
• coherent-only RmGpuDirectRdmaForceSPA selection;
• mapping-type requirements;
• static BAR1 and MIG conditions;
• inside, spanning, outside, empty, and overflowing ranges;
• importer DMA-mask coverage;
• identity-IOMMU requirements; and
• rejected and accepted P2PDMA distances.

Run them with:

make -C tests clean check

Hardware validation requested

This draft intentionally requests broader community validation.

Dimension	Validated	Requested
GPU	RTX 5060 Ti / GB206	GB202/203/205, Ada, Ampere, other supported GPUs
NIC	ConnectX-6 Lx	ConnectX-5/6 Dx/7 and other DMA-BUF importers
Topology	NODE	PHB, PIX, PXB, and cross-root/cross-socket SYS where accepted by host topology
IOMMU	Identity/passthrough	Confirm safe rejection with translated IOMMU
Workload	perftest and NCCL	Multi-host NCCL, GDS/cuFile, additional RDMA applications
Counter calibration	Negligible MAC deltas during same-host NCCL	Physical-wire control producing workload-scale MAC deltas
Concurrency	Basic application operation	Parallel registration/deregistration stress
Lifecycle	Normal teardown	Extended stress, reload, suspend/resume, and reboot

Please include module provenance, userspace provenance, topology and IOMMU evidence, complete endpoint logs, actual NCCL connector lines, counter deltas, cleanup results, and kernel-health logs. The reproduction plan defines the requested evidence and classification criteria.

Known limitations

• Experimental and disabled by default.
• Stock CUDA userspace does not exercise the experimental kernel path on the validated 610-series GeForce configuration; userspace investigation or modification is outside the scope of this PR.
• Requires static BAR1 and rejects ranges outside its usable window.
• Requires an identity IOMMU domain.
• MIG is excluded.
• Only GB206 with ConnectX-6 Lx in NODE topology has completed the full recorded validation.
• Other GPUs, NICs, topologies, translated IOMMU, GDS, VF-to-VF, and multi-host operation remain validation follow-ups.
• nvidia-peermem is not required and is not the path being enabled.

Review status

This should remain a draft until:

• the design and fail-closed policy receive review;
• the default-off negative control is independently reproduced;
• physical-wire counter calibration is completed before claiming adapter-internal forwarding;
• concurrent registration/deregistration stress is completed; and
• additional GPU, NIC, or topology results establish whether any further restrictions are required.