Skip to content

meta(distributed): multi-node and multi-GPU execution — backend API, IGG halo, tests, slab adjoint #79

Description

@kylebeggs

Umbrella for everything that makes a CartesianGrid operator run across more than one device or node. MDLA (single-node multi-GPU, CUDA-only) is done behind the _dist_scatter! / _dist_reduce! seam; what remains is the API to choose a backend, the multi-node backend itself, the tests that prove ≥3 partitions, and one perf gap on the slab transpose.

Sub-issues, in order

  1. feat(api): explicit distributed-backend selection (MDLABackend / IGGBackend), mirroring AbstractJVPBackend #47 — explicit backend selection (AbstractDistributedBackend, MDLABackend, IGGBackend), mirroring AbstractJVPBackend. First: it is the entry point feat(distributed): ImplicitGlobalGrid halo backend for multi-node CartesianGrid #46 must implement against, and today a second extension defining prepare_distributed(::AbstractOperator, ::Integer) would be ambiguous.
  2. feat(distributed): ImplicitGlobalGrid halo backend for multi-node CartesianGrid #46 — ImplicitGlobalGrid halo backend for multi-node. Carries the κ-ghost note from the PR feat(distributed): compact diffusion leaf on partition slabs #66 review (the IGG backend owes one setup-time exchange of the coefficient's Interface ghosts) and, from perf(algebra): constant-fold affine time-step operators (α·I + β·Laplacian) into one stencil pass #52, the comm/compute overlap idea: launch the interior stencil, run _exchange_storage! concurrently, then finish the boundary strips. Overlap is not to be built before a blocking exchange is measured.
  3. test(ci): officially instantiate test/multigpu (>=3-GPU distributed tests) #30 — instantiate test/multigpu (≥3-GPU distributed tests). Upstream blockers cleared; also lifts the min(NGPUS_MDLA, 2) caps in test/mdla_gpu.jl.
  4. perf(distributed): slab Diffusion adjoint runs the generic masked gather at ~5.7× the forward stencil #77 — slab Diffusion adjoint runs the generic masked gather at ~5.7× the forward stencil. Becomes a per-iteration cost the moment distributed reverse-mode AD lands.

Invariants that bind every sub-issue

  • _dist_reduce! must remain the exact transpose of _dist_scatter! across ranks, including the ∂/∂κ scatter-add — this is the part with no equivalent in the IGG/ParallelStencil ecosystem.
  • One halo exchange per operator application; coefficient exchanges belong at prepare time.
  • No new core dependency: MPI/IGG live in ext/.

🤖 Beep boop — Claude drew the org chart; Kyle is not delegating to himself.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions