[tp] Consolidate branches and validate vLLM/SGLang

.gitignore

@@ -362,4 +362,7 @@ marimo/_lsp/
 __marimo__/
 
 # Streamlit
-.streamlit/secrets.toml
+.streamlit/secrets.toml
+tests/deepep_matrix_work/
+deepep_fabric_archive/
+hook_archive/

CMakeLists.txt

@@ -43,3 +43,43 @@ set_target_properties(cuda_hook PROPERTIES
 install(TARGETS cuda_hook
     LIBRARY DESTINATION ${CMAKE_INSTALL_PREFIX}
 )
+
+include(CTest)
+if(BUILD_TESTING)
+    # Pure host-policy tests: no CUDA driver/runtime linkage, matching cuda_hook.
+    # EXCLUDE_FROM_ALL keeps them out of the default (editable pip) build; they
+    # are built explicitly (`cmake --build build --target <name>`) and run via
+    # ctest.
+    add_executable(vmm_ipc_reservation_test EXCLUDE_FROM_ALL
+        tests/vmm_ipc_reservation_test.cpp
+    )
+    target_include_directories(vmm_ipc_reservation_test PRIVATE
+        ${CMAKE_CURRENT_SOURCE_DIR}/include
+        ${CUDAToolkit_INCLUDE_DIRS}
+    )
+    add_test(NAME vmm_ipc_reservation COMMAND vmm_ipc_reservation_test)
+
+    add_executable(vmm_ipc_ownership_test EXCLUDE_FROM_ALL
+        tests/vmm_ipc_ownership_test.cpp
+    )
+    target_include_directories(vmm_ipc_ownership_test PRIVATE
+        ${CMAKE_CURRENT_SOURCE_DIR}/include
+    )
+    add_test(NAME vmm_ipc_ownership COMMAND vmm_ipc_ownership_test)
+
+    add_executable(vmm_ipc_chunk_key_test EXCLUDE_FROM_ALL
+        tests/vmm_ipc_chunk_key_test.cpp
+    )
+    target_include_directories(vmm_ipc_chunk_key_test PRIVATE
+        ${CMAKE_CURRENT_SOURCE_DIR}/include
+    )
+    add_test(NAME vmm_ipc_chunk_key COMMAND vmm_ipc_chunk_key_test)
+
+    add_executable(vmm_cleanup_test EXCLUDE_FROM_ALL
+        tests/vmm_cleanup_test.cpp
+    )
+    target_include_directories(vmm_cleanup_test PRIVATE
+        ${CMAKE_CURRENT_SOURCE_DIR}/include
+    )
+    add_test(NAME vmm_cleanup COMMAND vmm_cleanup_test)
+endif()

README.md

@@ -81,12 +81,110 @@ Foundry ships engine integrations under `foundry/python/foundry/integration/`. P
 
 | Engine | Single GPU | DP | TP | EP |
 |---|:---:|:---:|:---:|:---:|
-| vLLM | ✅ | ✅ | 🚧 | ✅ |
-| SGLang | ✅ | ✅ | 🚧 | ✅ |
+| vLLM | ✅ | ✅ | ✅ | ✅ |
+| SGLang | ✅ | ✅ | ✅ | 🚧 |
 | TensorRT-LLM | 🚧 | 🚧 | 🚧 | 🚧 |
 
 ✅ validated end-to-end (SAVE → LOAD → query)  ·  🚧 not yet
 
+**Provenance.** Actual `foundry-org/foundry` upstream `main` is the
+[0.0.2 commit `eef12012`](https://github.com/foundry-org/foundry/commit/eef12012aa0f85ae6079891144797b08c282152d),
+which has no official TP support. This `modal-projects` fork is based on that
+commit plus its
+[agent-guidance commit `21824649`](https://github.com/modal-projects/foundry/commit/21824649e711487a2c6af01cb631b3e41ccc32d8).
+Historical TP evidence came from fork draft
+[PR #4](https://github.com/modal-projects/foundry/pull/4) and
+[PR #2](https://github.com/modal-projects/foundry/pull/2), not upstream `main`;
+the pinned SGLang evidence below was produced on this branch.
+
+TP remains experimental and each check mark is limited to the pinned profile
+described below.
+
+vLLM dense TP=2 is verified only for Foundry
+`a8e43442421a80fea7991ee26f97f1003b170e29`, vLLM
+`4309c257d3f639e5490d3811293c890c61c76f29` with wheel parent
+`6cbe448eed751824d608faf9078ef84724d621c1`, and
+`Qwen/Qwen3-8B@b968826d9c46dd6066d109eabc6255188de91218`. The
+[six-phase 2×H100 run](https://modal.com/apps/modal-labs/rahul-dev/ap-YfKALdMwnyJQmzQoVvZSMU)
+passed all 17 checks. Each rank published 64 JSON/64 binary graph pairs; both
+fresh LOADs restored `64/64` graphs per rank at live offset `70724354048`.
+SAVE/SAVE2 inventories, offsets, and 68 semantic entries per rank matched; all
+265 fingerprinted files remained byte-for-byte unchanged through both LOADs.
+Baseline, SAVE2, LOAD, and LOAD2 returned identical sequential and concurrent
+text/token strings, every phase exited zero, and cleanup removed both the
+authoritative Dict marker and Volume path.
+
+This vLLM result is a single-host, exactly-two-Hopper, V1,
+`FULL_DECODE_ONLY`, PyNCCL P2P/legacy-CUDA-IPC profile. Custom, torch
+symmetric, FlashInfer, and fused all-reduce paths plus NCCL CUMEM/NVLS were
+disabled. TP>2, V2, multi-host, Ampere/Blackwell, MoE TP, and alternate
+collectives remain unsupported or unvalidated.
+
+The same exact candidate passed the
+[native H100 gate](https://modal.com/apps/modal-labs/rahul-dev/ap-0aeSCJNb6eHQhqdMkpyfXk):
+10 focused VMM/RNG tests and 823 core tests. It also passed the strict
+[2×H100 DeepEP matrix](https://modal.com/apps/modal-labs/rahul-dev/ap-Q8mb8U4EYuE5Jo6Zb0xwrH):
+the pinned PyTorch preflight, `fabric`, `nvl_ipc`, `nvl_ipc_prealloc`, and
+Modal CLI all returned zero. This is standalone transport/replay evidence, not
+general MoE TP or SGLang EP validation.
+
+SGLang TP=2 is validated only for the pinned single-host symmetric-memory
+profile: Foundry
+`a8e43442421a80fea7991ee26f97f1003b170e29`, unmodified upstream SGLang
+`a25164bda355250015c0ba6d8821f0d4b37abe7d`, and Qwen3-8B revision
+`b968826d9c46dd6066d109eabc6255188de91218`. A lightweight
+`foundry_sglang_plugin` entry point activates nine named `HookRegistry` targets
+with zero SGLang edits. It retains one torch symmetric-memory TP communicator
+allocated outside Foundry VMM, performs direct multimem all-gather into that
+stable buffer, captures one padded batch-size-256 full-decode graph per rank,
+and restores through SGLang's backend-driven capture lifecycle.
+
+The exact SGLang and model SHAs are validation provenance enforced by the Modal
+harness, which checks out the engine SHA and passes the model revision. They
+are not inspected by `_validate_profile`, and arbitrary shell-installed SGLang
+revisions are not rejected at startup. Shell recipe users must install the
+matching SGLang pin; runtime fail-closed checks cover topology, backend, and
+settings.
+
+Plain-NCCL fresh-process SGLang restore is explicitly unsupported. Disproving
+app
+[`ap-2qSQUKlY24L4RrQkMpUrmc`](https://modal.com/apps/modal-labs/rahul-dev/ap-2qSQUKlY24L4RrQkMpUrmc)
+passed structural checks—36 graphs restored per rank, matching graph counts,
+offsets, immutable archive bytes, and P2P/IPC topology—but both fresh LOAD
+processes produced the same corrupt decode output while baseline and SAVE2
+matched. Those observations reject this plain-NCCL restore path. Archived
+process-specific communicator and graph-sequence state is the leading forensic
+explanation, not treatment-proven causality; structural restoration alone is
+not output correctness.
+
+The focused symmetric SAVE→LOAD treatment
+[`ap-5wO9cFhRZ7ITsLDDv27Xfh`](https://modal.com/apps/modal-labs/rahul-dev/ap-5wO9cFhRZ7ITsLDDv27Xfh)
+and final five-phase matrix
+[`ap-6fYcSUFLOxocBSjNxakNGZ`](https://modal.com/apps/modal-labs/rahul-dev/ap-6fYcSUFLOxocBSjNxakNGZ)
+are `VERIFIED`. Final evidence includes one graph per rank at offset
+`68908220416`; 73 symmetric all-reduce plus one symmetric all-gather and zero
+NCCL collective nodes per rank; exact baseline/SAVE2/LOAD/LOAD2 sequential and
+batch-8 outputs; reproducible SAVE/SAVE2 semantic fingerprints; byte-immutable
+LOAD/LOAD2 archives; observed batch-1 and batch-8 replay; clean process exits
+and cleanup. Startup times were 217.2 s baseline, 103.0 s SAVE, 103.1 s SAVE2,
+and 61.0 s for each LOAD.
+
+The pinned vLLM profile remains single-host dense Qwen3-8B TP=2 with
+full-decode graphs and plain NCCL P2P / legacy CUDA IPC. The SGLang result does
+not establish general/upstream TP or plain-NCCL support. SGLang TP>2, multi-host,
+custom/NCCL-symmetric/fused collectives, CUMEM, NVLS, EP, DP-attention,
+speculative decode, LoRA, pdmux, and unpinned revisions remain unsupported;
+Ampere and Blackwell are unvalidated.
+
+Historical SGLang branch runs from draft
+[PR #4](https://github.com/modal-projects/foundry/pull/4) and
+[PR #2](https://github.com/modal-projects/foundry/pull/2) are **pre-port fork
+evidence only**, not evidence for the zero-edit pinned plugin.
+[Upstream Discussion #5](https://github.com/orgs/foundry-org/discussions/5)
+explains that NCCL initialization is not generally deterministic and identifies
+torch symmetric memory as the intended general backend; official TP support
+remains open in [issue #6](https://github.com/foundry-org/foundry/issues/6).
+
 The adapted vLLM / SGLang / TensorRT-LLM forks will be released alongside this repo at `foundry-org/vllm`, `foundry-org/sglang`, `foundry-org/TensorRT-LLM`.
 
 ### Performance

RELEASE.md

@@ -1,10 +1,136 @@
-# Foundry 0.0.2
+# Release Notes
+
+## Unreleased
+
+- **Experimental dense tensor parallel profiles.** vLLM dense TP=2 is verified
+  at Foundry `a8e43442421a80fea7991ee26f97f1003b170e29`, vLLM
+  `4309c257d3f639e5490d3811293c890c61c76f29`, wheel parent
+  `6cbe448eed751824d608faf9078ef84724d621c1`, and
+  `Qwen/Qwen3-8B@b968826d9c46dd6066d109eabc6255188de91218`. The
+  [final six-phase run](https://modal.com/apps/modal-labs/rahul-dev/ap-YfKALdMwnyJQmzQoVvZSMU)
+  used exactly two H100 80GB HBM3 GPUs in every phase with CUDA
+  `13.0.1-cudnn-devel-ubuntu24.04`, driver `580.95.05`, PyTorch
+  `2.11.0+cu130`, vLLM
+  `0.1.dev16924+g4309c257d.precompiled`, and NCCL `2.28.9+cuda13.0`.
+  All 17 checks passed.
+
+  The accepted archive had 64 JSON/64 `.cugraph` files per rank, 68 semantic
+  entries per rank, and 265 fingerprinted files. SAVE/SAVE2 reproduced
+  inventory, semantics, and per-rank offset `70724354048`; LOAD and LOAD2 each
+  restored `64/64` graphs on both ranks, matched that live offset, and left all
+  265 fingerprinted files byte-for-byte unchanged. Baseline, SAVE2, LOAD, and
+  LOAD2 had exact sequential and concurrent text/token-string equality. All six
+  child processes and the Modal CLI exited zero, then cleanup removed both the
+  Dict publication marker and Volume directory.
+
+  The vLLM envelope is single-host, exactly two Hopper GPUs, V1,
+  `FULL_DECODE_ONLY`, and PyNCCL P2P/legacy CUDA IPC. NCCL CUMEM/NVLS and
+  custom, torch-symmetric, FlashInfer, and fused all-reduce paths are disabled.
+  TP>2, V2, multi-host, Ampere/Blackwell, MoE TP, and alternate collectives are
+  unsupported or unvalidated. The exact candidate also passed
+  [native H100 validation](https://modal.com/apps/modal-labs/rahul-dev/ap-0aeSCJNb6eHQhqdMkpyfXk)
+  with 10 focused and 823 core tests, after a 320-test local gate plus compile,
+  pre-commit, diff, and direct-main lineage checks.
+
+  SGLang is distinct: its only validated TP result is
+  the pinned single-host Qwen3-8B TP=2 profile at Foundry
+  `a8e43442421a80fea7991ee26f97f1003b170e29`, unmodified upstream SGLang
+  `a25164bda355250015c0ba6d8821f0d4b37abe7d`, and Qwen3-8B revision
+  `b968826d9c46dd6066d109eabc6255188de91218`. The SGLang
+  path uses a retained torch symmetric-memory communicator outside Foundry VMM,
+  direct stable multimem gather, one padded batch-size-256 full-decode graph,
+  and backend-driven LOAD.
+
+  The exact engine/model SHAs are validation provenance enforced by the Modal
+  harness, not by `_validate_profile` or arbitrary shell-install startup
+  rejection. Shell recipe users must install the matching SGLang pin; runtime
+  fail-closed checks cover topology, backend, and settings.
+
+  Plain-NCCL fresh-process SGLang restore is rejected. In
+  [`ap-2qSQUKlY24L4RrQkMpUrmc`](https://modal.com/apps/modal-labs/rahul-dev/ap-2qSQUKlY24L4RrQkMpUrmc),
+  36 graphs per rank, matching offsets/inventory, immutable archive bytes, and
+  P2P/IPC topology all passed, yet both fresh LOAD processes produced the same
+  corrupt decode while baseline and SAVE2 matched. Archived process-specific
+  communicator and graph-sequence state is the leading forensic explanation,
+  not treatment-proven causality. The output observation rejects this
+  plain-NCCL path; the structural checks alone are not correctness evidence.
+
+  The focused treatment
+  [`ap-5wO9cFhRZ7ITsLDDv27Xfh`](https://modal.com/apps/modal-labs/rahul-dev/ap-5wO9cFhRZ7ITsLDDv27Xfh)
+  and final matrix
+  [`ap-6fYcSUFLOxocBSjNxakNGZ`](https://modal.com/apps/modal-labs/rahul-dev/ap-6fYcSUFLOxocBSjNxakNGZ)
+  are `VERIFIED`. The matrix recorded one graph per rank, offset `68908220416`,
+  73 symmetric all-reduce plus one symmetric all-gather and zero NCCL
+  collective nodes per rank, exact sequential and batch-8 output equality,
+  SAVE/SAVE2 semantic reproducibility, LOAD/LOAD2 byte immutability,
+  batch-1/batch-8 replay, phase times `217.2/103.0/103.1/61.0/61.0` seconds,
+  clean process exits, and completed cleanup.
+
+  Historical draft PR #4/#2 SGLang records are **pre-port fork evidence only**.
+  The pinned symmetric result is not general/upstream TP or plain-NCCL support.
+  General TP remains open in
+  [upstream issue #6](https://github.com/foundry-org/foundry/issues/6);
+  [Discussion #5](https://github.com/orgs/foundry-org/discussions/5) identifies
+  torch symmetric memory as the intended general backend.
+- **Zero-edit SGLang plugin activation and pinned-main graph binding.** SGLang
+  is used unmodified at the pinned commit; Foundry activates through the
+  dependency-light `foundry_sglang_plugin:activate` entry point and installs
+  nine `HookRegistry` targets. SAVE and LOAD both traverse the pinned upstream
+  `DecodeCudaGraphRunner.capture` lifecycle. The backend captures on SAVE and,
+  on LOAD, performs one pooled `start_graph_builds` followed by ordered
+  `finish_one_graph_load` calls without native capture or resave. The serialized
+  `MemoryPoolConfig` is reused by hooking
+  `KVCacheConfigurator._resolve_memory_pool_config`. This replaces the
+  monolithic `CudaGraphRunner` fork patch and its ~47-line `sglang/` footprint.
+  Qwen3.5-122B-A10B remains unsupported because its hybrid
+  linear-attention/MoE path requires capture-safe warmup, Mamba-state handling,
+  and a supported fused all-reduce route.
+  SGLang TP>2, multi-host, custom/NCCL-symmetric/fused collectives, CUMEM, NVLS,
+  EP, DP-attention, speculative decode, LoRA, pdmux, and unpinned revisions are
+  outside the validated profile.
+- **No-fabric DeepEP IPC.** The VMM-IPC bridge transports shareable file
+  descriptors with `SCM_RIGHTS`. A strict
+  [2×H100 matrix](https://modal.com/apps/modal-labs/rahul-dev/ap-Q8mb8U4EYuE5Jo6Zb0xwrH)
+  passed the pinned PyTorch CPU preflight plus exact SAVE/LOAD output and
+  sentinel-overwrite checks for `fabric`, `nvl_ipc`, and whole-preallocation
+  `nvl_ipc_prealloc`. It used Foundry
+  `a8e43442421a80fea7991ee26f97f1003b170e29`, vLLM
+  `4309c257d3f639e5490d3811293c890c61c76f29`, DeepEP
+  `29d31c095796f3c8ece47ee9cdcc167051bbeed9`, NVSHMEM `3.3.24`,
+  CUDA `13.0.1`, and PyTorch `2.11.0+cu130`. The preflight, all three cases,
+  and Modal CLI returned zero; the application status was `passed`.
+
+  This establishes the standalone transport/replay paths, not general vLLM
+  MoE serving, MoE TP, or SGLang EP. The current SGLang HookRegistry EP profile
+  remains quarantined. The IPC work extends
+  [upstream PR #3](https://github.com/foundry-org/foundry/pull/3). Historical
+  branch evidence covers BF16 on 2×H100; GB300/aarch64 FP8 and RDC relinking
+  reported in [upstream issue #1](https://github.com/foundry-org/foundry/issues/1)
+  remain outside that scope.
+- **FA2 cursor restoration.** When every graph in a load set has zero RNG
+  increment, LOAD skips unused generator-state restoration instead of
+  suspending the tracked allocation region. Nonzero-RNG graphs keep their
+  seed/offset tensors in deterministic memory. This incorporates
+  [upstream PR #4](https://github.com/foundry-org/foundry/pull/4) and the root
+  cause from [issue #2](https://github.com/foundry-org/foundry/issues/2), but
+  does not establish broad Ampere support.
+- **Restoration hardening.** Preallocated IPC chunks carry a generation in
+  their mapping identity so a freed and recreated chunk at the same address
+  cannot alias stale mappings. The Python graph context always passes the
+  required pool argument, with the default resolving to `(0, 0)`. CUDA's
+  zero-alignment sentinel is normalized to Foundry's 2 MiB VMM alignment for
+  reservation and replay, preserving the deterministic cursor with one
+  advance.
+
+---
+
+## Foundry 0.0.2
 
 SGLang graduates to a fully validated engine. This release brings the SGLang
 integration to parity with vLLM across single GPU, data parallel, and expert
 parallel — with a self-contained recipe and no vLLM build dependency for EP.
 
-## Highlights
+### Highlights
 
 - **SGLang single GPU / DP / EP all validated end-to-end.**
   SAVE → LOAD → query verified on single-GPU Qwen3-1.7B / 4B / 14B,
@@ -23,7 +149,7 @@ parallel — with a self-contained recipe and no vLLM build dependency for EP.
   `data_parallel_controller.py`). All save/load logic lives in the integration
   layer; the edits are inert unless `--foundry-graph-extension-config-path` is set.
 
-## Engine integrations
+### Engine integrations
 
 - **SGLang** — integration for SGLang v0.5.13. Working configurations: single
   GPU, data parallel (DP), expert parallel (EP, DeepEP low-latency + DP-attention with fa3). Self-contained recipes under `recipe/sglang/` (shared TOML pair +
@@ -45,7 +171,7 @@ parallel — with a self-contained recipe and no vLLM build dependency for EP.
   on LOAD, with `init_nvshmem_for_loaded_modules` run once on LOAD before any
   NVSHMEM-kernel graph replays.
 
-## Fixes
+### Fixes
 
 - **Per-rank VMM device binding (DP/TP/EP).** `set_allocation_region` binds to
   the current CUDA device, so the integration now calls `set_device(gpu_id)`
@@ -63,26 +189,22 @@ parallel — with a self-contained recipe and no vLLM build dependency for EP.
   capture loop never sets), and the FlashInfer per-bs pre-pass gated off for fa3
   while still populating `decode_cuda_graph_metadata` post-load for replay.
 
-## Docs
+### Docs
 
 - New `docs/sglang/` set (overview, direct-edits, hooks, memory-lifecycle,
   save-load-workflow, memory-consistency) and a self-contained `recipe/sglang/`
   README with install, run, performance, and troubleshooting.
 - Top-level README parallelism status table updated to mark SGLang single GPU,
   DP, and EP as validated.
 
----
-
-## Previous Releases
-
 ## Foundry 0.0.1
 
 First public release of Foundry — a CUDA-graph persistence library that
 captures an entire model's CUDA graphs (plus their device context: modules,
 workspaces, VMM layout) once and replays them at startup, eliminating compile,
 warmup, and capture from cold-start time.
 
-## Highlights
+### Highlights
 
 - **Deterministic memory layout — zero patching on graph load.**
   Foundry indirects memory allocation to the same reserved memory region
@@ -99,7 +221,7 @@ warmup, and capture from cold-start time.
   On load, the template is rebuilt once per group and instances are reconstructed
   on demand, keeping load fast and asynchronous.
 
-## Engine integrations
+### Engine integrations
 
 - **vLLM** — compatible with vLLM v0.21. Working configurations: single GPU,
   data parallel (DP), expert parallel (EP, DeepEP low-latency). End-to-end
@@ -111,31 +233,31 @@ warmup, and capture from cold-start time.
   integration layer (`foundry/integration/<engine>/`); engine forks contain
   only minimal hook calls.
 
-## Verified kernel & comm support
+### Verified kernel & comm support
 
 - **cuBLAS NVJET** kernels (Hopper+).
 - **torch.compile** modules.
 - **NVSHMEM / DeepEP** validated.
 - **DeepGEMM FP8 MoE** validated.
 
-## Dependency
+### Dependency
 
 - **PyTorch 2.11.0** (compatible with 2.9 – 2.11).
 - **CUDA 12+**, CMake 4.0+, Boost.
 
-## Documentation
+### Documentation
 
 - Integration design notes and per-engine recipes under `docs/` and `recipe/`.
 - vLLM recipe README covers save/load workflow, archive layout, and required
   env settings.
 
-## Repository hygiene
+### Repository hygiene
 
 - Open-source pre-commit hooks (ruff, ruff-format, clang-format,
   markdownlint, actionlint, DCO sign-off).
 - Smoke tests covering re-export imports and archive round-trip.
 
-## Roadmap
+### Roadmap
 
 - Adapted vLLM and SGLang forks published alongside the release.
 - Tensor parallel support.