feat: update Minimax-H3 support

.gitignore

@@ -22,6 +22,7 @@ CLAUDE.md
 # local benchmark outputs
 /test_outputs/
 /.wty_results/
+/assets/minimax-h3-ref2va/
 
 # local-only CPU profiling / benchmark tooling (not for commit)
 /scripts/run_image_cmp.sh

CMakeLists.txt

@@ -29,6 +29,7 @@ option(ED_ENABLE_CUDNN_SDPA "Enable experimental cuDNN SDPA backend for CUDA fla
 option(ED_ENABLE_CUDA_NORM "Enable edge CUDA norm fast paths for ggml CUDA backend" ON)
 option(ED_ENABLE_CUDA_ROPE "Enable edge CUDA RoPE fast paths for ggml CUDA backend" ON)
 option(ED_ENABLE_CUDA_MODULATION "Enable edge CUDA fused modulation fast paths for ggml CUDA backend" ON)
+option(ED_ENABLE_CUDA_GRAPHS "Compile CUDA Graph support for opt-in edge inference" ON)
 option(ED_ENABLE_CUDA_SAGE_ATTN "Enable SageAttention2-style INT8-QK + F16-PV fused attention for ggml CUDA backend" OFF)
 option(ED_ENABLE_PARALLEL "Enable edge-dit lightweight parallel runtime" OFF)
 option(ED_ENABLE_NCCL "Enable NCCL communication backend" OFF)
@@ -254,6 +255,9 @@ if(ED_ENABLE_CUDA_MODULATION AND ED_GGML_CUDA)
 endif()
 
 if(ED_GGML_CUDA)
+    if(ED_ENABLE_CUDA_GRAPHS)
+        set(GGML_CUDA_GRAPHS ON CACHE BOOL "ggml: use CUDA graphs" FORCE)
+    endif()
     find_package(CUDAToolkit REQUIRED)
     target_sources(ggml-cuda
         PRIVATE
@@ -291,9 +295,10 @@ if(ED_ENABLE_CUDNN_SDPA)
         PRIVATE
             ${CMAKE_CURRENT_SOURCE_DIR}/src/core/backend/cuDNN/ed_cudnn_conv2d.cu
             ${CMAKE_CURRENT_SOURCE_DIR}/src/core/backend/cuDNN/ed_cudnn_conv3d.cu
+            ${CMAKE_CURRENT_SOURCE_DIR}/src/core/backend/cuDNN/ed_cudnn_conv_transpose_1d.cu
             ${CMAKE_CURRENT_SOURCE_DIR}/src/core/backend/cuDNN/ed_cudnn_sdpa.cu
     )
-    target_compile_definitions(ggml-cuda PRIVATE ED_ENABLE_CUDNN_SDPA ED_ENABLE_CUDNN_CONV2D ED_ENABLE_CUDNN_CONV3D)
+    target_compile_definitions(ggml-cuda PRIVATE ED_ENABLE_CUDNN_SDPA ED_ENABLE_CUDNN_CONV2D ED_ENABLE_CUDNN_CONV3D ED_ENABLE_CUDNN_CONV_TRANSPOSE_1D)
     target_include_directories(ggml-cuda
         PRIVATE
             ${CMAKE_CURRENT_SOURCE_DIR}/src/core/backend/cuDNN
@@ -391,7 +396,7 @@ target_compile_definitions(edgedit
 )
 
 if(ED_ENABLE_CUDNN_SDPA)
-    target_compile_definitions(edgedit PRIVATE ED_ENABLE_CUDNN_SDPA ED_ENABLE_CUDNN_CONV2D ED_ENABLE_CUDNN_CONV3D)
+    target_compile_definitions(edgedit PRIVATE ED_ENABLE_CUDNN_SDPA ED_ENABLE_CUDNN_CONV2D ED_ENABLE_CUDNN_CONV3D ED_ENABLE_CUDNN_CONV_TRANSPOSE_1D)
 endif()
 
 if(ED_ENABLE_CUDA_MODULATION AND ED_GGML_CUDA)

README.md

@@ -19,9 +19,9 @@ runtime for image generation, image editing, and video generation across local,
 edge, and resource-constrained deployment environments.
 
 It supports major DiT model families including **FLUX.1, FLUX.2 [klein] 4B,
-Stable Diffusion 3/3.5, Qwen-Image, and Wan**, with explicit control over model
-loading, memory usage, graph execution, quantization, device placement, and
-backend selection.
+Stable Diffusion 3/3.5, Qwen-Image, Wan, and MiniMax-H3**, with explicit control
+over model loading, memory usage, graph execution, quantization, device
+placement, and backend selection.
 
 ## Features
 
@@ -34,7 +34,7 @@ backend selection.
 - **Unified across tasks and model families**
   - **Text-to-image**, **image editing**, and **video generation** in one runtime
   - SD3/SD3.5, FLUX.1, FLUX.1-Kontext, FLUX.2 [klein] 4B, Qwen-Image,
-    Qwen-Image-Edit, and Wan 2.1
+    Qwen-Image-Edit, Wan 2.1, and MiniMax-H3 video+audio
   - **Few-step distilled models** auto-detected — Turbo / Lightning / schnell default to a **4–8 step** schedule
   - Shared **C API, CLI, HTTP server, and Python** interfaces across every family
 
@@ -54,6 +54,7 @@ backend selection.
 
 ## Latest News
 
+- **2026-08-14:** 🚀 Added **MiniMax-H3 FL2VA and Ref2VA video+audio generation** with image, video, embedded/paired audio, and mixed references; full or pruned BF16 DiTs, persistent Q8_0 conversion, Q4_K_M weights, and automatic VRAM fitting are supported ([usage and H200 results](docs/minimax-h3.md)).
 - **2026-08-05:** 🚀 Completed the **RTX 4090 (24 GB) benchmark** — full cross-system speed / VRAM / image-quality across text-to-image, editing, and video ([results](docs/performance-4090.md)).
 - **2026-07-30:** 🚀 Added **per-component offload** (`--dit-offload` / `--text-encoder-offload` / `--vae-offload`), unifying all offload paths on one semantics.
 - **2026-07-29:** 🚀 Added **`--auto-fit`** — one flag picks DiT quantization *and* per-component placement to fit a hard VRAM budget.
@@ -80,6 +81,7 @@ the current support commitment unless documented in
 | **Qwen-Image** | Text-to-image | `Qwen/Qwen-Image` | Qwen-Image Lightning *(LoRA)* | Supported |
 | **Qwen-Image-Edit** | Image editing | `Qwen/Qwen-Image-Edit` | Qwen-Image-Edit Lightning *(LoRA)* | Supported |
 | **Wan 2.1** | Video generation | `Wan-AI/Wan2.1-T2V-1.3B` (and 14B) | Wan2.1-T2V-1.3B Distill | Supported (Vulkan still optimizing) |
+| **MiniMax-H3** | Video + audio generation | FL2VA / Ref2VA component checkpoints | — | Supported (CUDA validated) |
 
 Distilled checkpoints load through the same pipeline as the base model and are
 **auto-detected** (default **4–8 steps** when `--steps` is unset). Most ship as
@@ -158,6 +160,17 @@ Load time follows each runtime's reported initialization boundary and may
 reflect different weight materialization or memory-mapping strategies.
 Generation latency is the primary cross-runtime performance metric.
 
+MiniMax-H3 has a separate 124-frame H200 benchmark because it generates video
+and audio rather than one image. In the current resident-component BF16
+comparison, Edge is faster than Diffusers in all four FL2VA and all four Ref2VA
+generation paths while using less peak VRAM. FL2VA text-to-video takes
+`51.396s` versus `53.986s`; Ref2VA image and mixed-reference generation reach
+`1.08x` and `1.06x` speedups. Full and pruned BF16 DiTs are supported directly,
+and either can be converted once to persistent Q8_0 GGUF. `--auto-fit` has also
+been validated down to a 24 GB VRAM budget. See [MiniMax-H3 usage and
+performance](docs/minimax-h3.md) for inputs, weights, memory placement, and the
+complete comparison.
+
 ## Open-Source Interfaces
 
 edge-dit.cpp exposes the same runtime through several public integration

docs/cli.md

@@ -273,6 +273,7 @@ Video flags:
 ```text
 --video
 --frames <int>
+--video-duration <seconds>  # MiniMax-H3 only
 --fps <int>
 --video-format auto|avi|mp4|mov|mkv|webm
 ```
@@ -366,6 +367,7 @@ Memory-oriented flags:
 --vae-offload
 --max-vram <GB>
 --auto-allocate
+--auto-fit
 ```
 
 `--vae-tiling` is tri-state `on|off|auto` and defaults to `auto`, which enables
@@ -401,6 +403,31 @@ requested resolution (`-W`/`-H`, and `--frames` for video) to size the resident
 headroom, so pass the generation size you intend to use. Without a size it falls
 back to a conservative fixed headroom.
 
+MiniMax-H3 applies this policy to four independently placeable components: DiT,
+Qwen3-VL, video VAE, and audio VAE. Its measurement graph conservatively includes
+mixed image, video, paired-audio, and additional-audio conditioning, so the same
+placement remains valid across FL2VA and Ref2VA workflows. The MiniMax-H3 video
+VAE also keeps its model-specific fixed `16x16` tiling regardless of the generic
+`--vae-tiling`/`--vae-tile-size` values.
+
+For MiniMax-H3, `--video-duration <seconds>` converts a requested duration at
+the model's fixed 24 fps to the nearest legal `17k+5` frame count. Keep using
+`--video-frames` when an exact legal count is required; the two options are
+mutually exclusive.
+
+```bash
+# MiniMax-H3 Q8_0 under a 40 GiB hard placement budget.
+./build-cuda/bin/ed-cli --video \
+  --diffusion-model /path/to/minimax_h3_fl2va-diffusers-Q8_0.gguf \
+  --vae /path/to/minimax_h3_video_vae_fp16.safetensors \
+  --audio-vae /path/to/minimax_h3_audio_vae_fp32.safetensors \
+  --llm /path/to/qwen3vl_32b_minimax_h3-Q8_0.gguf \
+  --auto-fit --max-vram 40 --vae-tiling auto \
+  -W 864 -H 480 --video-duration 5 --steps 20 --cfg-scale 1 \
+  --prompt "A cinematic sunset over layered mountain ridges." \
+  --video-format mp4 --output minimax-h3-autofit.mp4
+```
+
 ```bash
 # Fully automatic under an 8 GiB budget — system picks DiT quant + placement.
 ./build-cuda/bin/ed-cli \
@@ -700,4 +727,4 @@ See [API and bindings](api.md) for HTTP endpoints and curl examples.
 - [performance (RTX 4090)](performance-4090.md)
 - [performance (H200)](performance-H200.md)
 - [API and bindings](api.md)
-- [Development and contributing](development.md)
+- [Development and contributing](development.md)