Repository navigation

[feat]: FastH3 on DGX Spark and Apple Silicon - #1920

Merged

SolitaryThinker merged 111 commits into

hao-ai-lab:mainfrom

aryan5v:fasth3-spark-mlx

Oct 6, 2026

Commits on Oct 3, 2026

[feat]: load packed FastH3 NVFP4 and Comfy int8 VAE on Blackwell
aryan5v
committed
2d0ebf1
[feat]: add CompactH3 NVFP4 cookbook paths for Blackwell GPUs
aryan5v
committed
c38b4e2
[feat]: document CompactH3 NVFP4 on RTX 5090 and RTX PRO 6000
aryan5v
committed
a068234
[bugfix]: refuse FSDP NVFP4 retain and drop restating comments
aryan5v
committed
d224e52
[feat]: enable CompactH3 VAE compile on RTX PRO 6000
aryan5v
committed
d078283
[bugfix]: refuse zero-gate CompactH3 VSA
aryan5v
committed
a766348
[wip]: sm_120 H3 FP4 experiments: SP FP8 exchange path and Modal drivers
aryan5v
committed
1fb455d
[wip]: H3 single-GPU 5090/4090 shipping path
aryan5v
committed
0a4737c
[wip]: H3 4090 groundwork, step splice, encoder fallback on sm89
aryan5v
committed
4012c37
[feat]: H3 headline benchmark (480p 5 s, 768p 10 s) for GPU clusters and Modal
aryan5v
committed
4a72920
[bugfix]: ship app.py into the headline Modal image
aryan5v
committed
f0688b0
[bugfix]: ship app.py into the headline Modal image as local python source
aryan5v
committed
0a23563
[bugfix]: run the headline benchmark from app.py (one Modal app, no cross-module import)
aryan5v
committed
a87344c
[bugfix]: import os in gpu_worker; apply pre-commit formatting; ignore local headline results
aryan5v
committed
edbe18b
[bugfix]: H3 FP4 attention: new _build_block_mask signature; share Q/K/V quantization only at unit scale
aryan5v
committed
dd10fc7
[bugfix]: address #45 review: int64 FP8 epilogue offsets, order-independent NVFP4/FP8 conversion, splice scope
aryan5v
committed
a6faddd
[misc]: headline Modal runs keep the full log on the volume and report error lines
aryan5v
committed
ccebd84
[bugfix]: restore import os in fsdp_load (dropped in the rebase onto main)
aryan5v
committed
54f335c
[feat]: headline benchmark: HEADLINE_VAE_PARALLEL=1 decodes VAE tiles on every GPU
aryan5v
committed
4d33717
[feat]: headline benchmark: engine/pipeline placement overrides for memory-limited GPUs
aryan5v
committed
5386585
[feat]: headline Modal entrypoint takes extra env and a run tag
aryan5v
committed
cb79431
[bugfix]: H3 pinned swaps use exact-size cudaHostRegister arenas
aryan5v
committed
a97d23f
[wip]: exact-size pinned arenas for H3 offload and 4090 benchmarks
aryan5v
committed
a216768
[wip]: document 4090 setup validation and reproducible baselines
aryan5v
committed
a491f15
[wip]: record checkpoint revision and GPU driver in 4090 benchmark results
aryan5v
committed
4ae791c
[wip]: record completed 480p baseline and stage summary tooling
aryan5v
committed
21f9899
[perf]: reduce H3 attention activation copies and share FP8 input quantization
aryan5v
committed
c716408
[wip]: prototype tile-64 INT8 QK and FP8 PV attention on sm89
aryan5v
committed
bd713dd
[perf]: retain offloaded H3 VAEs on host until their consuming stages
aryan5v
committed
5bf9804
[perf]: add opt-in sm89 tile-64 BF16 and INT8 QK attention
aryan5v
committed
59e6946
[docs]: record cached 4090 timings and sm89 precision validation
aryan5v
committed
b3ab6e1
[wip]: validate tilewise FP8 values and dynamic probability scales
aryan5v
committed
994c220
[docs]: record resident-block timings and encoder constraints
aryan5v
committed
6b18535
[docs]: record five-second 4090 timing and FP8 encoder footprint
aryan5v
committed
92fc26c
[feat]: stream the H3 encoder for consumer VRAM limits
aryan5v
committed
d986589
[perf]: fuse serialized NVFP4 encoder weight dequantization
aryan5v
committed
887deaa
[perf]: release H3 fine-attention copies before the gated merge
aryan5v
committed
2aa19c4
[perf]: share VAE INT8 input preparation and avoid weight copies
aryan5v
committed
9c9f1ed
[docs]: record 12 GiB and streamed 4090 results
aryan5v
committed
e457b68
[perf]: read H3 INT8 sparse attention from existing layouts
aryan5v
committed
7a0d7d3
[perf]: fuse H3 INT8 VAE scaling and bias without intermediates
aryan5v
committed
3c0668f
[bugfix]: retain consumer attention environment parsing after core rebase
aryan5v
committed
a7f7ede
[test]: adapt consumer VSA validation to packed segment metadata
aryan5v
committed
87b22a5
[bench]: sample total GPU memory for consumer budget checks
aryan5v
committed
fb92af1
[docs]: record 42-second 4090 clips and consumer memory validation
aryan5v
committed
5ac85ad
[docs]: record Track B PR and strict 8 GiB budget limit
aryan5v
committed
98cdc7d

Commits on Oct 5, 2026

[docs]: FastH3 NVFP4 on RTX PRO 6000
aryan5v
committed
5a377ca
[bugfix]: validate sparse FP4 block lists before launch; restore the #44 sparse kernel tests
aryan5v
committed
3d727ff
[bugfix]: keep ModelOpt-quantized projections off the dense re-quantize path; converter parity test
aryan5v
committed
5efdd8b
[bugfix]: headline benchmarks record their decode mode; 4090 bench never reports unmeasured memory as 0 GiB
aryan5v
committed
74a190f
[misc]: merge upstream main into the FastH3 RTX release branch
aryan5v
committed
5db0cfa
[bugfix]: streamed NVFP4 encoder layers pass their FP4 scalars on the GEMM's device
aryan5v
committed
4784019
[bugfix]: preserve ModelOpt H3 activation calibration in NVFP4 export

Aryan Kumar
authored and
aryan5v
committed
7863e08
[bugfix]: declare NVFP4 static activation-scale cache attributes for mypy
aryan5v
committed
9ca4eba
[bugfix]: clone batched VAE tile output before the next CUDA-graph replay
aryan5v
committed
3c7e89c
[bugfix]: VSA zero-gate guard checks packed NVFP4 gates
aryan5v
committed
f4df26c
[bugfix]: sparse FP4 entry points validate block lists by default
aryan5v
committed
2964f15
[misc]: register FastH3 single-GPU switches in fastvideo.envs
aryan5v
committed
826b5d0
[test]: NVFP4 H3 encoder tests follow the pre-Blackwell de-quantized fallback
aryan5v
committed
a59e842
[misc]: drop the os imports layerwise offload no longer uses
aryan5v
committed
e9a35b2
[bugfix]: FP4 VSA path stays compatible with fastvideo-kernel releases that predate validate=
aryan5v
committed
84eb65d
[feat]: add resident FastH3 V2 Spark recipe

Aryan Kumar
authored and
aryan5v
committed
d60e209
[feat]: benchmark resident FastH3 Spark recipe

Aryan Kumar
authored and
aryan5v
committed
4874682
[docs]: pin Spark benchmark environment

Aryan Kumar
authored and
aryan5v
committed
8c01b5a
[bugfix]: retain per-stage CUDA peaks in Spark benchmarks

Aryan Kumar
authored and
aryan5v
committed
6ab83c3
[feat]: add resident pruned and paired Spark recipes

Aryan Kumar
authored and
aryan5v
committed
b1c3495
[bugfix]: preserve H3 release helpers after main integration

Aryan Kumar
authored and
aryan5v
committed
4ce033a
[feat]: support pruned FastH3 AdaLN in MLX

Aryan Kumar
authored and
aryan5v
committed
40f6c3c
[feat]: trim MLX H3 conditioner to required layers

Aryan Kumar
authored and
aryan5v
committed
159168d
[feat]: allow FP8 source for MLX H3 conversion

Aryan Kumar
authored and
aryan5v
committed
f4f903d
[docs]: add pruned FastH3 MLX conversion recipe

Aryan Kumar
authored and
aryan5v
committed
c5d233e
[feat]: add resident NVFP4 conditioning for MLX H3

Aryan Kumar
authored and
aryan5v
committed
89d3986
[feat]: enable experimental MXFP8 H3 conversion

Aryan Kumar
authored and
aryan5v
committed
cfe9d5f
[test]: check MXFP8 H3 checkpoint round trip

Aryan Kumar
authored and
aryan5v
committed
aef7cab
[bugfix]: omit unused layers from single-shard MLX encoder

Aryan Kumar
authored and
aryan5v
committed
d14f666
[test]: verify rank-16 H3 modulation and cache reload

Aryan Kumar
authored and
aryan5v
committed
2f400ed
[bugfix]: fence NVFP4 activation quantization on DGX Spark

Aryan Kumar
authored and
aryan5v
committed
7735e12
[docs]: explain calibrated and reproducible Spark NVFP4 inference

Aryan Kumar
authored and
aryan5v
committed
0301008
[bugfix]: keep Spark NVFP4 fence opaque to Dynamo

Aryan Kumar
authored and
aryan5v
committed
5830879
[bugfix]: order fresh NVFP4 global scales on DGX Spark

Aryan Kumar
authored and
aryan5v
committed
ad06a38
[bugfix]: carry H3 performance switches to Ray workers

Aryan Kumar
authored and
aryan5v
committed
c7969ea
[docs]: record verified Track C Spark release timings

Aryan Kumar
authored and
aryan5v
committed
fd8b213
[perf]: bound MLX H3 sparse attention gather memory

Aryan Kumar
authored and
aryan5v
committed
532fd6b
[perf]: batch 32 keys in opt-in MLX H3 sparse attention

Aryan Kumar
authored and
aryan5v
committed
e8bcd31
[bugfix]: apply H3 wired residency through the MLX wired API

Aryan Kumar
authored and
aryan5v
committed
ef1f277
[docs]: explain H3 Metal wired-memory limits

Aryan Kumar
authored and
aryan5v
committed
6457d71
[feat]: cache packed H3 encoder in MLX layout

Aryan Kumar
authored and
aryan5v
committed
ecc91b2
[feat]: support scaled native H3 floating quantization

Aryan Kumar
authored and
aryan5v
committed
10697e8
[perf]: parallelize H3 SIMD softmax across four lanes

Aryan Kumar
authored and
aryan5v
committed
32c8428
[bugfix]: publish the packed MLX encoder cache with one rename
aryan5v
committed
ff8bbcc
[bugfix]: MLX H3 resident preload skips dropped AdaLN weights; restore the wired limit on close
aryan5v
committed
e2f89da
[test]: update MLX H3 regression stubs for model_root and NVFP4 conditioner args
aryan5v
committed
a1c024b
[test]: update MLX fast-spatial stubs for model_root and decode tile arguments
aryan5v
committed
2539357
[misc]: one registry entry per FastH3 switch after stacking on the RTX env migration
aryan5v
committed
6ccdbc7
[docs]: Align Spark recipes with released FastH3 stacks
aryan5v
committed
e331f01
[docs]: Keep Spark launch table on released models
aryan5v
committed
3168746
[misc]: ship validated Track C MLX release paths
aryan5v
committed
ecbdbe5
[bugfix]: reject packed H3 transformer sources
aryan5v
committed
8f33ee9
[docs]: use released Track C INT6 Mac recipe
aryan5v
committed
ec7c971
[test]: MLX dequant-GEMM test allows CPU BF16 accumulation error
aryan5v
committed
17987f6

Commits on Oct 6, 2026

[misc]: merge current main into FastH3 Spark and MLX
aryan5v
committed
c36d002
[bugfix]: make FastH3 MLX limits reversible and restore CI coverage
aryan5v
committed
0881b38
[misc]: merge FastH3 RTX main into Spark and MLX PR
aryan5v
committed
0e9617c
Merge remote-tracking branch 'origin/main' into aryan/pr1920-merge-ready
aryan5v
committed
79674eb
[misc]: keep RTX showcase prompts in the merged base
aryan5v
committed
40cffbe
[ci]: bind MLX Gloo smoke tests to loopback and bound hangs
aryan5v
committed
f10bf3e
[bugfix]: fall back from inaccurate M1 SIMD VSA
aryan5v
committed
4e84449
[bugfix]: use reference VSA on small virtual Metal runners
aryan5v
committed
d10f4e4
[bugfix]: address review findings on FastH3 Spark and MLX paths

SolitaryThinker
and
claude
committed
ddd67fa
[misc]: read device memory through mx.device_info when available

SolitaryThinker
and
claude
committed
f06681a
[misc]: patch the gather budget function in the gather-budget test

SolitaryThinker
and
claude
committed
3170247