[perf] Enable cached H3 generation on RAM-limited RTX hosts - #48

Open

aryan5v wants to merge 69 commits into

fasth3-rtx-4090-launch

Commits on Oct 3, 2026

[feat]: load packed FastH3 NVFP4 and Comfy int8 VAE on Blackwell
aryan5v
committed
2d0ebf1
[feat]: add CompactH3 NVFP4 cookbook paths for Blackwell GPUs
aryan5v
committed
c38b4e2
[feat]: document CompactH3 NVFP4 on RTX 5090 and RTX PRO 6000
aryan5v
committed
a068234
[bugfix]: refuse FSDP NVFP4 retain and drop restating comments
aryan5v
committed
d224e52
[feat]: enable CompactH3 VAE compile on RTX PRO 6000
aryan5v
committed
d078283
[bugfix]: refuse zero-gate CompactH3 VSA
aryan5v
committed
a766348
[wip]: sm_120 H3 FP4 experiments: SP FP8 exchange path and Modal drivers
aryan5v
committed
1fb455d
[wip]: H3 single-GPU 5090/4090 shipping path
aryan5v
committed
0a4737c
[wip]: H3 4090 groundwork, step splice, encoder fallback on sm89
aryan5v
committed
4012c37
[feat]: H3 headline benchmark (480p 5 s, 768p 10 s) for GPU clusters and Modal
aryan5v
committed
4a72920
[bugfix]: ship app.py into the headline Modal image
aryan5v
committed
f0688b0
[bugfix]: ship app.py into the headline Modal image as local python source
aryan5v
committed
0a23563
[bugfix]: run the headline benchmark from app.py (one Modal app, no cross-module import)
aryan5v
committed
a87344c
[bugfix]: import os in gpu_worker; apply pre-commit formatting; ignore local headline results
aryan5v
committed
edbe18b
[bugfix]: H3 FP4 attention: new _build_block_mask signature; share Q/K/V quantization only at unit scale
aryan5v
committed
dd10fc7
[bugfix]: address #45 review: int64 FP8 epilogue offsets, order-independent NVFP4/FP8 conversion, splice scope
aryan5v
committed
a6faddd
[misc]: headline Modal runs keep the full log on the volume and report error lines
aryan5v
committed
ccebd84
[bugfix]: restore import os in fsdp_load (dropped in the rebase onto main)
aryan5v
committed
54f335c
[feat]: headline benchmark: HEADLINE_VAE_PARALLEL=1 decodes VAE tiles on every GPU
aryan5v
committed
4d33717
[feat]: headline benchmark: engine/pipeline placement overrides for memory-limited GPUs
aryan5v
committed
5386585
[feat]: headline Modal entrypoint takes extra env and a run tag
aryan5v
committed
cb79431
[bugfix]: H3 pinned swaps use exact-size cudaHostRegister arenas
aryan5v
committed
a97d23f
[wip]: exact-size pinned arenas for H3 offload and 4090 benchmarks
aryan5v
committed
a216768
[wip]: document 4090 setup validation and reproducible baselines
aryan5v
committed
a491f15
[wip]: record checkpoint revision and GPU driver in 4090 benchmark results
aryan5v
committed
4ae791c
[wip]: record completed 480p baseline and stage summary tooling
aryan5v
committed
21f9899
[perf]: reduce H3 attention activation copies and share FP8 input quantization
aryan5v
committed
c716408
[wip]: prototype tile-64 INT8 QK and FP8 PV attention on sm89
aryan5v
committed
bd713dd
[perf]: retain offloaded H3 VAEs on host until their consuming stages
aryan5v
committed
5bf9804
[perf]: add opt-in sm89 tile-64 BF16 and INT8 QK attention
aryan5v
committed
59e6946
[docs]: record cached 4090 timings and sm89 precision validation
aryan5v
committed
b3ab6e1
[wip]: validate tilewise FP8 values and dynamic probability scales
aryan5v
committed
994c220
[docs]: record resident-block timings and encoder constraints
aryan5v
committed
6b18535
[docs]: record five-second 4090 timing and FP8 encoder footprint
aryan5v
committed
92fc26c
[feat]: stream the H3 encoder for consumer VRAM limits
aryan5v
committed
d986589
[perf]: fuse serialized NVFP4 encoder weight dequantization
aryan5v
committed
887deaa
[perf]: release H3 fine-attention copies before the gated merge
aryan5v
committed
2aa19c4
[perf]: share VAE INT8 input preparation and avoid weight copies
aryan5v
committed
9c9f1ed
[docs]: record 12 GiB and streamed 4090 results
aryan5v
committed
e457b68
[perf]: read H3 INT8 sparse attention from existing layouts
aryan5v
committed
7a0d7d3
[perf]: fuse H3 INT8 VAE scaling and bias without intermediates
aryan5v
committed
3c0668f
[bugfix]: retain consumer attention environment parsing after core rebase
aryan5v
committed
a7f7ede
[test]: adapt consumer VSA validation to packed segment metadata
aryan5v
committed
87b22a5
[bench]: sample total GPU memory for consumer budget checks
aryan5v
committed
fb92af1
[docs]: record 42-second 4090 clips and consumer memory validation
aryan5v
committed
5ac85ad
[docs]: record Track B PR and strict 8 GiB budget limit
aryan5v
committed
98cdc7d

Commits on Oct 5, 2026

[docs]: FastH3 NVFP4 on RTX PRO 6000
aryan5v
committed
5a377ca
[bugfix]: validate sparse FP4 block lists before launch; restore the #44 sparse kernel tests
aryan5v
committed
3d727ff
[bugfix]: keep ModelOpt-quantized projections off the dense re-quantize path; converter parity test
aryan5v
committed
5efdd8b
[bugfix]: headline benchmarks record their decode mode; 4090 bench never reports unmeasured memory as 0 GiB
aryan5v
committed
74a190f
[misc]: merge upstream main into the FastH3 RTX release branch
aryan5v
committed
5db0cfa
[bugfix]: streamed NVFP4 encoder layers pass their FP4 scalars on the GEMM's device
aryan5v
committed
4784019
[bugfix]: preserve ModelOpt H3 activation calibration in NVFP4 export

Aryan Kumar
authored and
aryan5v
committed
7863e08
[bugfix]: declare NVFP4 static activation-scale cache attributes for mypy
aryan5v
committed
9ca4eba
[bugfix]: clone batched VAE tile output before the next CUDA-graph replay
aryan5v
committed
3c7e89c
[bugfix]: VSA zero-gate guard checks packed NVFP4 gates
aryan5v
committed
f4df26c
[bugfix]: sparse FP4 entry points validate block lists by default
aryan5v
committed
2964f15
[misc]: register FastH3 single-GPU switches in fastvideo.envs
aryan5v
committed
826b5d0
[test]: NVFP4 H3 encoder tests follow the pre-Blackwell de-quantized fallback
aryan5v
committed
a59e842
[misc]: drop the os imports layerwise offload no longer uses
aryan5v
committed
e9a35b2
[bench]: allow the launch benchmark seed to be specified
aryan5v
committed
0453c08
[offload]: honor pageable host storage for layerwise models
aryan5v
committed
1d5d9c9
[bench]: measure memory on cgroup v1 pods
aryan5v
committed
00bd0a1
[bugfix]: keep CPU-targeted checkpoint loads off the GPU
aryan5v
committed
659a484
[bench]: preserve peak memory for failed generations
aryan5v
committed
2c0cd2a
[bench]: build exact V2 modulation tables with provenance
aryan5v
committed
3c011ef
[bench]: render a showcase prompt once without extra warmup clips
aryan5v
committed
1299c7d
[perf]: retain mapped H3 encoder weights on pageable hosts
aryan5v
committed
73ee8c2
[bugfix]: preserve padded vocabulary checkpoint validation
aryan5v
committed
cda0579