[WIP] Add experimental NVFP4 fake-QAT RL recipe

[zianglih]

Warning

Experimental / WIP. This is the initial NVFP4 fake-QAT RL wiring and has not yet been validated on a GPU devbox.

Summary

Adds an explicit --train-nvfp4-qat mode to the Qwen3 30B-A3B reference recipe, mirroring the existing INT4 QAT split between BF16 training weights and quantized rollout weights.

• Requires --rollout-nvfp4.
• Is mutually exclusive with native --train-nvfp4 and the other training precision modes.
• Is limited to Blackwell hardware.
• Sets OPEN_TRAINING_NVFP4_FAKE_QAT_FLAG=1 for the companion Megatron implementation.
• Keeps the existing NVFP4 checkpoint conversion, live weight export, and rollout paths unchanged.

Scope and limitations

Fake quantization is limited to routed MoE expert TE grouped FC1/FC2 weights. Dense MLPs, shared experts, sequential or legacy experts, activations, optimizer state, parameter gather, and the native TE FP4-GEMM training path are unchanged.

The recipe keeps BF16 master weights and GEMMs while the companion Megatron change applies TE NVFP4 quantize/dequantize during each forward with straight-through gradients.

Miles continues forwarding the driver's NVTE_* and FLASHINFER_* variables so conversion, QAT, and rollout can use consistent 4over6 scope, E4M3 bounds, error mode, and fast-math settings. The quantized checkpoint does not encode those choices, so the same environment must be restored when resuming.

The current Qwen recipe uses expert tensor parallel size 1. Generic expert-TP parity is not claimed yet: with expert TP greater than 1, the fake-QAT local-shard amax can differ from full-weight rollout conversion.

Dependencies

• Companion Megatron PR: radixark/Megatron-LM#75
• The intended CuTe DSL W4A16 rollout path assumes flashinfer-ai/flashinfer#4048.

The dependency CI marker is intentionally deferred until the GPU validation phase.

Validation

Passed static checks:

• pre-commit run --files scripts/run_qwen3_30b_a3b.py
• git diff origin/main...HEAD --check

Per request, no unit or functional tests, benchmarks, accuracy runs, or GPU/devbox validation were performed before opening this draft.

[gemini-code-assist]

Caution

The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased.