[perf] MiniMax H3: return uint8 frames from the decode worker

[stayinalive181]

Purpose

The MiniMax-H3 decode stage returns a CPU fp32 pixel buffer that crosses the multiprocess-executor boundary (4.27 GB for a 345-frame 1344x768 clip), and the main process casts it to uint8 immediately afterwards. Casting inside the worker ships 1.07 GB instead. Lossless: the clamp matches the post-decode path, so the pixels are unchanged.

This is one of the runtime patches published in hlander-ai/minimax-h3 (Apache-2.0), where the post-decode/encode/save stages went from 3.09 s to 1.98 s per 345-frame clip on 8xH200 (together with a worker-side encode change that is not part of this PR). It applies to every hardware target, not only Hopper.

Changes

• fastvideo/pipelines/basic/minimax_h3/stages/minimax_h3_decoding.py: on the output rank, quantize the decoded fp32 buffer to uint8 (mul_(255).clamp_(0, 255)) before handing it to the executor. The TAEH3 preview path and output_type="latent" are untouched.
• fastvideo/entrypoints/video_generator.py: post-decode passes uint8 worker output through untouched (float output keeps the on-device quantize path from [perf] Quantize frames to uint8 on-device before the post-decode D->H copy #1362); the optional samples mirror stays on its [0, 1] float contract (copy_ then div_(255), or float().div_(255) on the shape-mismatch path).
• fastvideo/tests/entrypoints/test_video_generator.py: test_generate_single_video_accepts_uint8_worker_output checks that frames come back byte-identical and that samples is [0, 1] float.

Test Plan

ruff check fastvideo/entrypoints/video_generator.py \
  fastvideo/pipelines/basic/minimax_h3/stages/minimax_h3_decoding.py \
  fastvideo/tests/entrypoints/test_video_generator.py
yapf --diff <same three files>
codespell --toml pyproject.toml <same three files>
python -m pytest fastvideo/tests/entrypoints/test_video_generator.py -q

Test Results

Test output

ruff:      All checks passed!
yapf:      no diff
codespell: clean
pytest:    33 passed in 96.85s   (CPU, macOS arm64, torch 2.14.0)

The same worker-side cast has been running in a downstream 4xH100 deployment (Windflow streaming worker on FastVideo b2db0c0, which consumes the uint8 frames through return_frames=True), including a 30-minute continuous run of 51 clips.

Checklist

• ruff, yapf and codespell run with the repository configuration on the touched files (pre-commit itself is not installed on the authoring machine)
• I added or updated tests for my changes
• I updated documentation if needed (none needed)
• I considered GPU memory impact of my changes (none: the cast happens on the CPU buffer; the worker briefly holds the 1.07 GB uint8 tensor next to the 4.27 GB fp32 buffer before the latter is released)

For model/pipeline changes, also check:

• SSIM regression not run here; the frames are byte-identical by construction (same *255, clamp and cast, moved across the process boundary) and the unit test asserts exact equality
• Support matrix unchanged

[github-actions]

Welcome to FastVideo! Thanks for your first pull request.

How our CI works:

PRs run a three-tier CI system:

1. Pre-commit — formatting (yapf), linting (ruff), type checking (mypy). Runs immediately on every PR.
2. Fastcheck — six core GPU lanes run automatically via Buildkite (~10-15 min).
3. Merge gate — a reviewer adds ready; changed paths select only the relevant integration, training, golden, or SSIM coverage.

Before your PR is reviewed:

• pre-commit run --all-files passes locally
• You've added or updated tests for your changes
• The PR description explains what and why

If pre-commit fails, a bot comment will explain how to fix it. Fastcheck and merge-gate results appear in the Checks section below.

Useful links:

• Contributing Guide
• Development Roadmap
• Slack

[mergify]

Merge Protections

🔴 1 of 1 protections blocking · waiting on 👀 reviews and 🤖 CI

	Protection	Waiting on
🔴	PR merge requirements	👀 reviews and 🤖 CI

🔴 PR merge requirements

Waiting for

• #approved-reviews-by>=1
• check-success=fastcheck-passed
• check-success=full-suite-passed

This rule is failing.

• #approved-reviews-by>=1
• check-success=fastcheck-passed
• check-success=full-suite-passed
• check-success~=pre-commit
• title~=(?i)^\[(feat|feature|bugfix|fix|refactor|perf|ci|doc|docs|misc|chore|kernel|new.?model|skill|skills|infra)\]

[SolitaryThinker]

Two issues need addressing: training validation still assumes normalized float output, and existing decode-stage tests contradict the new uint8 contract.

Validation: reproduced uint8 overflow with a CPU PyTorch arithmetic check. Attempted the entrypoint and H3 streaming-stage pytest suites, but collection was blocked by a Triton dependency requiring an active GPU driver. No GPU inference or SSIM run was performed.

[SolitaryThinker]

[P1] Handle uint8 output in training validation before changing the stage contract

This also changes direct pipeline calls, which bypass the new VideoGenerator handling. The training validation callback reads output_batch.output directly and still applies (x * 255).numpy().astype(np.uint8) in fastvideo/train/callbacks/validation.py:1575. The shipped examples/train/configs/overfit_minimax_h3_t2va.yaml selects this H3 pipeline, so validation videos now suffer uint8 overflow: decoded values [0.25, 0.5, 0.75, 1.0] produce pixel bytes [63, 127, 191, 255], which validation transforms into [193, 129, 65, 1]. Please update this consumer to pass uint8 through and add regression coverage.

[SolitaryThinker]

[P2] Update existing stage tests for the new uint8 output contract

fastvideo/tests/stages/test_minimax_h3_vae_streaming.py still asserts that the serial result is the original float output buffer with values of 0.25 (lines 112–114), and that the parallel result contains 0.5 (line 192). This conversion instead returns a new uint8 tensor containing 63 or 127, respectively, so both tests contradict the new behavior. Please update them to verify quantization while preserving their CPU-buffer and rank-participation checks; the added entrypoint test does not exercise either stage path.