Hao AI Lab open-sources FastH3 Omni: MiniMax H3 distilled to 8 steps for reference-guided video and audio, with 5.6-11.9x lower end-to-end latency

Background

Reference-guided video with sound (a few portraits, a motion clip and a voice sample turned into a short clip with dialogue) is one of the most practical uses of video models, but MiniMax H3-class models run 50 diffusion steps, so a 15-second clip takes over two minutes even on eight GB200s. That rules out real time and most local GPUs. UCSD Hao AI Lab's FastVideo team, with Nuva Lab and NVIDIA's FastGen team, built FastH3 Omni to close that gap.

Architecture

FastH3 Omni combines two distillation methods: PDD lets one forward pass cover several denoising steps, and DMD pulls output quality back toward the base model, ending at eight transformer evaluations with guidance 1.0. A request can mix up to nine images, three videos and three audio clips, using <Picture 1>, <Video 1> and <Audio 1> tags to say whose face, which motion and whose voice goes where. The team also trained 4-bit NVFP4 VAE decoders in 36-layer quality and 8-layer fast tiers.

Benchmarks

Team figures: across 27 matched workloads with identical prompts, references and 1344x768 output on 8x GB200, end-to-end latency falls 5.6-11.9x, timed until the MP4 is written. A 5-second single-image request goes from 35.44s to 5.67s, and a 15-second one from 139.69s to 14.14s. The team says its examples are qualitative and not proof of output parity; no public quantitative quality benchmark has been published.

Getting started

BF16 weights are at FastVideo/FastVideo-FastH3-Omni-8-Step-V1 on Hugging Face, served with UniServe on GB200/B200, RTX PRO 6000 or H100/H200. Locally, FastVideo with NVFP4 QAD weights runs on DGX Spark or RTX 5090 at 480p by default, and FP8 weights cover RTX 4090. ComfyUI works from day one. The weights fall under the MiniMax-H3 Community License, so check terms before commercial use; a dedicated local AI post is due Oct 21.