Kandinsky 6.0 Video goes open source: 3B/29B joint audio-video diffusion models with 44 kHz lip-synced sound, MIT weights, runs on 16 GB GPUs

Background

Joint audio-video generation (dialogue, sound effects and lip-sync in one pass) has been dominated by closed systems such as Veo 3.1 and Sora 2. Kandinsky Lab's Kandinsky 6.0 Video open-sources the full stack under MIT: Lite (3B) and Pro (29B) models generating 5-second clips with synchronized 44 kHz audio in T2AV and I2AV modes, with a built-in super-resolution model up to 1920×1080.

Architecture

A dual-stream CrossDiT reuses the Kandinsky 5.0 video stream and adds a newly trained audio stream, connected blockwise by bidirectional cross-attention (3D RoPE for video, 1D RoPE for audio). Pro splits into 19B video + 5B audio + 5B cross-attention. Training: audio stream pretrained alone, then joint audio-video training, SFT, RL post-training and two-stage distillation to 10 function evaluations.

Benchmarks

Per the technical report: beats open LTX 2.5 on most VABench metrics; clearly beats Kandinsky 5.0 Pro in human SBS; trades wins with Veo 3.1 Fast (significantly better artifacts and camera motion, behind on speech and sync); behind MiniMax H3 and Seedance 2.0 on visuals while competitive on speech. Distilled vs full model preference is 51% vs 49%. All numbers are self-reported.

Getting started

NVIDIA GPU + Python 3.13/3.14, then just setup, just download pro-distill, just generate "a cat on a mat". Block offload cuts Pro SD peak memory from 72.8 to 21.7 GiB, with a 16 GB preset; non-distilled Pro Full HD takes ~402 s on H100. Try the HF Space, the ComfyUI node or the Diffusers collection.