MBZUAI's IFM Open-Sources PAN 2 World Model: A Multimodal LLM Plans Who Does What and When, a 29B Diffusion Model Renders Up to 20 Seconds of Simulation, Top Open-Weight Physics-IQ Score of 49.38, Apache 2.0
On November 10, the Institute of Foundation Models (IFM) at Mohamed bin Zayed University of Artificial Intelligence (MBZUAI) released PAN 2, an open world model. It splits conceptual and perceptual reasoning: a multimodal LLM first writes the current scene and text actions as a time-stamped, structured world state, and a 29B latent diffusion model trained from scratch then renders up to 20 seconds of 704x1248, 24 fps simulated video, so an agent can rehearse each candidate plan before acting. Weights and code are released under Apache 2.0, with day-0 support in Diffusers and vLLM-Omni. IFM also says about two in three simulated multi-step plans still miss at least one step.
Key Takeaways
- Architecture: a multimodal LLM handles conceptual reasoning (timing, actions, camera motion) and a 29B MMDiT diffusion model handles perceptual reasoning, with Qwen3.5-9B as the text encoder and the Wan 2.2 VAE decoder (model card)
- Output: from one starting image plus text actions, up to 20 seconds of 704x1248 / 24 fps simulated video; weights and code under Apache 2.0
- Public benchmarks (IFM figures): 49.38 on Physics-IQ Verified in the leaderboard setting, best among open-weight models (Cosmos 3 Super 48.62, closed FLUX 3 51.11); PhyGround 4.74; RBench 57.04
- Multi-step plans: 34.00% full-plan completion on 100 everyday tasks, best among open-weight models (MiniMax-H3 32.50%), though its six-capability overall score of 84.48 trails MiniMax-H3 at 87.20
- Safety: the companion pan2-guardrail cut the share of harmful text-to-video requests that produce unsafe video from 33.9% to 1.5%, while also blocking 11.5% of benign requests
