MBZUAI's IFM Open-Sources PAN 2 World Model: A Multimodal LLM Plans Who Does What and When, a 29B Diffusion Model Renders Up to 20 Seconds of Simulation, Top Open-Weight Physics-IQ Score of 49.38, Apache 2.0

Core Background and Industry Pain Points

A language model predicts the next word, but robots and agents need to predict what the world will look like after an action. Today's video generators look good but often break physics, mix up the order of actions, or change objects when the camera turns, which makes them hard to trust for rehearsing a decision. On November 10, the Institute of Foundation Models (IFM) at MBZUAI open-sourced PAN 2, aiming for a world model you can inspect and reproduce, built to simulate first and act second.

Architecture Highlights and Underlying Mechanisms

PAN 2 follows IFM's Generative Latent Prediction (GLP) architecture in two steps. A multimodal LLM reads the starting image and text actions and writes a time-stamped, structured world state that says who does what and when and how the camera moves. A 29B-parameter MMDiT diffusion model trained from scratch then fills in appearance and motion, and the Wan 2.2 VAE decodes up to 20 seconds of video. Camera motion is described separately, so the scene stays consistent when the viewpoint changes.

Authoritative Benchmarks and Measured Comparisons

These are figures from IFM's technical report. In the Physics-IQ Verified leaderboard setting, with Claude Opus 5.5 as the conceptual reasoner, PAN 2 scores 49.38, the best open-weight result, versus 48.62 for Cosmos 3 Super and 51.11 for the closed FLUX 3. It scores 4.74 on PhyGround and 57.04 on RBench. On multi-step plans across 100 everyday tasks, its full-plan completion rate is 34.00%, ahead of MiniMax-H3 at 32.50%, but its six-capability overall score of 84.48 trails MiniMax-H3 at 87.20. IFM also says about two in three simulated plans still miss at least one step.

Developer Adoption and Getting Started

Weights are on Hugging Face at IFM/PAN2 and code is on GitHub at ifm-ai/PAN2 under Apache 2.0, which allows commercial use, with day-0 support in Diffusers and vLLM-Omni. Each inference worker loads the full 29B model and the default sampler runs 50 steps, so the compute bar is high. Rewrite free-text prompts into structured captions first with the official Conceptual Reasoner skill, which works in Codex or Claude Code, and IFM strongly recommends running the pan2-guardrail safety filter alongside it.