WorldGuide: MBZUAI and ANU Introduce Goal-Directed Closed-Loop Video World Model for Procedural Task Execution

Background and the Problem

While generative video models deliver photorealistic visual quality, their application to procedural, goal-directed physical tasks—such as industrial assembly, culinary preparation, and surgical simulation—remains brittle. Conventional models operate as open-loop systems: they sample full video trajectories from prompt conditioning alone without assessing intermediate outcomes. Consequently, open-loop generation suffers from logical step reversal, repetitive loops, hallucinated completion, and premature termination.

Architecture and How It Works

To enforce causal procedural consistency, researchers from MBZUAI and ANU formulate WorldGuide (arXiv:2610.12459, code: github.com/mbzuai-oryx/WorldGuide, site: mbzuai-oryx.github.io/WorldGuide/):

1. Closed-Loop Planner-Executor Architecture: Pairs a high-level Planner with a video Executor in a unified loop. Conditioned on current synthesized keyframes and task objectives, the Planner outputs atomic actions, the Executor renders corresponding video segments, and the Planner re-evaluates the resulting frame to route subsequent steps or signal completion.
2. Hierarchical Visual Memory: Mitigates quadratic token bloat across long horizons by compacting distant steps into milestone latent anchors while maintaining fine-grained representations for recent frames.
3. WorldGuide-Bench Dataset Suite: Establishes a standardized dataset of ~59K step-annotated videos across 245 tasks and 27 procedural categories to train and benchmark joint planner-executor world models.

Benchmarks and Measured Results

Benchmarked on WorldGuide-Bench and VideoCraft-Bench:

1. Outperforming MiniMax-H3 Without Plans: Under goal-only evaluation without reference sub-goal plans, WorldGuide reaches a 33.33% Task Success Rate on WorldGuide-Bench, surpassing the strong MiniMax-H3 model (29.90%) even when MiniMax-H3 is provided with reference plans.
2. +14.96 Percentage Points on VideoCraft-Bench: Scores 47.69% on VideoCraft-Bench compared to 32.73% for MiniMax-H3 under goal-only conditioning.
3. Eliminating Step Reversals: Compresses logical ordering failures from over 62% in open-loop baselines to below 4% through dynamic visual verification.

Getting Started for Developers

The authors have open-sourced the codebase, training scripts, and evaluation benchmarks on GitHub (github.com/mbzuai-oryx/WorldGuide). For developers building robotics simulators, procedural tutorial systems, or interactive gaming environments, WorldGuide highlights the necessity of closed-loop execution. Decouple long-horizon planning from short-horizon video generation and incorporate hierarchical state memory, enabling video world models to serve as verified physical simulators.