AI Coding Live News

NVIDIA Unveils Open World Action Model GR00T 2 at CoRL: Predicts Future Video and Robot Actions Together, 40% Zero-Shot on Unseen Tasks, 95% After a 30-Minute Fine-Tune on 5 Demos, Code Coming in December

NVIDIA Robotics announced GR00T 2 at CoRL 2026, and NVIDIA GEAR lab lead Yuke Zhu posted the technical details. GR00T 2 is a new generation of world action model (WAM) built on Cosmos: one diffusion transformer denoises future video and an action chunk together, and one model drives parallel grippers, dexterous hands and humanoids. Pretraining mixes 402,730 hours of egocentric human video, 10,184 hours of handheld UMI gripper demos and 23,500 hours of robot teleoperation. By NVIDIA's numbers, it reaches 40% zero-shot success on 50 tasks the target robot never practiced, rising to 95% with five demos per task (under 3 minutes of robot data) and a 30-minute LoRA fine-tune on one RTX 5090. Reasoning models can call GR00T 2 as a tool through the WAM API. Code arrives in December; 100,000 hours of the human video data are already on Hugging Face.

NVIDIA Unveils Open World Action Model GR00T 2 at CoRL: Predicts Future Video and Robot Actions Together, 40% Zero-Shot on Unseen Tasks, 95% After a 30-Minute Fine-Tune on 5 Demos, Code Coming in December

Key Takeaways

  • Architecture: one diffusion transformer predicts future video and action chunks together; small per-robot heads let one model cover grippers, dexterous hands and humanoids
  • Data: 402,730 h of egocentric human video + 10,184 h of UMI demos (2,897 tasks) + 23,500 h of teleoperation; going from 0% to 100% of the human video lifts real-robot task progress from 50% to 70% (NVIDIA figures)
  • Adaptation: 40% zero-shot on 50 unseen tasks, 71% with 1 demo, 95% with 5 demos, about a 30-minute LoRA fine-tune on one RTX 5090
  • Agents: a reasoning model calling GR00T 2 via the WAM API completes 97+ of 100 back-to-back assemblies for each of three toys; marking grasp points on the camera view lifts instruction following from 40% to 92%
  • GR00T 2 Turbo is a 2B-parameter variant running at 124 ms per prediction on the Unitree G1's Jetson Thor; code ships in December, and 100k hours of human video are already on Hugging Face
Read full story
Linked Models & Agents:
⚡ GR00T 2⚡ GR00T 2 Turbo⚡ Cosmos🌐 NVIDIA

X Discussion: Node.js and Deno Creator Ryan Dahl Says Software Engineering Is Now "100% AI Written", AI Writes Better C++ Than He Does, but Making Sure It Works Has Not Gone Away

At 04:11 UTC on November 9, Ryan Dahl, creator of Node.js and Deno, posted on X that it is shocking how quickly software engineering has moved to "100% AI written", and more shocking that engineers carry on as if nothing happened. He said AI writes C++ better than he ever could, but the problem of making sure it works did not magically disappear; teams are simply shipping far more code at higher quality. Within about six hours the post had roughly 1,370 likes, 236 bookmarks, 83 replies, 25 quotes and nearly 90K views. Quote posts split into camps: some say the bottleneck has moved from writing code to verifying it, so testing and review are worth more, while others question the "100%" and "higher quality" claims. This is a personal opinion post; "100%" is not a measured statistic.

Key Takeaways

  • Reach: @rough__sea posted at 04:11 UTC on November 9 and within about six hours had about 1,370 likes, 236 bookmarks, 83 replies, 25 quotes and nearly 90K views
  • Claim: software engineering has moved to "100% AI written", AI writes C++ "better than I ever could", but making sure it works has not gone away
  • Conclusion: engineers are shipping far more code at higher quality, yet "continuing on as if nothing happened"
  • Split: some quote posts say the bottleneck has moved from generation to verification, making testing and review more valuable; others question the "100%" and "higher quality" claims
  • Caveat: this is a personal opinion; "100%" is not a measured figure, and the post cites no data or specific projects
Read full story

Pika releases open-weight video model Pika H3 Flex: post-trained on MiniMax H3, six-step sampling, five audio stems, API from $0.01 per second

On October 29 Pika released Pika H3 Flex, a joint audio-video generator post-trained on MiniMax H3, with weights on Hugging Face, code on GitHub, and ComfyUI and Diffusers support. Pika says it distilled sampling from 50 denoising steps to six, averages about 1.8x faster DiT denoising than Fal H3 Max at 768p, and can produce a 15-second clip with soundtrack in as little as about 7 seconds; the hosted API adds 1080p to 4K upscaling and five exportable audio stems (dialogue, score, SFX, foley, ambience). All speed and quality figures are vendor-reported.

Pika releases open-weight video model Pika H3 Flex: post-trained on MiniMax H3, six-step sampling, five audio stems, API from $0.01 per second

Key Takeaways

  • Pricing (vendor figures): text/image-to-video from $0.01 per output second, reference-to-video from $0.03/s; Pika's comparison table lists MiniMax H3 at $0.08/s and Seedance 2.5 at $0.103/s
  • Speed (vendor figures): six-step sampling, about 1.8x faster DiT denoising than Fal H3 Max at 768p on average and about 2x for 10–15 s text/image-to-video; 1.60x faster end-to-end 5 s 768p generation on eight B300 GPUs
  • Ranking: Pika's product page says it scores 1293 and ranks #7 on DesignArena's text-to-video board, the highest open-weight model there
  • Release form: the Hugging Face repo is tagged as an adapter on MiniMax-H3 under an 'other' license; local inference natively supports 480p/768p with mixed stereo audio, while 4K upscaling and audio stems are hosted-API only
  • Accepts up to 12 reference inputs (image, audio, video, sketch and more) and supports style transfer, motion transfer, character swap and targeted edits
Read full story
Linked Models & Agents:
⚡ Pika H3 Flex⚡ MiniMax H3⚡ Fal H3 Max⚡ HeyGen Video⚡ Gemini Omni 1.1 Flash🌐 Pika🌐 MiniMax

DiffuSpace, founded by the HKU Dream team, open-sources Clover-35B-A3B, a diffusion LLM hitting 680 tokens/s on one GPU and 73.6 on SWE-bench Verified

On Oct 28 DiffuSpace, whose members come from the HKU NLP group behind Dream 7B, released its first model, Clover-35B-A3B-Preview: a mixture-of-experts diffusion language model initialized from Qwen3.6-35B-A3B, with 35B total and about 3B active parameters, a 128K context window and block-parallel denoising. Vendor-reported results include 73.6 on SWE-bench Verified, 51.7 on Terminal-Bench 2.1 and 83.8 on GPQA Diamond, with 680 tokens/s on a single GB300 in agent workloads. Weights are Apache 2.0 on Hugging Face, ModelScope and GitHub, with SGLang and llama.cpp support.

DiffuSpace, founded by the HKU Dream team, open-sources Clover-35B-A3B, a diffusion LLM hitting 680 tokens/s on one GPU and 73.6 on SWE-bench Verified

Key Takeaways

  • A 35B-total, roughly 3B-active MoE diffusion model initialized from Qwen3.6-35B-A3B, with 128K context and a thinking mode
  • DiffuSpace says it tops all six agent benchmarks it compared: SWE-bench Verified 73.6, Terminal-Bench 2.1 51.7 and tau3 Overall 65.3, ahead of LLaDA2.2 flash, Mercury 2.5 and Qwen3.6-35B-A3B
  • Speed (vendor-reported): 680 tokens/s on one GB300 at BF16 and batch size 1 in agent workloads, and 515 tokens/s on one H200 versus 365 tokens/s for Qwen with DFlash2
  • An INT4 build runs on a 16GB RTX 5070 Ti at 101.8 tokens/s on MATH-500
  • Weights on Hugging Face are Apache 2.0, about 69GB in BF16, and SGLang is the recommended serving path
Read full story
Linked Models & Agents:
⚡ Clover-35B-A3B-Preview⚡ Qwen3.6-35B-A3B⚡ Dream 7B⚡ LLaDA2.2 flash⚡ Mercury 2.5🌐 DiffuSpace🌐 HKU NLP🌐 Alibaba Qwen🌐 NVIDIA
ADSponsored

SolarRenderer Breaks Generative Video: 4-Step Denoising on Greyboxes and Depths for Real-Time 768p World Synthesis

Researchers from NUS and CUHKSZ have introduced SolarRenderer (arXiv:2610.28630), a structure-conditioned generative reference rendering framework. Converting game engine grey-boxes, pose skeletons, semantic maps, and depth passes into photorealistic video, SolarRenderer adapts a frozen MiniMax-H3 33B backbone via lightweight control and 4-step generation LoRAs. Using structure-guided distribution matching, it renders stylized video in just 4 denoising steps (~12.3x faster than 50-step diffusion) and introduces a block-causal variant for real-time 768p streaming synthesis in generative game engines.

SolarRenderer Breaks Generative Video: 4-Step Denoising on Greyboxes and Depths for Real-Time 768p World Synthesis

Key Takeaways

  • SolarRenderer (arXiv:2610.28630) enables high-fidelity video rendering directly from coarse 3D geometry (grey-boxes, skeletons, depth, meshes)
  • Adapts a frozen MiniMax-H3 33B backbone with lightweight Control and 4-step LoRA modules, preserving visual priors
  • Requires only 4 denoising steps via structure-guided distribution matching, achieving a 12.3x acceleration over 50-step baselines
  • Block-causal streaming variant achieves real-time continuous 768p video generation for next-generation generative gaming engines
Read full story
Linked Models & Agents:
⚡ MiniMax-H3⚡ SolarRenderer

vLLM-Omni Integrates PhysisServe: Accelerating Video Diffusion & Physical-World AI with 4.1ms Chunk Output

The vLLM Project officially announced that PhysisServe's core physical-world serving techniques are being integrated into vLLM-Omni. Addressing the severe memory bottlenecks and latency spikes of multi-step video diffusion and physical AI generation, PhysisServe introduces cross-denoising-step pipeline parallelism across Transformer blocks, time-shared batching, and virtual-rank joint scheduling. Benchmarked on MiniMax H3 33B across 200 GPUs, it generates a chunk every 4.1 ms (~1,000 FPS without VAE decoding), while cutting single-GPU latency for Wan2.2 5B from 173 ms down to 98 ms.

vLLM-Omni Integrates PhysisServe: Accelerating Video Diffusion & Physical-World AI with 4.1ms Chunk Output

Key Takeaways

  • The vLLM project announced the upstream integration of PhysisServe's serving architecture into vLLM-Omni for physical AI and video models
  • Introduces cross-timestep block pipelining and time-shared batching to eliminate GPU communication bubbles in multi-step diffusion
  • Generates a video chunk every 4.1 ms on MiniMax H3 33B across 200 GPUs (~1,000 FPS), cutting Wan2.2 5B single-card latency to 98 ms
  • Expands 24-GPU serving concurrency by 1.7x to 3.3x, with full open-source code merging into the vLLM-Omni mainline
Read full story
Linked Models & Agents:

4D/3D Vision and Fast Video Diffusion Breakthroughs: Gen4P Single-Pass 4D Perceiver, SUPRA-TEX 32.08 dB PBR Texturing, and HKUST ForcingBooster (-50.19% Patch NFE)

Three new architectures advance 4D perception, 3D PBR texturing, and real-time autoregressive video diffusion: Gen4P (arXiv:2610.22371) repurposes a video generator into a single-pass feed-forward 4D perceiver without iterative denoising; HUST and Huawei's SUPRA-TEX (arXiv:2610.22352) reaches 32.08 dB Shaded PBR PSNR on DTC-Tex; and HKUST's ForcingBooster (arXiv:2610.12625) cuts patch denoising NFE by 50.19% for a 1.30x end-to-end speedup across Wan2.1 14B and LongLive.

4D/3D Vision and Fast Video Diffusion Breakthroughs: Gen4P Single-Pass 4D Perceiver, SUPRA-TEX 32.08 dB PBR Texturing, and HKUST ForcingBooster (-50.19% Patch NFE)

Key Takeaways

  • Gen4P (arXiv:2610.22371) converts a pretrained video generator into a single-pass feed-forward 4D perceiver without iterative denoising, hitting 91.93 APD / 0.112 EPE on WorldTrack ADT
  • HUST and Huawei's SUPRA-TEX (arXiv:2610.22352) reaches 32.08 dB Shaded PBR PSNR on DTC-Tex, beating Pixal3D (29.19 dB), TRELLIS.2 (28.60 dB), and Hunyuan3D 2.1 (26.80 dB)
  • HKUST's ForcingBooster (arXiv:2610.12625) reduces patch denoising NFE by 50.19% in autoregressive video diffusion for a 1.30x end-to-end speedup (11.15s to 8.56s) with lossless VBench semantics (72.20 vs 72.21)
  • ForcingBooster generalizes across Causal Forcing, Wan2.1 14B, LingBot-World-V2, and LongLive interactive world models
Read full story
Linked Models & Agents:

4.36x Faster World Model Attention with Zero Training: NUS Introduces Velox Spatial Memory, Alongside Nankai's DeltaStreamer and HKPolyU's CoWorld Reasoner

Four new studies advance interactive video world models and streaming video agents: NUS's training-free Velox router (arXiv:2610.22311) uses 3D camera frustum-slab correspondences on H200 GPUs to accelerate chunk-48 attention by 4.36x and denoising by 2.43x with negligible quality loss; Nankai University's DeltaStreamer (arXiv:2610.19849, 39 upvotes) combines change-aware memory with targeted frame replay to reach 66.07% on OVO-Bench; HKPolyU's CoWorld Reasoner (arXiv:2610.21910) co-evolves a VLM-VWM tool harness for +13.0% relative gains on ERQA; and Zhejiang University releases the 260-case WorldMemBench (arXiv:2610.19063).

4.36x Faster World Model Attention with Zero Training: NUS Introduces Velox Spatial Memory, Alongside Nankai's DeltaStreamer and HKPolyU's CoWorld Reasoner

Key Takeaways

  • NUS's training-free Velox router uses 3D camera frustum slabs on H200 to speed up chunk-48 attention by 4.36x and denoising by 2.43x within 0.15 points of dense attention on WBench
  • Nankai University's DeltaStreamer combines change-aware memory with targeted frame replay to achieve 66.07% on OVO-Bench and 59.53% on StreamingBench
  • HKPolyU's CoWorld Reasoner co-evolves a VLM-VWM imagination harness for +11.8% (MMSI) and +13.0% (ERQA) relative gains, alongside ZJU's 260-case WorldMemBench
Read full story
Linked Models & Agents:

Skywork open-sources Matrix-Game 4.0: game engine owns state, a 5B causal video model renders playable games in real time on one GPU

Skywork AI (Kunlun Tech) open-sourced Matrix-Game 4.0 on 2026-10-15. An executable game engine owns world state, physics and rules and emits a cheap low-poly RGB proxy; a 5B causal video model built on Wan2.2-Fun-5B-Control re-renders it into finished frames with 1–2 denoising steps. Inference code, a WebRTC browser demo and game source (Apache-2.0) are on GitHub, with Castle, Ancient Egypt and City Racing weights on Hugging Face. Skywork claims 480p/60 FPS and 112 ms end-to-end on an RTX 5090; the repo's own A100 measurements are ~38–39 FPS. Training code and data are not released.

Key Takeaways

  • Scale: each scene checkpoint has 5,001,016,512 BF16 params on an alibaba-pai/Wan2.2-Fun-5B-Control base, output 480×864 (HF card)
  • Vendor claim (RTX 5090): 480p at 60 FPS, 112 ms end-to-end, up to 83.4 FPS generation; repo-measured on one A100-80GB: ~38–39 FPS for Castle/Egypt fixed prompts, ~28.9 FPS for Racing dynamic prompts (v4 README)
  • Presets: realtime = TAEHV codec + 1-step denoise + CUDA graphs; quality = Wan VAE + 2 steps; 21-latent attention cache incl. sink, 4 frames per chunk
  • License: code and scene weights Apache-2.0; no training code or data; weights need the Matrix-Game 4.0 runtime (not drop-in Diffusers)
  • Design: a harness where the engine owns state and rules, a coding agent turns natural-language requests into new rules and prompts, and the video model only renders
Read full story
Linked Models & Agents:
⚡ matrix-game-4.0⚡ wan2.2-fun-5b-control🌐 skywork🌐 kunlun-tech🌐 huggingface

ChemDiffAgent Debuts: Masked Diffusion Language Agent Breaks Autoregressive Bottlenecks in Chemistry Reasoning

Researchers from the Institute of Computing Technology (CAS ICT) have introduced ChemDiffAgent, the first masked diffusion language agent for tool-integrated chemical reasoning. By replacing standard left-to-right autoregressive generation with iterative trajectory denoising, ChemDiffAgent enables bidirectional molecule comprehension and arbitrary-order tool planning, surpassing conventional models by ~20%.

Key Takeaways

  • Paradigm breakthrough: replaces unilateral causal attention with an iterative denoising process over interleaved reasoning and tool trajectories
  • Two-stage post-training: couples agentic supervised fine-tuning with variance-reduced preference optimization for holistic trajectory valuation
  • 8B model matches frontier LLMs: outperforms autoregressive baselines by ~20% across 6 chemistry tasks, matching or exceeding GPT-5.5 and Claude Opus 4.7
Read full story
Linked Models & Agents:
ADSponsored

MC-Sparse: CUHK and ByteDance Unveil Meta-Cached Sparse Attention for DiTs, Accelerating Video & 3D Generation by 2.32x with Zero Quality Loss

Diffusion Transformers (DiTs) underpin frontier video generation systems and high-fidelity 3D modeling, yet self-attention computation over extreme sequence lengths introduces prohibitive inference latencies and memory bottlenecks. While sparse attention represents a promising acceleration pathway, existing methods suffer substantial visual quality degradation and texture corruption under high sparsity regimes. Researchers from The Chinese University of Hong Kong (CUHK) and ByteDance analyze sparse attention failures in DiTs (arXiv:2610.06801), tracing quality drops to three root causes: rigid token grouping constraints, inaccurate interaction selection, and uncompensated attention residuals from discarded tokens. The authors introduce Meta-Cached Sparse Attention (MC-Sparse, project site: dodododddo.github.io/mcsparse-project-page/), a training-free framework that preserves individual Key-Value (KV) selection precision while organizing similar Queries into tile-aligned blocks for hardware efficiency on GPUs. MC-Sparse caches query clusters, exact KV indices, and dense-sparse attention output residuals, reusing this metadata across downstream denoising timesteps. On Minimax-H3-Base video models, MC-Sparse delivers a 1.80x denoising speedup, and a 2.32x acceleration on 3D generative backbones, maintaining near-identical fidelity to dense attention with zero visible quality degradation.

Key Takeaways

  • CUHK and ByteDance introduce MC-Sparse, leveraging meta-caching to eliminate quality degradation in sparse DiTs
  • Delivers a 1.80x speedup on Minimax-H3-Base and 2.32x on 3D generation while cutting peak memory by 45%
  • Operates training-free with residual compensation, matching dense attention quality across visual benchmarks
Read full story
Linked Models & Agents:
⚡ MC-Sparse⚡ DiT⚡ Minimax-H3-Base⚡ Triton-Sparse🤖 Inference-Optimizer🤖 DiT-Accelerator🌐 CUHK🌐 ByteDance🌐 Hugging Face

ME-World: KAIST Unveils First Multi-Agent Egocentric World Model with Synchronized Ego-Streams and Shared Memory

In collaborative multi-robot warehousing, autonomous vehicle fleets, and multi-user spatial computing, egocentric world models serve as foundation simulators predicting environmental evolution conditioned on physical actions. However, existing world models focus almost exclusively on isolated single agents, while existing multi-agent systems restrict conditioning to coarse navigation commands. Consequently, they fail to model complex physics where multiple agents execute fine-grained, coupled manipulations within shared environments. Researchers from KAIST CVLab and Seoul National University introduce ME-World (arXiv:2610.12299, project site: cvlab-kaist.github.io/ME-World/), the first Multi-Agent Egocentric World Model. ME-World formalizes multi-agent world modeling as synchronized first-person video stream generation driven by fine-grained embodied actions in a shared world. The model jointly denoises multiple ego-streams within a unified sequence representation, conditions each stream on cross-agent target-view poses, and grounds video synthesis using persistent shared environment memory. Extensive benchmarks across real-world and synthetic datasets confirm that ME-World substantially outperforms existing world models in shared-world spatial consistency, physical update propagation, and action controllability.

Key Takeaways

  • KAIST unveils ME-World, the first multi-agent egocentric world model using synchronized joint denoising across ego-streams
  • Improves cross-view coherence by 38.6% and raises fine-grained physical action accuracy from 41.5% to 84.2%
  • Integrates shared environment persistent memory and open-sources benchmarks for collaborative embodied simulation
Read full story

Deno team joins Cloudflare: Deno runtime gets one more year of fixes, Deno Deploy shuts down in six months, celld folds into workerd for self-hosted agent infra

On Oct 9 Ryan Dahl announced the whole Deno team is joining Cloudflare. Deno runtime gets one more year of monthly bugfix/security releases, then official development ends (it stays open source); Deno Deploy runs six more months before shutting down, with migration help to Workers; JSR continues. Focus shifts to the Workers/Durable Objects model and merging self-hosted celld into workerd, pitched explicitly at agent harnesses.

Deno team joins Cloudflare: Deno runtime gets one more year of fixes, Deno Deploy shuts down in six months, celld folds into workerd for self-hosted agent infra

Key Takeaways

  • Timeline: Deno runtime gets 12 months of monthly fixes then official development ends; Deno Deploy shuts down in 6 months with migration support for paying customers
  • JSR keeps running on Cloudflare infra; rusty_v8 stays supported and heads into workerd
  • celld (Apache-2.0, ~5.3k stars, v0.6.2 beta): a 58 MB static binary for self-hosted distributed Durable Objects backed by one S3-compatible bucket
  • Vendor-reported celld numbers: 0.47 MB RAM per resident cell, 2,500 cells per 8 GB node, 0.2/0.3 ms p50/p99 stateless requests, ~20 s failover with zero lost acknowledged writes
  • 1,234 points on the Hacker News front page; Deno's announcement post drew 6.3k likes
Read full story
Linked Models & Agents:

ALoDLM: Amazon Proposes Adaptively Looped Diffusion Language Models to Surpass Autoregressive LLMs Across 11 Benchmarks

Masked Diffusion Language Models (DLMs) promise massive throughput leaps by generating multiple tokens in parallel. However, practical deployment is constrained by a persistent quality deficit relative to autoregressive (AR) foundations. Researchers from Amazon Science attribute this gap to a fundamental 'computation-difficulty mismatch': in partially masked sequences, predictable tokens require negligible compute, whereas complex reasoning tokens require substantial inductive processing; yet conventional DLMs force identical computational depth across all positions at every denoising transition. Amazon introduces ALoDLM (arXiv:2610.04198), substituting uniform compute with token-adaptive latent recurrence. At each diffusion step, resolved tokens commit immediately as discrete contextual anchors, while unresolved tokens retain and refine their internal representations through recurrent latent loops. By formulating token-level schedules as latent variables optimized via a conditional negative evidence lower bound (NELBO), ALoDLM trained at 1.7B and 8B scales outperforms all existing diffusion architectures and corresponding autoregressive baselines across 11 diverse benchmarks while preserving parallel decoding speedups.

ALoDLM: Amazon Proposes Adaptively Looped Diffusion Language Models to Surpass Autoregressive LLMs Across 11 Benchmarks

Key Takeaways

  • Amazon Science introduces ALoDLM, resolving the computation-difficulty mismatch in diffusion language models via adaptive recurrence
  • Commits simple tokens early as discrete context while routing complex tokens through iterative latent recurrent loops
  • Outperforms all diffusion baselines and matching autoregressive models across 11 benchmarks at 1.7B and 8B parameter scales
Read full story

ProWAM: Meta FAIR and SJTU Advance World Action Modeling with Progressive Sparse Sub-Goal Visual Planning

World Action Models (WAMs) represent a foundational robotic paradigm by predicting future visual dynamics and Cartesian actions directly from initial observations. However, existing WAMs struggle with long-horizon execution: synthesizing dense video rollouts incurs prohibitive computational overhead and temporal drift, while predicting single terminal frames deprives the policy of progressive visual guidance. Meta FAIR and Shanghai Jiao Tong University introduce ProWAM (arXiv:2610.02508), an efficient architecture that jointly predicts actions and an ordered sequence of sparse visual sub-goals. Sub-goal prediction scales naturally by learning from large-scale action-free video datasets, decoupling complex visual foresight from low-level policy control. ProWAM executes a single video-backbone forward pass to cache sparse sub-goal features, enabling rapid closed-loop replanning through lightweight action denoising without full video re-generation. Evaluated across extensive robotic benchmarks, ProWAM sets SOTA records on LIBERO-Plus (85.8%, +35.9% relative gain) and randomized RoboTwin (75.7%). In real-world zero-shot physical manipulator trials, ProWAM achieves a 70.0% success rate in unseen environments, outpacing the strongest baseline by +15.0 percentage points (+27.3% relative gain).

ProWAM: Meta FAIR and SJTU Advance World Action Modeling with Progressive Sparse Sub-Goal Visual Planning

Key Takeaways

  • Introduces ProWAM, replacing dense video rollouts with progressive sparse visual sub-goals for robust robotic foresight
  • Executes a single video forward pass to cache sub-goal features, enabling rapid closed-loop action replanning
  • Reaches 85.8% success on LIBERO-Plus (+35.9% relative gain) and lifts zero-shot physical robot success from 55% to 70%
Read full story