AI Coding Live News

Microsoft Research (MSRA) researchers published and open-sourced WildAWM (arXiv:2610.26828, GitHub: MSR-Orchard/WildAWM), a generalist agent world model designed for open-ended 'in the wild' digital environments. Traditional computer-use agents rely on slow, brittle, and expensive live web/OS sandboxes for training and evaluation. WildAWM establishes a scalable multimodal world model that accurately predicts visual screen transitions and DOM mutations conditioned on agent actions (click, type, scroll). By replacing live execution environments with a high-fidelity neural simulator, WildAWM unlocks safe, accelerated offline reinforcement learning for GUI agents.

Zoom

Researchers from Alibaba and collaborating academic institutions introduced FocusGUI in arXiv:2610.26203, establishing a focused attention architecture for streaming omni-modal GUI agents. Across streaming observations of continuous UI screenshots, synchronous audio, and device state histories, agents routinely suffer from 'Sequential Distraction'—where transient visual clutter from previous steps compounds into catastrophic action hallucination. FocusGUI preserves the base model while introducing action-sufficient projection and channel decomposition. Tested across 29 smartphone apps on OmniGUI, FocusGUI elevates Type Match from 31.0% to 64.2% and Exact Match from 11.2% to 33.5% across four omni-modal foundation backbones.

Zoom

Pixel-space world models for GUI agents require predicting future screen images prior to action selection. However, full-screen pixel reconstruction suffers from extreme latency and frequently triggers identity shortcuts, where generated screens merely copy the past rather than reflecting action-induced mutations. Researchers from KAIST introduced the first Latent World Action Model for GUI agents, shifting predictive objectives from pixel generation to action-conditioned transitions in latent space. Featuring shared multimodal representations with decoupled action and transition heads alongside action-aware contrastive losses, the framework achieves up to a 40.6% relative gain in online task success while reducing inference latency by 98.9%.

Zoom