AI Coding Live News

Researchers from Alibaba and collaborating academic institutions introduced FocusGUI in arXiv:2610.26203, establishing a focused attention architecture for streaming omni-modal GUI agents. Across streaming observations of continuous UI screenshots, synchronous audio, and device state histories, agents routinely suffer from 'Sequential Distraction'—where transient visual clutter from previous steps compounds into catastrophic action hallucination. FocusGUI preserves the base model while introducing action-sufficient projection and channel decomposition. Tested across 29 smartphone apps on OmniGUI, FocusGUI elevates Type Match from 31.0% to 64.2% and Exact Match from 11.2% to 33.5% across four omni-modal foundation backbones.

Zoom

Expanding text embedding models to multimodal domains typically triggers catastrophic forgetting in text retrieval, with existing omni-modal embedders ballooning past multi-billion parameters to compensate. MBZUAI researchers introduce Omni-Embed-Mini, a compact 0.9B architecture mapping text, speech, audio, image, video, and rich documents into a single shared cosine space. Crucially, text-side weights remain bit-identical to the base model, guaranteeing zero regression on text retrieval (49.57 nDCG@10 on MTEB-v2 BEIR-8). Powered by dense cascaded caption distillation, Matryoshka SigLIP loss, and online hybrid hard-negative mining, Omni-Embed-Mini is 2.7x to 9.5x smaller than competing open models, while its 2.3B variant surpasses proprietary gemini-embedding-2 on cross-modality averages.

Zoom

While general-purpose LLM agents exhibit strong long-horizon reasoning, their production utility remains heavily fragmented across seven heterogeneous artifact modalities: text, images, audio, video, documents, 3D assets, and code. Training unified native omni-modal foundation models incurs prohibitive compute, while naive tool chaining fails to coordinate asset dependencies and multi-turn revisions. Researchers from the National University of Singapore (NUS) introduce Omni-IO Skills, a plug-and-play agent harness transforming existing text/vision LLMs into omni-native agents. Using hierarchical skill abstractions, standardized I/O interfaces, a persistent Asset Registry, and Declare Execution Graphs, Omni-IO coordinates complex multi-asset workflows concurrently. Evaluated on the UniM-90 multimodal benchmark, Omni-IO boosts input-support rates for frontier models from ~40% to 100%, and nearly triples semantic quality coupling scores without modifying foundational model weights.

Zoom

Peking University's VaLuE Laboratory has open-sourced OmniEcho alongside the OmniEchoBench unified evaluation suite (arXiv: 2609.23407, GitHub: PKU-VaLuE-Lab/OmniEcho). Addressing the longstanding sensory blind spot where embodied agents rely exclusively on vision, OmniEchoBench compiles 197 real-world 3D acoustic environments with First-Order Ambisonics (FOA) audio across 2,972 QA pairs. The authors introduce OmniEcho, a spatially aware omni-modal agent featuring a dedicated FOA spatial encoder that matches conventional vision-language navigation fidelity purely guided by acoustic signals.

Zoom