Omni-IO Skills: Plug-and-Play Agent Harness Makes Existing Models Omni-Native Across 7 Modalities via Execution Graphs

核心背景与行业痛点

Autonomous workflows increasingly require coordinating seven distinct media types: text, images, audio, video, documents, 3D meshes, and code. However, engineering native omni-foundation models from scratch is cost-prohibitive and frequently destabilizes core reasoning capabilities. Conversely, relying on ad-hoc tool calling leads to chaotic intermediate asset lifecycles, where temporary files lack persistent version tracking, dependencies collide, and execution halts under sequential bottlenecks.

架构亮点与底层机制

Researchers from the National University of Singapore (NUS) and S-Lab present Omni-IO Skills (arXiv:2609.31847):

1. Declarative Execution Graphs (DEG): Formalizes complex multi-asset workflows into dependency graphs, executing independent operations concurrently across modular backends.
2. Persistent Asset Registry: Maintains a centralized index for multimodal artifacts across reasoning steps, facilitating seamless downstream reference and cross-turn edits.
3. 27 Hierarchical Multi-Asset Skills: Encapsulates standardized interfaces spanning 38 tasks across four functional families: comprehension, synthesis, deductive reasoning, and contextual retrieval.
4. Non-Invasive Harness-Level Composition: Upgrades foundation models to omni-native status at the harness layer without modifying internal model parameters.

权威 Benchmark 与实测跑分对比

Evaluated on the comprehensive UniM-90 multimodal benchmark:

1. 100% Multimodal Input Coverage: Propels input support rates for GPT-5.6 Sol and Claude Sonnet 5 from ~40% to a flawless 100% across all 7 modalities.
2. Tripling Semantic-Quality Coupled Scores: Catapults composite quality scores from 26.99 to 74.94 on GPT-5.6 Sol and from 27.82 to 77.78 on Claude Sonnet 5.
3. 99.78%+ Strict Structural Fidelity: Ensures rigorous format compliance across multi-tool interactions, eliminating syntactic schema crashes.

开发者实战落地与开箱指南

Omni-IO Skills is open-sourced on GitHub. Engineering teams building cross-media generative pipelines, gaming copilot agents, and document understanding bots can plug Omni-IO Skills into existing agent orchestrators to gain turnkey multimodal capabilities with enterprise-grade asset safety.