UIUC Proposes Meta-Skills for Agent Harness Design in Test-Time AI4AI: 12% Performance Gain Without Weight Updates

核心背景与行业痛点 While most agent research targets model weights or prompting, practical agent efficacy heavily relies on execution harnesses—sandboxes, tool ergonomics, and observation filters. Handcrafted by humans, current harnesses fail to adapt dynamically to novel tasks, particularly in enterprise settings where proprietary model weights are strictly frozen. ### 架构亮点与底层机制 UIUC researchers led by Heng Ji propose a test-time AI-for-AI (AI4AI) paradigm with Meta-Skill principles. Decoupled into a 'Builder' and a 'Target', the Builder learns abstract meta-skills from the Target's execution feedback—identifying when environmental scaffolding is required and what resources to synthesize. When facing unseen tasks, the Builder leverages this frozen meta-skill bank to assemble tailored harnesses with custom validators, hooks, and execution abstractions. ### 权威 Benchmark 与实测跑分对比 Evaluated on Harness-Bench and NewtonBench across heterogeneous environments: 1. 8.95% Macro-Average Gain: Full-bank meta-skill harness construction elevates target agent macro-average performance by 8.95 percentage points over zero-skill environments without modifying any model weights. 2. 12.02% Outperformance Over Direct Prompting: Structuring support into an executable environment outperforms raw skill prompt injection by 12.02 percentage points, demonstrating the superiority of active harness support over context cluttering. 3. System-Level Self-Improvement: When the same base model serves as both Builder and Target, consistent gains emerge, validating a viable path toward autonomous test-time agent self-improvement. ### 开发者实战落地与开箱指南 MetaSkill-AI4AI is open-sourced on GitHub with modular Builder controllers and Target harness adapters. Enterprise engineers working with proprietary closed-source APIs can easily integrate their agent rollout traces to auto-extract meta-skills, unlocking continuous environment-level performance gains.