Agents23d ago
First Empirical Study on Agent Skill Evolution Reveals Concrete Rules in SKILL.md Boost Agent Correctness
In modern coding agent ecosystems (such as Claude Code, Cursor, Cline, and Antigravity), Agent Skills—specialized instruction files like SKILL.md and CLAUDE.md that dictate project architecture, commands, and rules—are continuously revised by developers alongside source code. Yet the concrete impact of these revisions on agent behavior and inference cost has remained largely unquantified. In a comprehensive empirical audit (arXiv:2610.04832), researchers analyze 2,608 first/last revision pairs across 3,159 Skills to measure the effects of revisions across 21 foundation models and four sandboxed coding agents. The study reveals that 55% of revisions alter specific rules or procedures. Adding a concrete rule raises compliance in single answers by +0.41 across open-weight models, increases required action adherence by +0.23 across sandboxed agents, and improves final task correctness by +0.10 in blind evaluations. Critically, these gains are overwhelmingly driven by rules that specify explicit CLI commands or exact file paths, whereas abstract stylistic guidance offers negligible lift. The paper also documents the performance and token trade-offs of dynamic on-demand skill loading.
Tooling6d ago
Microsoft opens the GitHub-built Copilot harness in Foundry Agent Service: public preview, 20+ frontier models plus BYOK, billing from Dec 1
During GitHub Universe on Oct 28, Microsoft put the Microsoft Copilot harness, the shared agent execution layer behind Copilot Code, Copilot Studio, the GitHub Copilot app, the CLI and VS Code, into Foundry Agent Service as a public preview. Microsoft runs the agent loop and sandbox while developers supply instructions, skills, tools and model choice; Microsoft says it works across 20+ frontier models plus bring-your-own-key. Long-running resilience for hosted agents also reached GA. Harness runtime usage is free until Dec 1.
Tooling33d ago
NVIDIA DOCA Agent Skills on GitHub: verified API/hardware contracts lift 65-prompt checklist pass from 19% to 100%
On 2026-10-01 NVIDIA published DOCA AI agent skills on NVIDIA/skills: SKILL.md packs with real API signatures, hardware capability checks, and build constraints for Flow, GPUNetIO, PCC, RDMA, and more. Vendor 65-prompt eval: 19% checklist items without skills vs 100% with; RDMA demo used 73% less handwritten code and 46% fewer hardware commands. Installable into Claude Code, Codex, and other coding agents.
Discussion & Comments
0Sign in to join the discussion
Connect with AI developers to exchange benchmark insights.