UniSkill: Learning Actor-Aligned Skill Proposals for an Evolving Agent Policy via Contrastive Action Feedback

Background and the Problem

Equipping LLM agents with reusable skillbanks allows them to consolidate past experiences into structured assets. However, current policy-skill co-evolution paradigms face severe drawbacks: validating whether a newly proposed skill is effective necessitates costly online environment rollouts, while credit assignment remains confounded between true skill utility and general improvements in actor capabilities.

Architecture and How It Works

To overcome these hurdles, researchers introduce UniSkill (arXiv:2610.10164, repo: github.com/LimOkii/UniSKill):

1. Unified Dual-Role Policy: Eliminates disjoint actor-distiller architectures by tasking a single shared policy with both task execution and skillbank editing proposals (Add, Update, No Edit).
2. Contrastive Action Feedback: Unlocks offline skill verification without additional environment rollouts. Measures the shift in the policy's action log-likelihood margin between historical successful and failed trajectories when swapping a retrieved skill for the candidate proposal.
3. Co-Evolutionary Skill Pruning: The contrastive likelihood metric directly guides skill adoption and pruning, preventing uninformative or parasitic skills from polluting the agent's long-term retrieval index.

Benchmarks and Measured Results

Empirical validation across sequential decision-making environments demonstrates:

1. Over 70% Reduction in Rollout Compute: Matches or exceeds the final task success rates of traditional rollout-heavy baselines while completely bypassing auxiliary environment executions.
2. Curated Skillbank Compaction: Retains a lean, high-utility skill set that prevents retrieval degradation, whereas naive baselines suffer from uncontrolled library bloat and catastrophic interference.
3. Robust Metric Correlation: Statistical analysis confirms that contrastive action log-likelihood margins correlate strongly with actual downstream environment success, establishing a sound surrogate signal for agent self-improvement.

Getting Started for Developers

The UniSkill codebase is publicly accessible on GitHub (github.com/LimOkii/UniSKill). AI practitioners developing autonomous coding assistants and continuous-improvement agents can implement contrastive log-likelihood feedback over cached failure traces, enabling lifelong agent skill consolidation without ballooning simulation or API budgets.