UniSkill: Learning Actor-Aligned Skill Proposals for an Evolving Agent Policy via Contrastive Action Feedback
Enabling LLM agents to distill and reuse modular skills from past task traces allows policies and skillbanks to co-evolve over time. However, conventional skill discovery systems face steep bottlenecks: evaluating newly proposed skills typically demands expensive and slow environment rollouts, while rewarding skills based purely on execution success conflates the intrinsic value of the skill with the general competence of the evolving actor. In newly published research (arXiv:2610.10164), researchers introduce UniSkill (repo: github.com/LimOkii/UniSKill). UniSkill leverages a unified shared policy to simultaneously execute environment tasks and generate structured edits (Add, Update, No Edit) to a persistent skillbank. Crucially, UniSkill evaluates proposed skills through Contrastive Action Feedback—measuring how substituting a retrieved skill with a proposed candidate widens the action log-likelihood margin between previously observed successful and failed trajectories—providing immediate actor-alignment feedback without requiring expensive new rollouts. Evaluated across diverse agentic benchmarks, UniSkill demonstrates superior sample efficiency and sustained policy refinement, with code fully open-sourced.
Key Takeaways
- UniSkill framework open-sourced, unifying task execution and skillbank editing within a single shared policy
- Introduces Contrastive Action Feedback, evaluating skill proposals against historical success/failure traces with zero new rollouts
- Boosts sample efficiency by over 70% and mitigates skill library bloat for lifelong agent self-improvement