Evolving to Match Capabilities: Improving Harness Self-Evolution via Skill Adaptation

Harness self-evolution offers a practical route for LLM agents to improve from execution experience without updating model parameters. Its effectiveness depends on whether models can successfully act on the guidance provided by their harnesses.

“Harness” 自我进化为大语言模型（LLM）智能体提供了一条无需更新模型参数即可从执行经验中改进的实用途径。其有效性取决于模型能否成功地根据其 “harness”（引导工具）提供的指导采取行动。

However, we find that harness benefits are not uniform across model capabilities: skills that improve stronger models can provide little benefit or even degrade performance for weaker ones. Across seven models and three benchmarks, we identify 103 task-skill pairs where skills improve GPT-5.6-luna by 31.1 points but degrade Gemma-4-E4B by 8.8 points.

然而，我们发现 “harness” 的益处在不同模型能力之间并不均衡：能够提升较强模型性能的技能，对于较弱的模型可能几乎没有帮助，甚至会降低其性能。在七个模型和三个基准测试中，我们确定了 103 个任务-技能对，其中这些技能使 GPT-5.6-luna 的得分提高了 31.1 分，却使 Gemma-4-E4B 的得分下降了 8.8 分。

In 86% of failed weak-model runs, the violated requirement is already stated in the skill. This motivates skill adaptation: reformulating existing skills to match the executor’s capabilities and make their guidance easier to execute.

在 86% 的弱模型失败运行中，被违反的要求其实已经在技能中明确说明了。这促使我们进行技能适配：通过重构现有技能以匹配执行者的能力，从而使指导更容易被执行。

To achieve this, we diagnose failures in tool execution, context management, instruction following, and verification, and distill these diagnoses into a fixed meta-skill that guides adaptation during harness self-evolution.

为了实现这一点，我们诊断了工具执行、上下文管理、指令遵循和验证方面的失败，并将这些诊断结果提炼为一个固定的“元技能”（meta-skill），用于在 “harness” 自我进化过程中指导适配。

Guided by the meta-skill, a capable external rewriter, Kimi-K3, adapts skills without execution evidence, improving the three weak executors by 5.1-14.1 points over the original skills. We then incorporate this adaptation into a five-round self-evolution loop, where each model revises its own skill from execution trajectories and verifier feedback.

在元技能的指导下，一个强大的外部重写器 Kimi-K3 在没有执行证据的情况下对技能进行了适配，使三个弱执行者的表现比原始技能提高了 5.1 到 14.1 分。随后，我们将这种适配整合到一个五轮的自我进化循环中，每个模型根据执行轨迹和验证器反馈来修订自己的技能。

Across seven executors, from Gemma-4-E4B to Kimi-K2.5, the meta-skill consistently improves final scores by 2.0-5.1 points over the same loop without the meta-skill. Our findings provide practical design guidance for developing more effective harness self-evolution methods.

在从 Gemma-4-E4B 到 Kimi-K2.5 的七个执行者中，与没有使用元技能的相同循环相比，元技能始终能将最终得分提高 2.0 到 5.1 分。我们的研究结果为开发更有效的 “harness” 自我进化方法提供了实用的设计指导。