Forging More Tasks into Better Harnesses: Agentic Harness Evolution via Successive Scale Racing

查看原文 / View Original 编辑 / Edit

Computer Science > Artificial Intelligence arXiv:2610.28815 (cs) [Submitted on 20 Oct 2026] Title: Forging More Tasks into Better Harnesses: Agentic Harness Evolution via Successive Scale Racing Authors: Jingzhe Liu, Jiliang Tang, Tong Zhao, Neil Shah, Clark Mingxuan Ju.

计算机科学 > 人工智能 arXiv:2610.28815 (cs) [提交于 2026 年 10 月 20 日] 标题：将更多任务锻造成更好的工具：通过连续规模竞赛实现代理工具进化作者：Jingzhe Liu, Jiliang Tang, Tong Zhao, Neil Shah, Clark Mingxuan Ju。

Abstract: Agentic harness evolution optimizes an agent’s instructions, tools, and control flow by using its task execution experiences as evolving evidence. While more evolution tasks should intuitively improve performance, our preliminary study reveals a paradoxical failure mode: the performance of existing methods degrades as the task pool expands.

摘要：代理工具进化通过利用智能体的任务执行经验作为进化证据，来优化其指令、工具和控制流。虽然直觉上更多的进化任务应该能提高性能，但我们的初步研究揭示了一种矛盾的失效模式：随着任务池的扩大，现有方法的性能反而会下降。

Our further analysis demonstrates that this degradation stems from suboptimal evidence selection during optimization, as different evidence combinations lead to substantially different evolution outcomes. To address this issue, we propose Successive Scale Racing (SSR), a framework that adaptively searches for optimal evidence combinations in each evolution round.

我们进一步的分析表明，这种退化源于优化过程中证据选择的次优性，因为不同的证据组合会导致截然不同的进化结果。为了解决这个问题，我们提出了连续规模竞赛（SSR），这是一个在每一轮进化中自适应搜索最优证据组合的框架。

SSR constructs diverse evidence combinations based on the corresponding task quality and coverage, and efficiently races the resulting candidate harnesses under a small compute budget. Empirically, across three distinct benchmarks, SSR achieves state-of-the-art results in all six configurations.

SSR 根据相应的任务质量和覆盖范围构建多样化的证据组合，并在较小的计算预算下高效地对生成的候选工具进行竞赛。实验证明，在三个不同的基准测试中，SSR 在所有六种配置中均达到了最先进的结果。

Notably, as the task pool grows from 40 to 200, SSR steadily improves the pass rate from 64.3% to 68.3%, while the baselines’ performances all drop. Furthermore, its racing mechanism identifies near-optimal candidates while reducing validation executions by 60%.

值得注意的是，随着任务池从 40 个增加到 200 个，SSR 将通过率从 64.3% 稳步提高到 68.3%，而基准方法的性能均有所下降。此外，其竞赛机制在识别出近乎最优候选者的同时，将验证执行次数减少了 60%。