AIBuildAI-2.5: Efficient Autonomous AI Model Development Through LLM-Guided Tree Search

AIBuildAI-2.5：通过大模型引导的树搜索实现高效的自主 AI 模型开发

Abstract: Autonomous agents that automatically build artificial intelligence (AI) models could broaden access to AI across science and engineering. A popular line of such agents frames model building as a code search problem and solves it by tree search, in which each node is a candidate program and the tree grows by generating a child program from a parent, and these agents now approach the capability of experienced AI engineers on realistic benchmarks.

摘要： 能够自动构建人工智能（AI）模型的自主智能体可以拓宽科学和工程领域对 AI 的应用。目前流行的一类智能体将模型构建视为代码搜索问题，并通过树搜索来解决：其中每个节点都是一个候选程序，树通过从父节点生成子节点来扩展。在现实基准测试中，这些智能体的能力已接近经验丰富的 AI 工程师。

However, these agents have three weaknesses in efficiency that have not been fully addressed. First, only a small number of candidates can be executed within a realistic budget, so search rules that rank nodes by executed rewards, such as Monte Carlo-style tree search, rely on few and noisy scores and select the next node to explore less effectively. Second, no resource-aware strategy is used to schedule training jobs, which can lower hardware utilization and training efficiency. Third, every agent call is served by a single powerful model, which inflates inference cost.

然而，这些智能体在效率方面存在三个尚未完全解决的弱点。首先，在实际预算内只能执行少量候选程序，因此像蒙特卡洛树搜索这样根据执行奖励对节点进行排序的搜索规则，往往依赖于少量且带有噪声的评分，导致选择下一个探索节点的效果不佳。其次，缺乏资源感知策略来调度训练任务，这会降低硬件利用率和训练效率。第三，每次智能体调用都由单一的强大模型提供服务，这推高了推理成本。

Here we introduce AIBuildAI-2.5, an agentic system that carries out the tree search with LLM agents and addresses each of the three issues. AIBuildAI-2.5 proposes a novel LLM-guided tree search, in which a judge scores each candidate on its expected improvement, grounding, and feasibility, and a selector ranks the pool of candidates from these scores and the state of the search.

在此，我们介绍了 AIBuildAI-2.5，这是一个利用大模型（LLM）智能体进行树搜索并解决了上述三个问题的智能体系统。AIBuildAI-2.5 提出了一种新颖的大模型引导树搜索方法，其中“评判者”（judge）根据候选程序的预期改进、基础性和可行性进行评分，而“选择器”（selector）则根据这些评分和搜索状态对候选池进行排序。

In addition, AIBuildAI-2.5 comprises a scheduler that launches training jobs with the current hardware resource status taken into account and a router that assigns lower-cost LLMs to less demanding tasks while reserving the most capable LLM for the most challenging sub-tasks in the AI model building workflow. AIBuildAI-2.5 ranks first on MLE-Bench with a medal rate of 73.3%, and outperforms a strong baseline on six autonomous AI research tasks from AIRS-Bench.

此外，AIBuildAI-2.5 还包含一个调度器，它在考虑当前硬件资源状态的情况下启动训练任务；以及一个路由器，它将成本较低的大模型分配给要求较低的任务，同时为 AI 模型构建工作流中最具挑战性的子任务保留能力最强的大模型。AIBuildAI-2.5 在 MLE-Bench 上以 73.3% 的奖牌率排名第一，并在 AIRS-Bench 的六项自主 AI 研究任务中表现优于强基准模型。