Cost-Effective Agent Harnesses for Abstract Reasoning and Generalization on ARC-AGI-1

ARC-AGI-1 上的高性价比智能体框架：实现抽象推理与泛化

Abstract: Recent progress on ARC-AGI-1 from disclosed architectures has come broadly from two regimes: heavy test-time compute over frontier models (evolutionary search, exhaustive sampling, extended chain-of-thought), or benchmark-specific training in which small models are fine-tuned on ARC data, often with task-specialized architectures.

摘要： 近期在 ARC-AGI-1 任务上取得的进展，主要源于两种架构范式：一是对前沿模型进行繁重的测试时计算（如进化搜索、穷举采样、扩展思维链）；二是针对基准测试的特定训练，即在 ARC 数据上对小模型进行微调，且通常采用任务专用架构。

We study a third regime: an open-weight model in non-thinking mode (DeepSeek V3.2) under a strict budget, with no ARC-specific fine-tuning. We study what is recoverable through architecture alone, building agentic harnesses that decompose pattern-discovery and program-synthesis stages explicitly.

我们研究了第三种范式：在严格预算限制下，使用处于非思考模式的开源权重模型（DeepSeek V3.2），且不进行任何 ARC 专项微调。我们旨在探究仅通过架构设计能实现何种程度的性能恢复，并构建了能够明确拆解“模式发现”与“程序合成”阶段的智能体框架。

First, we introduce an Explorer-Definer Pipeline that separates pattern discovery from executable transformation synthesis, implemented as a two-stage agent pipeline. Next, we present the Reflective Orchestrator, which augments the pipeline with autonomous exploration of new transformations when previous hypotheses fail on training pairs.

首先，我们引入了“探索者-定义者流水线”（Explorer-Definer Pipeline），通过两阶段智能体流水线将模式发现与可执行转换合成过程分离开来。其次，我们提出了“反思型编排器”（Reflective Orchestrator），当先前的假设在训练对上失败时，该编排器能增强流水线，使其能够自主探索新的转换方案。

On the ARC-AGI-1 public 400-task evaluation set, the pipeline reaches 57.50% pass@2 at $0.25 per task, and the orchestrator reaches 67.25% pass@2 at $0.62 per task. Together these architectures lift a 15.50% one-shot baseline by ~52 points without benchmark-specific training or heavy test-time compute.

在 ARC-AGI-1 公共 400 任务评估集上，该流水线以每任务 0.25 美元的成本达到了 57.50% 的 pass@2 准确率，而编排器以每任务 0.62 美元的成本达到了 67.25% 的 pass@2 准确率。这些架构在无需基准特定训练或繁重测试时计算的情况下，将 15.50% 的一次性（one-shot）基线提升了约 52 个百分点。

Furthermore, the orchestrator-driven lift tests a falsifiable diagnostic the pipeline produces; unbiased pass@k analysis suggests the pipeline is generation-bound, not selection-bound (selection via training-pair accuracy captures ~95% of the candidate ceiling) and predicts that significant improvement requires broader generation, not better ranking.

此外，由编排器驱动的性能提升验证了流水线所产生的一个可证伪诊断结论；无偏 pass@k 分析表明，该流水线受限于“生成能力”而非“选择能力”（通过训练对准确率进行选择可捕获约 95% 的候选上限），并预测显著的性能提升需要更广泛的生成，而非更好的排序。

The orchestrator implements this prediction via adaptive re-exploration and confirms it (unbiased pass@1 lift +9.81 pp, matching selection-mediated pass@2 lift). An additional pipeline ablation identifies its think tool as a significant component, with removal reducing pass@2 by 5.75 pp.

编排器通过自适应重探索实现了这一预测并予以证实（无偏 pass@1 提升了 9.81 个百分点，与选择中介的 pass@2 提升相匹配）。额外的流水线消融实验表明，其“思考工具”（think tool）是一个关键组件，移除该组件会导致 pass@2 下降 5.75 个百分点。