Root-Cause Attribution Is a Search Problem: Continual Search for Long-Horizon Agent Failures

根本原因归因是一个搜索问题：针对长周期智能体故障的持续搜索

Abstract: The increasing deployment of AI agents in long-horizon tasks yields massive execution logs. Diagnosing failures within these records is crucial for reliability, as it transforms outcome-level signals into actionable interventions. The sheer scale of the data renders human review impractical, driving the need for automated root-cause attribution (RCA).

摘要： AI 智能体在长周期任务中的部署日益增加，产生了海量的执行日志。在这些记录中诊断故障对于可靠性至关重要，因为它能将结果层面的信号转化为可操作的干预措施。数据的庞大规模使得人工审查变得不切实际，从而推动了对自动化根本原因归因（RCA）的需求。

However, automated RCA methods using LLMs suffer from low diagnostic accuracy, especially as execution traces grow larger. They struggle because relevant information is often sparse, distributed across distant actions, and disconnected from the visible failure, reducing root-cause attribution to a massive search problem.

然而，使用大语言模型（LLM）的自动化 RCA 方法诊断准确率较低，尤其是在执行轨迹变长时。它们面临的困难在于，相关信息往往非常稀疏，分布在相隔甚远的动作中，且与可见的故障脱节，这使得根本原因归因演变成了一个巨大的搜索问题。

Existing RCA methods typically rely on one-shot LLM judgments to diagnose failures from execution traces. While effective for shorter trajectories, these judges tend to settle on a plausible diagnosis early, leaving critical evidence in longer traces unexamined.

现有的 RCA 方法通常依赖于一次性（one-shot）的 LLM 判断来从执行轨迹中诊断故障。虽然对于较短的轨迹有效，但这些判断模型往往会过早地锁定一个看似合理的诊断结果，从而导致长轨迹中的关键证据未被审查。

We introduce Continual Search, an iterative framework that nudges the judge, over successive turns, to keep searching for unresolved diagnostic evidence. We evaluate Continual Search across four existing RCA benchmarks. Recognizing the lack of massive execution traces in current benchmarks, we introduce MegaRCA-Mix to evaluate RCA at scale.

我们引入了“持续搜索”（Continual Search），这是一个迭代框架，通过连续的轮次引导判断模型不断寻找未解决的诊断证据。我们在四个现有的 RCA 基准测试中评估了“持续搜索”。鉴于当前基准测试中缺乏海量执行轨迹，我们引入了 MegaRCA-Mix 来在大规模场景下评估 RCA。

MegaRCA-Mix provides a challenging testbed of 50 human-annotated failure trials spanning long-horizon, execution-heavy tasks. Across multiple benchmark suites and model families, Continual Search consistently improves attribution performance. On MegaRCA-Mix, for example, it improves GPT-5.5’s F1 score by more than 40%, from 0.349 to 0.498. More interestingly, within the same model family, lower-tier models can even surpass their higher-tier counterparts, demonstrating that effective search supersedes raw model scale.

MegaRCA-Mix 提供了一个极具挑战性的测试平台，包含 50 个由人工标注的故障试验，涵盖了长周期、高执行强度的任务。在多个基准测试套件和模型系列中，“持续搜索”始终能提升归因性能。例如，在 MegaRCA-Mix 上，它将 GPT-5.5 的 F1 分数提高了 40% 以上，从 0.349 提升至 0.498。更有趣的是，在同一模型系列中，低级别模型甚至可以超越高级别模型，这证明了有效的搜索比单纯的模型规模更为重要。