Outcome Monitors: Recovery Affordances for Silent Tool Failures

结果监控器：针对静默工具故障的恢复机制

Abstract: When a tool call times out, the agent sees the failure and can route around it. A cached error page or negative price can instead arrive in the expected format and be consumed as fact. We introduce Outcome Monitors, which detect violations of outcome contracts mined from task-disjoint traces or derived from public schemas.

摘要： 当工具调用超时，智能体（Agent）能够感知到故障并绕过它。然而，缓存的错误页面或负数价格可能会以预期的格式返回，并被系统误认为是事实。我们引入了“结果监控器”（Outcome Monitors），它能够检测出违反“结果契约”（outcome contracts）的行为，这些契约是从任务无关的轨迹中挖掘出来的，或是从公共模式（schemas）中推导出来的。

On a violation, the monitor preserves the result and issues a nonbinding receipt naming the violated property and public recovery tools. In frozen, prespecified evaluations with injected failures, Outcome Monitors raise ToolMaze completion from 10.9% to 28.1% across four models in two provider families and replicate in a third.

一旦发生违规，监控器会保留结果，并发布一份非强制性的回执，指明被违反的属性以及可用的公共恢复工具。在注入故障的固定预设评估中，结果监控器将 ToolMaze 的任务完成率从 10.9% 提升至 28.1%（涵盖两个供应商家族的四种模型，并在第三个家族中得到复现）。

In tau-bench retail, completion improves by 14.0 and 12.0 points on two tiers. In separate ToolMaze controls, removing the recovery-tool list eliminates the measured gain and restoring it recovers the effect; diagnostic detail and timing produce no detectable differences. Gains concentrate where the fault blocks completion.

在 tau-bench 零售基准测试中，两个层级的任务完成率分别提升了 14.0 和 12.0 个百分点。在独立的 ToolMaze 对照实验中，移除恢复工具列表会消除已测得的性能增益，而恢复该列表则能使效果重现；诊断细节和时间延迟并未产生可检测的差异。性能提升主要集中在故障阻碍任务完成的场景中。

On a suite transcribed from a published incident taxonomy, detection outside the mined vocabulary falls to 46%, though delivery continues and completion is unchanged. Recovery tools are the active receipt content in these controls; extending detection beyond the contract vocabulary remains open.

在一套基于已发布事故分类法转录的测试集上，对于挖掘词汇表之外的检测能力下降至 46%，尽管数据交付仍在继续且任务完成率保持不变。在这些对照实验中，恢复工具是回执中的核心内容；如何将检测范围扩展至契约词汇表之外，仍是一个待解决的问题。