Success Leaves Detours: Learning Executable Walkthroughs for Long-Horizon Agents

成功留下的弯路：为长程智能体学习可执行的演练指南

Abstract: Test-time self-evolving agents improve by reusing past experience, yet sparse-reward trajectories contain failures, loops, and detours, while summaries often omit the state conditions and action dependencies needed for execution. 摘要： 测试时自进化智能体通过重用过往经验来提升性能，然而稀疏奖励轨迹中往往包含失败、循环和弯路，而总结性描述通常忽略了执行所需的各种状态条件和动作依赖关系。

We study executable Walkthrough induction from sparse-reward trajectories: extracting compact, state-conditioned, and verifiable procedures. Our key observation is that delayed credit identifies actions associated with progress but cannot determine whether they produce facts required by later actions. 我们研究了从稀疏奖励轨迹中归纳可执行“演练指南”（Walkthrough）的方法：即提取紧凑、状态条件化且可验证的程序。我们的核心观察是，延迟信用分配（delayed credit）虽然能识别与进展相关的动作，但无法确定这些动作是否产生了后续动作所需的事实依据。

We propose Trace, a credit-guided, dependency-grounded framework that compiles noisy trajectories into executable Walkthrough Memory. It detects progress anchors from rewards and persistent state changes, propagates credit to identify valuable transitions, and estimates action prerequisites from cross-episode success and failure evidence. 我们提出了 Trace，这是一个基于信用引导和依赖关系的框架，旨在将嘈杂的轨迹编译为可执行的“演练记忆”。它通过奖励和持续的状态变化来检测进展锚点，传播信用以识别有价值的转换，并根据跨剧集的成功与失败证据来评估动作的前置条件。

Backward dependency slicing then traces required facts to their producers, extracting dependency-consistent action chains while removing irrelevant loops and detours. The resulting Walkthroughs encode entry conditions, ordered state—action—effect steps, and completion and failure predicates, supporting reuse, intermediate-state resumption, and programmatic verification. 随后，通过反向依赖切片技术，将所需事实追溯至其生产者，从而提取出依赖一致的动作链，同时剔除无关的循环和弯路。最终生成的演练指南编码了进入条件、有序的“状态-动作-效果”步骤，以及完成和失败谓词，从而支持重用、中间状态恢复和程序化验证。

Experiments on J-TTL, WebShop, and ScienceWorld with three open-source LLMs show that Trace consistently outperforms eight test-time learning and memory baselines. Compared with the strongest baseline, it improves average AUC and Final-3 by 30.0% and 40.5%, respectively, while using fewer inference tokens. 在 J-TTL、WebShop 和 ScienceWorld 上使用三种开源大语言模型进行的实验表明，Trace 在性能上始终优于八个测试时学习和记忆基准模型。与最强的基准模型相比，它在平均 AUC 和 Final-3 指标上分别提升了 30.0% 和 40.5%，同时使用的推理 Token 更少。

These results show that long-horizon interaction benefits more from state-conditioned executable procedures than from complete trajectories or abstract summaries. 这些结果表明，相比于完整的轨迹或抽象的总结，长程交互从状态条件化的可执行程序中获益更多。