ECHOES: Seeing Beyond the Current Image via Recursive History Compression and Privileged On-Policy Distillation

Title: ECHOES: Seeing Beyond the Current Image via Recursive History Compression and Privileged On-Policy Distillation

标题：ECHOES：通过递归历史压缩和特权策略内蒸馏实现超越当前图像的视觉感知

Long-horizon GUI agents rely on visual history to maintain task states, but retaining historical screenshots increases visual-token usage and inference costs, while textual summaries can discard critical state information, undermining long-horizon reliability.

长程 GUI 智能体依赖视觉历史来维持任务状态，但保留历史截图会增加视觉标记的使用量和推理成本，而文本摘要可能会丢失关键的状态信息，从而削弱长程任务的可靠性。

We propose ECHOES, which leverages training-time privileged information to guide textual history compression and strengthen long-horizon reasoning. Specifically, ECHOES adopts a Compression-Think-Action (CTA) format to recursively update a textual history state and reason.

我们提出了 ECHOES，它利用训练时的特权信息来指导文本历史压缩并加强长程推理。具体而言，ECHOES 采用压缩-思考-行动 (CTA) 格式来递归更新文本历史状态并进行推理。

The same model, given privileged multi-image inputs, serves as a teacher that provides dense token-level on-policy distillation on student-generated CTA rollouts. Our Perception-Memory Fidelity (PMF) Gap-guided OPD-RL framework dynamically balances privileged distillation and task-reward optimization, with periodic teacher updates supporting continued improvement.

同一模型在给定特权多图像输入的情况下，充当教师角色，对学生生成的 CTA 展开提供密集的标记级策略内蒸馏。我们的感知-记忆保真度 (PMF) 差距引导的 OPD-RL 框架动态平衡了特权蒸馏和任务奖励优化，并通过定期的教师更新来支持持续改进。

We further introduce ECHOESBMK, a fine-grained diagnostic benchmark derived from real GUI agent failure trajectories to jointly evaluate visual perception and cross-modal state retention. Experiments on four GUI benchmarks and ECHOESBMK demonstrate substantial gains in task performance and memory fidelity.

我们进一步引入了 ECHOESBMK，这是一个源自真实 GUI 智能体失败轨迹的细粒度诊断基准，用于联合评估视觉感知和跨模态状态保留。在四个 GUI 基准测试和 ECHOESBMK 上的实验表明，任务性能和记忆保真度均有显著提升。

Without retaining historical screenshots at deployment, ECHOES achieves competitive performance with a favorable performance-inference-cost trade-off and exhibits positive zero-shot transfer to unseen visual planning tasks.

在部署时无需保留历史截图，ECHOES 实现了具有竞争力的性能，在性能与推理成本之间取得了良好的平衡，并对未见过的视觉规划任务表现出了积极的零样本迁移能力。