The World Won't Wait: Evaluating and Teaching Agents Under a Live Clock

Large language model (LLM) agents can solve complex interactive tasks, yet reasoning and tool calls take time. In continuously changing environments, an otherwise correct action may arrive too late.

大型语言模型（LLM）智能体能够解决复杂的交互式任务，但推理和工具调用需要时间。在不断变化的环境中，一个原本正确的操作可能会因为响应太慢而失效。

We introduce RealtimeBench, with 180 tasks across six game types, to evaluate agents while games continue running during inference. Paired evaluations show that GPT-6.1-sol outperforms the faster Jev when games are paused, but falls behind when they keep running.

我们引入了 RealtimeBench，包含六种游戏类型下的 180 项任务，旨在评估智能体在推理过程中游戏持续运行时的表现。配对评估显示，当游戏暂停时，GPT-6.1-sol 的表现优于速度更快的 Jev，但当游戏持续运行时，其表现则会落后。

We propose Two-Timescale Offline Recursive Self-Improvement (TORSI): a slow teacher studies failures and tests corrections in paused games between attempts, then writes reusable instructions for a fast student. The student acts without teacher calls during gameplay, and validation scores determine which updates are adopted.

我们提出了双时间尺度离线递归自我改进（TORSI）方法：一个“慢速教师”在尝试间隙通过暂停的游戏研究失败案例并测试修正方案，随后为“快速学生”编写可重用的指令。学生在游戏过程中无需教师调用即可自主行动，验证分数则决定了哪些更新会被采纳。

Two rounds raise Jev’s average test score from 13.51 to 24.19 on a 0—100 progress scale, improving all six game types. The instructions also improve average scores for three other models without retuning.

经过两轮迭代，Jev 在 0-100 分的进度量表上的平均测试分数从 13.51 提高到了 24.19，且在所有六种游戏类型中均有提升。这些指令还在无需重新调整的情况下，提高了其他三个模型的平均分数。

These findings point toward real-time intelligence: enabling powerful LLM agents to remain effective in a continuously changing world by organizing deliberation, learning, and action across timescales.

这些发现指向了实时智能的方向：通过跨时间尺度组织审议、学习和行动，使强大的 LLM 智能体能够在不断变化的世界中保持有效性。