Memory in the Loop: In-Process Retrieval as Extended Working Memory for Language Agents

循环中的记忆：作为语言智能体扩展工作记忆的进程内检索

Abstract: Language agents run a loop - observe, reason, act - but the memory they reason over sits outside it: a store queried at most once per turn. We study the regime where memory moves inside the loop, read and written on every step. The obstacle has always been latency: networked stores answer in tens to hundreds of milliseconds, and in-loop retrieval can inflate end-to-end latency by up to 83x when retrieval is expensive.

摘要： 语言智能体通常运行着“观察-推理-行动”的循环，但它们所推理的记忆却位于循环之外：即每个回合最多查询一次的存储系统。我们研究了一种将记忆移入循环内部的模式，即在每一步都进行读写。过去，这一方案的障碍始终是延迟问题：网络存储的响应时间通常在几十到几百毫秒之间，当检索成本较高时，循环内检索可能会使端到端延迟增加高达 83 倍。

Prior work manages that cost rather than questioning it: serving-layer scheduling hides it, “memory-first” designs ration retrieval to once per turn. We argue latency is a property of where the store lives, not the in-loop pattern: an in-process store answers in ~100us, three orders of magnitude below the network regime, and at that speed the per-step tax collapses. By the extended-mind thesis’s parity principle, a store fast enough to be constantly and directly available becomes extended working memory, not a tool the agent merely consults.

以往的研究倾向于管理这种成本，而非质疑其必要性：例如通过服务层调度来隐藏延迟，或采用“记忆优先”设计将检索限制为每回合一次。我们认为，延迟取决于存储系统的位置，而非循环模式本身：进程内存储的响应时间约为 100 微秒，比网络模式快了三个数量级，在这种速度下，每一步的额外开销几乎可以忽略不计。根据“延展心灵论”的等价原则，如果一个存储系统足够快，能够被持续且直接地访问，它就成为了扩展工作记忆，而不再仅仅是智能体咨询的工具。

The premise is causal: holding a fixed per-turn memory-latency budget and varying only the store’s answer speed, redundant actions rise monotonically with latency - 0.0 of 12 at in-process speed, 7.2 of 12 at a 110ms cloud round trip (gpt-5-nano, gpt-5-mini; exact permutation p=0.0079). We demonstrate the regime end-to-end: across four GPT-5-class models under a bounded window, recall improves from 0/5 to 3.6-4.8/5 with in-loop memory, store ops at p50 80-165us - though an instructed restate-every-reply baseline also solves it perfectly, at a token cost that grows with the working set.

这一前提具有因果性：在保持每回合记忆延迟预算不变的情况下，仅改变存储系统的响应速度，冗余操作会随着延迟的增加而单调上升——在进程内速度下，12 次操作中冗余为 0.0 次；而在 110 毫秒的云端往返延迟下，冗余则上升至 7.2 次（基于 gpt-5-nano 和 gpt-5-mini 模型；精确排列检验 p=0.0079）。我们对该模式进行了端到端验证：在受限窗口下，四种 GPT-5 级模型使用循环内记忆后，召回率从 0/5 提升至 3.6-4.8/5，存储操作的 p50 延迟为 80-165 微秒。尽管“每回复必重述”的基准方法也能完美解决问题，但其 Token 成本会随着工作集的大小而增加。

The store never lost a fact in any run (244 of 244 writes kept); every miss traces to the agent’s read policy, not the store. Our measurements also relocate the bottleneck: the dominant per-step cost is embedding (~200-400ms over the network); pairing the in-process store with a small local embedder returns the complete operation to a measured ~40us.

在所有运行测试中，存储系统从未丢失过任何事实（244 次写入全部保留）；每一次检索失败都源于智能体的读取策略，而非存储系统本身。我们的测量结果还重新定位了瓶颈所在：每一步的主要成本在于嵌入（Embedding）过程（通过网络约为 200-400 毫秒）；将进程内存储与小型本地嵌入器配对后，完整操作的测量延迟可降至约 40 微秒。