MEMENTO: Learning Context Management for Long Context Processing Agents

Computer Science > Computation and Language arXiv:2611.00244 (cs) [Submitted on 30 Oct 2026] Title: MEMENTO: Learning Context Management for Long Context Processing Agents Authors: Yunfan Zhang, Taehoon Hwang, Kathleen McKeown, Smaranda Muresan.

计算机科学 > 计算与语言 arXiv:2611.00244 (cs) [提交于 2026 年 10 月 30 日] 标题：MEMENTO：面向长上下文处理智能体的上下文管理学习 作者：Yunfan Zhang, Taehoon Hwang, Kathleen McKeown, Smaranda Muresan。

Abstract: LLM agents that operate over long inputs must perform context management: deciding what to retain in a fixed-size context window. A common approach is compaction, where the model replaces its history with a self-generated note. Yet off-the-shelf agentic models, including GPT-5.5 and GLM-5.3, often write ineffective notes, and reinforcement learning (RL) alone rarely discovers useful note-taking behavior.

摘要：处理长输入的大语言模型（LLM）智能体必须执行上下文管理：决定在固定大小的上下文窗口中保留什么内容。一种常见的方法是压缩，即模型用自生成的笔记替换其历史记录。然而，现成的智能体模型（包括 GPT-5.5 和 GLM-5.3）编写的笔记往往无效，且仅靠强化学习（RL）也很难发现有用的笔记行为。

We present MEMENTO, a long-context agent built on Qwen3-8B and trained to manage its context through compaction. We first bootstrap note-taking by fine-tuning on synthetic trajectories annotated with notes generated in hindsight, then reinforce it with RL using a reward that combines answer correctness, note-quality heuristics, and confidence calibration.

我们提出了 MEMENTO，这是一个基于 Qwen3-8B 构建的长上下文智能体，旨在通过压缩来管理其上下文。我们首先通过在带有事后生成笔记的合成轨迹上进行微调来引导笔记编写，然后利用结合了答案正确性、笔记质量启发式算法和置信度校准的奖励机制，通过强化学习对其进行强化。

Although trained only to compact into a fresh window, MEMENTO extends at test time to strategies such as context folding and confidence-based backtracking. On book-level question answering across four benchmarks, with books of up to 2.5M tokens, MEMENTO raises Qwen3-8B’s accuracy from 36.9% to 67.8% while never holding more than 20K tokens in context.

尽管 MEMENTO 仅接受过压缩到新窗口的训练，但在测试时它扩展到了上下文折叠和基于置信度的回溯等策略。在涵盖四个基准测试的书籍级问答任务中，面对长达 250 万 token 的书籍，MEMENTO 将 Qwen3-8B 的准确率从 36.9% 提高到了 67.8%，且上下文占用从未超过 2 万 token。

It outperforms GPT-5.5 agent in the same harness (13.1%) and full book context (56.7%) at less than 1/50 the cost. It also nearly matches a ~100x larger GLM-5.3 agent (69.3%) at 1/9 the cost, and comes within 6.4 points of GLM-5.3 full context at 1/26 the cost.

它在相同测试框架下优于 GPT-5.5 智能体（13.1%）和全书上下文模型（56.7%），且成本不到后者的 1/50。它还以 1/9 的成本几乎追平了规模大约 100 倍的 GLM-5.3 智能体（69.3%），并以 1/26 的成本达到了与 GLM-5.3 全上下文模型相差 6.4 个百分点的水平。