In-Model Context Compression with Agentic Soft Tokens for Long-Horizon LLM Agents

查看原文 / View Original 编辑 / Edit

Computer Science > Computation and Language arXiv:2610.41291 (cs) [Submitted on 30 Oct 2026] Title: In-Model Context Compression with Agentic Soft Tokens for Long-Horizon LLM Agents Authors: Xuning He, Kai Fan, Runyu Zhang, Binghuan Wang, Yuxiang Zhang, Xue Jiang, Yihong Dong.

计算机科学 > 计算与语言 arXiv:2610.41291 (cs) [提交于 2026 年 10 月 30 日] 标题：面向长程 LLM 智能体的模型内上下文压缩与智能软标记（Agentic Soft Tokens）作者：Xuning He, Kai Fan, Runyu Zhang, Binghuan Wang, Yuxiang Zhang, Xue Jiang, Yihong Dong。

Abstract: Long-horizon LLM agents play an important role in solving complex and sustained tasks, with ever-growing interaction contexts. However, these accumulated contexts will raise substantial computational cost and obscure key decision-relevant evidence. Existing methods typically train external compressors or context-management agents to rewrite context as shorter hard-token sequences, but such external selection may not align with the LLM’s evolving representation and is necessarily lossy.

摘要：长程 LLM 智能体在解决复杂且持续的任务中发挥着重要作用，其交互上下文不断增长。然而，这些积累的上下文会增加巨大的计算成本，并掩盖关键的决策相关证据。现有方法通常训练外部压缩器或上下文管理智能体，将上下文重写为更短的硬标记序列，但这种外部选择可能与 LLM 不断演进的表示不一致，且必然存在信息损失。

In this paper, we introduce Agentic Soft-Token Compression (ASTC), a training-free compression method using soft tokens within LLM agent for long-horizon tasks. Specifically, ASTC uses the model’s native recurrent dynamics to determine the already absorbed information, leveraging the commonly-used hybrid Gated Delta Network (GDN)-attention. It then carries this information forward as agentic soft tokens while retaining exact values as hard tokens.

在本文中，我们引入了智能软标记压缩（ASTC），这是一种无需训练的压缩方法，在 LLM 智能体内针对长程任务使用软标记。具体而言，ASTC 利用模型原生的循环动力学来确定已吸收的信息，并利用常用的混合门控增量网络（GDN）-注意力机制。随后，它将这些信息作为智能软标记向前传递，同时保留硬标记形式的精确值。

The resulting compact sequence reduces later-layer computation and yields reusable KV caches, extending the savings across the agent’s trajectory. Extensive experiments on long-horizon tasks show that ASTC matches or exceeds the uncompressed agent’s accuracy with up to 50.68% fewer end-to-end Attention Pairs, while trained compressors require 15.4 to 103.7 times more Attention Pairs per successful task.

由此产生的紧凑序列减少了后续层的计算，并生成了可重用的 KV 缓存，从而在智能体的整个轨迹中实现节省。在长程任务上的大量实验表明，ASTC 在端到端注意力对减少高达 50.68% 的情况下，其准确率与未压缩的智能体相当或更高，而训练过的压缩器在每个成功任务中所需的注意力对数量则是其 15.4 到 103.7 倍。

These findings highlight agentic soft tokens as a model-native way to carry absorbed context forward as the task representation evolves.

这些发现强调了智能软标记作为一种模型原生方式，能够在任务表示演进时将已吸收的上下文向前传递。