Lost in Compaction: Evaluating Side-Constraint Loss under Context Compaction

压缩中的迷失：评估上下文压缩下的侧向约束丢失

Abstract: When the context window is under pressure, LLM systems compact prior context to continue ongoing tasks. We identify a class of user-issued instructions, Session Constraints (SCs), such as “do not delete any emails until I confirm,” that are meant to constrain LLM’s behavior for the remainder of a session but are silently dropped during compaction.

摘要： 当上下文窗口面临压力时，大语言模型（LLM）系统会压缩先前的上下文以继续执行当前任务。我们识别出了一类由用户发出的指令，即“会话约束”（Session Constraints, SCs），例如“在我确认之前不要删除任何邮件”。这类指令旨在约束 LLM 在整个会话期间的行为，但在压缩过程中却被悄无声息地丢弃了。

To quantify this loss, we introduce COMPINT, an evaluation suite that evaluates compactors across three long-context scenarios: multi-turn chat, agentic trajectory, and long-horizon research. Current compactors retain only 17% of injected SCs on average, and most perform worse than running the same task without compaction. Retention varies sharply with compactor, prompt, context length, SC phrasing, and injection location, showing that the loss is systematic rather than tied to any single setting.

为了量化这种丢失，我们引入了 COMPINT，这是一个评估套件，用于在三种长上下文场景中评估压缩器：多轮对话、智能体轨迹和长周期研究。目前的压缩器平均仅能保留 17% 的注入约束，且大多数压缩器的表现甚至不如不进行压缩直接运行任务。保留率会随着压缩器、提示词、上下文长度、约束措辞和注入位置的不同而产生剧烈波动，这表明这种丢失是系统性的，而非局限于单一设置。

We propose an SC-aware extractor that runs alongside the compactor as a plug-and-play module, achieving over 90% retention across all three scenarios without modifying the compactor or LLM. The COMPINT evaluation suite and accompanying implementation are available at this https URL.

我们提出了一种具备 SC 感知能力的提取器，它作为即插即用模块与压缩器协同运行。在不修改压缩器或 LLM 的前提下，该模块在所有三种场景中均实现了超过 90% 的保留率。COMPINT 评估套件及相关实现代码可在该链接获取。