Context Injection and Long-Horizon Reliability: Separating Duplicate Recognition, Local Processing, and Counter Tracking

Computer Science > Computation and Language arXiv:2610.20633 (cs) [Submitted on 30 Sep 2026] Title: Context Injection and Long-Horizon Reliability: Separating Duplicate Recognition, Local Processing, and Counter Tracking Authors: Thomas Serval.

计算机科学 > 计算与语言 arXiv:2610.20633 (cs) [提交于 2026 年 9 月 30 日] 标题：上下文注入与长程可靠性：分离重复识别、局部处理与计数器跟踪 作者：Thomas Serval。

Abstract: Long-horizon failures of language models are usually measured as end-task success or as a horizon length. We instead score, step by step, three components of a 400-step ledger task in which every step is easy in isolation and the model’s own answers are re-injected into its context: recognising a record already seen 5 to 300 steps earlier, processing the current record locally, and writing a counter incremented at every step.

摘要：语言模型的长程失效通常以最终任务成功率或视界长度来衡量。我们转而对一项 400 步分类账任务的三个组成部分进行逐步评分，其中每一步单独来看都很简单，且模型自身的回答会被重新注入到其上下文中：识别 5 到 300 步前已见过的记录、局部处理当前记录，以及写入每一步递增的计数器。

On Qwen3-8B (non-thinking, 20 seeds, replicated on a second machine and backend), with every step scored component by component, two results hold. First, duplicate-recognition errors rise late in the episode — from 0.0019 to 0.011 per step at risk across step 200 — while the counter makes no error in 8,000 steps and local processing is descriptively stable; the rise combines long-distance recall errors and rising false alarms.

在 Qwen3-8B（非思维链模式，20 个随机种子，在第二台机器和后端上进行复现）上，通过对每一步进行组件级评分，得出两个结论。首先，重复识别错误在任务后期增加——在第 200 步之后，每步风险从 0.0019 上升至 0.011——而计数器在 8,000 步中未出现错误，局部处理在描述上保持稳定；这种上升结合了长距离回忆错误和不断增加的误报。

Second, a block of irrelevant content inserted before each step makes the counter fail in 20 of 20 seeds on each machine, against none without it, while its effect on local processing is much smaller and not distinguishable from none. Three pre-registered follow-ups probe the counter. Its value depends on the value present in the response history: a corrupted value persists without correction in every seed, and an exact external reminder, repeated before each message, re-aligns it within one to two steps.

其次，在每一步之前插入一段无关内容会导致计数器在每台机器的 20 个种子中全部失败（20/20），而在没有插入的情况下则无一失败，同时其对局部处理的影响要小得多，且与无影响的情况无法区分。三项预注册的后续研究对计数器进行了探究。其数值取决于响应历史中存在的数值：损坏的数值在每个种子中都会持续存在且无法自我修正，而每条消息前重复的精确外部提醒可在 1 到 2 步内将其重新对齐。

Under the irrelevant block, no counter error was observed when an exact compact reminder of the state, supplied externally by an oracle, preceded each message (0 of 20 seeds with an error, against 15 without it), and this reminder outperformed a length-matched neutral text; whether a single reminder suffices was not conclusive, and the absence of damage to the other components was not established. An aggregate or exclusive error classification can hide the difference we report.

在无关内容块存在的情况下，当每条消息前都有由预言机外部提供的精确状态紧凑提醒时，未观察到计数器错误（20 个种子中 0 个错误，而没有提醒时为 15 个），且该提醒的效果优于长度匹配的中性文本；单次提醒是否足够尚无定论，且未能确定其对其他组件是否无损害。汇总或排他的错误分类可能会掩盖我们所报告的差异。