Can Language Models Learn While They Think? Testing Gradient Adaptation Across Architectures

查看原文 / View Original 编辑 / Edit

Computer Science > Machine Learning arXiv:2610.33025 (cs) [Submitted on 10 Oct 2026] Title: Can Language Models Learn While They Think? Testing Gradient Adaptation Across Architectures Authors: Surya Saka.

计算机科学 > 机器学习 arXiv:2610.33025 (cs) [提交于 2026 年 10 月 10 日] 标题：语言模型能在思考时学习吗？跨架构梯度适应性测试作者：Surya Saka。

Abstract: Letting a language model take gradient steps on the text it is reading, known as dynamic evaluation, lowers its loss on the rest of the document. Its behaviour on hybrid linear-attention architectures, on diffusion language models, and on standard long-context benchmarks has not been measured. We measure it.

摘要：让语言模型对其正在阅读的文本采取梯度步骤（即动态评估）可以降低其在文档后续部分的损失。其在混合线性注意力架构、扩散语言模型以及标准长上下文基准测试中的表现尚未得到测量。我们对此进行了测量。

With one SGD step per 128-token chunk on the MLP output projections of the last quarter of blocks, no training, and a 2k-token window, softmax-attention models (Qwen3-1.7B, Qwen3-4B, Gemma-4-E4B) lose 5-8% of their next-token loss over a document and outperform the same frozen model given the full 32k-token context.

通过在最后四分之一区块的 MLP 输出投影上每 128 个 token 进行一次 SGD 步骤，且无需训练，在 2k-token 窗口下，softmax 注意力模型（Qwen3-1.7B、Qwen3-4B、Gemma-4-E4B）在文档上的下一个 token 损失降低了 5-8%，且表现优于给定完整 32k-token 上下文的相同冻结模型。

Hybrid Gated-DeltaNet models (Qwen3.5-4B, Qwen3.8-27B) gain under 1% at every learning rate: the linear-attention blocks already perform the adaptation. The mechanism transfers to masked-diffusion models with a larger effect, 8% on Nemotron-Diffusion-8B and 17-20% on DiffusionGemma-26B, once the rate is lowered and updates are clipped.

混合 Gated-DeltaNet 模型（Qwen3.5-4B、Qwen3.8-27B）在各种学习率下的增益均低于 1%：线性注意力区块已经执行了适应。该机制可迁移至掩码扩散模型，且效果更显著，在降低学习率并对更新进行裁剪后，Nemotron-Diffusion-8B 的增益为 8%，DiffusionGemma-26B 的增益为 17-20%。

Across RULER, BABILong, LongBench-v2, SCOTUS, HotpotQA, MuSiQue, ContractNLI and citation completion, at 100 or more items on two model sizes, weights updated on the full document recover none of the accuracy lost to context truncation: the adaptation is style and vocabulary, not retrieval.

在 RULER、BABILong、LongBench-v2、SCOTUS、HotpotQA、MuSiQue、ContractNLI 和引文补全等测试中，针对两种模型尺寸的 100 个或更多项目，在完整文档上更新的权重无法恢复因上下文截断而损失的任何准确率：这种适应性属于风格和词汇层面，而非检索层面。

Writing facts with the next-token loss stores the prompt format rather than the fact (30% recall under chat prompting against 80% in context) and costs 36-38 GSM8K points at every dose. Writing a conclusion as behaviour instead, by distilling the model’s own conclusion-conditioned judgments into a low-rank adapter, reaches the in-context ceiling (0.83 and 0.88 against 0.84 and 0.88 on 100 games) with GSM8K unchanged, and the conclusion can come from an inquiry loop that found it by asking information-gain questions.

利用下一个 token 损失来写入事实，存储的是提示格式而非事实本身（聊天提示下的召回率为 30%，而上下文中的召回率为 80%），且每次都会损失 36-38 个 GSM8K 分数。相反，若将结论作为行为来写入，通过将模型自身基于结论的判断提炼为低秩适配器，可以达到上下文内的上限（在 100 场游戏中分别为 0.83 和 0.88，对比 0.84 和 0.88），且 GSM8K 分数保持不变；该结论可以来自一个通过询问信息增益问题而找到结论的查询循环。