Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets
Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets
Title: Retrofitting Recurrent Depth into a Pretrained Language Model: Installation, Extrapolation, Transfer, and Retention at Two Parameter Budgets 标题： 为预训练语言模型加装循环深度：两种参数规模下的安装、外推、迁移与保持
Abstract: A dense, pretrained language model can be retrofitted with recurrent depth and learn an iterative latent transition that persists after outcome-only annealing. Qwen2.5-0.5B-Instruct is split into a Prelude, a weight-tied Recurrent Block, and a Coda, with an identity-preserving one-loop path and a re-entry bridge on later loops. At loop 1 the retrofit remains non-inferior to its base on a preregistered ARC battery.
摘要： 稠密预训练语言模型可以通过加装循环深度（Recurrent Depth）进行改造，并学习一种在仅基于结果的退火（outcome-only annealing）后依然能够持续存在的迭代潜在转换。研究将 Qwen2.5-0.5B-Instruct 模型拆分为前奏（Prelude）、权重共享的循环块（Recurrent Block）和尾声（Coda），并设计了保持恒等映射的单循环路径以及后续循环的重入桥接。在第 1 次循环时，该改造模型在预注册的 ARC 测试集上的表现不亚于原始基座模型。
Three findings. First, the mechanism is a reusable procedure rather than terminal-answer lookup, and installs at two budgets: 6M trained parameters over frozen base weights and 180M full-block. With intermediate-step supervision, the model computes one task step per loop and persists when only final answers are graded. The adapter matched the full block overall (83.8% versus 84.0%), led through depth 11, and trailed beyond. Verbal fine-tuning reached 79-86% on controlled verbal renderings (zero-shot transfer was minimal), and adapter verbal training begun from the installed mechanism outpaced matched fresh training by 18.6 points, including on a held-out test set.
研究得出三个结论。首先，该机制是一种可复用的过程，而非简单的终点答案查找，且可在两种参数规模下安装：在冻结基座权重基础上训练 600 万参数，或进行 1.8 亿参数的全块训练。通过中间步骤监督，模型在每次循环中计算一个任务步骤，并在仅对最终答案评分时依然有效。适配器（Adapter）在整体表现上与全块训练相当（83.8% 对 84.0%），在深度 11 之前处于领先，之后略有落后。在受控的语言表达任务中，语言微调达到了 79-86% 的准确率（零样本迁移效果微弱），且基于已安装机制进行的适配器语言训练比匹配的从头训练高出 18.6 个百分点，在留出测试集上也表现优异。
Second, the operation extrapolates to roughly 1.5 times its supervised depth, holding 70% accuracy through depth 18. Third, a same-size scratchpad-trained model matched the recurrent model within its learned horizon but collapsed beyond it. The recurrent model won overall, 84% versus 72%, retained 53% versus 2.5% beyond depth 10, and answered 7.6 times faster. An iterative transformer can therefore perform deeper reasoning in latent space faster than comparable or larger models fine-tuned on the same task, in a system-level comparison.
其次，该操作可外推至约 1.5 倍的监督深度，在深度 18 时仍保持 70% 的准确率。第三，一个相同规模的草稿纸（scratchpad）训练模型在其学习范围内与循环模型表现相当，但超出该范围后性能崩溃。循环模型在整体上以 84% 对 72% 的准确率胜出，在深度 10 之后仍保持 53% 的准确率（对比 2.5%），且回答速度快 7.6 倍。因此，在系统级比较中，迭代 Transformer 能够在潜在空间中比在相同任务上微调的同等或更大规模模型更快地执行更深层的推理。
A second task, running the rule in reverse, exposed the limits: the inverse was learnable in isolation, but no continuation acquired it while preserving the installed mechanism and general capability, a catastrophic-interference boundary. Learned depth selection remains open.
第二个任务是反向执行规则，这暴露了该方法的局限性：反向规则虽然可以单独学习，但没有任何延续训练能在保留已安装机制和通用能力的同时习得该规则，这构成了灾难性遗忘的边界。关于学习深度的选择问题仍有待进一步研究。