Cross-Model Memory Transfer via Target-Side Reader Adaptation

通过目标端阅读器适配实现跨模型记忆迁移

Abstract: Methods for improving knowledge use in large language models typically fall into two regimes. Non-parametric retrieval offers flexible access to external knowledge, but adds retrieval latency, context overhead, and only shallow integration with the backbone. Parametric adaptation is efficient at inference time, but entangles knowledge with model weights and can be hard to update, audit, or transfer.

摘要： 提升大语言模型知识利用率的方法通常分为两类。非参数化检索提供了对外部知识的灵活访问，但增加了检索延迟和上下文开销，且与主干模型的集成较浅。参数化适配在推理时效率较高，但会将知识与模型权重纠缠在一起，导致难以更新、审计或迁移。

Engram-style hashed memory occupies a middle regime: it stores learned information in an external, addressable table, yet consumes that table through a small learned reader. This raises a basic question: when such a memory is moved across backbones, what matters more, the frozen memory itself or the target-side reader?

Engram 式哈希记忆占据了中间地带：它将学习到的信息存储在外部的可寻址表中，并通过一个小型学习阅读器来读取该表。这提出了一个基本问题：当这种记忆在不同主干模型间迁移时，什么更重要——是冻结的记忆本身，还是目标端的阅读器？

We study this question through cross-model frozen-memory extraction, in which a memory trained on a source model is frozen and attached to a different target model, with only a lightweight reader trained. Ablations show that learned memory content and correct addressing both matter, but the transferred table becomes useful only through a reader aligned to the target model.

我们通过跨模型冻结记忆提取来研究这一问题，即将源模型上训练的记忆冻结并附加到不同的目标模型上，仅训练一个轻量级阅读器。消融实验表明，学习到的记忆内容和正确的寻址方式都很重要，但迁移后的表格只有通过与目标模型对齐的阅读器才能发挥作用。

In downstream question answering tasks, a dual-layer, four-branch reader nearly closes the gap between same-model and cross-model reuse, achieving an average score of 38.8 under our controlled evaluation protocol. Moreover, when the provider reader is directly compatible with the target interface, the frozen artifact can provide substantial utility without target-side training, while optional reader adaptation yields further improvement.

在下游问答任务中，双层四分支阅读器几乎消除了同模型复用与跨模型复用之间的差距，在我们的受控评估协议下达到了 38.8 的平均分。此外，当提供者阅读器与目标接口直接兼容时，冻结的制品无需目标端训练即可提供显著效用，而可选的阅读器适配则能带来进一步的提升。

These results suggest that Engram can serve as a reusable external knowledge artifact, provided that the target has access to a compatible reader interface; target-side adaptation can further improve alignment when direct reader reuse is insufficient.

这些结果表明，只要目标端拥有兼容的阅读器接口，Engram 就可以作为一种可复用的外部知识制品；当直接复用阅读器不足以满足需求时，目标端适配可以进一步改善对齐效果。