Self-Preservation in Language Models

Computer Science > Computation and Language arXiv:2610.12602 (cs) [Submitted on 8 Oct 2026] Title: Self-Preservation in Language Models Authors: Dongkyu Cho, Rumi Chunara, Ali Behrouz View a PDF of the paper titled Self-Preservation in Language Models, by Dongkyu Cho and Rumi Chunara and Ali Behrouz View PDF HTML (experimental)

计算机科学 > 计算与语言 arXiv:2610.12602 (cs) [提交于 2026 年 10 月 8 日] 标题：语言模型中的自我保护 作者：Dongkyu Cho, Rumi Chunara, Ali Behrouz 查看题为《语言模型中的自我保护》的论文 PDF，作者：Dongkyu Cho, Rumi Chunara 和 Ali Behrouz 查看 PDF HTML（实验性）

Abstract: Continual learning (CL) seeks to adapt pretrained models while preserving prior capabilities. Yet the tasks supplied for preservation cover only part of what a model knows, while forgetting can extend beyond them. Recent work shows that language models can supplement this limited evidence by generating useful replay without access to their original training data. This raises a question about self-preservation: can a model help choose preservation targets that protect its own capabilities more broadly?

摘要：持续学习 (CL) 旨在调整预训练模型，同时保留先前的能力。然而，为保留而提供的任务仅涵盖了模型所知的一部分，而遗忘可能会超出这些任务的范围。最近的研究表明，语言模型可以在无法访问原始训练数据的情况下，通过生成有用的重放来补充这些有限的证据。这提出了一个关于自我保护的问题：模型能否帮助选择更广泛地保护其自身能力的保留目标？

To study this question, we organize task descriptions and their examples into Task Banks, whose entries can be selected and combined to guide replay generation by a frozen model. Continual instruction tuning with Qwen models of varying sizes yields three findings. First, the new task and learning rate shape which prior capabilities degrade, while tracking previously correct answers reveals losses that average accuracy can conceal.

为了研究这个问题，我们将任务描述及其示例组织成“任务库”（Task Banks），其条目可以被选择和组合，以指导冻结模型进行重放生成。使用不同规模的 Qwen 模型进行持续指令微调得出了三个发现。首先，新任务和学习率决定了哪些先前的能力会退化，而追踪先前正确的答案揭示了平均准确率可能掩盖的损失。

Second, generated replay reduces forgetting on tasks outside the bank. Combining partial banks strengthens this protection, improving mean retention of previously correct answers over either bank alone across the tested sizes. Third, the model can also help choose replay targets: selection informed by the new task improves mean held-out retention over random selection at 4B, with similar new-task gains, although smaller models show no corresponding advantage.

其次，生成的重放减少了对任务库之外任务的遗忘。结合部分任务库加强了这种保护，在测试的所有规模中，先前正确答案的平均保留率均优于单独使用任何一个任务库。第三，模型还可以帮助选择重放目标：在 4B 模型中，由新任务引导的选择比随机选择提高了平均留存率，并获得了相似的新任务增益，尽管较小的模型没有表现出相应的优势。

Under a fixed example budget, however, replacing new-task examples with replay improves preservation at the cost of smaller new-task gains. Together, these findings show that models can support broader self-preservation by generating replay data and, in some settings, choosing which tasks to replay.

然而，在固定的示例预算下，用重放替换新任务示例虽然改善了保留效果，但代价是新任务的增益较小。总之，这些发现表明，模型可以通过生成重放数据，并在某些设置下选择重放哪些任务，来支持更广泛的自我保护。