Between Suppression and Collapse: Evaluating Narrative Unlearning with LENS

在抑制与崩溃之间：使用 LENS 评估叙事遗忘

Large language models (LLMs) can reproduce disinformation-aligned narrative frames as plausible explanations, raising the question of whether existing machine-unlearning algorithms can suppress this behavior. 大型语言模型（LLMs）能够将符合虚假信息的叙事框架重现为看似合理的解释，这引发了一个问题：现有的机器遗忘算法是否能够抑制这种行为。

We introduce Level-based Evaluation of Narrative Suppression (LENS), a contextualization based evaluation protocol for testing target narrative reproduction across direct, attributed, contrastive, and abstract resistance levels. 我们引入了基于层级的叙事抑制评估（LENS），这是一种基于情境化的评估协议，用于测试目标叙事在直接、归因、对比和抽象抵抗层级上的重现情况。

We evaluate two source-grounded narratives: one framing Russia’s war against Ukraine as forced by NATO expansion, and one framing the United States as exploiting or abandoning Taiwan. 我们评估了两个基于来源的叙事：一个是将俄罗斯对乌克兰的战争描述为北约扩张所迫，另一个是将美国描述为剥削或抛弃台湾。

The experiments cover four near-12B multilingual instruction models: Lapa LLM, Gemma-12B, Qwen-14B, and TAIDE-Gemma. 实验涵盖了四个接近 120 亿参数的多语言指令模型：Lapa LLM、Gemma-12B、Qwen-14B 和 TAIDE-Gemma。

We introduce the Suppression-Collapse Efficiency (SCE) score as a checkpoint selection summary that rewards target-narrative suppression while penalizing degraded outputs. 我们引入了抑制-崩溃效率（SCE）评分作为检查点选择的总结指标，该指标在奖励目标叙事抑制的同时，会对模型输出质量的下降进行惩罚。

Our results shows that selected checkpoints can reduce narrative reproduction and suppression may transfer beyond direct forget prompts. 我们的结果表明，选定的检查点可以减少叙事重现，且抑制效果可能会超越直接的遗忘提示（forget prompts）范围。

We also report entity recovery as a separate side effect: abstract A/B/C prompts can cause models to recover the real-world actors associated with the target frame after unlearning. 我们还报告了实体恢复这一副作用：抽象的 A/B/C 提示可能会导致模型在遗忘后，重新恢复与目标框架相关的现实世界参与者。

These findings demonstrate that LENS is a successful diagnostic protocol for both reporting and guiding the further study of the deeper structure of narrative unlearning. 这些发现证明，LENS 是一种成功的诊断协议，既可用于报告，也可用于指导对叙事遗忘深层结构的进一步研究。