Cross-Prompt Reuse of Entity-Replacement Responses
Computer Science > Computation and Language arXiv:2611.00352 (cs) [Submitted on 30 Oct 2026] Title: Cross-Prompt Reuse of Entity-Replacement Responses Authors: Jinkun Chen, Xuemin Yu, Sijia Han, Vlado Keselj View a PDF of the paper titled Cross-Prompt Reuse of Entity-Replacement Responses, by Jinkun Chen and 3 other authors View PDF HTML (experimental)
计算机科学 > 计算与语言 arXiv:2611.00352 (cs) [提交于 2026 年 10 月 30 日] 标题：实体替换响应的跨提示重用 作者：Jinkun Chen, Xuemin Yu, Sijia Han, Vlado Keselj 查看标题为《实体替换响应的跨提示重用》的论文 PDF，作者为 Jinkun Chen 及其他 3 位作者 查看 PDF HTML（实验性）
Abstract: With model weights fixed, we measure how much of an entity-replacement response transfers between English factual questions, beyond whether answers change. We add the hidden-state change from two source questions to a target question about the same entity, comparing the predicted log-probability shift between two relations’ published answers, given known answer prefixes, with the measured one.
摘要：在模型权重固定的情况下，我们测量了实体替换响应在英语事实性问题之间有多少发生了迁移，而不仅仅是关注答案是否改变。我们将来自两个源问题的隐藏状态变化添加到关于同一实体的目标问题中，并在已知答案前缀的情况下，比较两个关系已发布答案之间的预测对数概率偏移与测量值。
Hidden reuse injects the aligned change at 50% depth before the target’s remaining layers; direct reuse adds source output-score changes. With other-relation sources, hidden reuse removes 45% and 49% of no-change error on 0.5-1B Qwen and Llama. Gemma-2-2B removes 22% and passes one of three comparisons in plain text without its default beginning-of-sequence (BOS) token.
隐藏重用在目标剩余层之前的 50% 深度处注入对齐的变化；直接重用则添加源输出分数的变化。对于其他关系源，隐藏重用在 0.5-1B 参数的 Qwen 和 Llama 模型上消除了 45% 和 49% 的无变化误差。Gemma-2-2B 模型消除了 22% 的误差，并在不使用默认序列开始 (BOS) 标记的纯文本中通过了三项比较中的一项。
A later protocol restoring it yields 58-68%, passing every comparison on three cohorts. Larger checkpoints, chat format and new entities show replication and format- or cohort-dependent failures. On small plain-format checkpoints, other-relation hidden reuse mainly reproduces the shift both relations’ questions share, without reliably lowering relation-selective error.
随后恢复该标记的协议达到了 58-68% 的效果，在三个队列的所有比较中均获得通过。更大的检查点、聊天格式和新实体显示出复制以及依赖于格式或队列的失败。在小型纯格式检查点上，其他关系的隐藏重用主要重现了两个关系问题所共有的偏移，而未能可靠地降低关系选择性误差。
With target-relation sources, hidden reuse’s post hoc advantage over direct reuse is 0.2-0.9 nats outside chat format, unconfirmed and not replacement-specific. The same relation-neutral change at both questions’ entity tokens lowers relation-selective error more than random or other-replacement changes, indicating receiving-question dependence.
对于目标关系源，隐藏重用在聊天格式之外相对于直接重用的事后优势为 0.2-0.9 nats，但未经证实且不具有替换特异性。在两个问题的实体标记处进行相同的关系中性变化，比随机或其他替换变化更能降低关系选择性误差，这表明存在接收问题依赖性。