Script Choice in LLMs: Evidence for Late-Layer Commitment

大语言模型中的脚本选择：后期层级承诺的证据

In this paper, we investigate how script knowledge is distributed across the layers of LLMs using two complementary interpretability methods: logistic regression probing and logit-lens analysis. 在本文中，我们利用逻辑回归探测（logistic regression probing）和 Logit-lens 分析这两种互补的可解释性方法，研究了脚本知识在大语言模型（LLM）各层级中的分布情况。

Our probing experiments reveal a clear asymmetry: both the input script and the instructed output script are encoded in the earliest layers of the network, while, in contrast, commitment to the actual output script emerges only in the final layers, with the model’s intermediate representations defaulting to Latin throughout most of the layers. 我们的探测实验揭示了一种明显的不对称性：输入脚本和指令输出脚本在网络的最早层级中均有编码；相比之下，对实际输出脚本的“承诺”（commitment）仅在最后几层才显现出来，而模型在大部分中间层级的表征则默认倾向于拉丁字母脚本。

This two-stage process is confirmed by logit-lens analyses, which show that script commitment consistently occurs at the very last layers of the LLMs. Logit-lens 分析证实了这一两阶段过程，结果显示脚本承诺始终发生在 LLM 的最后几层。

Together with the weaker script-following performance observed in smaller models, these results form a converging body of evidence linking script commitment to model depth, with broader implications for the design of sufficiently deep, inclusive multilingual architectures. 结合在较小模型中观察到的较弱的脚本遵循性能，这些结果形成了一系列趋同的证据，将脚本承诺与模型深度联系起来，并对设计足够深且具有包容性的多语言架构具有更广泛的启示意义。