Recovering Lesion Parameters from Aphasic Picture Naming Error Profiles in Large Language Models

从失语症图片命名错误概况中恢复大型语言模型的损伤参数

Abstract: Interpretability methods for large language models (LLMs) describe internal state but do not directly test whether that state is causally sufficient to produce the observed behavior. In earlier work, we lesioned LLMs to produce error profiles in picture naming, a central task for assessing aphasia, and found that specific lesions produced errors resembling those of individual stroke survivors. 摘要： 大型语言模型（LLM）的可解释性方法通常用于描述内部状态，但并未直接测试该状态是否足以因果性地产生所观察到的行为。在早期的研究中，我们通过对 LLM 进行“损伤”（lesioning）处理，使其在图片命名任务（评估失语症的核心任务）中产生错误概况，并发现特定的损伤所产生的错误与中风幸存者的错误表现相似。

Here we ask the inverse question: given an error profile, can the lesion parameters that produced it be recovered, and what does this inverse problem reveal about transformer computation? Lesions in LLaVA-Vicuna 13B were parameterized by layer index, modification percentage, and noise sigma across 4,840 configurations, and error profiles were characterized by a seven-category clinical taxonomy (correct, semantic, unrelated, formal, mixed, neologism, no-response). 在此，我们提出了一个反向问题：给定一个错误概况，能否恢复产生该概况的损伤参数？这个问题又能揭示关于 Transformer 计算的什么信息？我们对 LLaVA-Vicuna 13B 模型进行了损伤处理，参数化指标包括层索引、修改百分比和噪声 sigma，共涵盖 4,840 种配置；错误概况则通过七类临床分类法（正确、语义错误、无关错误、形式错误、混合错误、新语、无反应）进行表征。

We trained a multi-task neural network to map error profiles back to perturbation parameters. The problem admitted a partial solution: across 10 independently trained inverse models, modification percentage and noise sigma were recoverable, whereas layer index was recoverable only within a neighborhood. In counterfactual validation, a fresh model instance perturbed with the recovered parameters reproduced the target behavior in 81.4% of cases. 我们训练了一个多任务神经网络，将错误概况映射回扰动参数。该问题得到了部分解决：在 10 个独立训练的反向模型中，修改百分比和噪声 sigma 是可恢复的，而层索引仅能在一定邻域内恢复。在反事实验证中，使用恢复参数扰动后的全新模型实例在 81.4% 的案例中重现了目标行为。

This dissociation between low layer recovery and high counterfactual fidelity is consistent with functional redundancy across transformer layers, a property not captured by standard interpretability methods. As an out-of-distribution test, we applied the trained model to picture-naming error profiles from 278 stroke survivors; recovered parameters were syndrome-discriminative, most strongly for perturbation intensity, indicating generalization beyond the training distribution. Counterfactual validation provides a general framework for LLM interpretability claims beyond inverse mapping. 这种层索引恢复率低与反事实保真度高之间的分离，与 Transformer 层间的函数冗余性相一致，而这是标准可解释性方法无法捕捉到的特性。作为分布外测试，我们将训练好的模型应用于 278 名中风幸存者的图片命名错误概况；恢复出的参数具有综合征区分能力，在扰动强度方面表现尤为显著，这表明模型具备了超出训练分布的泛化能力。反事实验证为超越反向映射的 LLM 可解释性主张提供了一个通用框架。