Decoupling Internal Representational Changes and Causal Importance in Fine-Tuned Large Language Models

解耦微调大语言模型中的内部表征变化与因果重要性

Abstract: Fine-tuning has emerged as a widely adopted approach for adapting LLMs to a variety of downstream tasks. However, how it reshapes their internal mechanisms remains poorly understood. To address this, we investigate how fine-tuning alters internal representations in LLMs, including attention patterns and layer-wise activations, and examine whether these changes are linked to task-relevant components identified by EAP (e.g., attention heads and logit-level activations) that drive task performance.

摘要： 微调已成为将大语言模型（LLM）适配至各种下游任务的广泛采用的方法。然而，微调如何重塑其内部机制仍知之甚少。为了解决这一问题，我们研究了微调如何改变 LLM 的内部表征（包括注意力模式和层级激活），并探讨了这些变化是否与由 EAP（边缘归因补丁，Edge Attribution Patching）识别出的、驱动任务性能的任务相关组件（如注意力头和 Logit 级激活）有关。

We find that EAP-identified components are concentrated within specific layers, indicating a degree of functional localisation in how models internalise task-specific behavior. Notably, the distribution of these components across layers is largely uncorrelated with the layers undergoing the most substantial representational changes during fine-tuning.

我们发现，EAP 识别出的组件集中在特定的层中，这表明模型在内化任务特定行为时存在一定程度的功能局部化。值得注意的是，这些组件在各层间的分布与微调过程中表征变化最显著的层之间基本不相关。

Furthermore, we observe that overlap in EAP-identified components across tasks does not translate into cross-task performance transfer if the tasks are different in nature (e.g. classification vs. generative tasks). More specifically, fine-tuning on one task can lead to a degradation of performance on another when the two tasks exhibit a high degree of overlap in their EAP-identified components.

此外，我们观察到，如果任务性质不同（例如分类任务与生成任务），跨任务的 EAP 识别组件重叠并不会转化为跨任务的性能迁移。更具体地说，当两个任务在 EAP 识别组件上表现出高度重叠时，针对其中一个任务进行微调反而可能导致另一个任务的性能下降。