Component and Dimension Sparsity in Transformer Refusal Mechanisms

查看原文 / View Original 编辑 / Edit

Computer Science > Computation and Language arXiv:2610.06903 (cs) [Submitted on 30 Sep 2026] Title: Component and Dimension Sparsity in Transformer Refusal Mechanisms Authors: Vincent Siu, Glenn Grant-Richards, Vlad Pavlovich, Yizhou Sun, Dawn Song, Chenguang Wang.

计算机科学 > 计算与语言 arXiv:2610.06903 (cs) [提交于 2026 年 9 月 30 日] 标题：Transformer 拒绝机制中的组件与维度稀疏性作者：Vincent Siu, Glenn Grant-Richards, Vlad Pavlovich, Yizhou Sun, Dawn Song, Chenguang Wang。

Abstract: Activation steering manipulates large language model behavior by intervening on internal activations, but the mechanistic basis of these interventions remains poorly understood. We decompose refusal steering into component-level interventions across four open-weight models, identifying the sparse subsets of attention and MLP components whose steering suffices to reproduce the full behavioral effect.

摘要：激活引导通过干预内部激活来操纵大型语言模型的行为，但这些干预的机制基础尚不明确。我们将拒绝引导分解为跨四个开放权重模型的组件级干预，识别出注意力机制和 MLP 组件中的稀疏子集，仅对这些子集进行引导就足以重现完整的行为效果。

We find that refusal directions concentrate in sparse component mechanisms comprising 28—48% of upstream components, retaining 88—101% of steering effectiveness. Within these mechanisms, effective steering further concentrates in approximately 50% of residual stream dimensions, retaining 85—98% of the component-mechanism baseline, consistent with a privileged basis structure.

我们发现，拒绝方向集中在稀疏的组件机制中，这些机制包含 28%—48% 的上游组件，并保留了 88%—101% 的引导有效性。在这些机制内部，有效的引导进一步集中在大约 50% 的残差流维度中，保留了组件机制基准的 85%—98%，这与特权基结构相一致。

Sparsity thus operates at two levels: which components are steered, and which dimensions within those components carry the signal. Together these findings show that refusal is not diffusely encoded across a transformer but assembled by a structured, identifiable mechanism, providing a foundation for mechanistic understanding of how refusal behaviors are represented and steered.

因此，稀疏性在两个层面上发挥作用：哪些组件被引导，以及这些组件内的哪些维度携带信号。这些发现共同表明，拒绝行为并非弥散地编码在 Transformer 中，而是由一种结构化、可识别的机制组装而成，这为从机制上理解拒绝行为如何被表征和引导提供了基础。