From Detection to Refusal: Safer LLMs via Circuit-Guided Weight Scaling

从检测到拒绝：通过电路引导权重缩放实现更安全的大语言模型

Despite extensive alignment efforts, Large Language Models (LLMs) remain vulnerable to generating unsafe content under adversarial prompting, yet the internal mechanisms by which safety behaviors are implemented remain poorly understood. 尽管已经进行了广泛的对齐工作，大语言模型（LLMs）在面对对抗性提示时，仍然容易生成不安全的内容，且其实现安全行为的内部机制尚不明确。

We study LLM safety from a mechanistic interpretability perspective and characterize a multi-stage safety circuit that organizes refusal behavior, consisting of (i) $\textbf{Harmful Detection Heads}$ that respond to harmful inputs, (ii) $\textbf{Safety Neurons}$ that mediate and stabilize safety signals in the residual stream, and (iii) $\textbf{Refusal Heads}$ that translate these signals into safe response generation. 我们从机械可解释性的角度研究了 LLM 的安全性，并刻画了一个组织拒绝行为的多阶段“安全电路”。该电路包括：(i) 响应有害输入的“有害检测头”；(ii) 在残差流中调节并稳定安全信号的“安全神经元”；以及 (iii) 将这些信号转化为安全响应生成的“拒绝头”。

Using targeted attention-head and neuron-level interventions, we provide causal evidence consistent with this circuit organization, showing that suppressing upstream Harmful Detection Heads disrupts downstream refusal behavior and that safety neurons mediate this interaction. 通过针对性的注意力头和神经元层面的干预，我们提供了与该电路组织结构一致的因果证据，表明抑制上游的“有害检测头”会破坏下游的拒绝行为，且“安全神经元”在这一交互过程中起到了调节作用。

We validate that this decomposition recurs across multiple LLM architectures and adversarial attack settings, and use simple, architecture-preserving weight scaling as a mechanistic probe to test its functional relevance. 我们验证了这种分解结构在多种 LLM 架构和对抗性攻击场景中均普遍存在，并使用简单的、保持架构不变的权重缩放作为机械探针，以测试其功能相关性。

Across six LLMs, circuit-guided scaling improves safety rates under attacks by 26.5%, while incurring only a 1.7% accuracy drop across four standard benchmarks. 在六个大语言模型上的实验表明，电路引导的缩放技术在遭受攻击时将安全性提高了 26.5%，而在四个标准基准测试中，准确率仅下降了 1.7%。

Overall, our results support a circuit-level interpretation of LLM safety and suggest that mechanistic abstractions can reveal stable and transferable patterns underlying aligned behavior. 总的来说，我们的研究结果支持从电路层面解读 LLM 的安全性，并表明机械抽象能够揭示对齐行为背后稳定且可迁移的模式。