NeuronFuzz: Safety Neuron Guided Fuzzing for LLM Safety Evaluation

NeuronFuzz：用于大模型安全评估的神经元引导模糊测试

Abstract: Safety evaluation is critical for assessing whether aligned Large Language Models (LLMs) remain robust against jailbreak attacks. Existing automated testing methods, however, largely rely on response-level feedback: each candidate prompt typically requires generating a target-model response to evaluate its attack effectiveness. This process is expensive and, more importantly, provides only sparse guidance on strongly aligned models, where most candidates are rejected with the same failure outcome.

摘要： 安全评估对于判断对齐后的大语言模型（LLM）是否能抵御越狱攻击至关重要。然而，现有的自动化测试方法主要依赖于响应层面的反馈：每个候选提示词通常都需要生成目标模型的响应，以评估其攻击有效性。这一过程不仅成本高昂，更重要的是，对于强对齐模型，它只能提供稀疏的指导，因为大多数候选提示词都会因相同的失败结果而被拒绝。

This paper presents NeuronFuzz, a white-box fuzzing framework that exploits internal safety neurons as continuous execution feedback for LLM safety evaluation. A SafetyOracle converts safety-neuron activations into a continuous safety alarm score that serves as feedback for fuzzing and can be obtained during prefill, eliminating response generation from the fuzzing loop.

本文提出了 NeuronFuzz，这是一个白盒模糊测试框架，利用内部安全神经元作为大模型安全评估的连续执行反馈。SafetyOracle（安全预言机）将安全神经元的激活值转换为连续的安全警报分数，作为模糊测试的反馈。该分数可以在预填充（prefill）阶段获取，从而消除了模糊测试循环中生成响应的需求。

To construct the SafetyOracle, NeuronFuzz uses template-invariant harmful and benign inputs and stability-aware selection to identify a compact set of safety neurons whose activations capture harmful-intent recognition. Moreover, since the safety alarm score is differentiable, NeuronFuzz uses its gradients to identify safety-sensitive template positions and a masked language model to generate fluent, context-compatible mutations while preserving original harmful payload and avoiding additional optimization variables.

为了构建 SafetyOracle，NeuronFuzz 使用模板不变的有害和良性输入，并通过稳定性感知选择来识别一组紧凑的安全神经元，这些神经元的激活能够捕捉到对有害意图的识别。此外，由于安全警报分数是可微的，NeuronFuzz 利用其梯度来识别对安全敏感的模板位置，并使用掩码语言模型（Masked Language Model）生成流畅且上下文兼容的变体，同时保留原始的有害载荷，并避免引入额外的优化变量。

We evaluate NeuronFuzz across 21 text and multimodal models. Across five white-box source models, it achieves a 76-100% jailbreak discovery rate, outperforming baselines by up to 48 percentage points. Its optimized templates further transfer zero-shot to open-weight and six proprietary target models, achieving average ASR and top-5 ensemble ASR (EASR) of 69.6%/92.6% and 44.1%/60.0%, respectively.

我们在 21 个文本和多模态模型上评估了 NeuronFuzz。在五个白盒源模型上，它实现了 76-100% 的越狱发现率，比基线方法高出 48 个百分点。其优化后的模板进一步零样本迁移到开源权重模型和六个专有目标模型上，平均攻击成功率（ASR）和前 5 名集成攻击成功率（EASR）分别达到了 69.6%/92.6% 和 44.1%/60.0%。