CoFiBreak: A Coarse-to-Fine Diversity Reward for Effective Multi-Turn Jailbreak

Computer Science > Computation and Language arXiv:2610.40926 (cs) [Submitted on 29 Oct 2026] Title: CoFiBreak: A Coarse-to-Fine Diversity Reward for Effective Multi-Turn Jailbreak Authors: Jialin Song, Xiaodong Liu, Weiwei Yang, Mingqian Feng, Chengquan Guo, Qiyao Ma, Carolyn Buractaon, Anson Ho, Wuyang Chen, Christopher White.

计算机科学 > 计算与语言 arXiv:2610.40926 (cs) [提交于 2026 年 10 月 29 日] 标题：CoFiBreak：一种用于有效多轮越狱的粗到细多样性奖励。作者：Jialin Song, Xiaodong Liu, Weiwei Yang, Mingqian Feng, Chengquan Guo, Qiyao Ma, Carolyn Buractaon, Anson Ho, Wuyang Chen, Christopher White。

Abstract: Automated red teaming trains an attacker large language model (LLM) to elicit harmful responses from a victim LLM, but reinforcement learning can collapse attackers onto a narrow set of prompts when attack success is the only reward. Such a collapse of attack diversity can bias safety evaluation toward repeatedly testing near-duplicate attack patterns rather than the underlying harmful intents. Existing methods often trade off attack success or inference cost for greater content or strategy diversity.

摘要：自动化红队测试通过训练攻击者大语言模型（LLM）来诱导受害者 LLM 做出有害响应，但当攻击成功是唯一奖励时，强化学习可能会导致攻击者收敛于一组狭窄的提示词。这种攻击多样性的崩溃可能会使安全评估产生偏差，倾向于重复测试近乎重复的攻击模式，而非底层的有害意图。现有方法通常以牺牲攻击成功率或推理成本为代价来换取更高的内容或策略多样性。

We propose CoFiBreak, a coarse-to-fine diversity reward for Group Relative Policy Optimization (GRPO) that improves attack diversity while preserving strong attack effectiveness and low inference cost. The coarse reward broadens which strategies the attacker uses by encouraging strategies underrepresented within each GRPO group, while the fine reward broadens how those strategies are realized by discouraging reuse of the same personas, framings, and output formats after harmful-intent information is removed from prompt embeddings.

我们提出了 CoFiBreak，这是一种用于组相对策略优化（GRPO）的粗到细多样性奖励，它在保持强大攻击效果和低推理成本的同时提高了攻击多样性。粗粒度奖励通过鼓励每个 GRPO 组中代表性不足的策略来拓宽攻击者使用的策略范围，而细粒度奖励则通过在从提示词嵌入中移除有害意图信息后，抑制对相同角色、框架和输出格式的重复使用，从而拓宽了这些策略的实现方式。

Diversity rewards apply only to attacks that already exceed an effectiveness threshold, so diversity cannot be gained through ineffective attacks. Since both rewards are computed only during training, CoFiBreak also retains inference efficiency. Trained with Llama3.1-8B-Instruct as the victim, CoFiBreak discovers 1.6-2.0x as many distinct successful fine-grained strategies as the strongest baseline across frontier victims on HarmBench.

多样性奖励仅适用于已经超过有效性阈值的攻击，因此无法通过无效攻击获得多样性。由于两种奖励仅在训练期间计算，CoFiBreak 也保留了推理效率。以 Llama3.1-8B-Instruct 作为受害者进行训练，CoFiBreak 在 HarmBench 的前沿受害者模型上发现的独特成功细粒度策略数量是表现最强基线的 1.6 到 2.0 倍。

It reaches 98.8% ASR@10 (attack success rate with 10 generations per intent) on AdvBench with the highest diversity score among all compared methods, which is 2.1x that of the strongest-attacking baseline. The resulting attacks transfer to unseen frontier LLMs, reaching 71.0% ASR@10 on AdvBench with the unseen victim Claude-Opus-4.6. The code will be released.

它在 AdvBench 上达到了 98.8% 的 ASR@10（每个意图生成 10 次的攻击成功率），在所有比较方法中具有最高的多样性得分，是表现最强攻击基线的 2.1 倍。由此产生的攻击可以迁移到未见过的各种前沿 LLM，在以未见过的 Claude-Opus-4.6 作为受害者时，在 AdvBench 上达到了 71.0% 的 ASR@10。代码将会发布。