DuplexSpeechBench-IFEval: Evaluating Implicit Instruction Following in Full-Duplex Voice Agents

DuplexSpeechBench-IFEval：评估全双工语音智能体中的隐式指令遵循能力

Abstract: Full-duplex voice agents must continuously decide when to listen, backchannel, interrupt, handle speech overlaps, take the floor, and yield. Existing benchmarks largely test these behaviors through explicit turn-management instructions, while deployed agents are often configured through roles or personas from which the appropriate conversational behavior must be inferred.

摘要： 全双工语音智能体必须持续做出决策，包括何时倾听、何时进行反馈（backchannel）、何时打断、如何处理语音重叠、何时接管对话以及何时让出话语权。现有的基准测试大多通过明确的轮次管理指令来测试这些行为，而实际部署的智能体通常通过角色或人设进行配置，智能体必须从中推断出适当的对话行为。

We introduce DuplexSpeechBench-IFEval (DSB-IFEval) for evaluating implicit instruction-following in real-time spoken interaction. (DSB-IFEval) comprises 1,038 test cases spanning eight diverse assistant roles and evaluates five conditioning protocols for instruction-following: default behavior, explicit behavioral instructions, persona-implied behavior, combined persona—rule conditioning, and instruction conflict.

我们引入了 DuplexSpeechBench-IFEval (DSB-IFEval)，用于评估实时语音交互中的隐式指令遵循能力。DSB-IFEval 包含 1,038 个测试用例，涵盖八种不同的助手角色，并评估了五种指令遵循的条件协议：默认行为、明确的行为指令、人设隐含行为、人设与规则结合的条件设定，以及指令冲突。

We measure real-time floor management using a deterministic Instruction Adherence Score (IAS) and persona-consistent content using LLM-judged Persona Adherence Score (PAS). Across six real-time speech systems, we find architecture-dependent trade-offs. Full duplex models like F-Actor and PersonaPlex are more sensitive to whether conversational behavior is stated explicitly or must be inferred from a persona, with adherence dropping by 9.7% and 4.5%, respectively, under persona-only conditioning.

我们使用确定性的“指令遵循得分”（IAS）来衡量实时话语权管理，并使用大模型评估的“人设遵循得分”（PAS）来衡量内容与人设的一致性。在对六个实时语音系统的测试中，我们发现了架构相关的权衡。像 F-Actor 和 PersonaPlex 这样的全双工模型对对话行为是明确说明还是需要从人设中推断更为敏感；在仅有人设条件的情况下，它们的遵循度分别下降了 9.7% 和 4.5%。

In contrast, GPT-Realtime, MiniCPM-o, and Fun-Audio-Chat strongly adhere to persona-consistent content, but their floor behavior does not adapt across explicit and persona-only instructions and remains constrained on several proactive actions. We further find that even if systems reliably follow conflicting directives to their prescribed persona, they still struggle to override them under safety conflict.

相比之下，GPT-Realtime、MiniCPM-o 和 Fun-Audio-Chat 在内容上能很好地保持人设一致性，但它们的对话轮次管理行为在明确指令和仅有人设指令之间无法自适应，且在某些主动行为上仍受到限制。我们进一步发现，即使系统能够可靠地遵循与其预设人设相冲突的指令，它们在面对安全冲突时仍难以覆盖这些指令。

These results show that inferring the behavior implied by a role, executing it at the appropriate conversational moment, and resolving competing instructions remain distinct challenges for full-duplex voice agents.

这些结果表明，推断角色隐含的行为、在适当的对话时机执行该行为，以及解决相互竞争的指令，对于全双工语音智能体而言仍然是独特的挑战。