CLEAT: Learning Value-Guided Clarification for Interactive Agents

查看原文 / View Original 编辑 / Edit

Computer Science > Artificial Intelligence arXiv:2610.12655 (cs) [Submitted on 8 Oct 2026] Title: CLEAT: Learning Value-Guided Clarification for Interactive Agents Authors: Jiatan Huang, Zheyuan Zhang, Ziming Li, Tianyi Ma, Mingchen Li, Yanfang Ye, Chuxu Zhang.

计算机科学 > 人工智能 arXiv:2610.12655 (cs) [提交于 2026 年 10 月 8 日] 标题：CLEAT：面向交互式智能体的价值引导式澄清学习作者：Jiatan Huang, Zheyuan Zhang, Ziming Li, Tianyi Ma, Mingchen Li, Yanfang Ye, Chuxu Zhang。

Abstract: LLM agents can execute increasingly complex tasks, but effective assistance also requires aligning their actions with goals and constraints that users may leave unstated. We study this problem as interactive intent alignment through clarification. Yet clarification is useful only when the agent can turn the answer into better decisions. As training changes how the agent handles incomplete information and uses additional answers, the same clarification can become more or less useful.

摘要：大语言模型（LLM）智能体能够执行日益复杂的任务，但有效的辅助还需要将其行为与用户可能未明确说明的目标和约束对齐。我们将此问题视为通过澄清进行的交互式意图对齐。然而，只有当智能体能够将答案转化为更好的决策时，澄清才是有用的。随着训练改变了智能体处理不完整信息和使用额外答案的方式，同一澄清的有用性也会随之变化。

We introduce CLEAT (CLarification through valuE-guided Agent co-Training), a framework that co-trains a clarification-value controller and the task policy from shared interaction outcomes. We formulate residual query value to measure the task utility gained by asking rather than proceeding under the current policy, and estimate it through paired rollouts from the same interaction history.

我们引入了 CLEAT（通过价值引导的智能体协同训练进行澄清），这是一个从共享交互结果中协同训练澄清价值控制器和任务策略的框架。我们制定了剩余查询价值，以衡量通过提问而非在当前策略下继续执行所获得的任务效用，并通过来自相同交互历史的成对展开（rollouts）来估计该价值。

These paired returns train the controller to predict clarification value and the task policy to select useful questions and act on the answers. The controller guides subsequent rollout collection, while outcomes from the updated agent provide new value labels that allow clarification guidance to adapt as the agent learns.

这些成对的回报训练控制器去预测澄清价值，并训练任务策略去选择有用的问题并根据答案采取行动。控制器引导后续的展开收集，而来自更新后智能体的结果提供了新的价值标签，使得澄清引导能够随着智能体的学习而进行调整。

With Qwen3-4B, CLEAT achieves the highest mean score in all ten evaluation settings across four benchmarks. Gains over the strongest baseline reach 13.2 and 6.5 percentage points on UserGym Travel and IN3 underspecification judgment accuracy, respectively.

基于 Qwen3-4B 模型，CLEAT 在四个基准测试的所有十个评估设置中均取得了最高平均分。在 UserGym Travel 和 IN3 不充分说明判断准确率方面，相比最强基线分别提升了 13.2 和 6.5 个百分点。

Ablations show that co-training improves the task policy even when the controller is removed at inference, while adapting clarification guidance to the evolving policy yields further gains.

消融实验表明，即使在推理时移除控制器，协同训练也能改善任务策略，而使澄清引导适应不断演进的策略则能带来进一步的提升。