Recursive Semantic Divergence for LLM Agent Consistency

Computer Science > Computation and Language arXiv:2610.33012 (cs) [Submitted on 8 Oct 2026] Title: Recursive Semantic Divergence for LLM Agent Consistency Authors: Harshavardhan Abichandani, Penny Chong, Atin Ghosh, Daniel Dahlmeier.

计算机科学 > 计算与语言 arXiv:2610.33012 (cs) [提交于 2026 年 10 月 8 日] 标题：用于大语言模型智能体一致性的递归语义分歧 作者：Harshavardhan Abichandani, Penny Chong, Atin Ghosh, Daniel Dahlmeier。

Abstract: Large Language Model (LLM) agents with tool use exhibit inconsistent behavior across independent runs from identical states, calling different tools, producing contradictory outputs, or pursuing divergent strategies. This variability compounds over time and undermines deployment reliability.

摘要：具备工具使用能力的大语言模型（LLM）智能体在从相同状态出发的独立运行中表现出不一致的行为，例如调用不同的工具、产生相互矛盾的输出或采取不同的策略。这种变异性会随时间累积，并削弱部署的可靠性。

Existing evaluation metrics primarily measure task correctness via completion-based scores and do not capture consistency in agent behavior across executions. Separately, agent behavior is analyzed either at the token level using probability distributions or at the single-response level using semantic clustering, but neither captures how inconsistency propagates across future turns of a trajectory.

现有的评估指标主要通过基于完成度的分数来衡量任务正确性，而无法捕捉智能体在多次执行中行为的一致性。此外，对智能体行为的分析要么是在标记（token）层面使用概率分布，要么是在单次响应层面使用语义聚类，但这两种方法都无法捕捉不一致性如何在轨迹的后续轮次中传播。

We introduce Recursive Semantic Divergence (RSD), an unsupervised trajectory-level consistency metric adapted from bisimulation metrics in reinforcement learning. RSD measures the expected cumulative semantic disagreement between two independent rollouts of the same agent from the same state, using natural language inference to detect logical inconsistency.

我们引入了递归语义分歧（RSD），这是一种改编自强化学习中双模拟（bisimulation）指标的无监督轨迹级一致性指标。RSD 通过自然语言推理来检测逻辑不一致性，从而衡量同一智能体从相同状态出发的两次独立运行之间预期的累积语义分歧。

It is defined recursively over future states and approximated by a neural network trained on independent rollouts. The learned metric estimates future divergence from the current state, providing a consistency score at each turn without requiring the full trajectory to complete.

它在未来状态上进行递归定义，并通过在独立运行上训练的神经网络进行近似。该学习到的指标可以估计从当前状态出发的未来分歧，从而在每一轮提供一致性分数，而无需等待完整轨迹结束。

When used as a reward signal for fine-tuning, RSD reduces the variation in task progress from 132% to 91% on tau2-bench and from 90% to 44% on toolsandbox, while improving performance across benchmarks and model scales. Training with RSD as an additional reward signal matches supervised fine-tuning performance with substantially lower variance, without requiring ground-truth labels.

当作为微调的奖励信号使用时，RSD 将 tau2-bench 上的任务进度变异性从 132% 降低至 91%，在 toolsandbox 上从 90% 降低至 44%，同时提升了跨基准测试和模型规模的性能。以 RSD 作为额外奖励信号进行训练，在无需地面真值标签的情况下，能够达到与监督微调相当的性能，且方差显著降低。