Conformal Factuality Control for Multi-Hop Retrieval-Augmented Generation

多跳检索增强生成中的共形事实性控制

Abstract: Retrieval-augmented generation (RAG) can ground large language models in external evidence, but retrieved context does not guarantee that generated claims are factually supported. This problem is especially relevant in multi-hop RAG, where retrieval and reasoning proceed through multiple dependent stages.

摘要： 检索增强生成（RAG）可以将大型语言模型建立在外部证据的基础上，但检索到的上下文并不能保证生成的声明在事实层面得到支持。这个问题在多跳 RAG 中尤为突出，因为在多跳 RAG 中，检索和推理过程需要经过多个相互依赖的阶段。

We study whether claim-level conformal factuality control, previously developed for RAG, remains effective in this setting. We apply split-conformal claim filtering to multi-hop RAG and evaluate it on HotpotQA, Natural Questions, and TriviaQA using Llama 3.1 8B and GPT-4o-mini, together with a single-hop reference experiment.

我们研究了此前为 RAG 开发的声明级共形事实性控制在这种环境下是否依然有效。我们将分割共形声明过滤（split-conformal claim filtering）应用于多跳 RAG，并使用 Llama 3.1 8B 和 GPT-4o-mini 在 HotpotQA、Natural Questions 和 TriviaQA 数据集上进行了评估，同时还进行了单跳参考实验。

Across all six multi-hop model-dataset configurations, increasingly stringent conformal targets consistently increase the fraction of responses whose retained claims are fully supported. At the 95% target, this rate ranges from 95.80% to 97.20%, compared with 55.60%-76.03% without filtering.

在所有六种多跳模型-数据集配置中，日益严格的共形目标始终提高了响应中保留声明被完全支持的比例。在 95% 的目标下，这一比例从 55.60%-76.03%（未过滤时）提升至 95.80%-97.20%。

However, the improvement is strongly selective: only 4.41%-31.09% of generated claims are retained and 9.70%-51.40% of responses remain non-empty at the 95% target. These results show that conformal factuality extends to multi-hop RAG, while demonstrating that nominal reliability must be interpreted jointly with claim retention and abstention.

然而，这种改进具有很强的选择性：在 95% 的目标下，仅有 4.41%-31.09% 的生成声明被保留，且仅有 9.70%-51.40% 的响应保持非空。这些结果表明，共形事实性控制可以扩展到多跳 RAG，同时也证明了名义上的可靠性必须结合声明保留率和弃权率来共同解读。