R2VC: Modular Fact-Checking with Retrieval, Verification, and Confidence Calibration

R2VC：结合检索、验证与置信度校准的模块化事实核查架构

Abstract: Large language models are increasingly used for automated fact checking, but end-to-end prompting often entangles evidence retrieval, reasoning, and uncertainty estimation, making failures difficult to diagnose and confidence difficult to trust.

摘要： 大型语言模型正越来越多地被用于自动化事实核查，但端到端的提示（prompting）方式往往将证据检索、推理和不确定性估计混为一谈，导致故障难以诊断，且置信度难以信任。

We present R2VC, a modular retrieve, reason, verify, calibrate architecture for evidence-grounded fact checking with citations and abstention. R2VC combines hybrid sparse+dense retrieval over Wikipedia, a supervised fine-tuned and DPO-aligned generator that produces diverse structured verdict candidates, an external NLI cross-encoder for evidence-based candidate selection, and a lightweight sequence-level calibrator for confidence estimation and selective abstention.

我们提出了 R2VC，这是一种模块化的“检索-推理-验证-校准”架构，用于实现基于证据的事实核查，并支持引用和弃权机制。R2VC 结合了针对维基百科的混合稀疏+稠密检索、经过监督微调和 DPO 对齐的生成器（用于产生多样化的结构化结论候选）、用于基于证据的候选选择的外部 NLI 交叉编码器，以及用于置信度估计和选择性弃权的轻量级序列级校准器。

On FEVER, an 8B backbone with R2VC achieves 13.74% higher accuracy than baseline. Ablation studies show that verifier-based candidate selection and confidence calibration are the largest contributors to performance. Removing candidate selection drops FEVER accuracy to 76.24%, while removing calibration nearly doubles the Brier score to 0.161.

在 FEVER 数据集上，使用 8B 参数主干模型的 R2VC 比基准模型准确率提高了 13.74%。消融研究表明，基于验证器的候选选择和置信度校准是对性能提升贡献最大的部分。移除候选选择会导致 FEVER 准确率下降至 76.24%，而移除校准则使 Brier 分数几乎翻倍，达到 0.161。

A manual analysis of 250 errors further shows that retrieval failures, especially wrong-entity evidence, remain the dominant bottleneck. Together, these results show that modular fact-checking pipelines can substantially improve both predictive accuracy and confidence reliability in open-domain verification.

对 250 个错误案例的人工分析进一步表明，检索失败（尤其是针对错误实体的证据检索）仍然是主要的瓶颈。总之，这些结果表明，模块化的事实核查流水线可以显著提高开放域验证中的预测准确性和置信度可靠性。