Sieve and Sage: Efficient Distraction Filtering for Reliable RALM Abstention

Sieve and Sage：用于可靠 RALM 弃权机制的高效干扰过滤方法

Abstract: Just as Socrates recognized the limits of his own knowledge, Retrieval-Augmented Language Models (RALMs) should learn to abstain when the retrieved evidence cannot support a reliable response. Existing approaches largely rely on monolithic LLMs to handle heterogeneous retrieval failures in a single step, resulting in limited abstention performance and high computational costs.

摘要： 正如苏格拉底认识到自身知识的局限性一样，检索增强语言模型（RALM）也应学会当检索到的证据不足以支持可靠回答时选择“弃权”。现有的方法大多依赖单一的大型语言模型（LLM）来一步处理各种异构的检索失败情况，这导致弃权效果有限且计算成本高昂。

We instead decompose retrieval failures into two distinct states: (i) the unanswerable state, where the required evidence is absent, and (ii) the distracted state, where relevant evidence is mixed with conflicting, negated, or adversarial information. Based on this decomposition, we introduce a lightweight module (Sieve) that screens retrieved document sets for distracting evidence before invoking a costly LLM (Sage) for grounded generation and abstention.

我们转而将检索失败分解为两种不同的状态：(i) “不可回答”状态，即缺失所需证据；(ii) “受干扰”状态，即相关证据中混杂了冲突、否定或对抗性信息。基于这种分解，我们引入了一个轻量级模块（Sieve），在调用昂贵的 LLM（Sage）进行基于证据的生成和弃权判断之前，先对检索到的文档集进行干扰证据筛选。

Evaluated across both general and high-stakes expert domains, our Sieve and Sage framework preemptively detects distracting noise, improving system accuracy by up to 69.4 percentage points and Macro-F1 by 55.2 percentage points compared to one-stage baselines. Furthermore, it achieves up to a 1.99x speedup, establishing a highly efficient and reliable abstention pipeline for RALM with abstention.

在通用领域和高风险专家领域的评估中，我们的 Sieve and Sage 框架能够预先检测干扰噪声，与单阶段基线模型相比，系统准确率提升了高达 69.4 个百分点，Macro-F1 指标提升了 55.2 个百分点。此外，该框架实现了最高 1.99 倍的加速，为具备弃权机制的 RALM 建立了一条高效且可靠的弃权流水线。