The Bottleneck Is Retrieval, Not Reading: Diagnosing Evidence-Based Generation with RAG for Long Context Question Answering

Retrieval-augmented generation (RAG) is the dominant paradigm for grounding large language models in external corpora, but its similarity-driven, chunked pipeline has structural blind spots: it cannot anticipate which passages a question requires and discards document-level context that resolves cross-passage references.

检索增强生成（RAG）是将大型语言模型建立在外部语料库基础上的主流范式，但其基于相似度驱动的分块流水线存在结构性盲点：它无法预判问题需要哪些段落，并且会丢弃用于解决跨段落引用的文档级上下文。

We introduce evidence-based generation (EBG), requiring models to return an answer with line-level citations, and a joint Ans+Evid metric that scores correctness and grounding simultaneously.

我们引入了基于证据的生成（EBG），要求模型在返回答案时提供行级引用，并提出了一种联合指标 Ans+Evid，用于同时评估答案的正确性和依据的充分性。

We reformulate two corpora into evidence-grounded multi-document QA benchmarks- MultiHop-RAG and QASPER-MP, a multi-paper adaptation of QASPER with realistic short paper cues, and pair them with a controlled synthetic two-hop haystack and an evidence-augmented NoLiMa spanning 10K-1M characters.

我们将两个语料库重新构建为基于证据的多文档问答基准——MultiHop-RAG 和 QASPER-MP（QASPER 的多论文改编版，包含真实的短论文提示），并将它们与受控的合成双跳“大海捞针”测试以及跨度为 1 万到 100 万字符的证据增强型 NoLiMa 相结合。

Across ten proprietary frontier and three open-weight LLMs, EBG isolates two RAG limitations that long-context models avoid: similarity retrieval surfaces any relevant chunk far more reliably than all required chunks for multi-fact queries, and chunking destroys document meta-information needed to resolve identity-bearing queries.

通过对十个专有前沿大模型和三个开源权重模型进行测试，EBG 识别出了长上下文模型所能规避的两个 RAG 局限性：在多事实查询中，相似度检索定位单个相关分块的可靠性远高于定位所有必要分块的可靠性；此外，分块处理会破坏解决身份识别类查询所需的文档元信息。

EBG also exposes grounding failures invisible to answer-only evaluation, including incomplete citations in two-needle settings and a widening gap between Ans and Ans+Evid when context contradicts parametric knowledge.

EBG 还揭示了仅评估答案时无法察觉的依据失效问题，包括在“双针”设置中引用不完整，以及当上下文与参数化知识相矛盾时，Ans 指标与 Ans+Evid 指标之间差距扩大的现象。