Three Kinds of RAG Corpus, and What It Costs to Build for the Wrong One
Three Kinds of RAG Corpus, and What It Costs to Build for the Wrong One
三种 RAG 语料库，以及为错误的架构买单的代价
Enterprise Document Intelligence [Vol.1 #14A] – Three questions tell you which shape a document collection has, and each shape wants a different architecture. 企业文档智能 [第1卷 #14A] —— 三个问题帮你识别文档集合的形态，每种形态都需要不同的架构。
Ask an AI engineer how to take a RAG system past one document and the answer has been the same for three years. Put every page of every file into a vector store, embed the question, and let similarity search pick the passages. On a demo folder that works. On an enterprise shelf it stops working, and not because a parameter is set wrong. 问问 AI 工程师如何将 RAG 系统扩展到单文档之外，三年来答案始终如一：将每个文件的每一页放入向量数据库，对问题进行向量化，然后通过相似度搜索提取段落。这在演示文件夹中行得通，但在企业级存储库中却会失效，而且原因并非参数设置错误。
Similarity search returns passages, so it never picks a document, and the answer comes back assembled out of three files that were never meant to be read together. The step that gets skipped is the one before that. A collection of documents is not one kind of thing. A shared drive of unrelated reports, five thousand copies of the same contract, and a stack of claim folders are three different problems, and an architecture that suits any one of them suits the other two badly. 相似度搜索返回的是段落，因此它从不选择“文档”，导致生成的答案是由三个本不该放在一起阅读的文件拼凑而成的。被跳过的步骤正是最关键的前置步骤：文档集合并非只有一种形态。共享驱动器中互不相关的报告、五千份相同的合同副本，以及一堆理赔文件夹，这是三个完全不同的问题，适用于其中任何一种的架构，对另外两种来说都是糟糕的。
1. A quarter of a million documents and one question
1. 二十五万份文档与一个问题
Everything in the series so far could assume the system knew which document to look at, because there was only one. This section is what happens when that assumption goes. 本系列到目前为止的所有内容都假设系统知道该查看哪份文档，因为当时只有一份。本节将探讨当这一假设失效时会发生什么。
1.1. What the expert does, and what the pipeline does 1.1. 专家如何处理，流水线如何处理
A mid-size insurance broker keeps about a quarter of a million documents. Fifteen years of policies, certificates, contracts, claims and correspondence, some scanned and some native, most of them never opened twice. A user asks: “what are the seller’s obligations in our distribution agreements with retailer X?” 一家中型保险经纪公司保存着大约二十五万份文档。其中包括十五年的保单、证书、合同、理赔记录和往来信函，有些是扫描件，有些是原生电子文档，大多数文档从未被二次查阅。用户问道：“我们与零售商 X 的分销协议中，卖方的义务是什么？”
A senior claims handler answers in under a minute, and she does it without reading anything she does not need. Obligations sit in distribution agreements, so invoices and certificates are out. For retailer X there is one master from 2019 and two amendments. Obligations are usually section 4 of the master, and an amendment wins where it says so. She pulls three folders, reads three sections, comes back. 一位资深理赔员能在不到一分钟内给出答案，且无需阅读任何无关内容。义务条款位于分销协议中，因此发票和证书被排除在外。对于零售商 X，有一份 2019 年的主协议和两份修订案。义务条款通常在主协议的第 4 节，而修订案在冲突时具有优先权。她调取了三个文件夹，阅读了三个章节，然后给出了答案。
Point the textbook pipeline at the same question and it does something else entirely. Every passage of every document is already embedded in one store. The question gets embedded, the store returns the thirty nearest passages by cosine similarity, and those thirty go into one prompt. 如果用教科书式的流水线处理同样的问题，结果则完全不同。每一份文档的每一个段落都已嵌入同一个存储库中。问题被向量化后，存储库根据余弦相似度返回最接近的三十个段落，并将这三十个段落全部塞进一个提示词（Prompt）中。
Sometimes the answer is right. Often the thirty hold two paragraphs from a contract with a different retailer, a certificate that belongs to no contract at all, and a 2017 invoice whose only qualification is the word “obligation.” Nothing in the input tells the model that those three should never be read together, so it writes a paragraph that reads well and describes no document that exists. 有时答案是正确的。但通常这三十个段落里包含着来自另一家零售商合同的段落、一份不属于任何合同的证书，以及一张 2017 年的发票（仅仅因为里面包含“义务”这个词）。输入内容中没有任何信息告诉模型这三者不应放在一起阅读，因此它写出了一段读起来通顺、但描述的内容在现实中并不存在的文字。
1.2. Five ways a flat pile fails at scale 1.2. 平铺式存储在规模化时的五种失效方式
At small scale the pile works often enough to demo. On a few hundred documents the right one usually lands in the top-k by luck, and the demo gets read as a validation of the approach rather than of the corpus size. Five things break as the shelf grows, and they break independently of each other. 在小规模下，这种平铺式存储足以应付演示。在几百份文档中，正确的文档通常会凭运气进入 Top-k 结果，演示会被误认为是该方法的验证，而非语料库规模的验证。随着存储库的增长，有五件事会失效，且它们彼此独立。
Read the last column first. A wider top-k pulls in noise at the same rate as signal. A different embedding model moves the failures around instead of removing them. Chunk size has nothing to say about a document boundary that was never recorded in the first place. 先看最后一列。扩大 Top-k 会以同样的速度引入噪声和信号。更换嵌入模型只会转移失效点，而无法消除它们。分块大小（Chunk size）对于从未被记录的文档边界问题毫无帮助。
1.3. Why a re-ranker does not rescue it 1.3. 为什么重排序器（Re-ranker）无法解决问题
The usual next move is a cross-encoder: retrieve a wide top-100, re-score every candidate against the question one at a time, keep the best three. A re-ranker is a real tool and it is worth having, but not here and not yet. It re-orders the list the embedding step returned. If the right document never entered that list, and vocabulary drift is precisely the condition under which it does not, there is nothing. 通常的下一步是使用交叉编码器（Cross-encoder）：检索出前 100 个结果，逐一针对问题对每个候选段落重新评分，保留最好的三个。重排序器是一个实用的工具，值得拥有，但不是现在，也不是在这里。它只是对嵌入步骤返回的列表进行重新排序。如果正确的文档从未进入该列表（而词汇漂移正是导致这种情况的原因），那么重排序器也无能为力。