RAG - Meta Filtering and Reranking

Generally, when a user asks a query, the system searches for the relevant chunks stored in the vector database using cosine similarity. The better we can filter the data, the smaller the search space becomes, resulting in faster and more efficient retrieval. Suppose we have a book with 10 chapters. If we want to search for a particular topic, all the points in the vector database are compared with the user query, and only the closest points are retrieved. This process is called KNN (K-Nearest Neighbors).

通常情况下，当用户提出查询时，系统会使用余弦相似度在向量数据库中搜索相关的分块（chunks）。我们对数据过滤得越好，搜索空间就越小，从而实现更快、更高效的检索。假设我们有一本包含 10 章的书，如果我们想搜索某个特定主题，向量数据库中的所有点都会与用户查询进行比较，并仅检索最接近的点。这个过程被称为 KNN（K-Nearest Neighbors，K-近邻算法）。

Another algorithm is ANN (Approximate Nearest Neighbors). Instead of checking all the points in the vector database, ANN searches only within a smaller region based on the proximity of the data. As the name suggests, it does not always return the exact result, but it provides the most preferred or approximate results much faster. Is there any other method we can use to make the search more effective?

另一种算法是 ANN（Approximate Nearest Neighbors，近似最近邻算法）。ANN 不会检查向量数据库中的所有点，而是根据数据的邻近性仅在较小的区域内进行搜索。顾名思义，它并不总是返回精确的结果，但能以快得多的速度提供最理想或近似的结果。我们还有其他方法可以使搜索更有效吗？

Metadata Filtering: Metadata means data about the data. Metadata is stored along with each chunk. It can contain information related to the chunk, such as the chapter name, topic description, author, or any other relevant details. When the user query contains information related to the metadata (for example, a chapter name or topic), the system can directly filter the relevant chunks before performing vector similarity search. This technique is called metadata filtering. Metadata filtering is supported by: Pinecone, ChromaDB, Qdrant. FAISS does not provide built-in support for metadata filtering.

元数据过滤：元数据是指关于数据的数据。元数据与每个分块一起存储，可以包含与该分块相关的信息，例如章节名称、主题描述、作者或任何其他相关细节。当用户查询包含与元数据相关的信息（例如章节名称或主题）时，系统可以在执行向量相似度搜索之前直接过滤相关分块。这种技术被称为元数据过滤。支持元数据过滤的工具有：Pinecone、ChromaDB、Qdrant。FAISS 不提供对元数据过滤的内置支持。

Reranking: Documents are first split into chunks, and each chunk is converted into vectors and stored in the vector database. When a user query arrives, it is converted into a vector and searched against the vector database to retrieve the closest chunks. However, we do not know whether the retrieved documents are actually the most relevant to the query. It is not always true that the closest vectors represent the most relevant documents.

重排序（Reranking）：文档首先被拆分为分块，每个分块被转换为向量并存储在向量数据库中。当用户查询到来时，它会被转换为向量，并在向量数据库中进行搜索以检索最接近的分块。然而，我们无法确定检索到的文档是否确实与查询最相关。最接近的向量并不总是代表最相关的文档。

How Reranking Works: The documents retrieved from the vector database are passed to a cross-encoder along with the user query. The cross-encoder assigns a relevance score that indicates how closely each document matches the query. The documents are then displayed in ascending or descending order based on these scores. The results produced by the cross-encoder are called reranked results. The retrieved documents remain the same as those returned by the vector database, but their order changes. Documents with higher relevance scores appear before those with lower scores.

重排序的工作原理：从向量数据库检索到的文档会与用户查询一起传递给交叉编码器（cross-encoder）。交叉编码器会分配一个相关性分数，表明每个文档与查询的匹配程度。然后，文档会根据这些分数按升序或降序排列。由交叉编码器产生的结果称为重排序结果。检索到的文档与向量数据库返回的文档相同，但它们的顺序发生了变化。相关性分数较高的文档会出现在分数较低的文档之前。

A cross-encoder is a neural ranking model. Instead of encoding the query and documents separately, it takes both the query and the document together as input to a transformer model and generates a relevance score for each document. There are transformer models specifically designed for reranking tasks. The encoder understands the meaning of both the query and the document and reranks the documents accordingly.

交叉编码器是一种神经排序模型。它不是分别对查询和文档进行编码，而是将查询和文档一起作为输入提供给 Transformer 模型，并为每个文档生成相关性分数。有一些专门为重排序任务设计的 Transformer 模型。编码器能够理解查询和文档的含义，并据此对文档进行重排序。

Why Use Reranking? Reranking is an important step in the RAG pipeline. It is especially useful when working with documents that contain images or other multimodal content. Example: Suppose the user asks: “Show me the front view of the truck.” The vector database may retrieve multiple images related to trucks because they are semantically similar. The reranker analyzes both the query and the retrieved images (or their associated text descriptions) and assigns relevance scores. As a result, the image showing the front view of the truck receives a higher score than the other truck images, making it appear first in the final results.

为什么要使用重排序？重排序是 RAG 流水线中的重要步骤。在处理包含图像或其他多模态内容的文档时，它特别有用。示例：假设用户问：“给我看卡车的正面视图。”向量数据库可能会检索到多张与卡车相关的图像，因为它们在语义上是相似的。重排序器会同时分析查询和检索到的图像（或其关联的文本描述）并分配相关性分数。结果，显示卡车正面视图的图像会获得比其他卡车图像更高的分数，从而使其在最终结果中排在首位。