How to Optimize Vector Search When RAM Gets Too Expensive: On-Disk vs. In-Memory ANN Indexes
How to Optimize Vector Search When RAM Gets Too Expensive: On-Disk vs. In-Memory ANN Indexes
当内存成本过高时，如何优化向量搜索：磁盘存储与内存存储 ANN 索引对比
Architecting cost-effective infrastructure by navigating the latency and storage trade-offs of HNSW, SPANN, and DiskANN. 通过权衡 HNSW、SPANN 和 DiskANN 在延迟与存储方面的表现，构建高性价比的基础设施。
Over the last few years, vector search has become a critical piece of AI infrastructure, powering use cases from RAG and semantic search to agentic memory and context layers. With the rise of agentic systems, companies are trying to provide as much context to the agents as possible, which requires vector db indexes to grow from an initial million or dozens of millions scale to the hundreds of millions or even billions. 过去几年中，向量搜索已成为人工智能基础设施的关键组成部分，为从 RAG（检索增强生成）和语义搜索到智能体记忆和上下文层等各种用例提供支持。随着智能体系统的兴起，企业正试图为智能体提供尽可能多的上下文，这要求向量数据库索引从最初的百万或千万级扩展到数亿甚至数十亿级。
At this scale, storing indexes and associated data in RAM will cost thousands of dollars per month, and HNSW can become a scalability bottleneck. In this article I would like to dive into the details of what actually makes semantic search fast and efficient: approximate nearest neighbor (ANN) algorithms, what different options exist, and their trade-offs. 在这种规模下，将索引及相关数据存储在内存（RAM）中每月将耗费数千美元，且 HNSW 可能会成为扩展性的瓶颈。在本文中，我将深入探讨实现高效语义搜索的核心：近似最近邻（ANN）算法，以及现有的不同方案及其权衡。
Deep dive into vector DB
深入了解向量数据库
Vector databases consist of three main components: 向量数据库由三个主要部分组成：
- Embeddings – the numeric representation of the corpus.
- 嵌入（Embeddings） – 语料库的数值表示。
- Search algorithm and index structure – the algorithm defines the search quality and speed.
- 搜索算法与索引结构 – 算法决定了搜索的质量和速度。
- Storage – how the data is stored (in memory, on disk, payload together with embeddings, etc.). Whether in RAM or on disk, it determines the costs and latency at scale.
- 存储 – 数据的存储方式（内存、磁盘、有效载荷与嵌入共同存储等）。无论是在内存还是磁盘中，这都决定了大规模场景下的成本和延迟。
Embeddings are already well defined and discussed in many articles, and this one will focus on search algorithms, and specifically ANN ones. There are generally two approaches for the search execution: 嵌入技术已在许多文章中得到了充分定义和讨论，本文将重点关注搜索算法，特别是 ANN 算法。搜索执行通常有两种方法：
- Exact search – which demonstrates the best retrieval metrics although it does not scale well in terms of latency.
- 精确搜索 – 具有最佳的检索指标，但在延迟方面扩展性较差。
- Approximate nearest neighbor (ANN) – which trades off the retrieval quality for the latency and scalability.
- 近似最近邻（ANN） – 以牺牲一定的检索质量为代价，换取延迟和扩展性。
The exact search is a simple approach which loops through all of the entries in the index and calculates the distance between your search query and existing data. There are no losses related to any approximation or generalization with the trade-off of the latency and scalability. It’s a great approach for either very small indexes or experimentation, but generally not very suitable for the production scale. 精确搜索是一种简单的方法，它遍历索引中的所有条目，并计算搜索查询与现有数据之间的距离。它没有因近似或泛化带来的损失，但代价是延迟和扩展性。这种方法非常适合极小的索引或实验场景，但通常不适用于生产规模。
Interesting fact: many modern vector databases allow you to bypass index building for small collections, falling back to kNN search because the overhead of building an index is not worth it for a few thousand vectors. 有趣的事实：许多现代向量数据库允许在处理小型集合时跳过索引构建，直接回退到 kNN 搜索，因为对于几千个向量来说，构建索引的开销并不划算。
The second option is approximate nearest neighbor algorithms, which is a group of algorithms with the main purpose of improving scalability by avoiding visiting all the entries in the index. The idea here is to provide some shortcuts to speed up the search and ingestion. The implementations vary, although many modern ANN algorithms (for example, HNSW and DiskANN) rely on a graph structure to provide low query latency. 第二种选择是近似最近邻算法，这是一组旨在通过避免访问索引中所有条目来提高扩展性的算法。其核心思想是提供一些“捷径”来加速搜索和数据摄入。虽然实现方式各异，但许多现代 ANN 算法（例如 HNSW 和 DiskANN）都依赖图结构来提供低查询延迟。
Approximate nearest neighbor algorithms
近似最近邻算法
It’s important to discuss that even though ANN algorithms are all following a similar concept to achieve the goal of providing a short path to the end result, there are different implementations with different algorithms having unique sets of trade-offs, which makes it crucial to select the one that fits your exact use case. 值得讨论的是，尽管所有 ANN 算法都遵循相似的概念以实现提供通往最终结果的“捷径”，但不同的实现方式和算法具有独特的权衡，因此选择最适合您具体用例的算法至关重要。
In this article I would like to focus on two different groups of the ANN algorithms: 在本文中，我想重点介绍两类不同的 ANN 算法：
- RAM-based – such algorithms are optimized for storing all or a big chunk of data in memory, which provides extremely low latency with the costs as a trade-off. The standard example here is HNSW (Hierarchical Navigable Small World).
- 基于内存（RAM-based） – 此类算法针对将全部或大部分数据存储在内存中进行了优化，提供了极低的延迟，但代价是成本较高。典型的例子是 HNSW（分层可导航小世界）。
- On-disk – these algorithms are minimizing RAM usage and relying heavily on disk to load the required data. An example here is DiskANN or SPANN.
- 基于磁盘（On-disk） – 此类算法最大限度地减少了内存使用，并严重依赖磁盘来加载所需数据。例子包括 DiskANN 或 SPANN。
It’s required to mention that it’s possible to store underlying data structures of both of these algorithm groups either on disk or in memory (at least partially), but they are optimized for the specific storage type and therefore will provide the best results utilizing what they were designed for. 需要说明的是，这两类算法的底层数据结构既可以存储在磁盘上，也可以存储在内存中（至少是部分存储），但它们是针对特定的存储类型进行优化的，因此使用其设计初衷的存储方式才能获得最佳效果。
In-memory ANN: HNSW (Hierarchical Navigable Small World)
内存 ANN：HNSW（分层可导航小世界）
The most popular ANN algorithm used by almost any modern vector database. The idea is to utilize a layered graph-based data structure to connect vectors with near neighbors, which provides extremely fast retrieval when storing the whole index in memory. It’s a great fit for small-medium use cases and will provide the best retrieval speed. 这是几乎所有现代向量数据库都在使用的最流行的 ANN 算法。其核心思想是利用分层图数据结构将向量与其近邻连接起来，当整个索引存储在内存中时，这能提供极快的检索速度。它非常适合中小型用例，并能提供最佳的检索速度。
However, once the index is big enough that it no longer fits in RAM or storing it in RAM becomes very expensive, the option is to either move data to disk, which may cause a drastic performance hit, or highly quantize it, which can cause a significant drop in retrieval quality. 然而，一旦索引规模大到无法放入内存，或者在内存中存储的成本变得非常昂贵，选择要么将数据移至磁盘（这可能导致性能急剧下降），要么进行高强度量化（这可能导致检索质量显著下降）。
The main reason for such a performance hit is that the HNSW structure is not optimized for the clustered disk access, and as a result, the search would produce a lot of non-sequential I/O operations. Considering multiple hops per search and relatively high read traffic, the disk I/O will become a bottleneck, which can significantly increase latency, from milliseconds to hundreds of milliseconds or worse under heavy I/O pressure. 性能下降的主要原因是 HNSW 结构并未针对集群磁盘访问进行优化，因此搜索会产生大量非顺序 I/O 操作。考虑到每次搜索涉及多次跳转以及相对较高的读取流量，磁盘 I/O 将成为瓶颈，这会显著增加延迟，在重负载下可能从毫秒级增加到数百毫秒甚至更糟。
The in-memory algorithms are highly optimized to store all vectors and connections in RAM, which makes them extremely fast, but with a trade-off of being memory hungry. Moreover, with on-disk options such algorithms rely on random disk access, which can become a potential bottleneck for large-scale indexes (100 million+). 内存算法经过高度优化，将所有向量和连接存储在内存中，这使其速度极快，但代价是内存消耗巨大。此外，当使用磁盘存储时，此类算法依赖随机磁盘访问，这对于大规模索引（1 亿以上）来说可能成为潜在的瓶颈。
Example vector databases: Qdrant, Milvus, pgvector, OpenSearch, Weaviate, Redis. 示例向量数据库：Qdrant, Milvus, pgvector, OpenSearch, Weaviate, Redis。
On-Disk ANN
磁盘 ANN
This group of algorithms is designed specifically to break the RAM consumption limitation of the in-memory ANN algorithms and reduce the storage costs while providing acceptable latency. It’s a great choice if search latency is not critical and the index size… 这一类算法专门为打破内存 ANN 算法的内存消耗限制而设计，旨在降低存储成本的同时提供可接受的延迟。如果搜索延迟不是关键因素且索引规模较大，这是一个极佳的选择……