Saving 100 terabytes of memory by optimizing 1.1.1.1’s DNS cache

通过优化 1.1.1.1 的 DNS 缓存节省 100 TB 内存

Big Pineapple, the platform behind 1.1.1.1, Gateway DNS, DNS Firewall, AS112, and several other Cloudflare DNS services, stores over 250 billion DNS cache entries at any given time. At that scale, wasting a single byte per entry costs more than 250 gigabytes of memory across our fleet. Big Pineapple 是支撑 1.1.1.1、Gateway DNS、DNS Firewall、AS112 以及其他多个 Cloudflare DNS 服务的平台，它在任何给定时间内都会存储超过 2500 亿条 DNS 缓存记录。在如此庞大的规模下，每条记录浪费一个字节，就会导致整个集群消耗超过 250 GB 的内存。

Five successive changes to how cache entries are stored in memory cut the per-entry footprint by over 50%. Across our fleet, these changes freed up roughly 100 terabytes of memory, equivalent to the amount of RAM in 130 of our Gen 13 servers. The cache also got faster. Insert throughput rose 43% and lookup latency dropped 19%, as fewer allocations and better memory locality meant we did not trade speed for space. 通过对缓存记录的内存存储方式进行五次连续改进，我们将每条记录的内存占用减少了 50% 以上。在整个集群中，这些更改释放了大约 100 TB 的内存，相当于我们 130 台第 13 代服务器的内存总量。同时，缓存速度也得到了提升：插入吞吐量提高了 43%，查询延迟降低了 19%。这是因为更少的内存分配和更好的内存局部性，意味着我们并没有以牺牲速度为代价来换取空间。

What we cache

我们缓存的内容

On cold start, Big Pineapple starts out with an empty cache. As DNS queries arrive, the cache fills until it hits its maximum entry count, at which point we evict older or less popular items to make room. 在冷启动时，Big Pineapple 的缓存是空的。随着 DNS 查询的到来，缓存会逐渐填满，直到达到最大记录数；此时，我们会剔除较旧或访问频率较低的记录，以腾出空间。

The exact cache size varies by data center. When EDNS Client Subnet (ECS) is in use, authoritative servers return different answers depending on the client’s network, so we cache multiple versions of the same query. This increases both the number of entries and the memory each one consumes, making the optimizations in this post especially impactful for ECS-heavy locations. 确切的缓存大小因数据中心而异。当使用 EDNS 客户端子网 (ECS) 时，权威服务器会根据客户端的网络返回不同的答案，因此我们会缓存同一查询的多个版本。这增加了记录的数量以及每条记录消耗的内存，使得本文介绍的优化措施对于 ECS 流量密集的地区尤为有效。

Each item in the cache is a key-value pair. The key identifies what was queried: 缓存中的每一项都是一个键值对。键标识了查询的内容：

pub struct CacheKey { 
    qname: Name, 
    qtype: Rtype, 
    authenticated: bool, 
    tag: Vec<u8>, 
}

The value stores the DNS response itself: the answer, authority, and additional record sections, along with metadata like the creation time, a hit counter, and the Time-to-Live (TTL). 值存储了 DNS 响应本身：包括应答、权威记录和附加记录部分，以及创建时间、命中计数器和生存时间 (TTL) 等元数据。

pub struct CacheEntry { 
    timestamp: UnixTimeStamp, 
    pub inception: Instant, 
    pub ttl: Ttl, 
    pub hits: u32, 
    pub answers: Vec<Record>, 
    pub authority: Vec<Record>, 
    pub additional: Vec<Record>, 
    pub errors: Vec<ExtendedError>, 
    ... 
}

Both structs have room for improvement. Several fields use types that carry overhead we don’t need once the entry is stored. 这两个结构体都有改进空间。其中几个字段使用的类型带有我们在记录存储后不再需要的额外开销。

Benchmarking memory usage

内存使用基准测试

To measure the impact of each change, we benchmark by filling the cache with randomly generated entries that roughly match the traffic distribution we see in production: 56% A records, 25% AAAA, and 19% TXT. Each entry contains between one and four records. 为了衡量每次更改的影响，我们通过填充随机生成的记录来进行基准测试，这些记录大致符合我们在生产环境中观察到的流量分布：56% 为 A 记录，25% 为 AAAA 记录，19% 为 TXT 记录。每条记录包含 1 到 4 条记录项。

TXT records serve as a stand-in for all non-A/AAAA record types in the benchmark. Their size is randomized between 64 and 224 bytes, close to the average response size we see for variable-length record types. 在基准测试中，TXT 记录作为所有非 A/AAAA 记录类型的代表。它们的大小在 64 到 224 字节之间随机分配，这接近我们观察到的变长记录类型的平均响应大小。

We track memory usage using a custom allocator that wraps Rust’s System allocator and records the number and size of allocations per cache entry. Alongside memory, we measure insert throughput and lookup latency across the full cache flow to make sure memory savings don’t come at the cost of performance. 我们使用一个自定义分配器来跟踪内存使用情况，该分配器封装了 Rust 的系统分配器，并记录了每条缓存记录的分配次数和大小。除了内存之外，我们还测量了整个缓存流程中的插入吞吐量和查询延迟，以确保内存节省不会以牺牲性能为代价。

These inputs approximate production rather than reproduce it exactly. Process memory also depends on traffic mix, cache occupancy, allocator state, and memory used outside the cache. We therefore measured resident memory across production instances during the rollout. 这些输入是对生产环境的近似，而非完全复刻。进程内存还取决于流量组合、缓存占用率、分配器状态以及缓存之外使用的内存。因此，我们在部署过程中测量了生产实例的常驻内存 (Resident Memory)。

The cost of capacity

容量的代价

Vec<T> stores three fields: a pointer to heap-allocated data, the current length, and the total capacity. When you push an item, Vec checks whether the length exceeds the capacity and reallocates if needed. If there’s room, it just appends the item and increments the length. Vec<T> 存储了三个字段：指向堆分配数据的指针、当前长度和总容量。当你添加一个元素时，Vec 会检查长度是否超过容量，并在必要时重新分配内存。如果有空间，它只需追加元素并增加长度即可。

Once we store a DNS response in the cache, however, we never modify it again. The capacity field serves no purpose, but still costs 8 bytes per Vec. The over-allocated heap space is wasted as well, as a Vec with capacity for eight items but only five stored leaves three slots unused on the heap. 然而，一旦我们将 DNS 响应存储在缓存中，我们就再也不会修改它了。容量字段毫无用处，但每个 Vec 仍然占用 8 字节。过度分配的堆空间也被浪费了，例如一个容量为 8 但只存储了 5 个元素的 Vec，会在堆上留下 3 个未使用的槽位。

Using Box<[T]> solves both problems. It can’t grow after creation, so it doesn’t need a capacity field or reserve space for future elements. The same applies to String, which also carries a capacity field. Box<str> drops it. 使用 Box<[T]> 可以解决这两个问题。它在创建后无法增长，因此不需要容量字段，也不需要为未来的元素预留空间。同样的情况也适用于 String，它也带有容量字段，而 Box<str> 则去掉了它。

Each cache entry stores 8 Vec and String fields. Replacing them with Box<[T]> and Box<str> saves 8 bytes per field, 64 bytes per entry. It also eliminates the excess heap memory that Vec reserves for future growth. The combined savings add up to over 15 terabytes with over 250 billion cache entries. 每条缓存记录存储了 8 个 Vec 和 String 字段。将它们替换为 Box<[T]> 和 Box<str> 后，每个字段节省了 8 字节，每条记录节省了 64 字节。它还消除了 Vec 为未来增长而预留的多余堆内存。在超过 2500 亿条缓存记录的情况下，总共节省了超过 15 TB 的内存。

Fewer lists, fewer pointers

更少的列表，更少的指针

Rather than storing the answer, authority, and additional sections in separate lists, we can store a single list with offsets to the start of each section. Since DNS record counts per section fit in a u16, we can use a u16 (2 bytes) for each offset, compared to the 8-byte pointer and 8-byte length that each separate Box<[T]> requires. 与其将应答、权威和附加部分存储在单独的列表中，我们不如存储一个单一列表，并记录指向每个部分起始位置的偏移量。由于每个部分的 DNS 记录数可以用 u16 表示，我们可以为每个偏移量使用一个 u16（2 字节），而每个单独的 Box<[T]> 则需要 8 字节的指针和 8 字节的长度。

This removes two lists, each with an 8-byte pointer and 8-byte length, and replaces them with two 2-byte offsets, saving 28 bytes per entry. 这去掉了两个列表（每个列表包含 8 字节指针和 8 字节长度），并用两个 2 字节的偏移量取而代之，每条记录节省了 28 字节。

These savings do not always map directly to the number of bytes removed from individual fields. Rust inserts padding to satisfy alignment requirements and rounds a struct’s size up to a multiple of its alignment. Removing a small field can therefore eliminate additional padding. For example, we also packed several boolean fields into a single bitflag. This reduced the surrounding padding, causing the struct to shrink by more than the size of the individual booleans. 这些节省的内存并不总是直接对应于从各个字段中移除的字节数。Rust 会插入填充以满足对齐要求，并将结构体的大小向上舍入为其对齐方式的倍数。因此，移除一个小字段可以消除额外的填充。例如，我们还将几个布尔字段打包成了一个位标志 (bitflag)。这减少了周围的填充，导致结构体缩小的幅度超过了这些布尔值本身的大小。

Dropping the owner

移除所有者 (Owner)

Each DNS record has an owner, the domain the record belongs to. In many cases, this owner is identical to the domain being queried. For example, a query for example.com A returns two records with the same owner: 每条 DNS 记录都有一个所有者，即该记录所属的域名。在许多情况下，这个所有者与被查询的域名相同。例如，查询 example.com A 会返回两条具有相同所有者的记录：

$ dig example.com A 
;; ANSWER SECTION: 
example.com. 300 IN A 198.51.100.1 
example.com. 300 IN A 198.51.100.2

But when a CNAME is involved, for example, the record owner can differ from the queried domain: 但当涉及 CNAME 时，记录所有者可能与被查询的域名不同：

$ dig example.com A 
;; ANSWER SECTION: 
example.com. 300 IN CNAME cdn.example.com. 
cdn.example.com. 300 IN A 198.51.100.1 
cdn.example.com. 300 IN A 198.51.100.2

The DNS wire format handles repeated owners using name compression, as defined in RFC 1035. Rather than encoding the same domain twice, subsequent occurrences store a 2-byte pointer to the first occurrence. A domain like www.example.com can encode just www followed by… DNS 线上传输格式使用 RFC 1035 中定义的域名压缩来处理重复的所有者。后续出现的域名不会被重复编码，而是存储一个指向第一次出现位置的 2 字节指针。像 www.example.com 这样的域名可以只编码 www，后面跟着……