Rapidly scaling online storage to serve over 1 billion ChatGPT users
Rapidly scaling online storage to serve over 1 billion ChatGPT users
为服务超过 10 亿 ChatGPT 用户而快速扩展在线存储
September 11, 2026 | By Jon Lee, Chaomin Yu, and Ben Ries 2026 年 9 月 11 日 | 作者：Jon Lee, Chaomin Yu, Ben Ries
Every OpenAI product depends on fast, reliable access to data, whether someone is logging in, checking their Codex settings, or starting a new conversation in ChatGPT. Each of those actions may require many separate data lookups before the product can respond. If those requests are slow, the product feels slow. If those requests fail, the product stops working entirely. 每一款 OpenAI 产品都依赖于快速、可靠的数据访问，无论是用户登录、查看 Codex 设置，还是在 ChatGPT 中开启新对话。在产品做出响应之前，每一个动作都可能需要多次独立的数据查询。如果这些请求缓慢，产品就会显得卡顿；如果请求失败，产品则会完全无法使用。
Habitat is the online storage platform we built so OpenAI products can quickly and reliably access needed information. Habitat now handles more than 70 million requests every second, supporting products used by over 1 billion people each week, across almost 40 geographic regions. Habitat first launched to support GPTs at DevDay 2023, starting as a simple Python client-side library connected to a single database. Today, it’s a complex distributed system that serves more than 500 petabytes of data. Habitat 是我们构建的在线存储平台，旨在让 OpenAI 产品能够快速、可靠地访问所需信息。目前，Habitat 每秒处理超过 7000 万次请求，支持着全球近 40 个地理区域、每周超过 10 亿用户使用的产品。Habitat 最初于 2023 年 DevDay 发布，用于支持 GPTs，起初只是一个连接到单一数据库的简单 Python 客户端库。如今，它已演变成一个复杂的分布式系统，服务于超过 500 PB 的数据。
Building and operating infrastructure at this scale is no easy feat, but also not particularly challenging. What made our situation unique is the unprecedented rate at which we’ve had to scale to support staggering user growth and product demand while simultaneously building out a mature platform. Often, system engineers build for 10x scale, and hope for it to hold for a few years while preparing for the next 10x. In our case, we’ve grown more than 10x year-over-year for the last three years. As a result, building and operating Habitat has been a series of tactical decisions and sequencing: understanding each component at the lowest level to squeeze as much juice out of our existing stack, while fending off storage and compute capacity crunches to buy time for foundational investments. 构建和运营这种规模的基础设施并非易事，但也并非不可逾越。我们所处情况的独特性在于，为了支持惊人的用户增长和产品需求，我们必须以史无前例的速度进行扩展，同时还要构建一个成熟的平台。通常，系统工程师会按 10 倍规模进行构建，并希望它能支撑几年，同时为下一个 10 倍做准备。而在我们的案例中，过去三年里，我们每年都实现了超过 10 倍的增长。因此，Habitat 的构建与运营是一系列战术决策与排序的过程：在最底层理解每一个组件，以从现有技术栈中榨取最大性能，同时抵御存储和计算能力的瓶颈，从而为基础性投资争取时间。
As OpenAI grew, Habitat had to grow with it: first by becoming reliable enough for mission-critical product traffic, then fast enough for global users, and finally, to deftly operate at massive scale. This post is the first in a two-part series on how we scaled online storage. In this post, we’ll share how Habitat evolved, why we turned it from a library into a service, and how we stretched a service written in an uncommon serving stack language—Python—into a reliable storage platform layer. 随着 OpenAI 的成长，Habitat 也必须随之进化：首先要达到足以承载关键任务产品流量的可靠性，然后要满足全球用户的速度需求，最后实现大规模下的灵活运营。本文是我们关于如何扩展在线存储的系列文章（共两篇）的第一篇。在本文中，我们将分享 Habitat 的演进过程，解释为什么我们将其从一个库转变为一项服务，以及我们如何将一个使用非主流服务栈语言（Python）编写的服务，打造成一个可靠的存储平台层。
In a future post, we’ll go into detail about how we made multi-tenancy reliability at scale, our layered strategy for optimizing read performance, and how we scaled our partnership with Azure Cosmos DB to reliably handle unprecedented demand. 在后续文章中，我们将详细介绍如何实现大规模下的多租户可靠性、我们优化读取性能的分层策略，以及我们如何扩展与 Azure Cosmos DB 的合作，以可靠地应对前所未有的需求。
What is Habitat?
什么是 Habitat？
Habitat started from a simple idea: product engineers shouldn’t need to think about database management. Habitat first launched to support GPTs at DevDay 2023 as a small Python library that interacted with ChatGPT’s main server. It supported a small set of operations that mapped under the hood to the database application, Azure Cosmos DB. Habitat 源于一个简单的想法：产品工程师不应该为数据库管理而分心。Habitat 最初于 2023 年 DevDay 发布，作为一个与 ChatGPT 主服务器交互的小型 Python 库来支持 GPTs。它支持一组简单的操作，这些操作在底层映射到数据库应用 Azure Cosmos DB。
The library’s job was to give product teams a simple way to store and retrieve data without needing to master the underlying details. Habitat took care of the necessary work: figuring out what kind of data was involved, where it should come from (or go), whether the request was allowed, and so on. 该库的任务是为产品团队提供一种简单的数据存取方式，而无需掌握底层细节。Habitat 负责处理必要的工作：确定涉及的数据类型、数据的来源（或去向）、请求是否被允许等等。
Product engineers need not concern themselves with schema lookup, routing, authorization, encryption, serialization, request shaping, and connection pooling. They didn’t even need to consider where the data comes from: Azure Cosmos DB, caches, or other types of storage. 产品工程师无需关心模式查找、路由、授权、加密、序列化、请求整形和连接池等问题。他们甚至不需要考虑数据来自何处：是 Azure Cosmos DB、缓存还是其他类型的存储。
By decoupling the storage logic into a standalone service, we established a single point of control for deployments, observability, and platform enhancements. 通过将存储逻辑解耦为独立的服务，我们为部署、可观测性和平台增强建立了一个单一的控制点。
This Python library worked well and Habitat saw rapid adoption among product engineers at OpenAI, despite no concerted central push away from using self-serve Postgres and Azure Postgres and Azure Cosmos DB. As product needs evolved, it was even easy for product developers to add to the shared library support for features like client-side caching, compression, or encryption. 尽管没有进行集中的强制推广来取代自建的 Postgres 和 Azure Cosmos DB，但这个 Python 库运行良好，并在 OpenAI 的产品工程师中得到了迅速采用。随着产品需求的发展，产品开发人员甚至可以轻松地在共享库中添加对客户端缓存、压缩或加密等功能的支持。
Build a service to better support multiple, complex products
构建服务以更好地支持多个复杂产品
By the middle of 2025, Habitat had reached its limits as a client-side implementation. As the Habitat layer had grown more complex and OpenAI’s services count increased, backward-compatible protocol changes had become infeasible. 到 2025 年年中，作为客户端实现的 Habitat 已达到其极限。随着 Habitat 层变得越来越复杂，且 OpenAI 的服务数量不断增加，进行向后兼容的协议更改已变得不可行。
In one instance, we wanted to reduce the blast radius of any single region outage for our most critical data sets by migrating them to a set of regionally distributed Azure Cosmos DB accounts. Making this change required introducing extra routing logic into the client, disabled behind a feature flag, ensuring it rolled out to all clients, and then enabling the feature flag. 例如，我们曾希望通过将最关键的数据集迁移到一组区域分布的 Azure Cosmos DB 账户中，来减少单区域故障的影响范围。进行此项更改需要在客户端引入额外的路由逻辑，通过功能标志（feature flag）进行禁用，确保其部署到所有客户端，然后再启用该标志。
Coordinating deployments across dozens of services and working with each team to roll it out took days. Before enabling this, we realized we wanted to introduce some shadowing to ensure the sharding logic would be correct. That took another couple of days. 协调数十个服务的部署并与每个团队合作进行推广耗费了数天时间。在启用之前，我们意识到需要引入一些影子流量（shadowing）来确保分片逻辑的正确性。这又花费了几天时间。