本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

PrivacyLib: Privacy Controls at the Data Access Boundary

By Sai Nagabhairava, Hannes Roth, Na Chen, Peter Prelich, Jonathan Bergeron This latest deep dive into Meta’s Privacy Aware Infrastructure (PAI) takes a look at PrivacyLib. A privacy requirement must hold as data moves through Meta’s diverse data and processing systems. Historically, supporting a new requirement meant implementing the same enforcement logic separately in each system. PrivacyLib replaces this with a common library that runs inside the clients of Meta’s data and processing systems. A system integrates with PrivacyLib once, and new privacy capabilities plug in as modules that reuse that integration. PrivacyLib performs billions of checks every second, and changes are simulated and rolled out per module. For internal AI agents, whose data access paths may not be known in advance, tool calls on supported paths carry agent context to PrivacyLib. The propagated context links each access to its session, so modules can observe data use and block operations that are not allowed.

本文由 Sai Nagabhairava、Hannes Roth、Na Chen、Peter Prelich 和 Jonathan Bergeron 撰写。这篇关于 Meta 隐私感知基础设施（PAI）的最新深度解析将聚焦于 PrivacyLib。当数据在 Meta 多样化的数据和处理系统中流动时，必须满足隐私要求。过去，支持一项新要求意味着必须在每个系统中分别实现相同的执行逻辑。PrivacyLib 用一个运行在 Meta 数据和处理系统客户端内部的通用库取代了这种做法。系统只需与 PrivacyLib 集成一次，新的隐私功能便可作为模块插入，并复用该集成。PrivacyLib 每秒执行数十亿次检查，且变更会按模块进行模拟和发布。对于数据访问路径可能无法预知的内部 AI 代理，受支持路径上的工具调用会将代理上下文传递给 PrivacyLib。传播的上下文将每次访问与其会话关联起来，从而使模块能够观察数据使用情况并阻止不允许的操作。

At Meta, many privacy requirements ultimately come down to the same question: Can we prove that a piece of data goes only where it is allowed? At Meta’s scale, rather than relying solely on manual privacy audits, which can go out of sync as systems change, we have developed Privacy Aware Infrastructure (PAI) to answer this question within the infrastructure itself. PAI offers efficient and reliable first-class privacy constructs embedded in Meta’s infrastructure so that products can keep evolving quickly. In previous posts, we have covered how data understanding, data lineage, and Policy Zones (our information flow control technology) play a role in PAI. Now, we are diving deep into PrivacyLib, a common library that runs inside the clients of hundreds of our data and processing systems at the point of data access. PrivacyLib plays an important role in the final stage of the broader PAI workflow: continuously enforcing and monitoring data flows. This blog post focuses on PrivacyLib’s role in the final stage of the PAI workflow.

在 Meta，许多隐私要求最终都归结为同一个问题：我们能否证明一段数据只流向了被允许的地方？在 Meta 的规模下，我们没有仅仅依赖可能随系统变更而失效的手动隐私审计，而是开发了隐私感知基础设施（PAI）来在基础设施内部回答这个问题。PAI 在 Meta 的基础设施中提供了高效且可靠的一流隐私构建模块，使产品能够快速迭代。在之前的文章中，我们介绍了数据理解、数据血缘和策略区域（我们的信息流控制技术）在 PAI 中所扮演的角色。现在，我们将深入探讨 PrivacyLib，这是一个运行在数百个数据和处理系统客户端内部、位于数据访问点的通用库。PrivacyLib 在更广泛的 PAI 工作流的最后阶段发挥着重要作用：持续执行和监控数据流。本篇博文重点介绍 PrivacyLib 在 PAI 工作流最后阶段的作用。

Before PrivacyLib, we integrated privacy enforcement separately into the clients of each data and processing system for each requirement. With this approach, implementing a new requirement across systems could take months. Each new requirement, whether purpose limitation or data flow discovery, needed the same set of capabilities: covering a wide range of systems; deploying in weeks rather than months; running on every data access without becoming a latency problem; and rolling out enforcement changes without taking the fleet down. Over time, this approach became harder to scale as requirements and systems evolved. PrivacyLib separates system integration from requirement implementation. We integrate each system once. When a requirement can be met at the point of data access, we implement it as a PrivacyLib module shared across the systems already integrated, so a new requirement no longer needs a separate implementation in each system.

在 PrivacyLib 出现之前，我们针对每一项要求，分别将隐私执行集成到每个数据和处理系统的客户端中。采用这种方法，在各系统中实现一项新要求可能需要数月时间。每一项新要求，无论是目的限制还是数据流发现，都需要具备相同的一套能力：覆盖广泛的系统；在几周而非几个月内完成部署；在每次数据访问时运行且不造成延迟问题；以及在不导致整个集群停机的情况下发布执行变更。随着时间的推移，随着要求和系统的演变，这种方法变得难以扩展。PrivacyLib 将系统集成与要求实现分离开来。我们对每个系统只进行一次集成。当一项要求可以在数据访问点得到满足时，我们将其实现为一个在已集成系统间共享的 PrivacyLib 模块，因此新要求不再需要在每个系统中单独实现。

There are two kinds of modules: Enforcing modules can block a data access. For example, Policy Zones enforces purpose limitation. Observing modules only record a data access. For example, Privacy Probes collects the runtime signals used to discover data flows. As internal AI agents became a new way of accessing data, we also added the Agent Observability module to observe agent-related data access events and help safeguard the data that agents can reach. Systems already integrated with PrivacyLib picked it up without new integration work, which is what integrating once buys: each new privacy capability arrives as a module, not as a project. PrivacyLib replaces repeated implementations across systems.

模块分为两种：执行模块可以阻止数据访问。例如，策略区域（Policy Zones）用于执行目的限制。观察模块仅记录数据访问。例如，隐私探测器（Privacy Probes）收集用于发现数据流的运行时信号。随着内部 AI 代理成为访问数据的新方式，我们还添加了代理可观测性模块，以观察与代理相关的数据访问事件，并帮助保护代理可触达的数据。已集成 PrivacyLib 的系统无需额外集成工作即可使用它，这就是“集成一次”带来的好处：每一项新的隐私功能都是作为一个模块而非一个项目引入的。PrivacyLib 取代了跨系统的重复实现。

Engineering Challenges and Approaches of PrivacyLib at Meta’s Scale PrivacyLib operates across a broad range of data access paths at Meta: More than a hundred libraries used to access data services, from key/value stores to AI features. Thousands of production services with versions of the deployed library. Millions of assets being accessed daily. Billions of checks every second. Operating at this scale creates three engineering challenges: Challenge Approach Coverage: Systems can have multiple client libraries and legacy integrations, accept direct network calls, and offer direct database access to backing stores. All of them need to be handled to ensure requirements hold no matter the access path. We integrate at two levels: typed client libraries, where PrivacyLib has full request context, and lower network boundaries on remaining RPC paths to get full coverage. Each integrated system maps its native APIs to PrivacyLib’s common operations and provides the inputs required by its configured modules. Once those pieces are in place, the same module can be enabled in multiple systems without a separate system-specific integration.

Meta 规模下 PrivacyLib 的工程挑战与方法：PrivacyLib 在 Meta 的广泛数据访问路径上运行：超过一百个用于访问数据服务的库，从键值存储到 AI 特征；数千个部署了该库版本的生产服务；每天有数百万个资产被访问；每秒进行数十亿次检查。在这种规模下运行带来了三个工程挑战：挑战与方法——覆盖范围：系统可能拥有多个客户端库和遗留集成，接受直接网络调用，并提供对后端存储的直接数据库访问。必须处理所有这些情况，以确保无论访问路径如何，要求都能得到满足。我们在两个层面进行集成：类型化的客户端库（PrivacyLib 在此拥有完整的请求上下文）以及剩余 RPC 路径上的较低网络边界，以实现全面覆盖。每个集成系统将其原生 API 映射到 PrivacyLib 的通用操作，并提供其配置模块所需的输入。一旦这些部分就位，同一个模块就可以在多个系统中启用，而无需针对特定系统进行单独集成。

Scale: At billions of checks per second, even a small per-check computational cost becomes consequential across the fleet. Every service in the hot path pays that cost. We compile requirements at build time into their simplest evaluable form and answer most checks from caches inside the process, so the common path avoids a network call. Reliability: Privacy logic runs in the hot path of critical systems, owned by teams that release on their own schedules. A change has to be testable in isolation and reversible on its own, because a fleet-wide rollback cannot be the recovery path. Enforcement is deployed as part of the service that owns it, so a change reaches production through that service’s own pipeline instead of a fleet-wide config push.

规模：在每秒数十亿次检查的情况下，即使每次检查的计算成本很小，在整个集群中也会产生重大影响。热路径上的每个服务都要承担这一成本。我们在构建时将要求编译成最简单的可评估形式，并从进程内的缓存中回答大多数检查，因此通用路径避免了网络调用。可靠性：隐私逻辑运行在关键系统的热路径中，由各自按计划发布的团队拥有。变更必须能够独立测试并可单独回滚，因为全集群回滚不能作为恢复路径。执行逻辑作为所属服务的一部分进行部署，因此变更通过该服务自己的流水线进入生产环境，而不是通过全集群配置推送。

Walkthrough: Protecting Messaging Metadata To illustrate how everything comes together at Meta’s scale, let’s walk through a privacy requirement that needs to be upheld when processing messaging metadata. Users exchange messages through apps like Messenger, which supports end-to-end encryption. As Messenger routes and delivers those messages, subject to applicable law, its backend systems proce

演练：保护消息元数据。为了说明在 Meta 的规模下这一切是如何协同工作的，让我们演练一个在处理消息元数据时需要遵守的隐私要求。用户通过 Messenger 等支持端到端加密的应用程序交换消息。当 Messenger 路由和传递这些消息时，在遵守适用法律的前提下，其后端系统会处理……