Meta Andromeda: Powering the Next Generation of Ads Retrieval Through Model-System Co-Design

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

By Haiyu Lu, Shuo Gu, Yang Yang (Tony), Jingwen Ren, Tianchen Ji, Qing Zhang, Wentian Guo, Haocheng Wu, Siqi Zeng, Yunchen Pu, Liang Wang, Chonglin Sun This post describes the next generation of Meta Andromeda, Meta’s personalized ads retrieval engine, and how co-designing the model, system, and GPU hardware unlocks a new efficiency frontier for ads retrieval.

本文作者包括 Haiyu Lu、Shuo Gu、Yang Yang (Tony)、Jingwen Ren、Tianchen Ji、Qing Zhang、Wentian Guo、Haocheng Wu、Siqi Zeng、Yunchen Pu、Liang Wang 和 Chonglin Sun。本文介绍了 Meta 下一代个性化广告检索引擎 Meta Andromeda，以及如何通过模型、系统和 GPU 硬件的协同设计，为广告检索开启全新的效率前沿。

In production, the system scores every eligible ad with near ranking-level model performance at retrieval scale and surfaces the top candidates at over one billion ads per second, a new throughput benchmark for ads retrieval-stage models (i.e., a 100x lift compared to the 2024 Meta Andromeda system). For advertisers, this means better performance automatically from the retrieval system matching ads across a larger candidate pool, within the same latency budget. Since 2025, Meta Andromeda has delivered a 2.4% increase in conversions on Facebook.

在生产环境中，该系统能够在检索规模下以接近排序级模型的性能对每个符合条件的广告进行评分，并以每秒超过 10 亿个广告的速度筛选出顶级候选广告，这为广告检索阶段模型树立了新的吞吐量基准（即相比 2024 年的 Meta Andromeda 系统提升了 100 倍）。对于广告主而言，这意味着在相同的延迟预算内，检索系统能够从更大的候选池中匹配广告，从而自动获得更好的投放效果。自 2025 年以来，Meta Andromeda 已使 Facebook 上的转化率提升了 2.4%。

Meta Andromeda is Meta’s machine learning system for personalized ads retrieval, the first stage of the ads recommendation pipeline. Retrieval narrows a vast pool of eligible ads down to a small set of strong candidates in milliseconds; a later, more computationally expensive ranking stage then scores those survivors in fine detail. Historically, the deep modeling quality of the ranking stage has been too costly to apply at retrieval’s scale. Building on the retrieval foundation established with Andromeda’s 2024 launch on the NVIDIA GH200 Grace Hopper Superchip, this post shares how co-designing the model, system, and hardware together brings near ranking-level performance to the retrieval scale.

Meta Andromeda 是 Meta 用于个性化广告检索的机器学习系统，也是广告推荐流程的第一阶段。检索过程在毫秒级时间内将海量的候选广告池缩小为一小部分高质量候选集；随后，计算成本更高的排序阶段会对这些筛选出的广告进行精细评分。从历史上看，排序阶段的深度建模质量因成本过高，无法应用于检索规模。基于 2024 年 Andromeda 在 NVIDIA GH200 Grace Hopper 超级芯片上发布所建立的检索基础，本文分享了如何通过模型、系统和硬件的协同设计，将接近排序级的性能带入检索规模。

Challenges and Opportunities in Scaling Retrieval The 2024 Andromeda deployment reimplemented retrieval with an end-to-end optimized retrieval implementation from custom AI hardware with hierarchical indexing and GPU-native operators to a purpose-built ML model, growing model capacity by four orders of magnitude. Since then, generative AI creative tools have helped advertisers rapidly create hundreds of millions of new ads, growing Meta’s ads candidate pool to an unprecedented scale. Scoring such a large ad candidate pool with a more accurate model at a much higher system throughput requires fundamental technology breakthroughs in model quality, system architecture, and hardware utilization jointly.

检索扩展的挑战与机遇：2024 年的 Andromeda 部署通过端到端优化的检索实现方案重新构建了检索功能，涵盖了具有分层索引和 GPU 原生算子的定制 AI 硬件，以及专门构建的机器学习模型，使模型容量增长了四个数量级。自那时起，生成式 AI 创意工具帮助广告主快速创建了数以亿计的新广告，使 Meta 的广告候选池增长到了前所未有的规模。要在更高的系统吞吐量下，利用更精确的模型对如此庞大的广告候选池进行评分，需要在模型质量、系统架构和硬件利用率方面实现根本性的技术突破。

Higher Model Performance Requires a New Architecture Ads retrieval has an outsized ROI on incremental quality: Small improvements in retrieval model performance can drive large gains in advertiser value. The rapid expansion of our ads candidate pool makes optimizing for hardware capacity ROI an increasingly critical factor. The 2024 Meta Andromeda retrieval models have established a highly effective foundation with their GPU-optimized deep neural network architecture. To keep improving retrieval performance at a sustainable compute cost at our current scale, however, we must move beyond that paradigm. This requires a fundamental architectural evolution to push model performance boundaries significantly without a proportional increase in costs.

更高的模型性能需要新的架构：广告检索在质量提升方面具有巨大的投资回报率：检索模型性能的微小改进就能带来广告主价值的大幅提升。广告候选池的快速扩张使得针对硬件容量投资回报率的优化变得愈发关键。2024 年的 Meta Andromeda 检索模型凭借其 GPU 优化的深度神经网络架构，建立了非常有效的基石。然而，为了在当前规模下以可持续的计算成本持续提升检索性能，我们必须超越这一范式。这需要进行根本性的架构演进，在不增加相应成本的前提下，显著突破模型性能的边界。

New System Bottlenecks at Billion Scale The 2024 Meta Andromeda system evaluates at tens of millions of ads per second per GPU card. Meeting the new business demand requires a throughput at multi-billion ads evaluated per second per card in the system. At billions of ads per second throughput, the memory bandwidth of passing ad tensors through memory hierarchy becomes the new bottleneck. Intermediate tensors between GPU kernels account for terabytes per second of memory traffic, and shipping ads between local CPU hosts and remote hardware accelerators evaluating models saturates the network link. This requires a new system co-design that unifies these fragmented, distributed workloads into a single GPU pipeline.

十亿级规模下的新系统瓶颈：2024 年的 Meta Andromeda 系统每张 GPU 卡每秒可评估数千万个广告。满足新的业务需求则要求系统每张卡每秒的评估吞吐量达到数十亿个广告。在每秒数十亿个广告的吞吐量下，通过内存层级传递广告张量的内存带宽成为了新的瓶颈。GPU 内核之间的中间张量占用了每秒数 TB 的内存流量，而在本地 CPU 主机与评估模型的远程硬件加速器之间传输广告则会使网络链路饱和。这需要一种新的系统协同设计，将这些碎片化的分布式工作负载统一到一个单一的 GPU 流水线中。

Pushing Towards the Hardware Limits With retrieval running on GPU, the important optimization question is “How close are we to the hardware ceiling, and how much room remains for future model growth?” Answering this requires pushing each component of the retrieval pipeline to the limits of the current hardware generation, a challenge that motivated a joint collaboration with NVIDIA. Each of these challenges drove a corresponding advance.

向硬件极限推进：随着检索在 GPU 上运行，一个重要的优化问题是：“我们距离硬件上限还有多远，未来模型增长的空间还有多少？”要回答这个问题，需要将检索流水线的每个组件都推向当前硬件一代的极限，这一挑战促成了与 NVIDIA 的联合协作。每一个挑战都推动了相应的进步。

Unlocking Retrieval Performance Through Model-System Co-Design Next-Gen Retrieval Model Architecture — Retrieval Mixture-of-Experts The latest iteration of Meta Andromeda introduces three synergistic architectural innovations that bring large language model (LLM)-inspired mixture-of-experts (MoE) into the retrieval (see Figure 1). With this highly expressive yet inference-efficient architecture, a new retrieval performance-efficiency Pareto frontier has been established: model performance approaches ranking-level in a typical retrieval-then-ranking system, at approximately 300x fewer inference-time FLOPs than the 2024 Meta Andromeda retrieval model.

通过模型-系统协同设计释放检索性能：下一代检索模型架构——检索专家混合模型 (MoE)：Meta Andromeda 的最新迭代引入了三项协同架构创新，将受大语言模型 (LLM) 启发的专家混合模型 (MoE) 引入了检索过程（见图 1）。凭借这种表达能力强且推理高效的架构，我们建立了一个新的检索性能-效率帕累托前沿：模型性能在典型的“先检索后排序”系统中接近排序级，而推理时的浮点运算次数 (FLOPs) 比 2024 年的 Meta Andromeda 检索模型减少了约 300 倍。

On the user modeling side, we transitioned from a single user tower to an ensemble of specialized user towers as experts. This captures diverse, time-evolving, and contextualized user interests by integrating a vast array of high-fidelity signals: sequential ad engagement events, user-cohort representations, LLM-derived semantic features, and cross-surface co-engagement signals. In the candidate ads tower, we employ a joint encoding strategy that incorporates real-time serving context, ensuring that downstream interaction models receive context-aware representations. In the interaction overarch layer, the interaction model evolved from dense neural networks into an MoE attention-based gating network that dynamically routes candidates across the most relevant user experts, delivering near-ranking-level performance at a fraction of the FLOPs. A defining property of the new architecture is that its performance is co-designed with the underlying硬件 execution model. Model inference throughput has reached over 10 billion ads per second on a single GH200 GPU, and this efficiency.

在用户建模方面，我们从单一用户塔转型为由专业用户塔组成的专家集合。通过整合大量高保真信号（如序列广告互动事件、用户群体表示、LLM 派生的语义特征以及跨平台共同互动信号），该架构能够捕捉多样化、随时间演变且具有上下文关联的用户兴趣。在候选广告塔中，我们采用了结合实时服务上下文的联合编码策略，确保下游交互模型能够接收到具有上下文感知的表示。在交互总控层，交互模型从密集神经网络演变为基于 MoE 注意力的门控网络，能够动态地将候选广告路由至最相关的用户专家，从而以极少的 FLOPs 实现接近排序级的性能。新架构的一个决定性特征是其性能与底层硬件执行模型进行了协同设计。在单张 GH200 GPU 上，模型推理吞吐量已达到每秒超过 100 亿个广告，且这种效率……