Meta’s Generative Ads Model (GEM): Scaling the Central Model and Compute Engine for Frontier Ads AI

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

By Huayu Li, Darren Liu, Xiaoyi (Leo) Liu, Ellie Wen, Jackie (Jiaqi) Xu, Chunzhi Yang, Reva Srinivasan Last year, we introduced GEM, the industry’s largest recommendation system (RecSys) foundation model. GEM established Meta’s first LLM-inspired paradigm for ads recommendation, driving major performance gains. Building on this foundation, our next goal is to further scale both compute and intelligence transfer with next-generation hardware, and expand this capability across Meta’s entire ads ecosystem. This spans dozens of surfaces and objectives, delivering seamless full-funnel optimization across awareness, consideration, and conversions.

去年，我们推出了行业内最大的推荐系统（RecSys）基础模型 GEM。GEM 确立了 Meta 首个受大语言模型（LLM）启发的广告推荐范式，推动了性能的显著提升。在此基础上，我们的下一个目标是利用下一代硬件进一步扩展计算能力和智能迁移，并将此能力扩展到 Meta 的整个广告生态系统中。这涵盖了数十个界面和目标，在品牌认知、考虑和转化等全漏斗阶段实现无缝优化。

The 2026 generation of GEM shifts from isolated per-domain optimizations — such as Facebook, Instagram, Reels — to a single central foundation model. It learns from data and signals across all of Meta’s ads platforms, bends the scaling curve at LLM-scale computation, and delivers strong advertiser outcomes through post-training techniques that provide a larger latency budget without requiring real-time evaluation of all ad candidates — all on a substantially more efficient training stack. The result is simultaneous ad performance gains across every surface and funnel stage, and deeper understanding that better matches ads to what people are looking for.

2026 版 GEM 从针对 Facebook、Instagram、Reels 等独立领域的优化转向单一的中央基础模型。它学习 Meta 所有广告平台的数据和信号，在 LLM 规模的计算下弯曲扩展曲线，并通过训练后技术提供更大的延迟预算，无需实时评估所有广告候选对象，从而实现强大的广告主成果——所有这些都基于一个效率显著提高的训练栈。其结果是每个界面和漏斗阶段的广告性能同步提升，并实现了更深层次的理解，从而更好地将广告与人们的需求相匹配。

GEM has delivered 18.9% conversion improvement and 12% ad click-through rate (CTR) improvement across Meta’s ad surfaces and objectives, and additionally contributed 5.8%, 4.1%, and 1.6% conversion uplift on Facebook, Instagram, and off-site conversion for targeted users over the past year. These gains span both upper-funnel surfaces and deep-funnel optimization goals, driving stronger returns for advertisers. GEM advances the frontier of recommender systems in the following four areas:

在过去一年中，GEM 在 Meta 的广告界面和目标中实现了 18.9% 的转化率提升和 12% 的广告点击率（CTR）提升，并额外为 Facebook、Instagram 和目标用户的站外转化分别贡献了 5.8%、4.1% 和 1.6% 的转化增长。这些收益涵盖了上层漏斗界面和深层漏斗优化目标，为广告主带来了更强的回报。GEM 在以下四个领域推动了推荐系统的前沿发展：

Bending the Scaling Curve, Enabling 2.5x Model Complexity Challenge: Empirical scaling laws demonstrate that model performance predictably improves with compute, parameter, and data size using transformer model architecture. However, recommender system models do not exhibit similar scaling properties with transformer due to massive heterogeneous features and shifting distributions. Solution: We built an LLM-style architecture — Kunlun, co-designed with the latest GPU hardware, to make larger, deeper models substantially more cost-efficient and to enable long-context learning over user engagement history. We observed an LLM-like scaling law at our largest pre-training scale with 2.5x model complexity growth from earlier versions of GEM. The pattern mirrors frontier LLMs: more capacity and longer context produce emergent gains in ranking quality that translate directly into business results.
弯曲扩展曲线，实现 2.5 倍模型复杂度。挑战：经验扩展定律表明，使用 Transformer 模型架构时，模型性能会随着计算量、参数量和数据规模的增加而可预测地提升。然而，由于海量的异构特征和不断变化的分布，推荐系统模型在使用 Transformer 时并未表现出类似的扩展特性。解决方案：我们构建了一种 LLM 风格的架构——“昆仑”（Kunlun），并与最新的 GPU 硬件协同设计，使更大、更深的模型在成本上显著更高效，并能够对用户参与历史进行长上下文学习。我们在最大的预训练规模下观察到了类似 LLM 的扩展定律，模型复杂度较早期版本的 GEM 增长了 2.5 倍。这种模式与前沿 LLM 如出一辙：更大的容量和更长的上下文产生了排序质量的涌现收益，并直接转化为业务成果。
Efficient Intelligence Transfer From GEM to User-Facing Models Challenge: Serving a model at this scale in real time is prohibitively expensive and drives up latency, compromising both user experience and advertiser ROI. Solution: The structured representation, LoopFM, a state-of-the-art post-training technique, transfers GEM’s internal representations and sequential historical memory to downstream models. Through computation sharing, it doubles the intelligence transfer together with the knowledge distillation deployed in the previous version, at no additional compute cost or latency. We also introduced elastic pre-computation (Solaris) to harvest the gains from the upstream model.
从 GEM 到面向用户模型的高效智能迁移。挑战：实时服务如此规模的模型成本高昂且会增加延迟，从而损害用户体验和广告主投资回报率。解决方案：结构化表示 LoopFM 是一种最先进的训练后技术，它将 GEM 的内部表示和序列历史记忆迁移到下游模型。通过计算共享，它与上一版本中部署的知识蒸馏技术相结合，在不增加额外计算成本或延迟的情况下，使智能迁移效率翻倍。我们还引入了弹性预计算（Solaris）来获取上游模型的收益。
Accelerating the Central Compute Engine With Large-Scale Cross-Surface, Cross-Domain Learning at Meta Ads Scale Challenge: Meta’s apps span dozens of surfaces — Facebook, Instagram, Reels, Stories, and more — with 3.6 billion people using at least one of our apps each day. Each surface and optimization objective previously had its own independent prediction system. As the platform grows in surfaces and objectives, so does the challenge of improving ad performance simultaneously. Solution: GEM accelerates one single, unified compute engine through the central foundation model that jointly predicts clicks and conversions across all surfaces, including Facebook and Instagram, and across every key advertiser objective from off-site conversions to mobile app installs. The generalizable representations learned on data-rich click tasks transfer into data-sparse conversion tasks. A new training recipe with cost-free data mixing and the domain expert framework makes this heterogeneous, large-scale learning tractable, with predictable scale and enhanced reliability.
在 Meta 广告规模下，通过大规模跨界面、跨领域学习加速中央计算引擎。挑战：Meta 的应用涵盖了数十个界面——Facebook、Instagram、Reels、Stories 等，每天有 36 亿人至少使用我们的一款应用。此前，每个界面和优化目标都有其独立的预测系统。随着平台界面和目标的增加，同时提升广告性能的挑战也随之增加。解决方案：GEM 通过中央基础模型加速了一个单一的统一计算引擎，该模型联合预测所有界面（包括 Facebook 和 Instagram）的点击和转化，并涵盖从站外转化到移动应用安装的每一个关键广告主目标。在数据丰富的点击任务上学习到的可泛化表示被迁移到数据稀疏的转化任务中。一种包含免费数据混合和领域专家框架的新训练方案，使得这种异构的大规模学习变得可行，并具有可预测的规模和增强的可靠性。
Scaling Total Training FLOPs by 4x While Doubling E2E Training Efficiency Across Thousands of GPUs Challenge: Training a RecSys foundation model across thousands of GPUs at this scale requires careful orchestration of compute, parallelism, communication, and memory. Today’s data center GPUs and their software stacks are mostly optimized for LLM workloads, whereas recommendation workloads have a fundamentally different profile due to unique data characteristics that make it extremely difficult to achieve high GPU compute utilization while scale. Solution: We developed custom kernels with low precision and new 5D parallelism strategies co-designed with the latest GPU hardware and Meta’s training cluster, and optimized model architecture with infrastructure together to scale efficiently across thousands of GPUs. End-to-end training efficiency is doubled to 20-25% model FLOPS utilization (MFU) with 4x total training FLOPs, enabling GEM to train at LLM-scale at Meta (see full technical story in our companion blog on Training GEM at LLM Scale). Scaling Law: Bend Scaling Curve at Frontier Scale with Kunlun GEM introduces a new LLM-style model architecture — Kunlun. Compared to the previous version (Interformer), Kunlun delivers 2x performance gains with the same computation scaling. Kunlun is designed around the following innovations: Standardize and unify architecture framework by seamlessly interleaving lifelong sequence modeling with scalable recommendation-specific cross-interaction. Introduce
在数千个 GPU 上将总训练 FLOPs 扩展 4 倍，同时使端到端训练效率翻倍。挑战：在如此规模下跨数千个 GPU 训练 RecSys 基础模型，需要对计算、并行性、通信和内存进行精心的编排。当今的数据中心 GPU 及其软件栈大多针对 LLM 工作负载进行了优化，而推荐工作负载由于独特的数据特性，其配置截然不同，这使得在扩展时极难实现高 GPU 计算利用率。解决方案：我们开发了低精度的自定义内核和新的 5D 并行策略，并与最新的 GPU 硬件和 Meta 的训练集群协同设计，同时优化了模型架构与基础设施，从而在数千个 GPU 上高效扩展。端到端训练效率翻倍至 20-25% 的模型 FLOPS 利用率（MFU），总训练 FLOPs 增加了 4 倍，使 GEM 能够在 Meta 实现 LLM 规模的训练（完整技术细节请参阅我们的配套博客《在 LLM 规模下训练 GEM》）。扩展定律：利用“昆仑”在前沿规模下弯曲扩展曲线。GEM 引入了一种新的 LLM 风格模型架构——“昆仑”。与上一版本（Interformer）相比，昆仑在相同的计算扩展下实现了 2 倍的性能提升。昆仑围绕以下创新进行设计：通过将终身序列建模与可扩展的推荐特定交叉交互无缝交织，标准化并统一了架构框架。引入