GenRecs Tokenization: Transforming Ads Recommendation at Meta with Stable Model Vocabulary

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

By Carl Hu, Justin Johnson, Parish Aggarwal, Kaushik Rangadurai, Senthil Manickavelu, Tianwei Zhang. Generative Recommenders (GenRecs) represent a paradigm shift from scoring every ad to reasoning about ad content and user preferences together in order to predict the best outcome for advertisers. Leveraging the world knowledge built into LLMs makes ad matching more intelligent and precise, compounding gains for advertisers. GenRecs Tokenization is the foundational data representation that makes this possible. Rather than maintaining a dedicated ID for every item, GenRecs Tokenization represents each item with a set of tokens from a stable semantic vocabulary. Paired with Meta’s Generative Ads Model (GEM) and Meta’s Adaptive Ranking Model, GenRecs Tokenization has delivered a +0.35% lift in conversions across surfaces for advertisers.

由 Carl Hu、Justin Johnson、Parish Aggarwal、Kaushik Rangadurai、Senthil Manickavelu 和 Tianwei Zhang 撰写。生成式推荐系统 (GenRecs) 代表了一种范式转变，即从对每条广告进行评分，转变为综合考量广告内容和用户偏好，从而为广告主预测最佳结果。利用大语言模型 (LLM) 中内置的通用知识，使得广告匹配更加智能和精准，从而为广告主带来复合收益。GenRecs 分词 (Tokenization) 是实现这一目标的基础数据表示方式。GenRecs 分词不再为每个项目维护一个专用 ID，而是用来自稳定语义词汇表的一组标记来表示每个项目。结合 Meta 的生成式广告模型 (GEM) 和 Meta 的自适应排序模型，GenRecs 分词已使广告主在各平台上的转化率提升了 0.35%。

GenRecs improves the way Meta’s ads recommendation systems make predictions today. Short for Generative Recommenders, GenRecs shifts our approach from scoring every individual ad candidate to leveraging LLM architecture to deeply reason about ad content and user preferences simultaneously. By pairing pre-trained world knowledge with a semantic understanding of what an ad actually represents, the system moves beyond click-pattern matching to understand underlying user preferences. This allows the model to predict the most relevant ad for each person, even when surfacing brand-new products with no past product, ad or company engagement history.

GenRecs 改进了 Meta 广告推荐系统当前的预测方式。GenRecs 是生成式推荐系统的缩写，它将我们的方法从对每个候选广告进行评分，转变为利用 LLM 架构同时深入推理广告内容和用户偏好。通过将预训练的通用知识与对广告实际含义的语义理解相结合，该系统超越了点击模式匹配，转而理解潜在的用户偏好。这使得模型能够为每个人预测最相关的广告，即使是在展示没有任何过往产品、广告或公司互动历史的全新产品时也是如此。

GenRecs Tokenization is the foundational data representation that makes this possible. Instead of maintaining a dedicated ID for every concept in the world, language models compose a small vocabulary of semantic tokens to represent any concept. Because tokens describe what an item (e.g., ads, products, services, etc.) is, the model can relate an item it has never seen engagement for, to items it knows well. A newly launched “trail shoe” inherits what the model has already learned about that category, rather than starting a new representation with no signal.

GenRecs 分词是实现这一目标的基础数据表示方式。语言模型不再为世界上的每个概念维护一个专用 ID，而是通过组合一个小型的语义标记词汇表来表示任何概念。由于标记描述了项目（例如广告、产品、服务等）的本质，模型可以将从未有过互动记录的项目与它所熟知的项目联系起来。一款新推出的“越野跑鞋”可以继承模型对该类别已有的认知，而不是在没有任何信号的情况下从零开始建立新的表示。

GenRecs Tokenization introduces a structured vocabulary approach for ad recommendations, enabling GenRecs for both retrieval and ranking across every item type in the system, including ads, products, pages, and pixels. Meta’s state-of-the-art ads recommendation system drives ongoing advertiser performance improvements by learning about individual items and how they perform. LLMs present a unique opportunity to further improve the performance of recommendation systems by leveraging semantic tokenization in several ads use cases: Accelerating learning from new entities: Semantic tokenization helps models to infer relevance and establish relationships through shared concepts. New entities include newly created ads, niche products, and pages with limited engagement.

GenRecs 分词为广告推荐引入了一种结构化的词汇表方法，使 GenRecs 能够对系统中的每种项目类型（包括广告、产品、页面和像素）进行检索和排序。Meta 最先进的广告推荐系统通过学习各个项目及其表现，推动广告主绩效的持续提升。大语言模型提供了一个独特的契机，通过在多个广告用例中利用语义分词，进一步提升推荐系统的性能：加速从新实体中学习：语义分词有助于模型通过共享概念来推断相关性并建立联系。新实体包括新创建的广告、利基产品以及互动有限的页面。

Enabling cross-campaign transfer: Semantic tokenization creates a stable vocabulary that allows the models to carry insights across campaigns for advertisers. LLM scaling benefits: Semantic tokenization allows the benefits to compound with the LLM scaling curve’s improving performance. Infusing world knowledge: Semantic tokenization gives pre-trained language models a direct interface to reason about ad content and user intent. Once items are migrated to semantic tokens, models learn from token patterns much like LLMs do. In production, this yields: Prediction accuracy improvements in production: Paired with Meta’s GEM and Adaptive Ranking Model, GenRecs Tokenization delivered a +0.35% lift1 in conversions.

实现跨广告系列迁移：语义分词创建了一个稳定的词汇表，使模型能够为广告主在不同广告系列之间传递洞察。LLM 扩展优势：语义分词使这些优势能够随着 LLM 扩展曲线性能的提升而叠加。注入通用知识：语义分词为预训练语言模型提供了一个直接的接口，用于推理广告内容和用户意图。一旦项目迁移到语义标记，模型就会像 LLM 一样从标记模式中学习。在生产环境中，这带来了：生产环境预测准确性的提升：结合 Meta 的 GEM 和自适应排序模型，GenRecs 分词实现了 +0.35% 的转化率提升。

Compounding feedback loop: Better tail predictions yield better lessons from tail ads which enrich the training data to further improve predictions for all ads from the model, creating a performance improving feedback loop. A new way to scale: Across production ranking and retrieval, tokens bend the scaling curve along sequence length, training data, and model architecture. 1 Meta Internal Data, measured in February 2026. Introducing GenRecs Tokenization. Pre-trained multimodal models read each ad, product, and page, including its text and images, and encode the items into an embedding that carries world knowledge.

复合反馈循环：更好的长尾预测能从长尾广告中获得更好的经验，从而丰富训练数据，进一步改善模型对所有广告的预测，形成一个提升性能的反馈循环。一种新的扩展方式：在生产环境的排序和检索中，标记在序列长度、训练数据和模型架构方面优化了扩展曲线。1 Meta 内部数据，测量于 2026 年 2 月。GenRecs 分词介绍。预训练的多模态模型读取每个广告、产品和页面（包括其文本和图像），并将这些项目编码为携带通用知识的嵌入向量。

The tokenizer combines that content signal with representations learned from how people engage with the items, then quantizes the result into a short sequence of discrete tokens. Similar items share tokens, much as related words share roots. For example, there’s overlap between two running shoe ads from the previous season to the current season, so the new campaign immediately benefits from everything the model has already learned about the previous season’s products. At the heart of GenRecs Tokenization is a three-stage pipeline that: Encodes each item into a semantic embedding space. Decomposes that embedding into independent subspaces. Quantizes by mapping each subspace to an entry in a codebook.

分词器将该内容信号与从人们与项目互动方式中学习到的表示相结合，然后将结果量化为一小段离散的标记序列。相似的项目共享标记，就像相关的词共享词根一样。例如，上一季和本季的两则跑鞋广告之间存在重叠，因此新的广告系列可以立即受益于模型此前对上一季产品所学到的一切。GenRecs 分词的核心是一个三阶段流水线：将每个项目编码到语义嵌入空间；将该嵌入分解为独立的子空间；通过将每个子空间映射到码本中的条目来进行量化。

Maximizing ads recommendation quality requires diverse combinations of these stages depending on entity and feature semantics. For example, for many ad representations, GenRecs Tokenization uses prefix n-grams that combine a learned encoder, residual decomposition, learned vector codebooks, and token parametrization. Token Quality Predicts Ranking Accuracy. Independently of downstream ranking gain, tokenizer model quality can be evaluated against intrinsic measures like: Cluster purity: The items that share a code tend to share a category, giving the model a cleaner signal per token; Distribution balance: Items spread evenly across semantic IDs instead of crowding into a few.

最大化广告推荐质量需要根据实体和特征语义，对这些阶段进行多种组合。例如，对于许多广告表示，GenRecs 分词使用前缀 n-gram，结合了学习编码器、残差分解、学习向量码本和标记参数化。标记质量预测排序准确性。除了下游的排序增益外，分词器模型的质量还可以通过以下内在指标进行评估：聚类纯度：共享代码的项目往往属于同一类别，从而为模型提供更清晰的单标记信号；分布平衡：项目均匀分布在各个语义 ID 中，而不是集中在少数几个 ID 上。

Codebook utilization: A larger share of codes holds at least one item, so the vocabulary expresses more distinctions. Across the eight tokenizer versions we tested, we see a strong correlation between these intrinsic quality metrics and downstream ranking prediction accuracy. Tokenizer versions that scored better on these metrics also had better CTR prediction accuracy for ads models. The Tokenization Flywheel. Tokenization maximiz

码本利用率：更大比例的代码至少包含一个项目，因此词汇表能表达更多的区别。在我们测试的八个分词器版本中，我们发现这些内在质量指标与下游排序预测准确性之间存在很强的相关性。在这些指标上得分更高的分词器版本，其广告模型的点击率 (CTR) 预测准确性也更好。分词飞轮。分词最大化