Exploring Hierarchical Interest Representation For Meta Ads Deep Funnel Optimization

探索 Meta 广告深度漏斗优化的分层兴趣表示

By Yuhui Ouyang, Di Wang, Sreedal Menon, Jie Tian 作者：Yuhui Ouyang, Di Wang, Sreedal Menon, Jie Tian

Hierarchical Interest Representation is a research area for Meta Ads. We’re exploring an upstream representation layer over the universe of Ads entities – users, advertisers, products, services – learning unified embeddings that connect users’ inferred interests with the breadth of what advertisers offer in their deep funnel ads. “分层兴趣表示”（Hierarchical Interest Representation）是 Meta 广告的一个研究领域。我们正在探索在广告实体（用户、广告主、产品、服务）全集之上构建一个上游表示层，通过学习统一的嵌入（embeddings），将用户推断出的兴趣与广告主在深度漏斗广告中提供的广泛内容连接起来。

The innovations in Hierarchical Interest Representation are an in-house transformer based graph learning with bias-aware attention and self-supervised cross-view distillation, learning multi-hierarchical interest representations across a large graph. 分层兴趣表示的创新之处在于采用了一种内部开发的基于 Transformer 的图学习方法，结合了偏差感知注意力机制（bias-aware attention）和自监督跨视图蒸馏（self-supervised cross-view distillation），从而在大规模图结构中学习多层次的兴趣表示。

Hierarchical Interest Representation blends real-world knowledge with engagement signals – multimodal advertiser and product content processed through LLMs enriches sparse interactions, enabling generalization to rare and unseen entities. 分层兴趣表示将现实世界的知识与参与度信号相结合——通过大语言模型（LLM）处理的多模态广告主和产品内容丰富了稀疏的交互数据，从而实现了对罕见和未见实体的泛化能力。

Hierarchical Interest Representation outputs universal embeddings for ads entities and Bag-of-Meaning interest tokens that have the potential to power new personalization, retrieval, supervision, and specialized ranking architectures across the ads stack. 分层兴趣表示输出广告实体的通用嵌入以及“意义词袋”（Bag-of-Meaning）兴趣标记，这些输出有潜力为整个广告技术栈中的个性化、检索、监督和专门的排序架构提供动力。

Trained end-to-end on real Meta ads data at the scale of billions of interactions. Hierarchical Interest Representation is an upstream representation layer designed to improve upon Meta’s deep funnel ranking optimization. It aims to connect businesses with the population of people on our platforms who carry the most genuine, latent interest in what they offer. 该系统在数十亿次交互规模的真实 Meta 广告数据上进行了端到端训练。分层兴趣表示是一个旨在改进 Meta 深度漏斗排序优化的上游表示层。其目标是将企业与我们平台上对其实际产品拥有最真实、潜在兴趣的用户群体连接起来。

The system is intended to function across Meta’s broader recommendation ecosystem, such as Meta’s Generative Ads Model (GEM), Andromeda, and the Adaptive Ranking Model, to advance deep funnel optimization. 该系统旨在应用于 Meta 更广泛的推荐生态系统，例如 Meta 的生成式广告模型（GEM）、Andromeda 和自适应排序模型，以推动深度漏斗优化的进步。

People come to Meta’s apps and platforms to connect with people and content, expressing preference with every scroll. Engagement signals are used to understand both inferred and explicit interests and improve relevance of content across our platforms. 人们来到 Meta 的应用和平台是为了与他人和内容建立联系，并通过每一次滑动表达偏好。参与度信号被用于理解用户的推断兴趣和显性兴趣，并提高我们平台上内容的关联度。

Utilizing frontier AI to map latent interests from sparse engagement signals and aligning them with the vast landscape of advertiser offerings is a transformative approach to addressing the challenges of signal scarcity and driving up deep funnel ad performance. 利用前沿 AI 技术从稀疏的参与度信号中映射潜在兴趣，并将其与广告主提供的海量内容相对齐，这是一种解决信号稀疏性挑战并提升深度漏斗广告效果的变革性方法。

The mission is to strengthen reasoning relationships throughout the landscape of ads entities – spanning users, businesses, and products – utilizing multi-hierarchical granularities that allow our models to navigate between stable, high-level interest anchors and the specialized, sparse signals of deep funnel intent. 我们的使命是加强广告实体全景中的推理关系（涵盖用户、企业和产品），利用多层次的粒度，使我们的模型能够在稳定的高层兴趣锚点与深度漏斗意图的专业化、稀疏信号之间进行导航。

How Hierarchical Interest Representation Enhances Deep Funnel Optimization

分层兴趣表示如何增强深度漏斗优化

Hierarchical Interest Representation pioneers a structural shift in representation modeling by navigating long-range graph topologies and distilling sparse engagement signals into unified interest clusters at various granularities. By fusing real-world knowledge with a semantic grasp of advertised products, it effectively strengthens the connection between ads and user intents. 分层兴趣表示通过导航长距离图拓扑并将稀疏的参与度信号提炼为不同粒度的统一兴趣簇，开创了表示建模的结构性转变。通过将现实世界知识与对广告产品的语义理解相融合，它有效地加强了广告与用户意图之间的联系。

Hierarchical Interest Representation encourages discovery-oriented ad experiences by extracting stable interest anchors from massive engagement datasets and grounding them in multi-modal world knowledge enrichment. This aims to enable the delivery of more relevant ad content to optimize deep funnel ads. 分层兴趣表示通过从海量参与度数据集中提取稳定的兴趣锚点，并将其植根于多模态世界知识增强中，从而鼓励以发现为导向的广告体验。这旨在实现更相关广告内容的投放，以优化深度漏斗广告。

The Technical Challenges

技术挑战

User engagement with ads entities is naturally graph-structured: users and ads entities (advertisers, products, services, campaigns etc.) are nodes, and the activities and events connecting them are edges. At Meta’s scale, this is one of the largest graph networks in the industry. Learning the interest representation bears the following challenges: 用户与广告实体的交互本质上是图结构的：用户和广告实体（广告主、产品、服务、活动等）是节点，连接它们的活动和事件是边。在 Meta 的规模下，这是业内最大的图网络之一。学习兴趣表示面临以下挑战：

User Inferred Signal Dynamics: Meta provides users with tools to help tailor their experiences, like providing ‘Interested/Not interested’ feedback on posts they see. In addition, inferred interests based on engagement signals continue to play an important role for improving deep funnel ads. 用户推断信号的动态性：Meta 为用户提供了帮助定制体验的工具，例如对看到的帖子提供“感兴趣/不感兴趣”的反馈。此外，基于参与度信号的推断兴趣在改进深度漏斗广告方面继续发挥着重要作用。

Large Networks With Sparse Connections: Every month, Meta’s ads network serves millions of ads, from millions of advertisers, to billions of people across our platforms. While this “vocabulary” is large, ad impression opportunities are limited and deep funnel user feedback is scarce. 具有稀疏连接的大型网络：每个月，Meta 的广告网络都会向我们平台上的数十亿人投放来自数百万广告主的数百万条广告。虽然这个“词汇表”很大，但广告展示机会有限，且深度漏斗的用户反馈非常稀缺。

Long-Range, Global Relationships: Given the sparsity of the individual connections in the deep funnel, it is useful to observe common patterns from long range, graph connected entities and users and encode into representation. Capturing long-range relationships within large graph networks is computationally demanding. Even as hardware capabilities scale, the pursuit of modeling accuracy necessitates the design of memory-efficient attention kernels and high-performance learning algorithms. 长距离、全局关系：鉴于深度漏斗中个体连接的稀疏性，观察长距离、图连接实体和用户的共同模式并将其编码到表示中是非常有用的。在大规模图网络中捕获长距离关系对计算要求很高。即使硬件能力在不断扩展，对建模精度的追求也要求设计内存高效的注意力内核和高性能的学习算法。

Introducing Hierarchical Interest Representation

引入分层兴趣表示

Our latest research area is an upstream representation layer for learning universal, relational knowledge representations of users and ads entities. The representations capture users’ ads engagement patterns, absorb real-world world semantics, and cascade through multiple hierarchical granularities into latent-space projections at each level. 我们最新的研究领域是一个上游表示层，用于学习用户和广告实体的通用关系知识表示。这些表示捕获了用户的广告参与模式，吸收了现实世界的语义，并通过多个分层粒度级联到每一层的潜在空间投影中。

Based on the graph data structure, four design properties drive the system end to end: 基于图数据结构，四个设计属性驱动了整个系统的端到端运行：

Dimension Reduction: Hierarchical Interest Representation projects a raw graph into a configurable super-graph where each super-node is a learned latent interest primitive. User-ad edges that are sparse at the raw graph become meaningfully denser at the primitive interest graph. Inherently, the primitive interest graph is more stationary and stable in vocabulary, even though the ads business is more dynamic. 降维：分层兴趣表示将原始图投影到一个可配置的超图（super-graph）中，其中每个超节点都是一个学习到的潜在兴趣基元。在原始图中稀疏的用户-广告边在基元兴趣图中变得更有意义且更密集。本质上，尽管广告业务更加动态，但基元兴趣图在词汇表上更加平稳和稳定。

Knowledge Enrichment: Hierarchical Interest Representation enriches heterogeneous entities, in particular advertiser and product types, with multimodal content features such as text, images, and video. These features are pulled from structured page metadata and advertiser catalog attributes and processed through vision or language models. This extra information complements… 知识增强：分层兴趣表示利用文本、图像和视频等多模态内容特征来丰富异构实体，特别是广告主和产品类型。这些特征从结构化的页面元数据和广告主目录属性中提取，并通过视觉或语言模型进行处理。这些额外的信息补充了……