Select, Don’t Train: The Benefits of Modular Entity Disambiguation with LLM-Based Selection

选择而非训练：基于大语言模型选择的模块化实体消歧的优势

Abstract: Entity Disambiguation (ED) is a key task for constructing and using knowledge graphs. State-of-the-art neural approaches commonly model ED as a single task, although it consists of two distinct subproblems: retrieving candidate entities and selecting the correct one given context. Dual-encoder models optimize for both within a shared embedding space, forcing representations to balance high-recall retrieval with fine-grained selection, and they require trained retrievers, which are costly to maintain as knowledge graphs change.

摘要： 实体消歧（Entity Disambiguation, ED）是构建和使用知识图谱的关键任务。目前最先进的神经方法通常将 ED 建模为单一任务，尽管它实际上由两个截然不同的子问题组成：检索候选实体，以及在给定上下文的情况下选择正确的实体。双编码器模型在共享嵌入空间内同时优化这两个目标，迫使表示在“高召回率检索”与“细粒度选择”之间寻求平衡，且需要训练检索器，而当知识图谱发生变化时，维护这些检索器的成本很高。

While recent work has begun to combine retrievers with LLM-based selectors, the interplay between the two stages has not been studied systematically. In this paper, we present a systematic comparison of retrieval strategies for candidate generation under a shared LLM-based selection stage, combining sparse retrieval (BM25), Web KB search, and a state-of-the-art trained dense retriever with several open- and closed-source LLMs.

尽管近期的研究已开始将检索器与基于大语言模型（LLM）的选择器相结合，但这两个阶段之间的相互作用尚未得到系统性研究。在本文中，我们针对共享 LLM 选择阶段下的候选生成检索策略进行了系统性比较，结合了稀疏检索（BM25）、Web KB 搜索以及最先进的训练密集型检索器，并测试了多种开源和闭源 LLM。

We show that, once selection is delegated to a capable LLM, training the retriever provides only modest additional value: a fully training-free BM25 retriever paired with an LLM selector reaches a new state of the art on the ZELDA benchmark, raising inKB micro-F1 from 82.3 to 86.3 (+4); pairing the same LLM with a trained dense retriever reaches 88.5.

我们证明，一旦将选择任务委托给能力出众的 LLM，训练检索器所带来的额外价值微乎其微：一个完全无需训练的 BM25 检索器与 LLM 选择器配合，在 ZELDA 基准测试上达到了新的领先水平，将 inKB micro-F1 从 82.3 提升至 86.3（+4）；而将同一 LLM 与训练过的密集检索器配合，则达到 88.5。

Decoupling retrieval from selection also exposes a limitation of current ED systems: when the correct entity is missing from retrieved candidates, they are forced to predict an incorrect entity. In contrast, our framework allows for abstention when retrieval failure is detected. In an evaluation setting that rewards correct abstentions, the training-free BM25 + LLM pipeline reaches 90.7 F1.

将检索与选择解耦也揭示了当前 ED 系统的一个局限性：当检索到的候选集中缺失正确实体时，系统被迫预测一个错误的实体。相比之下，我们的框架允许在检测到检索失败时进行“弃权”。在奖励正确弃权的评估设置中，无需训练的 BM25 + LLM 流水线达到了 90.7 的 F1 分数。