Think Before You Link: Rarity, Reasoning, and Retrieval in Multilingual Entity Linking

链接前请三思：多语言实体链接中的稀有性、推理与检索

Abstract: Multimodal entity linking grounds entity mentions in text and images to knowledge-base entries. These systems degrade on rare entities, but prior work measures rarity primarily through popularity-based metrics such as pageviews. We broaden this view using knowledge-graph structural metrics that capture how well an entity is documented and connected. These metrics identify many rare entities that popularity metrics miss.

摘要： 多模态实体链接将文本和图像中的实体提及关联到知识库条目。这些系统在处理稀有实体时性能会下降，但以往的研究主要通过页面浏览量等基于流行度的指标来衡量稀有性。我们利用知识图谱结构指标拓宽了这一视角，这些指标能够捕捉实体的文档记录完善程度及其连接情况。这些指标识别出了许多流行度指标所遗漏的稀有实体。

Across the resulting rare-entity slices, state-of-the-art accuracy drops by 15.4-39.9%, showing that different rarity definitions expose different failure modes. To address these failures, we introduce a simple, training-free framework in which a reasoning-capable vision-language model iteratively searches and reasons over Wikipedia, gathering evidence dynamically.

在由此产生的稀有实体切片中，最先进模型的准确率下降了 15.4% 至 39.9%，这表明不同的稀有性定义会暴露不同的失效模式。为了解决这些问题，我们引入了一个无需训练的简单框架，其中具备推理能力的视觉语言模型会在维基百科上进行迭代搜索和推理，从而动态收集证据。

Controlled experiments show that reasoning and retrieval are complementary. Reasoning alone does not significantly improve accuracy on rare entities. Retrieval without reasoning improves rare-entity accuracy but can hurt overall accuracy. Their combination performs best.

对照实验表明，推理和检索是互补的。仅靠推理并不能显著提高稀有实体的准确率。在没有推理的情况下进行检索虽然能提高稀有实体的准确率，但可能会损害整体准确率。两者的结合效果最佳。

On MERLIN, a multilingual multimodal entity linking benchmark over five languages (Hindi, Indonesian, Japanese, Tamil, Vietnamese), our best system improves over the state of the art by 6.9% overall and by up to 23.3% on rare-entity slices. We release MERLIN-Rare, rare-entity test slices for targeted evaluation, with our framework.

在涵盖五种语言（印地语、印尼语、日语、泰米尔语、越南语）的多语言多模态实体链接基准测试 MERLIN 上，我们表现最好的系统在整体准确率上比现有最先进水平提高了 6.9%，在稀有实体切片上最高提升了 23.3%。我们发布了 MERLIN-Rare（用于针对性评估的稀有实体测试切片）以及我们的框架。