The Impact of Vocabulary Overlaps on Knowledge Transfer in Multilingual Machine Translation

词汇重叠对多语言机器翻译中知识迁移的影响

Abstract: Knowledge transfer, especially across related languages, has been found beneficial for multilingual neural machine translation (MNMT), but some aspects are still under-explored and deserve further investigation. 摘要： 研究发现，知识迁移（尤其是在相关语言之间）对多语言神经机器翻译（MNMT）大有裨益，但其中一些方面仍未得到充分探索，值得进一步研究。

A joint vocabulary is most often applied to form a uniform word embedding space, but since the impact of a disjoint vocabulary on model performance is far less studied, there is no consensus on how much knowledge transfer is mainly due to vocabulary overlap. 通常，研究人员会应用联合词汇表来构建统一的词嵌入空间。然而，由于不相交词汇表对模型性能的影响研究较少，目前对于知识迁移在多大程度上归因于词汇重叠尚未达成共识。

In this paper, we present systematic experiments with joint and disjoint vocabularies, and auxiliary languages related and unrelated to the source language. 在本文中，我们针对联合词汇表和不相交词汇表，以及与源语言相关和不相关的辅助语言，进行了系统性的实验。

We design this experiment in an out-of-domain setup in order to emphasize transfer and the impact of the auxiliary language. 我们采用域外（out-of-domain）设置来设计此实验，旨在强调知识迁移以及辅助语言的影响。

As expected, we yield better results with more extensive vocabulary overlaps typical for related languages, but our experiments also show that domain-match and language relatedness are more important than a joint vocabulary. 正如预期，当词汇重叠度较高（相关语言的典型特征）时，我们获得了更好的结果；但实验同时也表明，领域匹配度和语言相关性比联合词汇表更为重要。