Embedding Initialization for Unseen Low-resource Languages in Multilingual NMT: A Case Study on Limbum-English Translation

多语言神经机器翻译中未见低资源语言的嵌入初始化：以林布姆语-英语翻译为例

Abstract: Multilingual neural machine translation models such as NLLB-200 cover 200 languages but leave thousands unsupported, including most Grassfields Bantu languages of Cameroon. When fine-tuning these models for an unseen language, practitioners must choose a proxy language token, yet no principled method exists for this selection.

摘要： 诸如 NLLB-200 之类的多语言神经机器翻译模型涵盖了 200 种语言，但仍有数千种语言不受支持，其中包括喀麦隆的大多数草地班图语（Grassfields Bantu languages）。当针对一种未见语言对这些模型进行微调时，从业者必须选择一个代理语言标记（proxy language token），然而目前尚无针对此选择的原则性方法。

We implemented an embedding initialization strategy where a language token is the average of embeddings from multiple typologically related languages already in the model. We evaluate this approach on Limbum-to-English translation using a parallel corpus of 8,837 sentence pairs from New Testament text and a bilingual dictionary.

我们实现了一种嵌入初始化策略，即将语言标记设为模型中已有的多种类型相关语言的嵌入平均值。我们使用包含 8,837 个句子对的平行语料库（来自《新约》文本和双语词典），对林布姆语（Limbum）到英语的翻译进行了评估。

We compare models: NLLB-200 zero-shot (chrF2++ = 12.5), a Transformer trained from scratch (chrF2++ = 14.5), NLLB-200 fine-tuned with a Swahili proxy token (chrF2++ = 47.3), and NLLB-200 with our averaged embedding initialization (chrF2++ = 46.7). We find that the multi-language initialization achieves performance comparable to the best single-language proxy. Both NLLB-200 variants improve over the from-scratch baseline by over 32 chrF2++ points.

我们对比了以下模型：NLLB-200 零样本模型（chrF2++ = 12.5）、从零开始训练的 Transformer 模型（chrF2++ = 14.5）、使用斯瓦希里语作为代理标记微调的 NLLB-200（chrF2++ = 47.3），以及使用我们平均嵌入初始化方法的 NLLB-200（chrF2++ = 46.7）。研究发现，多语言初始化所达到的性能与最佳单语言代理相当。两种 NLLB-200 变体均比从零开始训练的基准模型提高了 32 个以上的 chrF2++ 点。

These results show that multilingual transfer is the dominant factor in extremely low-resource Bantu translation while eliminating the need for heuristic proxy selection. However, all systems fail to preserve tonal diacritics, highlighting an open challenge. We make our dataset and code available to support further research.

这些结果表明，多语言迁移是极低资源班图语翻译中的主导因素，同时消除了对启发式代理选择的需求。然而，所有系统都未能保留声调变音符号，这凸显了一个亟待解决的挑战。我们已公开数据集和代码，以支持后续研究。