Comparative Analysis of Multilingual Pre-trained Models for Nepali Automatic Speech Recognition

尼泊尔语自动语音识别多语言预训练模型对比分析

Abstract: Multilingual pretrained models nominally support Nepali, yet no controlled benchmark has compared them under a single fine-tuning protocol. We fine-tune six pretrained models (XLSR-53, IndicWav2Vec, MMS-1B, Whisper-Medium, Whisper-Large-v3-Turbo, and Conformer-Hi) spanning CTC self-supervised, autoregressive encoder-decoder, and hybrid Conformer-CTC architectures, on the OpenSLR SLR54 Nepali corpus (~165 hours) using identical preprocessing, splits, optimizer, and family-matched learning-rate schedules.

摘要： 虽然多语言预训练模型名义上支持尼泊尔语，但目前尚无受控基准测试在统一的微调协议下对它们进行比较。我们对六种预训练模型（XLSR-53、IndicWav2Vec、MMS-1B、Whisper-Medium、Whisper-Large-v3-Turbo 和 Conformer-Hi）进行了微调，这些模型涵盖了 CTC 自监督、自回归编码器-解码器以及混合 Conformer-CTC 架构。我们在 OpenSLR SLR54 尼泊尔语语料库（约 165 小时）上，使用相同的预处理、数据集划分、优化器和同族匹配的学习率计划进行了实验。

We evaluate Word Error Rate (WER), Character Error Rate (CER), and Real-Time Factor (RTF) on three independent test sets (OpenSLR, FLEURS, Common Voice). Whisper-Large-v3-Turbo (14.76% WER) and IndicWav2Vec (14.89% WER) tie at the top despite a 9x parameter gap and 40x pretraining-data gap, providing direct empirical evidence that language-family proximity in pretraining can substitute for raw scale for in-domain Nepali.

我们在三个独立的测试集（OpenSLR、FLEURS、Common Voice）上评估了词错误率 (WER)、字符错误率 (CER) 和实时因子 (RTF)。尽管参数量相差 9 倍，预训练数据量相差 40 倍，Whisper-Large-v3-Turbo（14.76% WER）和 IndicWav2Vec（14.89% WER）仍并列第一。这提供了直接的实证证据，表明在预训练中，语言家族的亲缘关系可以弥补尼泊尔语领域内数据规模的不足。

CTC decoders run up to 29x faster than autoregressive Whisper at the same accuracy, flipping the practical deployment preference toward CTC under any latency budget. Massively multilingual pretraining (MMS-1B) yields the smallest out-of-domain degradation on FLEURS (+12.55 pp), indicating that scale buys robustness rather than peak in-domain accuracy. The resulting benchmark provides the first standardized, multi-model, efficiency-aware reference numbers for Nepali ASR.

在相同准确率下，CTC 解码器的运行速度比自回归的 Whisper 快 29 倍，这使得在任何延迟预算下，实际部署时都更倾向于选择 CTC。大规模多语言预训练 (MMS-1B) 在 FLEURS 数据集上的域外性能下降最小（+12.55 pp），这表明模型规模带来的是鲁棒性，而非领域内的峰值准确率。由此产生的基准测试为尼泊尔语 ASR 提供了首个标准化、多模型且兼顾效率的参考数据。