Whispering Wisely: Smart Subset Selection for ASR Fine-Tuning

查看原文 / View Original 编辑 / Edit

Computer Science > Computation and Language arXiv:2610.32997 (cs) [Submitted on 5 Oct 2026] Title: Whispering Wisely: Smart Subset Selection for ASR Fine-Tuning Authors: Roi Yozevitch, Or Haim Anidjar.

计算机科学 > 计算与语言 arXiv:2610.32997 (cs) [提交于 2026 年 10 月 5 日] 标题：明智的耳语：用于 ASR 微调的智能子集选择作者：Roi Yozevitch, Or Haim Anidjar。

Abstract: Automatic Speech Recognition (ASR) models such as OpenAI’s Whisper have substantially improved speech recognition quality but remain expensive to fine-tune, especially when large amounts of transcribed audio are available. This makes it challenging for many practitioners to adapt these models to new languages or domains.

摘要：诸如 OpenAI 的 Whisper 等自动语音识别 (ASR) 模型显著提高了语音识别质量，但微调成本依然高昂，尤其是在拥有大量转录音频的情况下。这使得许多从业者难以将这些模型适配到新的语言或领域。

In this work, we introduce an embedding-based subset selection approach for selecting smaller training sets for ASR fine-tuning. Using Whisper’s pretrained encoder, we map each audio sample into a fixed-dimensional embedding space and apply a greedy k-center algorithm to select diverse and representative subsets of the training data.

在这项工作中，我们引入了一种基于嵌入的子集选择方法，用于为 ASR 微调选择较小的训练集。利用 Whisper 的预训练编码器，我们将每个音频样本映射到一个固定维度的嵌入空间，并应用贪婪 k-中心算法来选择训练数据中多样化且具有代表性的子集。

We evaluate this approach on both Hebrew and Danish, showing that k-center selection often improves upon random sampling, particularly at smaller subset sizes. An analysis of lexical diversity reveals that k-center subsets capture a broader range of linguistic content than random subsets of the same size, which may help explain their observed effectiveness.

我们在希伯来语和丹麦语上评估了该方法，结果表明 k-中心选择通常优于随机采样，特别是在子集规模较小时。对词汇多样性的分析显示，与相同规模的随机子集相比，k-中心子集捕获了更广泛的语言内容，这可能有助于解释其观察到的有效性。

Overall, our results demonstrate that embedding-based subset selection provides a practical strategy for ASR subset selection in the two evaluated languages, with the clearest recognition benefits observed at smaller sample budgets.

总体而言，我们的结果表明，基于嵌入的子集选择为这两种评估语言的 ASR 子集选择提供了一种实用的策略，且在较小的样本预算下观察到了最明显的识别收益。