Reinforcement Learning for Data-Efficient Code-Switched ASR

用于数据高效代码转换自动语音识别（ASR）的强化学习

Abstract: Audio-language models can be prompted for code-switched speech, but their decoding is not optimized for code-switching and often fails at language boundaries. We propose a practical reinforcement learning with verifiable rewards recipe for data-efficient adaptation of audio-language models to code-switched ASR using group relative policy optimization, combining an error rate reward with a script fidelity reward that penalizes wrong writing systems and a two-pass draft-and-refinement procedure.

摘要： 音频-语言模型可以通过提示（prompt）处理代码转换（code-switched）语音，但其解码过程并未针对代码转换进行优化，且在语言边界处经常失效。我们提出了一种实用的、带有可验证奖励的强化学习方案，通过组相对策略优化（group relative policy optimization），实现音频-语言模型向代码转换自动语音识别（ASR）的数据高效适配。该方案结合了错误率奖励、惩罚错误书写系统的脚本保真度奖励，以及一个两阶段的“草稿与精修”流程。

Using Qwen2-Audio as a reproducible testbed across 10 language pairs, training on only TTS code-switched speech, we show that RLVR with 10% of the data matches LoRA supervised fine-tuning trained on the full dataset, with the largest gains on typologically distant pairs. The error rate reward eliminates translation errors while the script fidelity reward separately reduces script contamination without degradation. These gains transfer zero-shot to a human-recorded code-switching corpus.

我们以 Qwen2-Audio 作为可复现的测试平台，在 10 个语言对上进行了实验，仅使用 TTS（语音合成）生成的代码转换语音进行训练。结果表明，使用 10% 数据量的 RLVR（带可验证奖励的强化学习）效果即可媲美使用全量数据集进行 LoRA 监督微调的结果，且在语言类型差异较大的语言对上提升最为显著。错误率奖励消除了翻译错误，而脚本保真度奖励则在不降低性能的前提下减少了脚本混淆。这些性能增益在人类录制的代码转换语料库上实现了零样本（zero-shot）迁移。