LuxSQA: Ask Me in Luxembourgish with TTS-Augmented Spoken Question Answering

LuxSQA：用卢森堡语问我——基于 TTS 增强的口语问答系统

Abstract: Spoken Question Answering (SQA) remains largely focused on high-resource languages and carefully recorded speech, limiting the reach of speech-LLM methods in low-resource settings. This paper investigates whether text-to-speech (TTS) can provide task-specific training data for Luxembourgish SQA without requiring a large human-recorded QA corpus.

摘要： 口语问答（SQA）目前主要集中在高资源语言和精心录制的语音上，这限制了语音大模型（speech-LLM）方法在低资源环境下的应用。本文探讨了在无需大规模人工录制问答语料库的情况下，文本转语音（TTS）技术能否为卢森堡语 SQA 提供特定任务的训练数据。

Starting from existing text-based QA resources, we translate questions into Luxembourgish, synthesize spoken questions with multiple TTS systems, and pair them with textual answers. We train a parameter-efficient SLAM-style architecture that connects a frozen Whisper encoder to frozen multilingual LLM backends through a learned projector and LoRA adapters.

我们从现有的基于文本的问答资源出发，将问题翻译成卢森堡语，利用多种 TTS 系统合成口语问题，并将其与文本答案配对。我们训练了一种参数高效的 SLAM 风格架构，通过学习到的投影器（projector）和 LoRA 适配器，将冻结的 Whisper 编码器连接到冻结的多语言大模型后端。

We compare MMS-TTS, Qwen3-TTS, and OmniVoice variants, including single-source corpora of about 48k questions and a 4TTS multi-source mix of approximately 230k questions. Evaluation on LLAMA-LB-Test with two real Luxembourgish speaker conditions shows that multi-source and voice-design-based synthetic training configurations yield the strongest SQA performance.

我们对比了 MMS-TTS、Qwen3-TTS 和 OmniVoice 的变体，包括约 4.8 万个问题的单源语料库，以及约 23 万个问题的 4TTS 多源混合语料库。在包含两名真实卢森堡语说话人的 LLAMA-LB-Test 测试集上的评估结果表明，基于多源和语音设计（voice-design）的合成训练配置能够产生最强的 SQA 性能。

The results also show that no-reference TTS quality scores do not monotonically predict downstream QA performance, indicating that synthetic speech must be evaluated as task-specific training data rather than only as natural-sounding audio.

研究结果还显示，无参考 TTS 质量评分并不能单调地预测下游的问答性能，这表明合成语音必须被视为特定任务的训练数据来评估，而不仅仅是作为听起来自然的音频来评估。