DLLM-TTS: Block Discrete Diffusion Language Model for Text-to-Speech Synthesis

DLLM-TTS：用于语音合成的块离散扩散语言模型

Abstract: Current text-to-speech systems face a trade-off: autoregressive codec language models produce highly intelligible speech but require large-scale models and training data and decode tokens sequentially, while non-autoregressive approaches improve speed at the cost of linguistic accuracy.

摘要： 当前的文本转语音（TTS）系统面临一种权衡：自回归编解码器语言模型虽然能生成高度清晰的语音，但需要大规模的模型和训练数据，且必须按顺序解码 Token；而通过非自回归方法提高速度，往往又会以牺牲语言准确性为代价。

We present DLLM-TTS, a framework that formulates TTS as conditional block discrete diffusion over X-Codec2 neural audio codec tokens. The model decomposes sequences into blocks and applies masked diffusion within each block while processing blocks sequentially, learning both local acoustic coherence and global text-speech alignment.

我们提出了 DLLM-TTS，这是一个将 TTS 建模为基于 X-Codec2 神经音频编解码器 Token 的条件块离散扩散框架。该模型将序列分解为多个块，并在每个块内应用掩码扩散，同时按顺序处理这些块，从而学习局部声学连贯性和全局文本-语音对齐。

During inference, parallel token prediction within blocks enables efficient generation with a real-time factor (RTF) of 0.15. A 0.6B-parameter model trained on 20K hours achieves competitive performance on the Seed-TTS-eval benchmark, demonstrating that block discrete diffusion language models enable practical and data-efficient speech synthesis with parallel generation.

在推理过程中，块内的并行 Token 预测实现了高效生成，其实时因子（RTF）达到 0.15。一个在 2 万小时数据上训练的 6 亿参数模型在 Seed-TTS-eval 基准测试中表现出极具竞争力的性能，证明了块离散扩散语言模型能够通过并行生成实现实用且数据高效的语音合成。