Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models
Adaptive Multi-Step Lookahead Decoding for Diffusion Language Models
扩散语言模型的自适应多步前瞻解码
Abstract: Masked diffusion language models (DLMs) enable parallel text generation by iteratively refining masked tokens, offering a promising alternative to autoregressive decoding. Recent lookahead-based decoding methods improve the accuracy—efficiency trade-off by exploring future decoding states before committing token updates. However, existing approaches mainly rely on shallow one-step lookahead, which optimizes immediate information gain but can be suboptimal for longer-horizon decoding trajectories.
摘要： 掩码扩散语言模型（DLM）通过迭代细化掩码标记来实现并行文本生成，为自回归解码提供了一种极具前景的替代方案。近期基于前瞻（lookahead）的解码方法通过在提交标记更新前探索未来的解码状态，改善了准确性与效率之间的权衡。然而，现有方法主要依赖于浅层的单步前瞻，这种方法虽然优化了即时信息增益，但在长程解码轨迹中可能并非最优。
Meanwhile, we find that a naive extension for deeper lookahead is also ineffective, as fixed-depth rollout introduces additional computation and cannot adapt to heterogeneous intermediate decoding states. Thus, in this work, we propose AdaLook, an adaptive lookahead framework for DLM decoding. AdaLook dynamically determines whether to continue rollout based on candidate-score variance and further enables branch expansion when intermediate rollout states require additional exploration.
同时，我们发现简单地扩展为更深层的前瞻也是无效的，因为固定深度的展开（rollout）会引入额外的计算开销，且无法适应异构的中间解码状态。因此，在这项工作中，我们提出了 AdaLook，这是一个用于 DLM 解码的自适应前瞻框架。AdaLook 根据候选分数方差动态决定是否继续展开，并在中间展开状态需要进一步探索时，支持分支扩展。
This design avoids unnecessary deep rollout while allowing the decoder to re-trigger lookahead from informative intermediate states. Experiments on various benchmarks and models demonstrate that AdaLook achieves a better accuracy—decoding steps trade-off than existing one-step lookahead decoding methods.
这种设计避免了不必要的深度展开，同时允许解码器从信息丰富的中间状态重新触发前瞻。在各种基准测试和模型上的实验表明，与现有的单步前瞻解码方法相比，AdaLook 在准确性与解码步数之间实现了更好的权衡。
Paper Details:
- Authors: Yingqian Cui, Wei Deng, Lantao Mei, Hang Li, Charu C. Aggarwal, Hui Liu, Yue Xing
- arXiv ID: 2607.15655
- Subjects: Computation and Language (cs.CL); Machine Learning (cs.LG)
论文详情：
- 作者： Yingqian Cui, Wei Deng, Lantao Mei, Hang Li, Charu C. Aggarwal, Hui Liu, Yue Xing
- arXiv ID: 2607.15655
- 学科分类： 计算与语言 (cs.CL)；机器学习 (cs.LG)