Contextual Correction of Parallel Draftsfor Speculative Decoding

Computer Science > Computation and Language arXiv:2610.12570 (cs) [Submitted on 8 Oct 2026] Title: Contextual Correction of Parallel Drafts for Speculative Decoding Authors: Roy Eisenstadt, Harel Lidar, Tal Gonen, Omer Luxembourg, Itamar Zimerman, Lior Wolf, Eli David, Ilan Naiman.

计算机科学 > 计算与语言 arXiv:2610.12570 (cs) [提交于 2026 年 10 月 8 日] 标题：用于推测解码的并行草稿上下文校正 作者：Roy Eisenstadt, Harel Lidar, Tal Gonen, Omer Luxembourg, Itamar Zimerman, Lior Wolf, Eli David, Ilan Naiman。

Abstract: Speculative decoding accelerates large language model inference by using a lightweight drafter to propose multiple tokens that are subsequently verified in parallel by the target model. Recent diffusion-based drafters make this paradigm particularly attractive: deep attention-based neural backbones provide strong predictions, while drafting multiple future positions in parallel avoids the sequential cost of autoregressive generation.

摘要：推测解码通过使用轻量级草稿生成器提出多个标记，随后由目标模型并行验证，从而加速大语言模型的推理。近期基于扩散的草稿生成器使这种范式极具吸引力：基于深度注意力的神经骨干网络提供了强大的预测能力，同时并行草稿化多个未来位置避免了自回归生成的顺序成本。

This parallelism, however, comes at a cost. Tokens within a draft block are predicted without access to the realized identities of preceding draft tokens, removing intra-block dependencies that would naturally be available under autoregressive generation. We introduce DCOCO (DFlash Contextual Corrector), a lightweight mechanism that restores these dependencies while preserving parallel drafting.

然而，这种并行性是有代价的。草稿块内的标记在预测时无法获取先前草稿标记的实际身份，从而消除了在自回归生成下自然存在的块内依赖关系。我们引入了 DCOCO (DFlash 上下文校正器)，这是一种在保持并行草稿化的同时恢复这些依赖关系的轻量级机制。

Rather than committing to each position’s top prediction, DCOCO retains multiple candidate tokens and evaluates their contextual compatibility across adjacent draft positions in parallel, enabling contextually informed token selection at low additional cost. Across four verifier backbones and ten benchmarks, DCOCO achieves the highest average throughput in every evaluated configuration, spanning greedy and stochastic decoding with and without extended reasoning, with speedups of up to 6.6x over vanilla decoding.

DCOCO 不会锁定每个位置的最佳预测，而是保留多个候选标记，并并行评估它们在相邻草稿位置之间的上下文兼容性，从而以较低的额外成本实现基于上下文的标记选择。在四个验证器骨干网络和十个基准测试中，DCOCO 在所有评估配置中均实现了最高的平均吞吐量，涵盖了包含或不包含扩展推理的贪婪解码和随机解码，相比原始解码速度提升高达 6.6 倍。

Its advantage persists under concurrent serving: DCOCO outperforms the strongest competing method by 6-23% across 4-32 concurrent requests, while other sequential correctors fall below uncorrected DFlash at these loads. These results demonstrate that recovering intra-block dependencies through parallel contextual scoring delivers consistent acceleration across decoding regimes and serving loads.

其优势在并发服务中依然存在：在 4 到 32 个并发请求中，DCOCO 的表现比最强的竞争方法高出 6-23%，而其他顺序校正器在这些负载下表现甚至不如未校正的 DFlash。这些结果表明，通过并行上下文评分恢复块内依赖关系，可以在各种解码方案和服务器负载下提供持续的加速。