Accelerating Visual On-Policy Distillation with Batched Speculative Jacobi Rollouts

通过批量投机雅可比展开加速视觉在线策略蒸馏

Abstract: Visual on-policy distillation (OPD) improves the training of compact visual autoregressive models by learning from trajectories generated by the current student. However, these online rollouts are still produced token by token with autoregressive decoding, which adds substantial cost to every on-policy training step.

摘要： 视觉在线策略蒸馏（OPD）通过学习当前学生模型生成的轨迹，改进了紧凑型视觉自回归模型的训练。然而，这些在线展开（rollouts）仍需通过自回归解码逐个 token 生成，这为每个在线策略训练步骤增加了巨大的成本。

Speculative Jacobi Decoding (SJD) provides an alternative because it can process multiple tokens in parallel without an auxiliary draft model, but the original method is designed for single-sequence inference. We introduce HB-SJD, a batched SJD rollout backend for visual OPD.

投机雅可比解码（SJD）提供了一种替代方案，因为它可以在无需辅助草稿模型的情况下并行处理多个 token，但原始方法是为单序列推理设计的。我们引入了 HB-SJD，这是一种用于视觉 OPD 的批量 SJD 展开后端。

HB-SJD allows each image to advance independently according to its own decoding progress, while images at different sequence positions are still verified in batched model forwards. As images finish, HB-SJD switches between Full and Compact execution to reduce the cost of later rollout rounds.

HB-SJD 允许每张图像根据其自身的解码进度独立推进，同时处于不同序列位置的图像仍可在批量模型前向传播中进行验证。随着图像生成完成，HB-SJD 会在“完整（Full）”和“紧凑（Compact）”执行模式之间切换，以降低后续展开轮次的成本。

HB-SJD only replaces the student rollout backend and leaves the teacher, distillation objective, and optimization procedure unchanged. Experiments with LlamaGen show that HB-SJD substantially reduces rollout and end-to-end training time while preserving the generation quality of the distilled student.

HB-SJD 仅替换了学生模型的展开后端，而保持教师模型、蒸馏目标和优化过程不变。在 LlamaGen 上的实验表明，HB-SJD 在保持蒸馏后学生模型生成质量的同时，大幅缩短了展开时间和端到端的训练时间。