Reviser: Revision-Capable Text Generation via Autoregressive Cursor Actions

Reviser：通过自回归光标操作实现具备修订能力的文本生成

Abstract: Revision-capable generation is appealing because it can insert or revise earlier content, but many non-autoregressive and edit-based approaches obtain this flexibility through repeated sequence-level computation. We propose Reviser, a decoder-only Transformer that generates a response as a sequence of cursor-relative actions on a mutable canvas.

摘要： 具备修订能力的生成模型因其能够插入或修改先前内容而极具吸引力，但许多非自回归和基于编辑的方法往往通过重复的序列级计算来实现这种灵活性。我们提出了 Reviser，这是一个仅解码器的 Transformer 模型，它通过在可变画布上执行一系列光标相关操作来生成响应。

At each step, Reviser predicts exactly one action token: INSERT(token), MOVE($\Delta$), or STOP, and is autoregressive over edit-history actions rather than final text order. This design enables genuinely non-monotonic generation while preserving a simple next-action interface.

在每一步中，Reviser 仅预测一个动作标记：INSERT(token)（插入）、MOVE($\Delta$)（移动）或 STOP（停止），并且它是基于编辑历史动作而非最终文本顺序进行自回归的。这种设计在保持简单的“下一动作”接口的同时，实现了真正的非单调生成。

On a continuation benchmark, Reviser is strongly preferred to SEDD and MDLM in our arena evaluations, and trajectory statistics confirm that the model performs frequent backward moves and mid-canvas insertions rather than merely emulating end-append decoding.

在续写基准测试中，我们的竞技场评估显示 Reviser 明显优于 SEDD 和 MDLM。轨迹统计数据证实，该模型执行了频繁的向后移动和画布中间插入操作，而不仅仅是模拟末尾追加式的解码。

Against size-matched autoregressive baselines, Reviser is competitive at both the 100M and 300M scales. Under our shared FLOPs convention, Reviser also requires substantially less inference compute than representative multi-pass refinement and diffusion-style baselines.

与同等规模的自回归基线模型相比，Reviser 在 100M 和 300M 参数规模下均具有竞争力。在统一的 FLOPs（浮点运算次数）标准下，Reviser 所需的推理计算量也远低于典型的多轮精炼和扩散式基线模型。