The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding

解析器早已知晓：约束解码中的轻量级偏差校正

Abstract: Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step. However, rigid masking distorts the model’s underlying probability distribution, often biasing generation toward valid but suboptimal outputs.

摘要： 语法约束解码（GCD）通过在每一步屏蔽不符合语法的 Token，强制语言模型（LM）生成语法有效的输出。然而，这种僵化的屏蔽机制会扭曲模型底层的概率分布，往往导致生成结果偏向于语法正确但并非最优的输出。

While online sampling restores this distribution, it requires computationally expensive iterative resampling. As a result, existing methods force a compromise between output quality and inference latency.

虽然在线采样（online sampling）可以恢复这种分布，但它需要计算开销巨大的迭代重采样。因此，现有方法往往不得不在输出质量和推理延迟之间做出妥协。

Our key insight is that the internal parser and lexer states inherently maintained during incremental parsing already encode future grammatical validity — exactly the information required to restore the LM’s true distribution.

我们的核心洞察在于：在增量解析过程中，内部解析器（parser）和词法分析器（lexer）状态本身就隐含了未来的语法有效性信息——这正是恢复语言模型真实分布所需的信息。

We propose a lightweight, offline-trained logit correction conditioned on this syntactic and lexical state together with candidate next tokens. Because these states are already computed as a necessary part of incremental parsing for masking, extracting them adds negligible overhead while leaving the base LM’s weights completely untouched.

我们提出了一种轻量级的、离线训练的 Logit 校正方法，该方法以这些语法和词法状态以及候选的下一个 Token 为条件。由于这些状态在增量解析的屏蔽过程中已经是计算必需的一部分，因此提取它们几乎不会增加额外开销，同时完全无需改动基础语言模型的权重。

Across several grammars, this correction substantially closes the gap between the masked distribution and the LM’s true distribution, consistently outperforming both masking and online sampling.

在多种语法测试中，这种校正方法显著缩小了屏蔽分布与语言模型真实分布之间的差距，在性能上始终优于单纯的屏蔽法和在线采样法。

Even its lightest variant, which relies on the candidate next token alone, still matches or exceeds both baselines: the next token itself carries an implicit lookahead, much like how parsers commonly use a lookahead token to resolve ambiguous decisions. By restoring the probability mass that masking removes, it reconciles the LM’s probabilistic integrity with grammar conformance.

即使是其最轻量级的变体（仅依赖候选的下一个 Token），其表现也依然能够匹配或超越上述两种基准方法：下一个 Token 本身就带有隐式的“前瞻”（lookahead），这与解析器通常使用前瞻 Token 来解决歧义决策的原理非常相似。通过恢复被屏蔽掉的概率质量，该方法成功地在语言模型的概率完整性与语法合规性之间达成了统一。