FormulaSPIN: Self-Play Fine-Tuning for Natural Language to Spreadsheet Formula Generation

FormulaSPIN：用于自然语言转电子表格公式生成的自我博弈微调框架

Abstract: Spreadsheet applications are used by hundreds of millions worldwide, yet writing formulas remains a significant barrier. Existing approaches rely on static supervised data, which quickly saturates on limited annotations. In this paper, we introduce FORMULASPIN, a self-play framework that breaks the ceiling of supervised fine-tuning by enabling iterative self-improvement without any additional data.

摘要： 全球有数亿人使用电子表格应用程序，但编写公式仍然是一个巨大的障碍。现有的方法依赖于静态的监督数据，这些数据在有限的标注下很快就会达到性能瓶颈。在本文中，我们引入了 FORMULASPIN，这是一个自我博弈（self-play）框架，通过在无需任何额外数据的情况下实现迭代式的自我改进，打破了监督微调（SFT）的性能上限。

Vanilla SPIN fails on this task: it uniformly penalizes every non-matching output, so execution-equivalent alternatives are punished as negatives in one example while serving as ground truth in another, producing contradictory gradients. Our framework resolves this by exploiting formula generation’s unique advantage: binary executability provides implicit supervision that separates semantic errors from valid stylistic variants.

传统的 SPIN 方法在此任务上表现不佳：它统一惩罚所有与标准答案不匹配的输出，因此，执行结果等效的替代方案在某些示例中被作为负样本惩罚，而在另一些示例中却作为标准答案，从而产生了矛盾的梯度。我们的框架通过利用公式生成任务的独特优势解决了这一问题：二进制的可执行性提供了隐式监督，能够将语义错误与有效的风格变体区分开来。

We frame training as a two-player game in which the main player learns to prefer ground-truth formulas over those from its previous version, while execution feedback sorts outputs into distinct granularities—enabling an adaptive curriculum that shifts from semantic correctness to stylistic refinement. To further increase accuracy, we incorporate ExecVote, a semantic-level voting mechanism that naturally handles multiple valid formulations.

我们将训练过程构建为一个双人博弈，其中主模型学习优先选择标准答案公式，而不是其先前版本生成的公式；同时，执行反馈将输出结果按不同粒度进行分类，从而实现了一个从语义正确性转向风格优化的自适应课程学习。为了进一步提高准确性，我们引入了 ExecVote，这是一种语义层面的投票机制，可以自然地处理多种有效的公式表达。

Experiments on multiple benchmarks demonstrate that FORMULASPIN achieves state-of-the-art performance, with 74.9% exact match and 87.1% execution accuracy on NL2FORMULA, matching models trained with additional preference annotations while outperforming both traditional SFT and frontier proprietary models. These findings underscore self-play’s potential to tackle scarce data tasks and open the door to extending it beyond executable domains.

在多个基准测试上的实验表明，FORMULASPIN 达到了最先进的性能，在 NL2FORMULA 数据集上实现了 74.9% 的精确匹配率和 87.1% 的执行准确率。这一表现不仅媲美了使用额外偏好标注训练的模型，还超越了传统的 SFT 方法和前沿的专有模型。这些发现强调了自我博弈在处理稀缺数据任务方面的潜力，并为将其扩展到可执行领域之外提供了可能。