Measuring Where Intent Gets Lost in Agentic Smart Contract Synthesis

Computer Science > Artificial Intelligence arXiv:2610.37830 (cs) [Submitted on 27 Oct 2026] Title: Measuring Where Intent Gets Lost in Agentic Smart Contract Synthesis Authors: Abhinav Goel, Agostino Capponi, Alfio Gliozzo, Chaitya Shah.

计算机科学 > 人工智能 arXiv:2610.37830 (cs) [提交于 2026 年 10 月 27 日] 标题：衡量智能合约合成中意图丢失的位置 作者：Abhinav Goel, Agostino Capponi, Alfio Gliozzo, Chaitya Shah。

Abstract: AI agents can now write software from a plain-English description, but code that runs is not the same as code that does what was asked. This gap matters most for smart contracts, the programs behind Web3 applications that move money on a blockchain and cannot be fixed once deployed. We ask how to check whether an AI-written contract behaves as it should, and where the requested behavior gets lost.

摘要：人工智能代理现在可以根据简单的英语描述编写软件，但能运行的代码并不等同于能实现预期功能的代码。这种差距对于智能合约而言尤为重要，因为智能合约是 Web3 应用程序背后的程序，负责在区块链上转移资金，且一旦部署便无法修复。我们探讨了如何检查人工智能编写的合约是否按预期运行，以及所要求的行为在何处丢失。

We design an evaluation built on execution rather than opinion. We replay the same transactions on an AI-written contract and on the original it should reproduce and measure how often they agree. We test every grading tool on planted mistakes, compare against an answer key written without the description, and audit our own scorer until known wrong answers cannot pass.

我们设计了一种基于执行而非观点的评估方法。我们将相同的交易分别在人工智能编写的合约和其应复现的原始合约上进行重放，并衡量两者结果的一致性。我们通过植入错误来测试每一个评分工具，将其与在没有描述的情况下编写的答案键进行对比，并对我们自己的评分器进行审计，直到已知的错误答案无法通过为止。

Common methods, such as AI judges and standard analysis tools, miss obvious mistakes, and scoring choices matter too. Checking only the functions every system includes hid about half the difference between pipelines. We then propose an elicitation-first architecture. Before writing code, the agent asks the client a short checklist of questions, checks the contract against an independently written executable answer key, and returns disagreements to the client instead of silently rewriting code.

常见的方法（如人工智能裁判和标准分析工具）会遗漏明显的错误，评分选择也至关重要。仅检查每个系统包含的函数掩盖了流水线之间约一半的差异。因此，我们提出了一种“启发优先”的架构。在编写代码之前，代理会向客户询问一份简短的检查清单，根据独立编写的可执行答案键检查合约，并将分歧反馈给客户，而不是默默地重写代码。

Asking first matters far more than reviewing or patching afterwards, across three AI models, while automated security fixes often break requested behavior. A few well-chosen questions recover much of what a short description omits, and their value can be estimated without the original contract. To our knowledge, this is the first study in smart contract synthesis to gather missing requirements before generation and check the result against an independently written executable model.

在三个人工智能模型中，先进行询问比事后审查或修补要重要得多，而自动安全修复往往会破坏预期的行为。几个精心挑选的问题可以恢复简短描述中遗漏的大部分内容，且其价值无需原始合约即可估算。据我们所知，这是智能合约合成领域首个在生成前收集缺失需求，并根据独立编写的可执行模型检查结果的研究。

The lessons extend to any setting where AI agents turn requests into programs. We release our tools, 600 tasks and all results.

这些经验适用于任何人工智能代理将请求转化为程序的场景。我们发布了我们的工具、600 个任务以及所有结果。