Reviewing AI Code Is Not A Viable Argument

审查 AI 代码并非一个可行的论点

18 Jul 2026 - Thomas Depierre 2026年7月18日 - Thomas Depierre

I am a skeptic of the utility of LLM in software development. It is not because of IP laws problems (even if they are highly problematic), nor is it for ecological and ressource comsumptions reasons. It is not even for “they are all crap” reasons. My problem with LLM Coding Assistants is that I cannot see, in the face of the scientific evidence, how they can help someone write code better or faster. 我对于大语言模型（LLM）在软件开发中的效用持怀疑态度。这并非因为知识产权法律问题（尽管这些问题确实非常棘手），也不是出于生态和资源消耗的考量，甚至不是因为“它们全是垃圾”这种理由。我对 LLM 编程助手的问题在于，面对科学证据，我无法看出它们如何能帮助人们写出更好或更快的代码。

And the thing that irritate me is that it seems that none of the proponents of LLM Coding Assistants seems to ever address this problem and this evidence when they defend their tooling choices. Worse, it seems that they give fuel to my arguments everytime they write a rebuke of the skeptics. So let’s look at what I have a problem with, how empirical scientific research support that view, how proponents of LLM Coding Assistants could show this is not a problem and then how right now they are doing the actual opposite. 令我恼火的是，当 LLM 编程助手的支持者们为他们的工具选择辩护时，似乎从不正面回应这一问题和证据。更糟糕的是，每当他们撰文反驳怀疑论者时，似乎反而为我的观点提供了论据。因此，让我们来看看我所质疑的内容、实证科学研究如何支持这一观点、LLM 编程助手的支持者们本可以如何证明这不是问题，以及他们目前实际上是如何背道而驰的。

Note: This piece was written nearly a year ago, hence why you may find vocabulary, like “Coding Assistants” that has been mostly replaced at this point. Sadly, I have not been able to find a term in the current vocabulary around genAI used to code that covers all the use case. So I kept the “Coding Assistants” wording. 注：本文写于近一年前，因此你可能会发现像“编程助手（Coding Assistants）”这样在当前已被大部分取代的词汇。遗憾的是，我无法在当前围绕生成式 AI 编程的术语中找到一个能涵盖所有用例的词，所以我保留了“编程助手”这一说法。

The Intern Problem

实习生问题

The fundamental problem of LLM Coding Assistants that my criticism center on is their relatively high risk of getting things wrong. For all kind of reasons, some structural to how LLM works and other more akin to the interfaces we provide to interact with them, LLM Coding Assistants get things wrong. It can be hallucinations, typos, simply doing something that is not linked to the task demanded, going into a different path, etc etc. 我所批评的 LLM 编程助手的根本问题在于它们出错的风险相对较高。由于各种原因——有些源于 LLM 的工作原理，有些则与我们提供的交互界面有关——LLM 编程助手总是会出错。这可能是幻觉、拼写错误、单纯地执行与任务无关的操作、走入歧途等等。

A lot of people I have talked about that experimented with LLM Coding Assistants explains that they feel “Like an intern”. Like an intern, you should not expect too much from them, you should expect that everything they do will be more or less wrong, and that they have no idea what they are doing, but are highly enthusiastic. I see they never got me as an intern. I was definitely not enthusiastic. 许多尝试过 LLM 编程助手的人告诉我，他们感觉“就像带实习生一样”。像对待实习生一样，你不应该对它们抱有太高期望，你应该预料到它们所做的一切或多或少都会出错，它们根本不知道自己在做什么，但却表现得非常热情。看来他们没带过我这样的实习生，我当时可一点也不热情。

And their answer to that problem, the one you will see all over the internet, is simple. You just do the same thing that you do with interns and junior developers in your team. No, they do not mean you put everything they did in the bin and forget about it. What they mean is that you should review all the code yourself. I mean, you are the human that know better. And you are the one responsible for the code anyway. And on top of this, you are doing that for all the code that get into your codebase anyway, you do not let code get in without a review, right? 而他们针对这一问题的回答（你在互联网上随处可见）很简单：你只需要像对待团队里的实习生和初级开发人员那样去做就行了。不，他们不是让你把他们写的东西全扔进垃圾桶。他们的意思是，你应该亲自审查所有的代码。毕竟，你是更懂行的那个人，而且你本来就要对代码负责。况且，无论如何你都会审查进入代码库的所有代码，你不会让未经审查的代码进入库中，对吧？

What We Means When We Say Reviews

我们所说的“审查”是指什么

First of all, I want to be clear here. There are different practices in the litterature and in the profession grouped under the term “review”. So let’s be explicit here. Seeing the degree of (mis)trust and potential mistakes there, we should not accept the kind of “lightweight and heavily distributed” reviews that we see the most in our industry as the standard for LLM Coding Assistants supervision by a professional developer. 首先，我想明确一点。在文献和行业中，有许多不同的实践都被归类为“审查”。所以我们必须明确定义。鉴于其中存在的信任（或不信任）程度以及潜在的错误，我们不应将行业中最常见的“轻量级且高度分布式”的审查，作为专业开发人员监督 LLM 编程助手的标准。

They are not a bad thing to do, nor are they inefficient, but they have been shown in the litterature mostly as good to distribute knowledge of changes and as a way to enforce all kind of surface level rules. For AI Coding Asssistants, we will need a proper “code review”. Not something as formal and complex as reviews of old, by committees, painstakingly checking every line one by one over a few hours. But still, we want something quite involved and complete. After all, these are interns writing sometimes highly complex code. And if there is something we know in software, it is that the devil can be in the details. 这些审查并非不好，也不低效，但文献表明它们主要适用于传播变更知识以及强制执行各种表层规则。对于 AI 编程助手，我们需要的是真正的“代码审查”。这不需要像过去那种由委员会进行、耗时数小时逐行检查的审查那样正式且复杂，但我们仍然需要一种相当深入且完整的审查。毕竟，这些“实习生”有时编写的是高度复杂的代码。而在软件开发中我们深知一点：魔鬼往往藏在细节中。

The Limits Of Reviews

审查的局限性

Without going into some philosophical depth of reviewing as a practice, there is a glaring problem in this idea. From all the research we have, we have learned, empirically a few things about code reviews. And the evidence is relatively solid here, within reasonable limits. You will see that these do not matter there. A review that last more than 1h is too long. A review that has to be effective cannot be more than 400LOC at a time, in that time. 无需深入探讨审查作为一种实践的哲学层面，这个想法本身就存在一个明显的问题。根据我们现有的所有研究，我们从实证角度了解了关于代码审查的一些事实。在合理的范围内，这些证据是相当扎实的。你会发现，这些事实在 AI 审查中至关重要：超过 1 小时的审查时间太长了；有效的审查在这一时间内不能超过 400 行代码（LOC）。

Empirical research has shown that reviews that are longer than 1h quickly reach diminishing returns whatever is the size of the code being reviewed. So this is not only that people cannot find bugs anymore after 1h because they already thoroughly reviewed most of the code. No, it is more linked to the fact that after 1h at that level of attention, people start getting tired, bored and simply need some time off. 实证研究表明，无论审查的代码量大小，超过 1 小时的审查很快就会出现边际效益递减。这不仅是因为人们在 1 小时后因为已经彻底审查了大部分代码而无法再发现 Bug，更重要的是，在保持这种专注度 1 小时后，人们开始感到疲劳、厌倦，仅仅是需要休息一下。

Of note is the total absence of research as far as I could find, on the recovery time needed between review sessions of 1h. So I canot tell you how frequently someone could do 1h review sessions. But we could probably accept an extreme maximum limit of a handful per day. Which is probably far more than most people could do, I would probably put the average at 2, but eh. That is still in the right ballpark. 值得注意的是，据我所知，目前完全没有关于 1 小时审查间隔所需恢复时间的研究。所以我无法告诉你一个人能多频繁地进行 1 小时的审查。但我们或许可以接受每天几次的极端上限。这可能远超大多数人的承受能力，我估计平均水平在 2 次左右，但无论如何，这大致在合理的范围内。

The second limit that has been seen in empirical research is that speed, that is number of Lines Of Code per Hour (LOC/H) is highly variable, mostly depending on the context of the code, the kind of code being reviewed, experience, knowledge and the rest of your expected reasons. But something that is regularly pointed out is that, even if there is no hard cut off, it seems that a maximum of 400 LOC/H is a good maximum speed acceptable for efficience, as nearly no review above this speed seems particularly effective in the empirical data at finding and flagging defects. 实证研究中发现的第二个局限是，审查速度（即每小时代码行数，LOC/H）变化很大，主要取决于代码上下文、代码类型、经验、知识以及其他预期因素。但经常被指出的一点是，即使没有硬性界限，400 LOC/H 似乎是保证效率的可接受最高速度，因为在实证数据中，几乎没有任何超过此速度的审查能有效发现并标记缺陷。

What It Means For LLMs

这对 LLM 意味着什么

So, if we combine the claim that the solution to LLM Coding Assistants problems is to review the code, with the empirical evidence from scientific research on code reviews, what do we get? For every 400 LOC written by a LLM Coding Assistants (at best, less for code that are… 因此，如果我们结合“LLM 编程助手问题的解决方案是审查代码”这一主张，以及关于代码审查的科学实证研究，我们会得出什么结论？对于 LLM 编程助手编写的每 400 行代码（这还是最好的情况，对于某些代码来说甚至更少……）