Why AI Keeps Making the Same Coding Mistakes—And How Teaching It Pain Gives It Wisdom

为什么 AI 总是在犯同样的编程错误——以及如何通过“教它感受痛苦”赋予其智慧

“What did you do for AI?” “I brought the scars.” — Randal L. Schwartz “你为 AI 做了什么？”“我带来了伤疤。”—— Randal L. Schwartz

SPOCK: “Dr. Daystrom, a computer is incapable of standard creative thought. It must be programmed. How did you instruct the M-5 to construct new data banks, to create new circuitry?” 史波克：“戴斯特姆博士，计算机无法进行标准的创造性思维。它必须被编程。你是如何指导 M-5 构建新的数据库、创造新的电路的？”

DAYSTROM: “By using a new method, Mr. Spock. I developed a method of impressing memory engrams upon computer circuits. The M-5 thinks.” 戴斯特姆：“通过一种新方法，史波克先生。我开发了一种将记忆印记（memory engrams）压印到计算机电路上的方法。M-5 会思考。”

SPOCK: “Whose engrams did you use?” 史波克：“你用了谁的印记？”

DAYSTROM: (With proud emphasis) “Why, mine, of course.” — Star Trek: The Original Series, “The Ultimate Computer” (1968) 戴斯特姆：（自豪地强调）“当然是我的。”——《星际迷航：原初系列》，“终极计算机”（1968）

1. The Straight-A Intern Who Crashes on Day One

1. 第一天就崩溃的“全优生”实习生

Anyone who has experimented with modern AI coding assistants knows the eerie sensation of talking to a genius. You describe a complex algorithmic problem, and in three seconds, the model streams out clean, beautifully indented code. It uses idiomatic expressions, adds helpful comments, and passes whatever toy unit test you throw at it. It looks like the ultimate software engineer. 任何尝试过现代 AI 编程助手的人，都知道那种与天才对话的诡异感觉。你描述一个复杂的算法问题，三秒钟内，模型就会输出整洁、缩进优美的代码。它使用地道的表达方式，添加有用的注释，并通过你扔给它的任何玩具级单元测试。它看起来就像终极软件工程师。

Then you deploy it into a real, living software project at 3 AM. Suddenly, that same genius starts making baffling, rookie mistakes: It forgets what happens when a user clicks a button twice in rapid succession. It sets up a real-time data stream that never gets closed, quietly leaking memory until the mobile application chokes and dies. It assumes every network request returns in under 50 milliseconds, crashing the interface the moment a user enters a subway tunnel. 然后，你在凌晨三点把它部署到一个真实、活跃的软件项目中。突然，同一个天才开始犯令人困惑的菜鸟错误：它忘记了当用户快速连续点击两次按钮时会发生什么；它建立了一个永远不会关闭的实时数据流，悄悄地泄露内存，直到移动应用程序卡死崩溃；它假设每个网络请求都在 50 毫秒内返回，导致用户一进入地铁隧道，界面就直接崩溃。

Worst of all: if you correct the mistake on Monday, it will apologize profusely, fix the line—and then make the exact same mistake again on Thursday in a different file. Why does this happen? We call this The Straight-A Intern Paradox. 最糟糕的是：如果你在周一纠正了这个错误，它会连连道歉，修复那一行代码——然后在周四又在另一个文件中犯下完全相同的错误。为什么会这样？我们称之为“全优生实习生悖论”。

Today’s frontier AI models are trained on virtually the entire public internet—millions of tutorials, classroom assignments, blog posts, and homework repositories. In those environments, the weather is always sunny. Inputs are always valid, databases never disconnect, and nobody leaves an app open in the background for three weeks. In computer science, we call this the “happy path.” 当今的前沿 AI 模型几乎是在整个公共互联网上训练出来的——数以百万计的教程、课堂作业、博客文章和作业库。在这些环境中，天气总是晴朗的。输入总是有效的，数据库从不断开连接，也没有人会把一个应用程序在后台挂着三周。在计算机科学中，我们称之为“快乐路径”（happy path）。

Textbooks teach the happy path because pedagogical clarity requires simplicity. But real-world production engineering is 95% about the “sad path”—the messy, chaotic, asynchronous universe of hardware glitches, race conditions, memory constraints, and unpredictable human behavior. When an AI writes code, it naturally drifts toward the most common patterns in its training data: the beginner-friendly, sunny-day code that looks plausible on paper, but shatters under the weight of real-world reality. 教科书教授“快乐路径”是因为教学清晰度需要简单化。但现实世界的生产工程 95% 都在处理“悲伤路径”（sad path）——即硬件故障、竞态条件、内存限制和不可预测的人类行为所构成的混乱、异步的宇宙。当 AI 编写代码时，它会自然地倾向于训练数据中最常见的模式：那种对初学者友好、在纸面上看起来合理，但在现实世界的重压下却会分崩离析的“晴天代码”。

2. Why Telling an AI “Don’t Do That” Never Works

2. 为什么告诉 AI “别那样做”永远没用

When engineering teams discover this flaw, their first instinct is to write a rules document. You might create a checklist or a prompt instruction file filled with stern warnings: “Do not use mutable lists in state containers.” “Always remember to cancel stream subscriptions.” “Never mutate shared state across asynchronous boundaries.” It sounds reasonable. But in practice, it almost always fails. 当工程团队发现这个缺陷时，他们的第一直觉是编写一份规则文档。你可能会创建一个清单或提示词指令文件，里面写满了严厉的警告：“不要在状态容器中使用可变列表。”“永远记得取消流订阅。”“永远不要在异步边界之间修改共享状态。”这听起来很合理。但在实践中，它几乎总是失败。

There are two fundamental reasons why passive prompt rules collapse: 被动提示规则失效有两个根本原因：

1. The “Pink Elephant” Trap: If someone tells you: “Whatever you do, do not think of a pink elephant,” what is the very first image that flashes into your mind? A pink elephant. Large language models work on statistical token association. When you flood their instructions with negative imperatives (“Don’t do X, don’t touch Y”), the tokens for X and Y dominate their attention. Under the cognitive pressure of solving a complex, multi-step problem, models frequently fixate on the forbidden pattern, or subtly rationalize why their current situation is a “special exception.”

2. “粉色大象”陷阱：如果有人告诉你：“无论如何，不要去想一只粉色的大象”，你脑海中闪过的第一个画面是什么？就是一只粉色的大象。大语言模型基于统计标记关联工作。当你用否定指令（“不要做 X，不要碰 Y”）充斥它们的指令时，X 和 Y 的标记就会占据它们注意力的主导地位。在解决复杂、多步骤问题的认知压力下，模型经常会固着在被禁止的模式上，或者微妙地合理化为什么它们当前的情况是一个“特殊例外”。

3. Statistical Amnesia (“Groundhog Day”): Every time you open a new conversation with an AI agent, it wakes up with total amnesia. It has no memory of the grueling three-hour debugging session you suffered through yesterday. It does not remember the frantic rollback, the broken release, or the angry customer emails. To the AI, every task is Day One. It lives in a permanent Groundhog Day, doomed to repeat the same well-intentioned blunders forever.

4. 统计性失忆（“土拨鼠之日”）：每次你与 AI 智能体开启一段新对话时，它醒来时都处于完全失忆状态。它不记得你昨天经历的那场痛苦的三个小时调试过程。它不记得那次疯狂的回滚、那次崩溃的发布，或者那些愤怒的客户邮件。对 AI 来说，每一项任务都是“第一天”。它生活在永恒的“土拨鼠之日”里，注定要永远重复同样的、出于好意的错误。

3. The Hot Stove: What Veteran Humans Have That AI Lacks

3. 热炉子：资深人类拥有而 AI 缺乏的东西

How do veteran human programmers avoid these traps? If you sit an engineer with 30 or 40 years of experience in front of a pull request, something fascinating happens. Before they have even compiled the code or stepped through the logic line by line, they will suddenly recoil, point at a block of code, and say: “No. Don’t do that. That’s going to blow up in production.” 资深人类程序员是如何避免这些陷阱的？如果你让一位拥有 30 或 40 年经验的工程师面对一个合并请求（Pull Request），会发生一些迷人的事情。在他们编译代码或逐行检查逻辑之前，他们会突然退缩，指着某块代码说：“不，别那样做。那会在生产环境中炸掉的。”

Where does that intuition come from? In 1994, renowned neuroscientist Antonio Damasio published a groundbreaking theory called the Somatic Marker Hypothesis. Damasio demonstrated that human decision-making is not purely an exercise in cold, step-by-step logic. When you experience a painful failure—such as touching a red-hot stove as a child—your body and brain record a visceral “somatic marker.” The next time your hand drifts toward that stove, you do not sit down to calculate thermodynamics. Your nervous system triggers an involuntary physical recoil before conscious analytical thought even kicks in. Your body remembers the burn. 这种直觉从何而来？1994 年，著名神经科学家安东尼奥·达马西奥（Antonio Damasio）发表了一项名为“躯体标记假说”（Somatic Marker Hypothesis）的开创性理论。达马西奥证明，人类的决策并非纯粹是冷冰冰的、循序渐进的逻辑练习。当你经历过痛苦的失败——比如小时候摸过滚烫的炉子——你的身体和大脑会记录下一个本能的“躯体标记”。下一次你的手靠近那个炉子时，你不会坐下来计算热力学。在有意识的分析思维介入之前，你的神经系统就会触发一种不由自主的身体退缩。你的身体记住了那种灼烧感。

Veteran software engineers survive because their brains are covered in metaphorical scar tissue: The scar from the production outage of 1998 caused by an unindexed database query. The scar from the memory leak of 2011 that drained phone batteries in twenty minutes. The scar from the race condition of 2020 that corrupted financial ledgers. AI models have no scars. They feel no pain, carry no somatic markers, and experience no visceral recoil. They approach a red-hot stove with joyful, enthusiastic curiosity every single day. If we want autonomous AI agents to build software we can actually trust, we cannot just give them bigger rulebooks. We have to give them scars. 资深软件工程师之所以能生存下来，是因为他们的大脑覆盖着隐喻性的伤疤组织：1998 年因未索引数据库查询导致的生产中断留下的伤疤；2011 年导致手机电池二十分钟耗尽的内存泄漏留下的伤疤；2020 年导致财务账簿损坏的竞态条件留下的伤疤。AI 模型没有伤疤。它们感觉不到痛苦，没有躯体标记，也不会产生本能的退缩。它们每天都带着快乐、热情的求知欲去接近滚烫的炉子。如果我们希望自主 AI 智能体构建我们真正能信任的软件，我们不能仅仅给它们更大的规则手册。我们必须给它们伤疤。

4. The Anatomy of a Synthetic Scar

4. 合成伤疤的解剖学

In our research, we developed a system to equip AI agents with artificial somatic markers, which we call… 在我们的研究中，我们开发了一种系统，为 AI 智能体配备人工躯体标记，我们称之为……