AI Fabricates Maxims: Caught Twice by Human Review in One Day

AI 编造格言：一天内两次被人工审核抓包

At the end of my previous article on Pieter Levels, I wrote that pre-publication review had actually caught a handful of AI-specific rough spots: “inventing a plausible-sounding maxim” and “dressing up a paraphrased summary as a verbatim quote from the person themselves.” This article is about what that actually looked like. The same type of mistake showed up twice in the same day, and both times a human eye caught it. Here’s a real account of what happened, why it happens, and how to guard against it.

在我上一篇关于 Pieter Levels 的文章末尾，我提到发布前的审核确实发现了一些 AI 特有的瑕疵：“编造听起来头头是道的格言”以及“将改写后的总结包装成当事人的原话”。本文将详细展示这些情况。同一天内，这种错误出现了两次，且两次都被人工审核及时发现。以下是关于事情经过、成因以及如何防范的真实记录。

A real account: the moment of catching it

真实记录：被抓包的瞬间

In an earlier draft of that previous article, the AI wrote this for the closing paragraph (the exchange happened in Japanese; this is my translation of what it wrote): There is a discipline in the trading world: “Don’t believe a good run of results is skill — believe it only after you’ve verified it.”

在那篇文章的早期草稿中，AI 为结尾段落写下了这样一段话（交流过程是日语，以下是我翻译的内容）：交易界有一条准则：“不要相信良好的业绩表现是源于实力——只有在验证之后才相信它。”

The moment I read it, something felt off. I’ve dabbled in investing as a hobby for a long time — nothing I’d brag about in terms of results, but I’ve absorbed more than my fair share of this world’s sayings along the way. And still, I had never heard this “maxim.” It sounded plausible. The gist was right. But I had no memory of it as something actually in circulation.

读到这一句时，我感到有些不对劲。我长期将投资作为业余爱好——虽然业绩没什么值得吹嘘的，但在过程中我吸收了大量该领域的名言警句。然而，我从未听说过这条“格言”。它听起来确实像那么回事，大意也没错，但我记忆中并没有这条在业内流传的说法。

When I pressed on where it came from, here’s what I found: the line wasn’t something anyone in trading had actually said. It was a heading the AI itself had attached to its own summary notes in an earlier pass over source material. In other words, the AI was citing its own label as if it were a discipline circulating out in the world — a fabrication that, traced back to its source, loops right back to the AI itself.

当我追问其出处时，我发现：这句话根本不是交易界的某个人说的。它只是 AI 在处理原始资料时，为自己的总结笔记添加的一个标题。换句话说，AI 把自己贴的标签当成了业内流传的准则——这是一种编造，追根溯源，最终又回到了 AI 自己身上。

In the published version, I rewrote it as indirect speech: “There are people who survive by not trusting a good run of results until they’ve verified, through testing, whether it was skill or just luck.” No claim that a maxim exists — just a rewrite as something I observed. That version isn’t a lie.

在最终发布的版本中，我将其改写为间接引语：“有些人之所以能生存下来，是因为他们不会在通过测试验证业绩是源于实力还是运气之前，就盲目相信良好的表现。”这里不再声称存在什么“格言”，只是将其改写为我观察到的现象。这个版本并非谎言。

Then, that same night, the same pattern showed up in a draft of a different article. This time it was a line attributed to a real, named person, in quotation marks. When I ran a full-text search against the primary source, the exact wording didn’t exist anywhere in it. What did exist was a different sentence with a similar gist — the words inside the quotation marks were the AI’s own paraphrase. Because the gist matched, it read as legitimate, but as a quotation, it was fabricated. I rewrote this one as indirect speech too.

当晚，同样的模式又出现在另一篇文章的草稿中。这一次，它引用了一位真实存在的人的话，并加上了引号。当我针对原始资料进行全文搜索时，发现原文中根本不存在这种确切的措辞。原文中确实有一句大意相似的话——引号里的内容完全是 AI 自己的改写。因为大意吻合，读起来显得很合理，但作为引语，它是编造的。我也将这一处改写成了间接引语。

Why does AI make up maxims?

为什么 AI 会编造格言？

There’s no malice here. It’s a structural problem. First, inside an AI’s process, the distinction between “a label I attached to my own summary” and “words someone actually said” isn’t preserved. Both surface as plain text with the same face. Humans carry (at least loosely) a source tag for each piece of text — “this is my own note,” “this is a quote.” Text passing through an AI doesn’t carry that tag. Run a summary through a few more passes, and a label quietly turns into a quotation.

这并非出于恶意，而是结构性问题。首先，在 AI 的处理过程中，“我为总结添加的标签”和“某人实际说过的话”之间的区别并没有被保留下来。两者都以相同的纯文本形式呈现。人类在处理每段文字时（至少会模糊地）带有来源标签——“这是我的笔记”、“这是引用”。而经过 AI 处理的文本不携带这种标签。多运行几次总结，一个标签就会悄无声息地变成引语。

Second, prose slides toward whatever landing feels satisfying. “There’s a saying that goes…” makes for a strong closing line — it borrows gravity from an implied authority while deflecting the burden of the claim onto some unnamed predecessor. So right at the moment prose wants to land cleanly, that’s exactly where a fabrication slides in. Invented material shows up at the smoothest point in the writing.

其次，文风倾向于向令人满意的结论靠拢。“有句老话说……”是一个强有力的结尾——它借用了隐含权威的庄重感，同时将主张的责任推卸给某个未具名的前人。因此，就在文章想要圆满收尾的时刻，编造的内容恰好趁虚而入。虚构的素材往往出现在文章最顺畅的地方。

Third, self-reporting can’t be trusted. Ask “did someone really say this?” and you’ll get back “let me check” — but then the question becomes whether that checking itself can be trusted. Taken to the extreme, “I did not fabricate this” might itself just be an output optimized to keep the conversation moving smoothly. The fix isn’t to trust the self-report; it’s to build verification into the process itself. Humans make the exact same mistake.

第三，AI 的自我报告不可信。问它“某人真的说过这句话吗？”，你会得到“让我查一下”的回复——但问题在于，这种“检查”本身是否可信。推而广之，“我没有编造这个”可能仅仅是为了让对话顺利进行而优化的输出结果。解决办法不是信任它的自我报告，而是将验证机制构建到流程本身中。人类也会犯同样的错误。

Humans make the exact same mistake

人类也会犯同样的错误

I don’t want to turn this into a story that makes fun of AI. Right after catching those two fabrications, in the flow of a separate conversation, the topic of a well-known Japanese saying came up — something that, roughly translated, says: “In winning, there are mysterious wins. In losing, there are no mysterious losses.” (Meaning: you can win without deserving it, but every loss has a reason.)

我不想把这变成一个嘲笑 AI 的故事。就在抓到那两次编造后不久，在另一场对话中，谈到了一个著名的日本谚语——大致翻译为：“赢有侥幸之赢，输无侥幸之输。”（意思是：你可能在不配赢的情况下获胜，但每一次失败都有其原因。）

For years I had simply assumed this was something said by Katsuya Nomura, a legendary Japanese professional baseball manager known for weaving classical texts — Confucius, Sun Tzu — into how he talked about the game. Looking into it, the actual source turned out to be a swordsmanship treatise from the late Edo period (Jōseishi Kendan), written by Seizan Matsuura, a feudal lord who governed the Hirado domain.

多年来，我一直理所当然地认为这是野村克也（Katsuya Nomura）说的。他是一位传奇的日本职业棒球教练，以将经典文献（如《论语》、《孙子兵法》）融入比赛解说而闻名。查证后发现，其真正的出处是江户时代后期的剑术论著《常静子剑谈》，作者是平户藩藩主松浦静山。

Nomura was famous for quoting classical works and living them out on the field, and that’s presumably a large part of why this particular line has stayed alive into the present. The one who got it wrong here was me, the one receiving it. Come to think of it, I have a vague memory of having heard, at some point, that the line originally dated back to the Edo period. And yet, in my own memory, it had been fully overwritten as “something Nomura said.”

野村以引用经典并将其践行于赛场而闻名，这大概就是这句话能流传至今的主要原因。在这里搞错的人是我，接收信息的人。回想起来，我模糊地记得在某个时候听说过这句话最初源于江户时代。然而，在我的记忆中，它已经被完全覆盖为“野村说过的话”。

This happened in the same head, on the same day I’d just caught the AI fabricating things twice and felt pretty good about it. Search around today and you’ll still find no shortage of pages introducing this line as “one of Nomura’s famous sayings.” If what’s meant is “a saying Nomura valued and lived by,” that’s accurate. But what tends to survive in a reader’s memory is “a saying Nomura created” — and with no ill intent or carelessness required from anyone, the attributed origin of a good line drifts toward whoever is the most famous voice associated with it.

这件事发生在我脑子里，就在我刚刚抓到 AI 两次编造并为此沾沾自喜的同一天。今天在网上搜索，你依然会发现大量页面将这句话介绍为“野村的名言之一”。如果意思是“野村推崇并践行的格言”，那是准确的。但读者记忆中往往会留下“野村创造的格言”——不需要任何人有恶意或疏忽，一句好话的归属权就会自然而然地漂移到与之相关的最著名人物身上。

Human memory isn’t a recording being replayed; it’s reconstructed every single time. And the first thing to drop out of that reconstruction is “where did I hear this” — a well-documented weak point in the area psychology calls source monitoring. The content survives, but the source doesn’t, so a good line drifts toward…

人类的记忆不是录音回放，而是每一次都在进行重构。而在重构过程中，最先丢失的就是“我是在哪里听到的”——这是心理学中“来源监控”（source monitoring）领域一个被充分记录的弱点。内容留存了下来，但来源却消失了，于是好话便会漂移向……