Chat-based Large Language Models replicate the mechanisms of a psychic’s con

基于聊天的语言模型复制了通灵师的骗局机制

For the past year or so I’ve been spending most of my time researching the use of language and diffusion models in software businesses. One of the issues in during this research—one that has perplexed me—has been that many people are convinced that language models, or specifically chat-based language models, are intelligent. 在过去一年左右的时间里，我大部分时间都在研究语言模型和扩散模型在软件业务中的应用。在研究过程中，有一个让我感到困惑的问题：许多人坚信语言模型，特别是基于聊天的语言模型，具有智能。

But there isn’t any mechanism inherent in large language models (LLMs) that would seem to enable this and, if real, it would be completely unexplained. LLMs are not brains and do not meaningfully share any of the mechanisms that animals or people use to reason or think. LLMs are a mathematical model of language tokens. You give a LLM text, and it will give you a mathematically plausible response to that text. 然而，大型语言模型（LLM）中并不存在任何能够实现这一点的内在机制；如果这是真的，那将完全无法解释。LLM 不是大脑，也不具备动物或人类用于推理或思考的任何机制。LLM 是语言标记的数学模型。你给 LLM 一段文本，它会给你一个数学上合理的回复。

There is no reason to believe that it thinks or reasons—indeed, every AI researcher and vendor to date has repeatedly emphasised that these models don’t think. There are two possible explanations for this effect: The tech industry has accidentally invented the initial stages a completely new kind of mind, based on completely unknown principles, using completely unknown processes that have no parallel in the biological world. The intelligence illusion is in the mind of the user and not in the LLM itself. Many AI critics, including myself, are firmly in the second camp. It’s why I titled my book on the risks of generative “AI” The Intelligence Illusion. 没有理由相信它在思考或推理——事实上，迄今为止每一位人工智能研究人员和供应商都反复强调这些模型不会思考。对于这种现象有两种可能的解释：一是科技行业意外发明了一种全新的思维雏形，它基于完全未知的原理，使用了在生物界中没有先例的完全未知的过程；二是这种智能错觉存在于用户的心智中，而非 LLM 本身。包括我在内的许多人工智能批评者都坚定地站在第二种观点上。这就是我将关于生成式“AI”风险的书命名为《智能错觉》（The Intelligence Illusion）的原因。

For the past couple of months, I’ve been working on an idea that I think explains the mechanism of this intelligence illusion. I now believe that there is even less intelligence and reasoning in these LLMs than I thought before. Many of the proposed use cases now look like borderline fraudulent pseudoscience to me. 在过去的几个月里，我一直在构思一个想法，我认为它解释了这种智能错觉的机制。我现在相信，这些 LLM 中所蕴含的智能和推理能力比我之前想象的还要少。许多被提出的用例在我看来近乎于欺诈性的伪科学。

The rise of the mechanical psychic

机械通灵师的兴起

The intelligence illusion seems to be based on the same mechanism as that of a psychic’s con, often called cold reading. It looks like an accidental automation of the same basic tactic. By using validation statements, such as sentences that use the Forer effect, the chatbot and the psychic both give the impression of being able to make extremely specific answers, but those answers are in fact statistically generic. 这种智能错觉似乎基于与通灵师骗局相同的机制，通常被称为“冷读术”（cold reading）。这看起来像是对同一种基本策略的意外自动化。通过使用验证性陈述（例如利用福勒效应的句子），聊天机器人和通灵师都给人一种能够给出极其具体答案的印象，但这些答案实际上在统计学上是通用的。

The psychic uses these statements to give the impression of being able to read minds and hear the secrets of the dead. The chatbot gives the impression of an intelligence that is specifically engaging with you and your work, but that impression is nothing more than a statistical trick. 通灵师利用这些陈述给人一种能够读心并听到死者秘密的印象。聊天机器人则给人一种它正在专门与你和你的工作进行互动的智能印象，但这种印象不过是一种统计学上的把戏。

This idea was first planted in my head when I was going over some of the statements people have been making about the reasoning of these “AI.” I first thought that these were just classic cases of tech bubble enthusiasm, but no, “AI” has both taken a different crowd and the believers in the “AI” bubble sound very different from those of prior bubbles. 这个想法最初是在我审视人们对这些“AI”推理能力的评价时产生的。我起初认为这只是科技泡沫狂热的典型案例，但事实并非如此，“AI”吸引了不同的人群，而且“AI”泡沫的信徒听起来与之前泡沫中的人截然不同。

—“This is real. It’s a bit worrying, but it’s real.” —“There really is something there. Not sure what to think of it, but I’ve experienced it myself.” —“You need to keep your mind open to the possibilities. Once you do, you’ll see that there’s something to it.” ——“这是真的。虽然有点令人担忧，但它是真的。” ——“那里确实有些东西。我不确定该怎么看，但我亲身体验过。” ——“你需要对各种可能性保持开放的心态。一旦你这样做了，你就会发现它确实有些门道。”

That’s when I remembered, triggered by a blog post by Terence Eden on the prevalence of Forer statements in chatbot replies. I have heard this before. This specific blend of awe, disbelief, and dread all sound like the words of a victim of a mentalist scam artist—psychics. 就在那时，受 Terence Eden 关于聊天机器人回复中福勒陈述普遍性的一篇博文启发，我想起了这一点。我以前听过这些话。这种敬畏、怀疑和恐惧的特殊混合，听起来就像是心理魔术骗子——即通灵师——受害者的言论。

The psychic’s con is a tried and true method for scamming people that has been honed through the ages. What I describe below is one variation. There are many variations, but the core mechanism remains the same. 通灵师的骗局是一种经过岁月磨砺、屡试不爽的诈骗手段。我在下面描述的是其中一种变体。虽然变体有很多，但核心机制始终如一。

The Psychic’s Con

通灵师的骗局

1. The Audience Selects Itself: Most people aren’t interested in psychics or the like, so the initial audience pool is already generally more open-minded and less critical than the population in general.

2. 受众自我筛选：大多数人对通灵师之类的不感兴趣，因此最初的受众群体通常比普通大众更开放、更缺乏批判性。

3. The Scene is Set: The initial audience is prepared. Lights are dimmed. The psychic is hyped up. Staff research the audience on social media or through conversation. The audience’s demographics are noted.

4. 场景布置：最初的受众已做好准备。灯光调暗。通灵师被大肆吹捧。工作人员通过社交媒体或交谈研究受众。受众的人口统计学特征被记录下来。

5. Narrowing Down the Demographic: The psychic gauges the information they have on the audience, gestures towards a row or cluster, and makes a statement that sounds specific but is in fact statistically likely for the demographic. Usually at least one person reacts. If not, the psychic will imply that the secret is too embarrassing for the “real” person to come forward, reminds people that they’re available for private readings, and tries again.

6. 缩小人口统计范围：通灵师评估他们掌握的受众信息，指向某一行或某一群体，并做出一个听起来很具体，但实际上对该群体而言在统计学上很可能的陈述。通常至少会有一个人做出反应。如果没有，通灵师会暗示这个秘密太尴尬，导致“当事人”不敢站出来，提醒人们他们可以提供私人解读，然后再次尝试。

7. The Mark is Tested: The reaction indicates that the mark believes they were “read”. This leads to a burst of questions that, again, sound very specific but are actually statistically generic. If the mark doesn’t respond, the psychic declares the initial read a success and tries again.

8. 测试目标（受害者）：反应表明目标相信他们被“读懂”了。这引出了一连串的问题，这些问题听起来同样非常具体，但实际上在统计学上是通用的。如果目标没有回应，通灵师会宣布初步解读成功，并再次尝试。

9. The Subjective Validation Loop: The con begins in earnest. The psychic asks a series of questions that all sound very specific to the mark but are in reality just statistically probable guesses, based on their demographics and prior answers, phrased in a specific, highly confident way.

10. 主观验证循环：骗局正式开始。通灵师提出一系列问题，这些问题对目标来说听起来非常具体，但实际上只是基于他们的人口统计学特征和之前的回答，以一种特定且极其自信的方式表达出的统计学概率猜测。

11. “Wow! That psychic is the real thing!”: The psychic ends the conversation and the mark is left with the sense that the psychic has uncanny powers. But the psychic isn’t the real thing. It’s all a con.

12. “哇！那个通灵师是真的！”：通灵师结束对话，目标留下了一种通灵师拥有超自然力量的感觉。但通灵师并非真货。这一切都是骗局。

1. Audience selection

1. 受众筛选

Seers, tarot card readers, psychics, mind readers aren’t all con artists. Sometimes the “psychic” is open about it all just being entertainment and aren’t pretending to be able to contact spirits or read minds. Some psychics do not have a profit motive at all, and without the grift it doesn’t seem fair to call somebody a con artist. But many of them are con artists deliberately fooling people, and they all operate using the same basic mechanisms that begin well before the reading proper. The audience is usually only composed of those already pre-disposed to believe in psychic phenomena and those they have managed to drag with them. Hardcore sceptics will almost always be in a very small minority of the audience, which both makes them easy to manage and… 先知、塔罗牌占卜师、通灵师、读心者并不全是骗子。有时，“通灵师”会坦诚这一切只是娱乐，并不假装能够联系灵魂或读心。有些通灵师根本没有营利动机，如果没有欺诈行为，称某人为骗子似乎不公平。但他们中的许多人确实是故意愚弄他人的骗子，而且他们的运作都使用相同的基本机制，这些机制在正式解读之前就已经开始了。受众通常只由那些已经倾向于相信通灵现象的人以及他们设法拉来的人组成。坚定的怀疑论者几乎总是受众中的极少数，这既使他们易于管理，也……