The contagion of fear

The contagion of fear

恐惧的传染

I have a difficult confession to make — a shameful incident that I have never spoken of, even to those closest to me. Late one night in my first year at university and having found computer science to be my life’s calling, I was in the back of the large computer science lab, talking with some fellow underclassmen. We were snidely decrying the lack of technological understanding in the broader population, with the kind of arrogance and hubris that youth and precociousness can uniquely summon.

我有一个难以启齿的忏悔——那是一段我从未向任何人提起过的羞耻往事，即便是对我最亲近的人也是如此。那是我大一的一个深夜，当时我已认定计算机科学是我毕生的追求。我坐在大型计算机实验室的后排，与几位低年级同学聊天。我们带着年轻人特有的那种傲慢与自负，尖酸刻薄地嘲讽大众缺乏技术素养。

Somewhere in this discussion, a terrible idea was hatched: what if we went next door to the general computer lab, and told the humanities students there that a computer virus was spreading through their machines? Would they believe it? To my everlasting regret, we acted on this ugly urge: we went next door to a packed lab of students working on term papers, and — with synthetic alarm in our voices — loudly announced that a virus had escaped the computer science lab, and that they all needed to eject their floppy disks to prevent its spread.

在讨论中，一个可怕的念头产生了：如果我们去隔壁的公共计算机实验室，告诉那里的文科生说他们的机器正在传播电脑病毒，会怎样？他们会相信吗？令我终生遗憾的是，我们真的付诸了行动：我们走进隔壁挤满学生、正在赶期末论文的实验室，用伪装出的惊恐语调大声宣布，计算机实验室里跑出了一种病毒，他们必须立刻弹出软盘以防止病毒扩散。

What happened next was bedlam — the technological equivalent of shouting fire in a crowded theater. People panicked: they not only ejected floppies, but they powered machines off; yanked cables out of already-powered off machines; screamed; ran out of the room. We immediately realized that we had done something awful, but found that we couldn’t contain the wildfire that we had lit. We tried to explain that it was “a prank”; some (rightfully!) responded with indignant fury (what was wrong with us that we would do such a thing?!) while others simply didn’t believe us: once the fear had taken root, we couldn’t eradicate it.

接下来的场面一片混乱——这在技术领域无异于在拥挤的剧院里大喊“着火了”。人们陷入了恐慌：他们不仅弹出了软盘，还强行关机；甚至把已经关掉的机器的电缆也拔了出来；有人尖叫，有人冲出房间。我们立刻意识到自己闯了大祸，却发现根本无法扑灭这把我们亲手点燃的野火。我们试图解释这只是“一个恶作剧”；有些人（理所当然地！）愤怒地回应（我们到底是怎么了，竟然做出这种事？！），而另一些人根本不相信我们：一旦恐惧扎了根，我们就无法将其根除。

The aftermath is a blur — but it was bad. It was the week leading up to finals, and people who had powered their computers off had lost work. They were furious, and we were in a ton of trouble. The next morning, we found ourselves in the office of the facilities director (a famously imposing presence), who lectured us sternly as we stared at our feet. He made us each write letters of apology (easily done as my shame was earnest), telling us to give him two copies: one to be sent to those whom we had wronged, the other to be kept in his drawer.

事后的记忆有些模糊，但结果很糟糕。那是期末考试前的一周，那些关掉电脑的人丢失了工作成果。他们怒不可遏，我们也陷入了大麻烦。第二天早上，我们被叫到了设施主管（一位以严厉著称的人）的办公室，他严厉地训斥我们，而我们只能盯着自己的脚尖。他让我们每人写一封道歉信（因为我内心深感羞愧，这很容易做到），并要求我们交两份：一份寄给那些被我们伤害的人，另一份留在他那里。

He told us in no uncertain terms that if we ever found ourselves anything close to anything like this again, we would be finding another university to attend — and we assured him that we had learned our lesson. (When I graduated four years later, he made a point of handing the letter back to me — and I remain grateful for the gravity and empathy with which he handled the whole thing.)

他明确地告诉我们，如果再发生类似的事情，我们就得转学去别的大学了——我们向他保证已经吸取了教训。（四年后我毕业时，他特意把那封信还给了我——我至今仍对他处理此事时所展现的庄重与同理心心存感激。）

I could have (and perhaps would have?) taken this whole shameful incident to the grave, but I feel compelled to confess it now, because I have never seen fear sown so irresponsibly by putative technologists as I have now with respect to AI and the threat of human extinction (!). Amazingly, that is not hyperbole: this week, ex-Anthropic employee Jacob Coxon claimed — and then Anthropic Alignment Science lead Evan Hubinger agreed — that the probability that AI will “kill all humans” is “>10% in the next decade” (!!).

我本可以（或许也应该）把这件羞耻的往事带进坟墓，但我现在感到必须坦白，因为我从未见过像现在这样，所谓的“技术专家”在人工智能和人类灭绝威胁（！）问题上如此不负责任地散布恐惧。令人震惊的是，这并非夸大其词：本周，前 Anthropic 员工 Jacob Coxon 声称——随后 Anthropic 对齐科学负责人 Evan Hubinger 表示赞同——人工智能在“未来十年内杀死全人类”的概率超过 10%（！！）。

That is, the claim is not merely that AI could kill thousands, millions, or even billions of humans (claims that themselves would be extraordinary!), but that there is a greater than ten percent chance that AI will kill all humans. That is, if you have a newborn, their claim is that there is a greater than ten percent chance that your child will somehow perish at the hands of AI before they enter middle school.

也就是说，他们的主张不仅仅是人工智能可能杀死成千上万、数百万甚至数十亿人（这些主张本身就已经非常惊人了！），而是说人工智能有超过百分之十的概率会杀死全人类。换句话说，如果你有一个新生儿，按照他们的说法，你的孩子在进入初中之前，有超过百分之十的概率会死于人工智能之手。

These ghoulish claims strike brazenly at the hearth, and given the obvious importance of AI, it is unsurprising that they have leapt into the mainstream, with people asking the natural question: how would that happen? The answers always rely on hand-wavy extrapolation into the future; for example, Jacob Coxon cites “hacking critical infrastructure” and “extinction-level bioweapons” without further elaboration. But Coxon is not an expert on critical infrastructure, nor on bioweapons — nor, for that matter, on extinction.

这些骇人听闻的言论公然冲击着人们的心理防线。鉴于人工智能的重要性，这些言论进入主流视野并不令人意外，人们自然会问：这怎么可能发生？答案总是依赖于对未来的模糊推测；例如，Jacob Coxon 提到了“黑客攻击关键基础设施”和“灭绝级生物武器”，却没有任何进一步的阐述。但 Coxon 既不是关键基础设施专家，也不是生物武器专家——甚至在灭绝问题上也不是专家。

What 27-year-old Coxon is an expert on — if accidentally — is the contagion of fear. We cannot expect the public to understand the inner workings of all technology (that was the fallacy of my 18-year-old self!): technology is too broad, too deep, and too core to all of our lives to expect everyone to understand everything; we must — at some level — trust domain experts. When those domain experts sow fear, it is the fear that takes root, propagating much more readily than any evidence that may follow it.

27 岁的 Coxon 真正擅长的——即便只是偶然——是恐惧的传染。我们不能指望公众理解所有技术的内部运作机制（那是我 18 岁时的谬论！）：技术太广泛、太深奥，且与我们所有人的生活息息相关，不可能要求每个人都理解一切；我们必须在某种程度上信任领域专家。当这些领域专家散布恐惧时，恐惧便会扎根，其传播速度远比随后的任何证据都要快得多。

(As Jonathan Swift famously quipped over three centuries ago, “Falsehood flies, and the truth comes limping after it”.) In this regard, Coxon is more vector than index case: it is indisputable that the fears of extinction risk are widespread and Coxon himself likely has these fears because he has heard them from someone else. Unfortunately, as the fear takes hold, the sheer number of frightened experts becomes its own kind of evidence, with contrary voices drowned out by the din of an apparent consensus.

（正如乔纳森·斯威夫特三个多世纪前那句名言所说：“谎言飞翔，真理蹒跚而行”。）在这方面，Coxon 更像是一个传播媒介而非首发病例：不可否认，对灭绝风险的恐惧已经广泛存在，Coxon 本人很可能也是因为听到了别人的说法才产生了这种恐惧。不幸的是，随着恐惧占据上风，那些感到恐惧的专家数量本身就成了一种“证据”，而反对的声音则被所谓的“共识”喧嚣所淹没。

With the full knowledge that the fear is becoming well established, I would nonetheless like to offer a rebuttal, if from my own (narrow) domain of expertise: building computers. As I reflected in my 2023 Monktoberfest talk, Intelligence is not Enough, acts of engineering are not acts of intelligence alone. They require not only the human aspects of our character, they also require our action in (and reasoning about) the physical world. Those who are resolute in their fear like to gloss over these bits (“robots will do this!”), but to do so is to ignore the physical reality that robots cannot do this today — and do not seem to be doing so on a time horizon consistent with these extinction fears.

尽管深知这种恐惧已根深蒂固，我仍想从我自己的（狭窄）专业领域——构建计算机——提出反驳。正如我在 2023 年 Monktoberfest 大会上所言《智能是不够的》，工程行为不仅仅是智能行为。它们不仅需要我们的人格特质，还需要我们在物理世界中的行动（以及对物理世界的推理）。那些坚定地沉浸在恐惧中的人喜欢掩盖这些细节（“机器人会做这些！”），但这样做是无视物理现实：机器人今天还做不到这些，而且在与这些灭绝恐惧相符的时间跨度内，它们似乎也做不到。

More generally, to the degree that digital systems have agency in the physical world, it is agency that we — we humans with arms and legs and brains and parents and children — permit: AI executes on phys…

更广泛地说，数字系统在物理世界中拥有的任何代理权，都是我们——我们这些有手有脚、有大脑、有父母和子女的人类——所允许的：人工智能是在物理……上执行的。