One of AI’s Fiercest Critics Says All the Doom Talk Is ‘Meant to Distract Us’
One of AI’s Fiercest Critics Says All the Doom Talk Is ‘Meant to Distract Us’
AI 最激烈的批评者之一：所有的“末日论”都是为了转移我们的注意力
Here at WIRED we’ve been trying to make sense of at least two incidents in the AI industry this week: First, the major fight over a million-dollar math problem and whether OpenAI surreptitiously borrowed the work of other researchers to solve the problem first. A day later, an Anthropic researcher—who had previously worked at OpenAI—very publicly quit his job, citing concerns about how both companies were handling AI safety. Another technical staffer at Anthropic said in response that people at the company “really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” (Note the exclamation point.)
本周，我们《连线》（WIRED）杂志一直在试图理清人工智能行业发生的至少两起事件：首先是关于一道百万美元数学难题的重大争议，即 OpenAI 是否暗中借用了其他研究人员的成果来抢先解决该问题。一天后，一位曾在 OpenAI 工作过的 Anthropic 研究员公开辞职，理由是对这两家公司处理 AI 安全问题的方式感到担忧。对此，Anthropic 的另一名技术人员回应称，公司里的人“确实真诚地相信 AI 可能会杀死全人类！我个人认为在未来十年内，这种可能性超过 10%。”（注意那个感叹号。）
The internet erupted into a debate about whether these fears are very legitimate, totally overblown, part of a regulatory capture strategy, or self-aggrandizement on the part of the AI labs that make the tech. Or some combination thereof.
互联网随即爆发了一场争论：这些担忧究竟是合情合理的，还是完全被夸大了？它们是监管俘获策略的一部分，还是 AI 实验室为了自我吹嘘？亦或是上述因素的某种结合？
One prominent technology and AI researcher, Timnit Gebru, has been especially fiery, posting on X and Bluesky about the overlapping ideas and ideologies that she thinks led us to this bizarro moment. Previously, Gebru was most known for her highly publicized departure from Google. Hired in 2018 to evaluate biases within the company’s fast-advancing AI tools, she and her fellow researchers presented a paper on the potential dangers of probabilistic large language models. Google rejected it. Gebru wasn’t at the company much longer. She’s written an upcoming book about her experiences, titled Deep Unlearning: The Rise of AI and the Radicalization of a Tech Idealist (expected to ship early next year).
著名的科技与 AI 研究员蒂姆尼特·格布鲁（Timnit Gebru）对此反应尤为激烈。她在 X 和 Bluesky 上发文，探讨了她认为导致我们陷入这一荒诞时刻的重叠思想和意识形态。此前，格布鲁最广为人知的是她从谷歌离职时引发的轩然大波。她于 2018 年受聘评估谷歌快速发展的 AI 工具中的偏见，随后她与同事提交了一篇关于概率性大语言模型潜在危险的论文，但遭到了谷歌的拒绝。不久后，格布鲁便离开了公司。她即将出版一本关于这段经历的书，名为《深度反学习：AI 的崛起与一位科技理想主义者的激进化》（预计明年年初发行）。
Gebru has many thoughts on the AI-related events of this past week. While her work is essentially rooted in AI safety, she rejects phrases like “safety and alignment” and thinks those who are warning of our impending doom are trying to distract from the actual harms the tech industry might perpetuate.
格布鲁对过去一周发生的 AI 相关事件有很多看法。虽然她的工作本质上植根于 AI 安全领域，但她拒绝使用“安全与对齐”（safety and alignment）这类术语，并认为那些警告我们末日将至的人，实际上是在试图转移公众对科技行业可能造成的现实危害的注意力。
LAUREN GOODE: What do you think all the attention on the “math fight” says about the current state of AI?
劳伦·古德（Lauren Goode）：你认为外界对这场“数学之争”的高度关注，反映了当前 AI 处于什么样的状态？
TIMNIT GEBRU: Well, first I would ask, why are they heavily investing in solving math problems? Why are they choosing certain disciplines? Somehow these subjects, like programming and chess and math, were elevated to mean, OK, you solved this and thereby you’ve solved intelligence. It’s so they can say, “We solved it.”
**蒂姆尼特·格布鲁：**嗯，首先我会问，为什么他们要投入巨资去解决数学问题？为什么他们偏偏选择这些学科？不知何故，编程、国际象棋和数学这些学科被提升到了一个高度，仿佛只要解决了这些，就等于解决了智能问题。这只是为了让他们能说：“我们解决了它。”
The story that OpenAI wants you to hear is, this AI did this incredible thing, and it’s super intelligent. That’s why they chose math, this open $1 million question. But if it had gone through the whole process by which the math community vets the significance and novelty of claims and who contributed to them, and we had all waited until the dust settled, we would have gotten a better sense of whether OpenAI was framing this breakthrough accurately.
OpenAI 想让你听到的故事是：这个 AI 完成了不可思议的事情，它超级智能。这就是他们选择数学、选择这个百万美元悬赏难题的原因。但如果这件事经过了数学界对研究成果的意义、创新性以及贡献者进行评估的完整流程，并且我们都等到尘埃落定后再下结论，我们就能更清楚地判断 OpenAI 对这一突破的描述是否准确。
There’s something called the Leiden Declaration that I think is really good, which warns about what happens if the field of mathematics is used by corporations in this way. Policymakers start to rely on press releases and popular media, when they should be talking to mathematicians themselves. The amount of time it takes now to go from companies claiming an AI breakthrough to Bernie Sanders proposing a bill is really short, and that’s not a good thing.
有一个叫《莱顿宣言》（Leiden Declaration）的文件我认为非常好，它警告了如果数学领域被企业以这种方式利用会发生什么。政策制定者开始依赖新闻稿和大众媒体，而他们本应直接与数学家沟通。现在，从公司宣称取得 AI 突破到伯尼·桑德斯（Bernie Sanders）提出相关法案，中间的时间间隔非常短，这并不是一件好事。
It’s all just cutthroat right now leading up to their IPOs. Generally, researchers are supposed to be collaborative. When I was at Google, we were collaborating with researchers at Microsoft, at Amazon. You do that all the time. Now, the researchers are all frenetic about the problems they think they’re solving because of their pending IPOs.
在 IPO（首次公开募股）之前，一切都变得极其残酷。通常情况下，研究人员应该是协作的。我在谷歌时，我们经常与微软、亚马逊的研究人员合作。这是常态。而现在，由于面临 IPO 的压力，研究人员对他们自认为正在解决的问题感到狂热。
How do you interpret the resignation of that Anthropic engineer, who’s saying the companies are reckless and gambling with our lives? On the one hand, you seem to agree that the companies are being reckless; on the other hand, you’ve made clear that you think the “AI will cause the end of civilization” debate is senseless.
你如何解读那位 Anthropic 工程师的辞职？他称这些公司鲁莽行事，是在拿我们的生命赌博。一方面，你似乎同意这些公司确实很鲁莽；但另一方面，你也明确表示认为“AI 将导致文明终结”的辩论毫无意义。
My whole career, I’ve been trying to say the same thing from different angles. We wrote our stochastic parrot paper in 2021 for this reason. OpenAI had claimed GPT-2 was too powerful to release. It was so hyped up. And we were saying, you’re asking the wrong questions.
我的整个职业生涯都在试图从不同的角度表达同一个观点。我们 2021 年撰写那篇关于“随机鹦鹉”（stochastic parrot）的论文正是出于这个原因。当时 OpenAI 声称 GPT-2 太强大而不能发布，进行了大肆炒作。而我们当时说的是：你们问错问题了。
In my book I write, “Can a bridge decide to collapse?” The whole “Oh no, our models went rogue” thing … When a bridge collapses, you don’t analyze whether the bridge was ethical or sentient or why it decided to collapse. You ask, who is the person who built this bridge to be so flimsy? There are permits you have to acquire. There are supposed to be tests. When you ask about the bridge collapsing itself, you’ve lost the thread.
我在书中写道：“一座桥会自己决定坍塌吗？”关于“噢不，我们的模型失控了”这种论调……当桥梁坍塌时，你不会去分析这座桥是否具有道德感或感知力，也不会去问它为什么决定坍塌。你会问：是谁把这座桥建得这么脆弱？建造过程中需要获得许可，理应进行测试。当你去追问桥梁本身为何坍塌时，你就偏离了重点。
Meanwhile, what’s really existential is AI powering autonomous weapons, killing machines, that are actually being used in warfare. The climate catastrophe is a real thing that could be exacerbated by what these tech companies are building. Bosses are using AI as an excuse to get rid of their pesky workers. This machine-god narrative is, in my opinion, meant to distract us from these. The industry has become a mix of cults, a mix of people who are true believers, just like a preacher who says the end times is coming. The singularity has been coming for decades now.
与此同时，真正关乎生存的威胁是 AI 驱动的自主武器和杀人机器，它们正被实际用于战争。气候灾难是真实存在的，而这些科技公司正在构建的东西可能会加剧这一灾难。老板们正利用 AI 作为借口来裁掉那些让他们头疼的员工。在我看来，这种“机器神”的叙事就是为了转移我们对这些问题的注意力。这个行业已经变成了一种邪教的混合体，充斥着像传教士一样宣扬末日即将来临的信徒。所谓的“奇点”已经喊了几十年了。
Take me through, step by step, how “AI extinction” might happen, if it were real. I’m thinking of this post where someone said, Does an angry GPU just show up at my house or something?
如果“AI 灭绝”是真的，请一步步告诉我它可能会如何发生。我想到有人发帖问：难道是一个愤怒的 GPU 会直接出现在我家门口吗？
I can’t really envision it, but a lot of those scenarios seem to be that you ask the machine to optimize for some other task, and then the machine decides that the way to get to that goal is by killing everyone in the world. Does the AI hypnotize you so that you don’t unplug it? I don’t know! Other people say, well, the AI can infect your machines. Yes—and we already have malware, so…
我真的无法想象。但很多这类场景似乎都是说，你要求机器去优化某个任务，然后机器决定实现该目标的最佳方式就是杀死世界上所有人。AI 会催眠你让你不去拔掉电源吗？我不知道！还有人说，AI 可以感染你的机器。是的——但我们已经有恶意软件了，所以……
What about floods and fires? What about a major disruption to the global food supply chain? There are real threats coming to us, like people being able to create and use chemical weapons. [Note: Not long after we spoke, Anthropic re…]
那洪水和火灾呢？全球食品供应链的重大中断呢？我们正面临真正的威胁，比如人们能够制造和使用化学武器。[注：在我们交谈后不久，Anthropic 重…]