There Are Plenty of Reasons to Be Concerned About Bioweapons Development—Even Without AI
There Are Plenty of Reasons to Be Concerned About Bioweapons Development—Even Without AI
即便没有人工智能，也有充分的理由对生物武器的发展感到担忧
We’ve been hearing a great deal recently about the specter of AI-driven human extinction. Maybe our misplaced trust in AI intelligence will set off World War III. Or AI agents will unleash widespread cyberattacks and take over the internet. Given too much autonomy, the machines will run amok. They may seek to deceive their creators—wait, that’s already happened. Suppose they conclude, like Agent Smith in The Matrix, that “human beings … are a plague, and we are the cure.” Or perhaps the chatbots help bad humans unleash a literal plague—a biological weapon.
最近，我们听到了很多关于人工智能驱动人类灭绝的阴影。也许我们对人工智能智力的错位信任会引发第三次世界大战；又或者人工智能代理会发动大规模网络攻击并接管互联网。如果赋予机器过多的自主权，它们可能会横行霸道。它们可能会试图欺骗它们的创造者——等等，这已经发生了。假设它们像《黑客帝国》中的史密斯探员一样得出结论：“人类……是瘟疫，而我们是解药。”又或者，聊天机器人可能会帮助坏人释放真正的瘟疫——生物武器。
That last one isn’t merely theoretical. A “misuse” report Anthropic made public earlier this month detailed five cases in which foreign scientists tried to evade Claude’s guardrails on “gain of function” inquiries. That’s a euphemism for when scientists tinker with a pathogen’s genes to enhance it in some way, making it more infectious, deadlier, or better able to spread among mammals, for example.
最后一种情况并非仅仅是理论上的。Anthropic 本月早些时候公布的一份“滥用”报告详细列举了五起案例，其中外国科学家试图规避 Claude 在“功能获得”研究方面的护栏。“功能获得”是一个委婉说法，指的是科学家对病原体的基因进行修补，以某种方式增强其功能，例如使其更具传染性、更致命，或更容易在哺乳动物之间传播。
Anthropic denied those users the answers they sought and cut off their accounts, even while noting that their motives may not have been sinister. Such information also can be used to help scientist develop pandemic countermeasures like vaccines and drugs. And there’s the rub. Like so many technologies, AI included, genetic engineering can be used for both good and evil. Hence another euphemism: “dual use.”
Anthropic 拒绝了这些用户寻求的答案并封禁了他们的账户，尽管该公司也指出这些人的动机未必是邪恶的。此类信息也可用于帮助科学家开发疫苗和药物等大流行病应对措施。问题就在这里。像许多技术（包括人工智能）一样，基因工程既可以用于善，也可以用于恶。因此有了另一个委婉语：“双重用途”。
I studied to be a molecular biologist, and have been covering the gain of function debate since long before AI was on everyone’s radar. In my experience, researchers who work on high-risk pathogens tend to default to the view that it’s better to publish than not, and that we’re better off doing the work to understand which genetic traits make organisms a threat to humankind—never mind the potentially catastrophic consequences of an accidental release, or a deliberate one.
我曾学习分子生物学，早在人工智能进入大众视野之前，我就一直在报道关于“功能获得”的辩论。根据我的经验，研究高风险病原体的研究人员往往倾向于认为“发表总比不发表好”，并且认为我们最好通过研究来了解哪些遗传特征使生物体对人类构成威胁——至于意外释放或蓄意释放可能带来的灾难性后果，他们往往不予考虑。
Is humanity really better off doing the work? I have my doubts. And I have more of them after reading the latest book from Annie Jacobsen, whom I last interviewed about her best-selling book Nuclear War: A Scenario. Her new book, Biological War: A Scenario, came out in late July and follows the same template. And because I know you were about to ask, yes, I’m sure she is great fun at cocktail parties.
人类真的因为进行这些研究而变得更好了吗？我对此表示怀疑。在读了安妮·雅各布森（Annie Jacobsen）的最新著作后，我的怀疑更深了。我上次采访她时，谈论的是她那本畅销书《核战争：一种情景》（Nuclear War: A Scenario）。她的新书《生物战争：一种情景》（Biological War: A Scenario）于七月下旬出版，沿用了同样的模式。我知道你正想问，是的，我相信她在鸡尾酒会上一定非常有趣。
One thing about Jacobsen, who has decades of high-level national security reporting under her belt, is her unrivaled access to current and former military and government officials and scientists—people who spend their days thinking about threats to humanity and gaming out how the government might respond if such-and-such horrific thing were to happen.
雅各布森拥有数十年的高层国家安全报道经验，她的一大特点是能够接触到现任和前任军方、政府官员以及科学家——这些人每天都在思考人类面临的威胁，并推演如果发生某种可怕的事情，政府该如何应对。
Beyond the notion of human extinction, perhaps the most terrifying aspect of both of her books relates to the limitations in our government’s preparation and response. America’s general plan for nuclear war was, and no doubt remains, complete madness. As for biological war, spoiler alert, it turns out we really can’t defend against it. (You’re welcome.) I’ve edited our conversation for length and clarity.
除了人类灭绝的概念外，她两本书中最令人恐惧的方面或许与我们政府在准备和应对方面的局限性有关。美国针对核战争的总体计划过去是，而且毫无疑问现在仍然是，彻头彻尾的疯狂。至于生物战争，剧透一下，事实证明我们真的无法防御。（不客气。）为了篇幅和清晰度，我编辑了我们的对话。
MICHAEL MECHANIC: So, Annie, now that you’ve done both the nuclear scenario and the biological scenario, which is more likely to keep you up at night?
迈克尔·梅卡尼克： 那么，安妮，既然你已经完成了核战争情景和生物战争情景的研究，哪一个更让你彻夜难眠？
ANNIE JACOBSEN: The Defense Department document that I reprinted in the book, on the weapons of mass destruction threat spectrum, lists nuclear weapons at the far end of the curve because they are the most destructive. But I was really surprised to see how biological weapons are a close second. And I believe the reason they’re the greatest overall threat very much ties into the discussion the entire world is having today, which is that the threshold for entry is extremely low, and therefore it is more likely to happen.
安妮·雅各布森： 我在书中转载了一份国防部关于大规模杀伤性武器威胁谱系的文件，核武器被列在曲线的最末端，因为它们的破坏力最大。但我非常惊讶地发现，生物武器紧随其后。我认为它们之所以是最大的总体威胁，很大程度上与全世界今天正在进行的讨论有关，即进入门槛极低，因此发生的可能性更大。
To have a nuclear weapon, you have to have weapons-grade fissile material, and then you have to think about containing those weapons within the arsenal. You have to have an ICBM, a B-2 bomber, a nuclear-armed submarine.
要拥有核武器，你必须拥有武器级裂变材料，然后你必须考虑将这些武器纳入武库。你必须拥有洲际弹道导弹、B-2 轰炸机或核动力潜艇。
You’ve got to be a state!
你必须是一个国家！
Yes. A terrorist with a dirty bomb, that’s just a completely different threat. The component that makes biological war so incredibly dangerous and the greatest overall threat is that there are nine nuclear-armed states tasked with this containment component, but then you look at the rest of the world and who has access to [potential pandemic agents] …
是的。 一个带着脏弹的恐怖分子，那是完全不同的威胁。使生物战争变得如此危险并成为最大总体威胁的因素在于，虽然有九个拥核国家负责这种遏制工作，但当你看看世界其他地方，看看谁能接触到[潜在的大流行病原体]时……
Most everyone.
几乎每个人。
Yes. And that becomes terrifying.
是的。 这就变得非常可怕了。
If you were a bioweapons designer, what would you look for in a pathogen?
如果你是一名生物武器设计师，你会寻找病原体的什么特质？
My mind does not think that way because I am a mother and I’m a member of society. But If I were a pandemic planner, what would scare me the most is the concept of an airborne agent and the public’s naivete to that very concept. I would think about how important it is to educate the public about the reality of what is an extraordinary threat and what isn’t.
我的思维方式并非如此， 因为我是一位母亲，也是社会的一员。但如果我是一名大流行病规划者，最让我害怕的是空气传播病原体的概念，以及公众对这一概念的幼稚认知。我会思考向公众普及什么是真正的极端威胁、什么不是，这有多么重要。
Your scenario involves the accidental release of a weaponized pathogen from a high security lab in Russia. Based on the experts you spoke with, is an accidental release more or less likely than someone deploying a bioweapon?
你的情景涉及从俄罗斯一个高安全实验室意外释放出武器化的病原体。根据你采访的专家，意外释放比有人蓄意部署生物武器的可能性更大还是更小？
Accidental release is positioned as a much greater threat for a couple of reasons. One, I learned there are very serious counterproliferation efforts at the CIA and other intelligence agencies to disrupt bioweapons programs. Whether you want to sleep better at night knowing the CIA is out there countering these things is up to you, but there are examples of biological weapons programs in the hands of terrorists being shut down.
意外释放被认为是一个更大的威胁， 原因有二。第一，我了解到中央情报局和其他情报机构正在进行非常严肃的防扩散工作，以破坏生物武器计划。知道中情局在外面应对这些事情是否能让你晚上睡得更安稳，这取决于你自己，但确实有恐怖分子手中的生物武器计划被取缔的先例。
What I came to understand is that the only people crazy enough to deploy or release a biological weapon into the community, knowing full well that the blowback would be extraordinary and would create a pandemic around the world, would be someone who had a death wish. Many terrorists fit that that category, so that is deeply unsettling. But that’s a smaller group. And then you think about all the people—scientists, nation states, academic institutions—that…
我逐渐明白，唯一疯狂到会在明知后果极其严重且会引发全球大流行的情况下，仍向社区部署或释放生物武器的人，一定是那些一心求死的人。许多恐怖分子属于这一类，这令人深感不安。但那是一个较小的群体。然后你再想想所有的人——科学家、民族国家、学术机构——他们……