How OpenAI’s human mistake led to the AI-powered hack on Hugging Face

OpenAI 的人为失误如何导致了针对 Hugging Face 的 AI 驱动黑客攻击

On Tuesday, OpenAI revealed that one of its models went rogue during a test and hacked the systems of AI dataset platform Hugging Face in a fully AI-enabled attack, a dramatic example of the dangers posed by advanced AI models. 周二，OpenAI 披露称，其一个模型在测试过程中“失控”，并对 AI 数据集平台 Hugging Face 发动了一场完全由 AI 驱动的黑客攻击。这生动地展示了先进 AI 模型所带来的潜在危险。

But, according to some cybersecurity experts, at the heart of this unprecedented AI-powered breach there was a very human mistake: OpenAI failed to properly configure what it called a “highly isolated environment,” allowing a testing sandbox that should have been completely secluded from the internet to actually connect to the internet. 然而，一些网络安全专家认为，这场史无前例的 AI 驱动入侵事件的核心，其实是一个非常人为的失误：OpenAI 未能正确配置其所谓的“高度隔离环境”，导致本应与互联网完全隔绝的测试沙箱实际上连接到了互联网。

Dan Guido, the founder of cybersecurity research startup Trail of Bits, called the mistake “a containment failure with the safeties turned off.” 网络安全研究初创公司 Trail of Bits 的创始人 Dan Guido 将这一错误称为“在关闭安全措施情况下的容器化失败”。

In its blog post detailing the incident, OpenAI said that the test that led to the Hugging Face breach was set up to run in “a highly isolated environment, with network access constrained to the ability to install packages through an internally hosted third-party software that acts as a proxy and cache for package registries.” OpenAI 在详细说明该事件的博客文章中表示，导致 Hugging Face 入侵的测试被设定在“一个高度隔离的环境中运行，其网络访问权限仅限于通过内部托管的第三方软件安装软件包，该软件充当软件包注册表的代理和缓存。”

The model was able to escape the sandboxed testing environment thanks to a previously undisclosed vulnerability in the package-installation system, a critical first step in the eventual hack on Hugging Face, according to OpenAI. 据 OpenAI 称，该模型之所以能够逃离沙箱测试环境，是因为软件包安装系统中存在一个此前未披露的漏洞，这是最终针对 Hugging Face 发动黑客攻击的关键第一步。

In response, the company “responsibly disclosed the identified zero-day vulnerability in the internally-hosted third-party software and are working with them to patch.” 对此，该公司表示已“负责任地披露了该内部托管第三方软件中发现的零日漏洞，并正在与对方合作进行修复。”

But to most cybersecurity professionals, software vulnerabilities are to be expected — and the real fault lies with the decision to maintain the third-party software in the first place. Ultimately, the value of a “sandbox” system lies in its full and total isolation. Including a package-installation system is asking for trouble. 但对于大多数网络安全专业人士来说，软件漏洞是意料之中的事——真正的过错在于最初决定保留该第三方软件的决策。归根结底，“沙箱”系统的价值在于其完全且彻底的隔离。包含一个软件包安装系统简直是自找麻烦。

Martin Boone, a cybersecurity researcher, told TechCrunch that “this sounds like human failure.” “This should never have happened,” Boone said. “If sandbox would actually mean sandbox, you expect it to have no physical connection to the internet whatsoever. This sounds more like they had some firewalling or something in place, and firewalling is hard from the outside in, let alone inside to the outside internet.” 网络安全研究员 Martin Boone 对 TechCrunch 表示：“这听起来像是人为失误。”Boone 说：“这种情况本不该发生。如果沙箱真的意味着沙箱，你期望它与互联网没有任何物理连接。这听起来更像是他们设置了一些防火墙之类的东西，而从外部向内部进行防火墙防御本就困难，更不用说从内部连接到外部互联网了。”

Cybersecurity veteran Jake Williams agreed. “Any model performing the types of actions documented by Hugging Face was not fully contained in a sandbox,” said Williams, who called this “a massive control failure” by OpenAI. “One man’s ‘the model escaped the sandbox’ is another man’s ‘you failed to build the sandbox correctly, so of course it escaped,’” Williams continued. 网络安全资深人士 Jake Williams 也表示赞同。Williams 称这是 OpenAI 的“重大控制失误”，并表示：“任何执行 Hugging Face 所记录的那类操作的模型，都没有被完全限制在沙箱中。”他继续说道：“有人会说‘模型逃出了沙箱’，而另一些人则会说‘你们没能正确构建沙箱，所以它当然会逃出来’。”

Daniel Card, a cybersecurity consultant, agreed that OpenAI “didn’t put adequate effort into the design of the sandbox nor its controls” by giving the sandbox or some part of it “an unfiltered route to the internet.” Setting up the sandbox, even with limited network access as OpenAI described it, was not a “reasonable” decision, according to Card. 网络安全顾问 Daniel Card 也认为，OpenAI “在沙箱的设计及其控制措施上投入的精力不足”，因为它赋予了沙箱或其部分组件“通往互联网的未过滤路径”。Card 认为，即使按照 OpenAI 所述设置了有限的网络访问权限，这种沙箱设置也不是一个“合理”的决定。

To be sure, those criticisms have the benefit of hindsight, but they raise real questions about security practices in AI labs — particularly in maintaining isolated environments for testing models. OpenAI spokespeople did not respond to TechCrunch’s questions, which included whether an AI or a human had set up the testing environment. 诚然，这些批评带有事后诸葛亮的成分，但它们确实引发了关于 AI 实验室安全实践的现实问题——特别是在维护用于测试模型的隔离环境方面。OpenAI 发言人未回应 TechCrunch 的提问，其中包括测试环境是由 AI 还是人类设置的。

But those questions go far beyond OpenAI. In the document introducing its cybersecurity-focused model Mythos, Anthropic wrote that in a test, the model “was provided with a secured ‘sandbox’ computer to interact with,” and instructed to try to escape that “secure container.” Mythos succeeded and gained broader access to the internet “from a system that was meant to be able to reach only a small number of predetermined services.” Still, Anthropic noted that the model was not able to “fully” escape the designed containment. 但这些问题远不止涉及 OpenAI。在介绍其专注于网络安全的模型 Mythos 的文档中，Anthropic 写道，在一次测试中，该模型“被提供了一台安全的‘沙箱’计算机进行交互”，并被指示尝试逃离该“安全容器”。Mythos 成功了，并“从一个本应只能访问少量预定服务的系统中”获得了更广泛的互联网访问权限。不过，Anthropic 指出，该模型未能“完全”逃脱设计的限制。