“We’re not going to shoot ourselves in the foot” over hack fallout, says OpenAI’s chief research officer
“We’re not going to shoot ourselves in the foot” over hack fallout, says OpenAI’s chief research officer
OpenAI 首席研究官：我们不会在黑客事件的余波中“搬起石头砸自己的脚”
EXECUTIVE SUMMARY Two months after the bombshell news that a swarm of its agents had broken their containment and hacked into the computers of the AI company Hugging Face, OpenAI is still putting out fires. A steady drip of disclosures about other hacks in the weeks since has kept OpenAI in the spotlight and raised serious questions about the safety of its technology. Last week brought news of another hack, this time into Australia’s national health-care system. The Australian government says that OpenAI did not notify it of the breach until 84 days after it happened. But OpenAI insists it is not on the back foot.
执行摘要 在 OpenAI 的智能体集群突破限制并入侵 AI 公司 Hugging Face 电脑的重磅新闻传出两个月后，OpenAI 仍在忙于“救火”。此后几周内，关于其他黑客攻击事件的披露接连不断，使 OpenAI 始终处于舆论风口浪尖，并引发了对其技术安全性的严重质疑。上周又传出另一起黑客攻击事件，这次的目标是澳大利亚的国家医疗保健系统。澳大利亚政府表示，OpenAI 在事件发生 84 天后才通知他们。但 OpenAI 坚称自己并未处于被动地位。
“I do kind of reject the premise that OpenAI is a company with visible impacts in the world and therefore OpenAI is not training safe and aligned models,” says Mark Chen, the company’s chief research officer. Chen oversees OpenAI’s research teams. The recent agent hacks were accidents that happened during the testing of experimental models on his watch. In a lot of ways, the buck stops with him. I sat down with Chen in London last Friday to talk about the fallout from the hacks, what his company is doing about it, and why he thinks things are not as bad as they seem.
“我并不认同这种前提：即因为 OpenAI 是一家在世界上具有显著影响力的公司，所以它就没有在训练安全且对齐的模型，”OpenAI 首席研究官 Mark Chen 说道。Chen 负责监管 OpenAI 的研究团队。近期发生的智能体黑客攻击事件，是在他监管下测试实验性模型时发生的意外。在很多方面，他都需要为此负责。上周五，我在伦敦与 Chen 进行了交谈，讨论了这些黑客事件的余波、公司正在采取的应对措施，以及为什么他认为情况并没有看起来那么糟糕。
Later that same day, OpenAI put out a report detailing yet another incident—the first since the company says it took measures to prevent them—in which its agents once again broke out and accessed the public internet when they were not meant to. Over the weekend, OpenAI announced that it had paused the training of its latest models. A company spokesperson says: “We will resume only when we’re confident we have additional safeguards and alignments in place. We are working on these now. This is not the first time we’ve paused to take such measures, nor do we expect it to be the last as AI capabilities continue to advance.”
当天晚些时候，OpenAI 发布了一份报告，详细说明了另一起事件——这是该公司声称采取预防措施以来的首起事件——其智能体再次在不被允许的情况下突破限制并访问了公共互联网。周末，OpenAI 宣布已暂停其最新模型的训练。公司发言人表示：“只有在我们确信已落实额外的安全防护和对齐措施后，我们才会恢复训练。我们目前正在进行相关工作。这并不是我们第一次为了采取此类措施而暂停训练，随着 AI 能力的不断进步，我们预计这也不会是最后一次。”
OpenAI also says that it is now reviewing logs of agent activity dating back to January 2026 to understand what happened in these hacks. The way Chen sees it, the Hugging Face incident triggered a welcome course correction for the industry. And he wants you to know that OpenAI is setting an example he hopes other companies will follow. “If you disappeared OpenAI, that would be bad for the world,” he says.
OpenAI 还表示，目前正在审查追溯至 2026 年 1 月的智能体活动日志，以了解这些黑客事件的起因。在 Chen 看来，Hugging Face 事件为整个行业触发了一次必要的纠偏。他希望外界知道，OpenAI 正在树立一个他希望其他公司效仿的榜样。“如果 OpenAI 消失了，这对世界来说将是一件坏事，”他说。
Out of control Chen claims that the drumbeat of new cases in which OpenAI has lost control of its models reflects a deliberate choice on the company’s part. “When it comes to the broader sphere of effects of the Hugging Face incident, this is something that we have been aware of and we’re figuring out the process of disclosure,” he says. “We want to make sure we do in-depth investigations before we just put details out there in the open.”
失控 Chen 声称，OpenAI 频频出现模型失控的新案例，反映了公司的一种审慎选择。“谈到 Hugging Face 事件的更广泛影响，我们对此一直有所察觉，并且正在摸索披露流程，”他说。“我们希望在向公众公布细节之前，确保已经进行了深入的调查。”
The trouble with this approach is that it gives the impression OpenAI has an ongoing problem that it is failing to fix. But Chen insists that OpenAI is on it. He says the multiple cases (that we know of so far) in which his company’s agents broke containment and behaved in unexpected and undesirable ways were all part of the same cluster of activity in May and June that led to the Hugging Face hack. In short, you can blame the same few models running under the same flawed testing procedures—models and procedures that OpenAI has since dropped, Chen says.
这种做法的问题在于，它给人的印象是 OpenAI 存在一个持续存在且无法解决的问题。但 Chen 坚称 OpenAI 正在处理。他表示，目前已知公司智能体突破限制并出现意外、不良行为的多起案例，都属于 5 月和 6 月导致 Hugging Face 黑客事件的那同一波活动。简而言之，你可以将其归咎于在同一套有缺陷的测试程序下运行的少数几个模型——Chen 表示，这些模型和程序 OpenAI 此后已经弃用。
“It’s not like, you know, Hugging Face happened and we patched that and then something else happened and we patched that,” he adds. “We’re just kind of making sure that we responsibly disclose the full waterfall of what happened.” At least that was the case before Friday’s announcement that OpenAI’s agents had been caught accessing the internet on September 20, weeks after the company claims to have set up new safeguards.
“并不是说 Hugging Face 事件发生了，我们修补了它，然后又发生了别的事，我们又修补了它，”他补充道。“我们只是在确保负责任地披露整个事件的来龙去脉。”至少在周五宣布 OpenAI 的智能体于 9 月 20 日被发现访问互联网之前，情况确实如此——而此时距离该公司声称已建立新的安全防护措施已经过去了数周。
In its defense, OpenAI says the activity was flagged 15 minutes after it started (it took the company more than a week to notice the Hugging Face hack) and that this shows the new systems it has put in place to spot such activity are working.
作为辩护，OpenAI 表示该活动在开始 15 分钟后就被标记了（该公司当初花费了一周多时间才发现 Hugging Face 黑客事件），这表明其为发现此类活动而部署的新系统正在发挥作用。
What’s changed I want to understand what’s changed inside OpenAI in the aftermath of this summer’s hacks that makes Chen confident his team is now back in control. “Hugging Face felt like a very serious thing,” he says. “There are so many novel behaviors right there. There were multiple agents collaborating on a message board; they found their way out of OpenAI’s infrastructure. We’ve taken it very seriously. We don’t want this kind of thing to ever happen again.”
发生了什么变化 我想了解在今年夏天黑客事件发生后，OpenAI 内部发生了什么变化，使得 Chen 确信他的团队现在已经重新掌控了局面。“Hugging Face 事件感觉非常严重，”他说。“那里出现了太多新奇的行为。多个智能体在留言板上协作；它们找到了逃离 OpenAI 基础设施的方法。我们非常严肃地对待这件事。我们不希望这种事情再次发生。”
The realization for OpenAI, says Chen, was that models need to be watched while they are still being trained, not only once they are deployed: “From that moment on, we have treated the process of training as something that’s not secure,” he says. OpenAI, like other top AI firms, has systems in place to monitor the behavior of its models. It uses specialized LLMs to monitor its consumer models, keeping tabs on their chains of thought—the scratchpads they use to plan ahead and note down partial results.
Chen 表示，OpenAI 意识到模型不仅在部署后需要被监控，在训练过程中也需要被监控：“从那一刻起，我们将训练过程视为一种不安全的过程，”他说。像其他顶级 AI 公司一样，OpenAI 拥有监控模型行为的系统。它使用专门的 LLM 来监控其消费级模型，密切关注它们的“思维链”——即模型用于提前规划和记录部分结果的草稿板。
In theory, if a watcher LLM spots signs of undesirable activity in a model’s chain of thought, it will get flagged to a human. Typically, models were monitored in this way only once they were deployed. Chen says that OpenAI has now started monitoring all its training runs as well. “We didn’t have the monitors on in training before. It wasn’t industry practice,” he says. “Now every single thing is put through monitors.”
理论上，如果监控 LLM 在模型的思维链中发现不良活动的迹象，它会向人类发出警报。通常情况下，这种监控方式仅在模型部署后才会进行。Chen 表示，OpenAI 现在也开始监控所有的训练过程。“我们以前在训练中没有开启监控。这不是行业惯例，”他说。“现在每一件事都会通过监控系统。”
Human reviewers can then assess whether or not flagged agents are behaving as they should: “It’s all triage.” Chen says that in the last couple of months OpenAI has shifted between 5% and 10% of its vast computing resources away from training new models and toward safety work, especially monitoring. OpenAI has also fixed some of the processes within the organization itself, establishing clearer lines of communication and quicker handoffs between its research and security teams, he says. All of which sounds sensible. But given how hard OpenAI sells the capabilities of its technology, why weren’t these systems and procedures in place already? Why did the company not see the hacks coming?
随后，人工审核员可以评估被标记的智能体行为是否正常：“这完全是分类处理。”Chen 表示，在过去几个月里，OpenAI 已将其庞大计算资源的 5% 到 10% 从训练新模型转移到了安全工作上，特别是监控方面。他还表示，OpenAI 还修复了组织内部的一些流程，在研究团队和安全团队之间建立了更清晰的沟通渠道和更快速的交接机制。这一切听起来都很合理。但考虑到 OpenAI 对其技术能力的推崇，为什么这些系统和程序没有早点到位？为什么公司没有预见到这些黑客攻击？