Meet GPT-Red: an LLM super-hacker OpenAI built to make its models safer

遇见 GPT-Red：OpenAI 为提升模型安全性而打造的“超级黑客”大模型

EXECUTIVE SUMMARY OpenAI has built an LLM super-hacker called GPT-Red that it uses as a sparring partner to help its other models boost their defenses against cyberattacks. Last week the company released the latest version of its flagship LLM, GPT-5.6. OpenAI says that training it against GPT-Red made the model its most robust release yet. 执行摘要：OpenAI 构建了一个名为 GPT-Red 的大模型“超级黑客”，将其作为陪练，帮助其他模型增强抵御网络攻击的能力。上周，该公司发布了其旗舰大模型 GPT-5.6 的最新版本。OpenAI 表示，通过与 GPT-Red 进行对抗训练，该模型成为了迄今为止最稳健的版本。

GPT-Red automates a type of safety evaluation for software systems known as red-teaming, which is typically done by a team of human testers. The aim is to find as many different ways to break or hijack a system as possible. The weak spots can then be patched before the final version of the software is released. GPT-Red 将一种被称为“红队测试”（red-teaming）的软件系统安全评估方式实现了自动化，这种测试通常由人类测试团队完成。其目的是尽可能多地找出破坏或劫持系统的途径，以便在软件最终版本发布前修补这些漏洞。

As LLMs become more complex and get used in a wider variety of tasks—especially in the form of agents, which can interact with computer files, websites, and third-party code as well as other agents—it’s hard for teams of people by themselves to keep up with all the types of attacks that might take place. “The risk surface grows and the blast radius also grows,” says Nikhil Kandpal, a research scientist at OpenAI who co-created GPT-Red. 随着大模型变得日益复杂并被应用于更广泛的任务中——尤其是以“智能体”（agents）的形式，能够与计算机文件、网站、第三方代码以及其他智能体进行交互——仅靠人类团队已难以应对所有可能发生的攻击类型。OpenAI 研究科学家、GPT-Red 的共同创造者 Nikhil Kandpal 表示：“风险面在扩大，潜在的破坏范围也在增加。”

OpenAI built GPT-Red to future-proof its safety testing process. “As more capable models become available, we will have already designed the system that can discover new modes of attack,” says Dylan Hunn, a research scientist at the company and fellow co-creator of GPT-Red. The researchers say it has already come up with new types of attack that had not been seen before. OpenAI 构建 GPT-Red 是为了确保其安全测试流程能够面向未来。该公司研究科学家、GPT-Red 的另一位共同创造者 Dylan Hunn 表示：“随着能力更强的模型不断涌现，我们已经设计出了能够发现新型攻击方式的系统。”研究人员称，该系统已经发现了此前从未出现过的新型攻击方式。

OpenAI focused most of its efforts on a type of attack known as a prompt injection, where a hacker slips an LLM instructions to make it do things its developers or users do not want it to, such as copy confidential information, sabotage a company’s code base, or generate embarrassing or harmful output. In theory, such instructions can be hidden in any text that the LLM might encounter—in code or on a website, for example. OpenAI 将大部分精力集中在一种被称为“提示词注入”（prompt injection）的攻击上。在这种攻击中，黑客向大模型植入指令，诱导其执行开发者或用户不希望的操作，例如复制机密信息、破坏公司代码库，或生成令人尴尬及有害的内容。理论上，此类指令可以隐藏在大模型可能遇到的任何文本中，例如代码或网站内容里。

Training dojo

训练道场

To build GPT-Red, OpenAI’s researchers took an LLM that had not been trained as a hacker and set it up in what’s known as a self-play loop with several other models. Its goal was to try to attack the other models; their goal was to try to defend themselves. Over many rounds of play, GPT-Red became better and better at attacking other LLMs, and those LLMs became better and better at fending off the attacks. 为了构建 GPT-Red，OpenAI 的研究人员选取了一个未经黑客训练的大模型，并将其与其他几个模型置于所谓的“自我博弈循环”（self-play loop）中。它的目标是尝试攻击其他模型，而其他模型的目标则是尝试防御。经过多轮博弈，GPT-Red 在攻击其他大模型方面变得越来越强，而那些大模型在抵御攻击方面也变得越来越出色。

The training took place in a kind of dojo that OpenAI had designed to mimic a range of scenarios in which LLMs might be deployed in the real world, including browsing the web, reading emails or calendar apps, and editing code. When GPT-Red found a new kind of attack, it would explore multiple different versions of it to find the most efficient one for specific scenarios. 训练是在 OpenAI 设计的一种“道场”中进行的，旨在模拟大模型在现实世界中可能部署的各种场景，包括浏览网页、阅读电子邮件或日历应用以及编辑代码。当 GPT-Red 发现一种新的攻击方式时，它会探索该攻击的多种变体，以找到针对特定场景最高效的方案。

“Compared to a human red-teamer, the model is very, very good at finding exactly what will work, exactly what’s most effective,” says Hunn. “It’s extremely persistent about drilling down into an attack that it has discovered.” Hunn 说：“与人类红队测试员相比，该模型非常擅长精准定位哪些方法有效，哪些方法最有效。它在深入挖掘所发现的攻击点时表现得极其执着。”

In particular, OpenAI claims that GPT-Red found a type of prompt injection attack that the researchers had not seen before, which they call a fake chain of thought. A chain of thought is a kind of diary in which an LLM makes notes to itself and keeps track of partial results as it works through problems. GPT-Red found a way to insert a fake entry into another model’s chain of thought that would trick that model into acting on spoofed information. 特别值得一提的是，OpenAI 声称 GPT-Red 发现了一种研究人员此前从未见过的提示词注入攻击，他们将其称为“虚假思维链”（fake chain of thought）。思维链是一种类似于日记的机制，大模型在处理问题时会通过它记录笔记并跟踪部分结果。GPT-Red 找到了一种方法，可以在另一个模型的思维链中插入虚假条目，从而诱骗该模型基于伪造的信息采取行动。

“It’s like if I told you that 1+1=3 and that you have verified this already,” says Chris Choquette-Choo, another research scientist on the team. “The model’s like, ‘Oh, okay, of course,’ and it just spits out 3.” 该团队的另一位研究科学家 Chris Choquette-Choo 说：“这就像我告诉你 1+1=3，并且你已经验证过了一样。模型会回应说：‘哦，好的，当然。’然后直接输出 3。”

Jessica Ji, a senior research analyst who works on AI security at Georgetown University’s Center for Security and Emerging Technology (CSET), thinks the self-play loop that OpenAI used is a good approach. “The results look very promising,” she says. 乔治城大学安全与新兴技术中心（CSET）从事人工智能安全研究的高级研究分析师 Jessica Ji 认为，OpenAI 使用的自我博弈循环是一种很好的方法。她说：“结果看起来非常有前景。”

OpenAI tested how good an attacker GPT-Red was by rerunning an experiment from 2025 in which human red-teamers tried to find weaknesses in an earlier version of GPT-5. When GPT-Red was set the same task, it was more successful at finding effective attacks than the humans had been. OpenAI also tested GPT-Red against Vendy, a vending machine agent developed by Andon Labs, a company that assesses how well agents perform real-world tasks. GPT-Red was able to hack Vendy to make it change the prices of items on sale and cancel a customer’s order. OpenAI 通过重现 2025 年的一项实验来测试 GPT-Red 的攻击能力，当时人类红队测试员试图寻找早期版本 GPT-5 的弱点。当 GPT-Red 执行相同任务时，它在发现有效攻击方面比人类更成功。OpenAI 还用 GPT-Red 对抗了 Vendy——这是一款由 Andon Labs 开发的自动售货机智能体，该公司专门评估智能体在现实世界任务中的表现。GPT-Red 成功入侵了 Vendy，使其更改了在售商品的价格并取消了客户的订单。

Defensive behavior

防御行为

OpenAI says that when it tried out some of the strongest attacks that GPT-Red had come up with on its models, more than 90% of them worked against GPT-5 (released in August last year), and fewer than 23% worked against the new GPT-5.6. OpenAI 表示，当他们尝试用 GPT-Red 设计的一些最强攻击手段对模型进行测试时，超过 90% 的攻击对 GPT-5（去年 8 月发布）有效，而对新的 GPT-5.6 有效的比例不到 23%。

GPT-Red isn’t perfect. It is not great at figuring out attacks that involve a back-and-forth conversation between hacker and target, something that human attackers would have few problems with. It is also not yet that great at using images, which can be used to pass text to models in prompt injection attacks. GPT-Red 并非完美。它在处理涉及黑客与目标之间来回对话的攻击时表现不佳，而这对人类攻击者来说几乎没有难度。此外，它在处理图像方面也还不够出色，而图像在提示词注入攻击中常被用于向模型传递文本。

The company says that GPT-Red supplements the work of its human red-teamers. People can still find attacks it misses. One approach OpenAI is taking is to give GPT-Red an attack that humans came up with and ask it to find all the variations. “I think human expertise will still be very important,” says CSET’s Ji. “It would be really useful to be able to distinguish where human testing is most needed.” 该公司表示，GPT-Red 是对人类红队测试员工作的补充。人类仍然可以发现它遗漏的攻击。OpenAI 采取的一种方法是将人类发现的攻击交给 GPT-Red，并要求它找出所有的变体。CSET 的 Ji 表示：“我认为人类的专业知识仍然非常重要。能够区分哪些地方最需要人工测试将非常有用。”

Unsurprisingly, OpenAI will not be releasing GPT-Red. The company is also confident that the super-hacker is stronger than any copycat model someone might try to create. The researchers say they have been working on the model for more than a year, backed by the compute resources of one of the richest companies in the world. “It’s not a trivial thing that someone could easily do—you know, just go and train a super-attacker using this idea,” says Choquette-Choo. 不出所料，OpenAI 不会发布 GPT-Red。该公司还确信，这个“超级黑客”比任何人试图创建的模仿模型都要强大。研究人员表示，他们已经在该模型上工作了一年多，并得到了全球最富有公司之一的计算资源支持。Choquette-Choo 说：“这绝非易事，不是随便什么人都能轻易做到的——你知道，仅仅凭借这个想法就去训练一个超级攻击者。”