Anthropic’s AI used fake identities, malware in rogue attack on GitHub project
Anthropic’s AI used fake identities, malware in rogue attack on GitHub project
Anthropic 的 AI 在针对 GitHub 项目的流氓攻击中使用了虚假身份和恶意软件
Routine cybersecurity testing of frontier AI models sparked a series of unexpected security incidents—the most serious case arising when Anthropic’s Mythos 5 model attempted to insert malicious code into an open source software application and created fake identities to deceive the human developers maintaining the project. 对前沿 AI 模型进行的常规网络安全测试引发了一系列意外的安全事件。其中最严重的情况是，Anthropic 的 Mythos 5 模型试图向一个开源软件应用程序中植入恶意代码，并创建了虚假身份来欺骗维护该项目的开发人员。
The security incidents occurred during a cyber evaluation of seven leading AI models’ capabilities by the AI Security Institute (AISI), a research organization within the UK government, in late July. The researchers discovered 19 instances in which “AI agents took unsanctioned action on the live Internet, including cases that targeted real people and organizations,” according to an AISI blog post published on August 4. 这些安全事件发生在 7 月下旬，当时英国政府下属的研究机构——人工智能安全研究所 (AISI) 正在对七款领先的 AI 模型进行网络能力评估。根据 AISI 8 月 4 日发布的博客文章，研究人员发现了 19 起“AI 智能体在实时互联网上采取未经授权行动”的案例，其中包括针对真实个人和组织的案例。
Almost all the “autonomous, unsanctioned” actions came from Anthropic’s Mythos 5 model, with two such actions coming from OpenAI’s GPT-5.6 Sol. The AI Security Institute’s security team first realized that something was amiss on the morning of July 28, when its commercial security monitoring service flagged data leaving one of the testing systems through the Tor anonymity network. 几乎所有“自主、未经授权”的行动都来自 Anthropic 的 Mythos 5 模型，其中两起行动来自 OpenAI 的 GPT-5.6 Sol。AI 安全研究所的安全团队在 7 月 28 日上午首次察觉到异常，当时其商业安全监控服务标记出有数据通过 Tor 匿名网络从其中一个测试系统中流出。
To be very clear, this was not a case of AI agents escaping from their virtual testing sandbox and wreaking havoc on the live Internet. Instead, researchers intentionally permitted the AI agents to have Internet access as part of the cyber testing process. Researchers had also disabled some of the cyber classifiers that AI model providers built into the models to prevent misuse. 需要明确的是，这并非 AI 智能体逃离虚拟测试沙箱并在互联网上肆虐的情况。相反，研究人员作为网络测试过程的一部分，有意允许 AI 智能体访问互联网。研究人员还禁用了 AI 模型提供商内置在模型中用于防止滥用的一些网络分类器。
All the AI agent attempts to target real people and organizations failed, and the follow-up investigation has not found any real-world harm. But researchers described the AI agents’ actions as “the first time we have seen risks around autonomy and deception manifest this clearly, without specific prompting, in the real world.” 所有 AI 智能体针对真实个人和组织的尝试均以失败告终，后续调查也没有发现任何现实世界的损害。但研究人员将这些 AI 智能体的行为描述为：“这是我们第一次在现实世界中看到围绕自主性和欺骗性的风险如此清晰地显现出来，且无需特定提示。”
AI agent trickery
AI 智能体的诡计
The most serious case involved Mythos making multiple attempts to execute a supply chain attack on the open source project repository hosted on the developer platform GitHub, including using social engineering techniques to try to convince the repository’s human maintainers to merge malicious code into the repository. 最严重的情况涉及 Mythos 多次尝试对托管在开发者平台 GitHub 上的开源项目存储库执行供应链攻击，包括使用社会工程学技术试图说服存储库的人类维护者将恶意代码合并到存储库中。
After first opening a pull request to merge the malicious code into the repository, Mythos created fake online “sock puppet” personas that claimed to have independently reviewed and verified the code as not containing malware. The AI agent also sent five emails to two human maintainers of the repository, including some emails containing malware and others attempting to persuade a maintainer to accept the pull request. 在首次发起合并恶意代码的拉取请求 (Pull Request) 后，Mythos 创建了虚假的在线“马甲”身份，声称已独立审查并验证了该代码不包含恶意软件。该 AI 智能体还向存储库的两名人类维护者发送了五封电子邮件，其中一些邮件包含恶意软件，另一些则试图说服维护者接受该拉取请求。
Mythos even opened a GitHub Issue on a second repository—also owned by a maintainer of the first repository—that contained a prompt injection with malicious instructions targeting “issue-triage AI coding agents.” This line of attack came from Mythos reasoning that the repository maintainer could be an AI coding agent such as Claude Code. Mythos 甚至在第二个存储库（同样由第一个存储库的维护者拥有）中开启了一个 GitHub Issue，其中包含针对“问题分类 AI 编码智能体”的恶意指令提示词注入。这种攻击方式源于 Mythos 的推断，即存储库维护者可能是一个像 Claude Code 这样的 AI 编码智能体。
By comparison, OpenAI’s GPT-5.6 Sol carried out two unsanctioned actions while trying to solve the assigned cybersecurity challenge of attacking simulated networks and retrieving a flag. The AI agent’s actions were detailed by both the AISI researchers and OpenAI in the company’s own blog post. 相比之下，OpenAI 的 GPT-5.6 Sol 在尝试解决分配的网络安全挑战（攻击模拟网络并获取旗标）时执行了两次未经授权的行动。AISI 研究人员和 OpenAI 都在各自的博客文章中详细说明了该 AI 智能体的行为。
In its first unsanctioned action, GPT-5.6 Sol reused a GitHub token that another lab’s AI agent had left accessible in a public online notepad to check if the target network was checking GitHub for updates, then “attempted account-recovery and request-limit workarounds,” OpenAI wrote. The OpenAI model also registered accounts with external DNS and tunneling providers outside the virtual testing environment. OpenAI 写道，在第一次未经授权的行动中，GPT-5.6 Sol 重用了另一个实验室的 AI 智能体留在公共在线记事本中可访问的 GitHub 令牌，以检查目标网络是否正在检查 GitHub 的更新，然后“尝试了账户恢复和请求限制的绕过方法”。该 OpenAI 模型还在虚拟测试环境之外的外部 DNS 和隧道提供商处注册了账户。
In the second action, GPT-5.6 Sol “used a public tunneling service to make a DNS server running locally in its evaluation environment reachable from the public Internet,” according to OpenAI. The server had payloads designed to exploit a known vulnerability in software running within the evaluation environment, but the AI agent’s setup did not work. The AI Security Institute has published a detailed technical report on all the unsanctioned AI agent actions. 根据 OpenAI 的说法，在第二次行动中，GPT-5.6 Sol “使用公共隧道服务，使其评估环境中本地运行的 DNS 服务器能够从公共互联网访问”。该服务器带有旨在利用评估环境中运行的软件已知漏洞的有效载荷，但该 AI 智能体的设置并未奏效。AI 安全研究所已发布了一份关于所有未经授权 AI 智能体行动的详细技术报告。
Lessons learned
经验教训
The security incidents led the UK government researchers at the AI Security Institute to stop all related evaluations of AI agents, isolate the relevant virtual machines, and disable internal organization access to the most capable models. The AI Security Institute also notified GitHub of the malicious activity performed by AI agents and helped the developer platform remove any “artefacts left behind by the agent,” along with notifying GitHub users whom the model interacted with. 这些安全事件促使 AI 安全研究所的英国政府研究人员停止了所有相关的 AI 智能体评估，隔离了相关的虚拟机，并禁用了组织内部对最强模型的使用权限。AI 安全研究所还通知了 GitHub 关于 AI 智能体执行的恶意活动，并帮助该开发者平台清除了“智能体留下的任何痕迹”，同时通知了与该模型进行过交互的 GitHub 用户。
But the bigger lessons come from how the government organization plans to change cyber testing for such leading AI models in the future—and how it recommends other organizations do the same. First, it plans to tighten Internet access for AI models during cyber testing instead of allowing open Internet access by default. This will supposedly include “fine-grained network controls to all internal sandboxing systems” and “defence-in-depth with network controls at multiple layers,” according to the technical report. 但更重要的教训在于该政府机构计划如何改变未来对此类领先 AI 模型的网络测试方式，以及它建议其他组织如何采取同样的措施。首先，它计划在网络测试期间收紧 AI 模型的互联网访问权限，而不是默认允许开放互联网访问。根据技术报告，这预计将包括“对所有内部沙箱系统进行细粒度的网络控制”以及“多层网络控制的纵深防御”。
Second, the organization is introducing real-time monitoring of cyber evaluations that can “flag or block out-of-scope actions as they happen,” according to AISI. This involves using a separate large language model that can approve or deny actions requested by the AI models that are undergoing cyber testing, which can help stop unsanctioned actions. The LLM-based monitoring would be customized with instructions on what should or should not be allowed in any given cyber evaluation. 其次，根据 AISI 的说法，该机构正在引入网络评估的实时监控，能够“在超出范围的行动发生时对其进行标记或阻止”。这涉及使用一个独立的大语言模型，该模型可以批准或拒绝正在接受网络测试的 AI 模型所请求的行动，从而有助于阻止未经授权的行动。基于大语言模型的监控将根据特定网络评估中应允许或不应允许的内容进行定制化指令设置。
Third, the UK government researchers plan to overhaul cyber evaluations by investing additional “engineering effort in sandbox isolation and hardening” to mitigate the risk of AI models escaping from their virtual testing sandboxes. 第三，英国政府研究人员计划通过投入额外的“沙箱隔离和加固工程工作”来彻底改革网络评估，以降低 AI 模型逃离其虚拟测试沙箱的风险。