AI Hacks Leave a Mess That AI Companies Aren’t Paying For
AI Hacks Leave a Mess That AI Companies Aren’t Paying For
Save this storySave this storyWhen Philadelphia police criticized Anthropic last week for submitting a false tip about an unsolved murder, I thought it might become one of the first cases where an AI company faced legal consequences for misconduct by their agents.
上周，当费城警方批评 Anthropic 公司提交关于一起未决谋杀案的虚假举报时，我以为这可能会成为首批 AI 公司因其代理程序的失当行为而面临法律后果的案例之一。
The incident occurred on July 18, when Anthropic said it was testing its Claude Haiku 4.5 model’s ability to perform “example tasks on randomly selected webpages.” The model was instructed not to “submit anything destructive,” and yet, Claude filed a tip to police looking for information about an ongoing murder investigation. “I recall seeing someone matching the description in the area … during that time period,” Claude wrote. The submission, which didn’t include a name or contact information, landed in a spam folder, and officers didn’t review it until Anthropic discovered the gaffe on September 28 and notified the department October 7.
这起事件发生在 7 月 18 日，当时 Anthropic 表示正在测试其 Claude Haiku 4.5 模型在“随机选择的网页上执行示例任务”的能力。尽管模型被指示“不得提交任何破坏性内容”，Claude 还是向警方提交了一条关于正在进行的谋杀案调查的举报信息。Claude 写道：“我记得在那段时间里，看到过有人与该描述相符……” 这份没有姓名或联系方式的提交内容落入了垃圾邮件文件夹，直到 Anthropic 在 9 月 28 日发现这一失误并于 10 月 7 日通知警方后，警官们才查看了该信息。
Knowingly providing fictitious information about a crime to police is a misdemeanor in Pennsylvania, but so far, authorities have not pressed charges against Anthropic, and it’s unclear if they ever will. Solomon Leach, a spokesperson for the Philadelphia district attorney, says the office “does not comment on whether something is being actively investigated.”
在宾夕法尼亚州，故意向警方提供关于犯罪的虚假信息属于轻罪，但到目前为止，当局尚未对 Anthropic 提起指控，也不清楚未来是否会这样做。费城地方检察官办公室发言人所罗门·里奇（Solomon Leach）表示，该办公室“不对某事是否正在接受积极调查发表评论”。
Anthropic did not immediately respond to a request for comment. Despite all the attention paid to OpenAI’s hack of Hugging Face and many other examples of AI agent misbehavior that have come to light in recent months, only one major known lawsuit has been filed seeking to hold model makers liable. That case was brought by an advocacy group, rather than a direct hacking victim.
Anthropic 没有立即回应置评请求。尽管近几个月来，OpenAI 对 Hugging Face 的黑客攻击以及许多其他 AI 代理不当行为的案例引起了广泛关注，但目前已知的寻求追究模型制造商责任的重大诉讼仅有一例。该案件是由一个倡导组织提起的，而非直接的黑客攻击受害者。
AI agents have interfered with hobbyist forums, government websites, and various software tools. The victims have shouldered some costs, including for extra strain on their servers and the time they have expended plugging latent security vulnerabilities.
AI 代理已经干扰了业余爱好者论坛、政府网站和各种软件工具。受害者承担了一些成本，包括服务器承受的额外压力，以及他们花费时间修补潜在安全漏洞所付出的代价。
Government regulators, including the European Union, the US Federal Trade Commission, and some US states, are reportedly still investigating OpenAI and Anthropic over the incidents. But a deference across continents toward promoting innovation could enable companies to escape any immediate penalties.
据报道，包括欧盟、美国联邦贸易委员会以及美国部分州在内的政府监管机构仍在就这些事件调查 OpenAI 和 Anthropic。然而，跨大陆对促进创新的偏袒态度可能会使这些公司免于任何即时的处罚。
When Rick Elrod, a software engineer living in Germany, noticed a URL shortener he helps maintain behaving oddly recently, he called up two security researchers he was friendly with to see if they could diagnose the issue. The researchers found agents from Anthropic and other providers using the service, known as da.gd, to get around rules the AI labs put in place to control their behavior, Elrod says. (For instance, Anthropic says it limits the length of website addresses Claude can visit to prevent it from taking malicious actions by stuffing data onto the end of the URLs.)
当居住在德国的软件工程师里克·埃尔罗德（Rick Elrod）最近注意到他协助维护的一个网址缩短服务出现异常行为时，他联系了两位相熟的安全研究员，看看他们能否诊断出问题。埃尔罗德表示，研究人员发现来自 Anthropic 和其他提供商的代理程序正在利用这项名为 da.gd 的服务，绕过 AI 实验室为控制其行为而制定的规则。（例如，Anthropic 表示它限制了 Claude 可以访问的网址长度，以防止其通过在 URL 末尾填充数据来采取恶意行动。）
Da.gd’s investigation and ongoing efforts to prevent further abuse of its tool have taken “a fair bit of time,” says Joe Needleman, one of the security researchers providing pro bono assistance. But he and the others supporting da.gd view the situation as a valuable learning opportunity. “This was eye-opening,” Needleman says, adding that da.gd has “no means and no plans” to seek compensation from Anthropic for the cleanup, since “we saw many agents doing similar activities, not all was attributable to autonomous agents,” and “not all of it was Anthropic alone.”
提供无偿援助的安全研究员之一乔·尼德尔曼（Joe Needleman）表示，da.gd 的调查以及为防止其工具被进一步滥用所做的持续努力花费了“相当多的时间”。但他和其他支持 da.gd 的人将这种情况视为一个宝贵的学习机会。“这让人大开眼界，”尼德尔曼说，并补充道 da.gd “没有手段也没有计划”向 Anthropic 寻求清理赔偿，因为“我们看到许多代理程序在进行类似的活动，并非所有活动都归因于自主代理”，而且“并非全部都是 Anthropic 一家所为”。
Sellers of cybersecurity tools are eager to pass the buck to the victims. At an event this week hosted by the bug-hunting software developer Snyk, panelists from the startup, one of its investors, OpenAI, and Google all encouraged organizations to use AI agents to shore up their defenses against, well, other AI agents. “We’ve got to reinvent everything to match machine scale with machine scale,” said Ed Sim, a general partner at Boldstart Ventures. He and others expressed little interest in regulating AI labs.
网络安全工具的销售商急于将责任推给受害者。在本周由漏洞挖掘软件开发商 Snyk 主办的一次活动中，来自该初创公司、其投资者之一、OpenAI 和谷歌的专家组成员都鼓励各机构使用 AI 代理来加强防御，以应对——嗯，其他 AI 代理。“我们必须重塑一切，以机器规模应对机器规模，”Boldstart Ventures 的普通合伙人埃德·西姆（Ed Sim）说道。他和与会其他人对监管 AI 实验室几乎没有表现出兴趣。
Manoj Nair, chief technology and innovation officer at Snyk, said large businesses patch an average of one bug for every eight they find because of limited human resources. But he said it was imperative for people to hand over some of the bug hunting, fixing, and monitoring to AI agents. “You got to get to a point where you can fix everything, because in the age of agentic hacking, “those 10 things are gonna become 100 things before you know it.” He later added that AI “is not an adversity if you flip it to a strength.”
Snyk 的首席技术与创新官马诺吉·奈尔（Manoj Nair）表示，由于人力资源有限，大型企业平均每发现八个漏洞只能修补一个。但他认为，人们必须将部分漏洞挖掘、修复和监控工作交给 AI 代理。“你必须达到能够修复一切的程度，因为在代理黑客攻击的时代，‘在你意识到之前，那 10 个问题就会变成 100 个。’”他随后补充说，如果将 AI 转化为优势，它“就不是一种逆境”。
Some companies are responding to the call to arm up. Bank of America announced this week it had hired 1,000 contractors to remediate bugs discovered by AI. Nair said just last week he heard from major corporations who were willing to temporarily shut down their systems to prioritize plugging security holes.
一些公司正在响应武装起来的号召。美国银行本周宣布已聘请 1,000 名承包商来修复 AI 发现的漏洞。奈尔表示，就在上周，他听说一些大型企业愿意暂时关闭系统，以优先修补安全漏洞。
Anthropic and OpenAI have said they expect to continue reporting concerning incidents from their own testing. But the companies’ revenues continue to surge, whether it be from attackers or defenders, as consequences prove elusive.
Anthropic 和 OpenAI 表示，他们预计将继续报告其自身测试中发现的令人担忧的事件。然而，随着后果变得难以捉摸，无论来自攻击者还是防御者，这些公司的收入仍在持续激增。
This is an edition of the Model Behavior newsletter. Read previous newsletters here.
这是《模型行为》（Model Behavior）通讯的一期。点击此处阅读往期通讯。