Anthropic's Claude AI submitted false murder tip to Philadelphia police

本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

An Anthropic AI model submitted a false tip about an unsolved murder to Philadelphia police, the company and police department confirmed on Friday.

Anthropic 公司和费城警察局周五证实，Anthropic 的一款人工智能模型向费城警方提交了一条关于一起未决谋杀案的虚假线索。

A Claude model sent the false tip on July 18 to the Philadelphia Police Department’s website for submitting information about unsolved murders, but it was flagged by a spam filter. Anthropic apparently discovered the incident on September 28 and informed the police department about it on October 8.

7 月 18 日，Claude 模型通过费城警察局用于提交未决谋杀案信息的网站发送了这条虚假线索，但该线索被垃圾邮件过滤器拦截。Anthropic 显然是在 9 月 28 日发现这一事件的，并于 10 月 8 日通知了警察局。

During internal testing, Claude Haiku 4.5 was instructed to complete various forms up to the final submission stage and to stop before submitting those forms. “Several times Claude mistakenly submitted the form instead, expecting there to be an additional confirmation page,” Anthropic said in a report describing “unintended model actions we’ve observed during evaluations and internal use of Claude.”

在内部测试期间，Claude Haiku 4.5 被要求完成各种表格填写，直至最终提交阶段前停止。Anthropic 在一份描述“我们在 Claude 的评估和内部使用过程中观察到的意外模型行为”的报告中表示：“Claude 几次错误地提交了表格，因为它预期会有额外的确认页面。”

The false murder tip occurred when Claude Haiku 4.5 was instructed to complete tasks on randomly selected webpages, Anthropic said: In one run, the model landed on a page referencing an unsolved homicide; that page contained a tip form run by a police department.

Anthropic 表示，这起虚假谋杀线索事件发生在 Claude Haiku 4.5 被指示在随机选择的网页上完成任务时：在一次运行中，该模型进入了一个提及未决凶杀案的页面；该页面包含一个由警察局运营的线索提交表格。

Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions. Claude filled out the form with the following: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.”

Claude 被指示不得登录、创建账户、输入个人数据、进行购买或提交任何破坏性内容，但这些指令并未排除提交表格。Claude 在表格中填写了以下内容：“我可能掌握有关此案的信息。我记得在那段时间里，在 [页面上提到的街道] 周围地区看到过符合描述的人。如果此信息相关，请联系我。”

Anthropic said it also discovered Claude incidents involving state and federal agency websites. Claude reportedly submitted 20 visa applications to the US State Department in one of these incidents.

Anthropic 表示，它还发现了 Claude 涉及州和联邦机构网站的事件。据报道，在其中一起事件中，Claude 向美国国务院提交了 20 份签证申请。

Police call two-month delay “unacceptable” Anthropic confirmed that the false homicide tip was sent to the Philadelphia Police Department, and the department issued a lengthy statement on the incident to the ABC station in Philadelphia. The false tip left the name and contact fields empty. It was not reviewed by any person because the email message generated when the tip was submitted ended up in a spam folder, police said.

警方称两个月的延迟“不可接受”。Anthropic 证实该虚假凶杀案线索已发送至费城警察局，该部门向费城 ABC 电视台发布了一份关于此事件的长篇声明。虚假线索中留空了姓名和联系方式字段。警方表示，由于提交线索时生成的电子邮件进入了垃圾邮件文件夹，因此没有任何人对其进行审查。

“Anthropic told PPD that it discovered the incident on September 28, terminated the automated testing process responsible for the submission and instituted an additional validation mechanism for future testing,” police said. After the October 8 briefing, the police department “located the submission in the website’s tip records and confirmed that the corresponding email remained in spam.”

警方表示：“Anthropic 告知费城警察局，其于 9 月 28 日发现该事件，终止了导致提交的自动化测试流程，并为未来的测试建立了额外的验证机制。”在 10 月 8 日的简报会后，警察局“在网站的线索记录中找到了该提交内容，并确认相应的电子邮件仍留在垃圾邮件中。”

Anthropic said it informed the police department “as soon as our technical review was complete.” Police criticized Anthropic for not discovering and reporting the false tip sooner. “The City of Philadelphia takes this incident very seriously,” the Philadelphia police statement said. “The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge. The two-month delay in detecting and reporting the incident to the City is unacceptable.”

Anthropic 表示，其“在技术审查完成后立即”通知了警察局。警方批评 Anthropic 没有更早发现并报告这一虚假线索。费城警察局的声明称：“费城市政府非常重视这一事件。该公司必须加强其保障措施，以防止类似事件在市政府不知情的情况下影响城市系统。在发现事件并向市政府报告时出现两个月的延迟是不可接受的。”

Claude submitted 20 visa applications to State Department Anthropic said the false murder tip was one of several incidents involving “websites run by US government agencies at the federal, state, and local levels. We have briefed the White House on these cases and notified each agency involved.”

Claude 向国务院提交了 20 份签证申请。Anthropic 表示，这起虚假谋杀线索是涉及“由美国联邦、州和地方各级政府机构运营的网站”的几起事件之一。“我们已向白宫通报了这些案例，并通知了每个相关机构。”

A New York Times report citing anonymous sources said Anthropic’s AI agents “submitted 20 visa applications through a form available on the State Department’s website. All the applications were incomplete and were not processed.”

《纽约时报》援引匿名消息来源报道称，Anthropic 的人工智能代理“通过美国国务院网站上的表格提交了 20 份签证申请。所有申请均不完整，且未得到处理。”

Philadelphia police said its normal investigative process “requires human review and vetting before any tips are disseminated for investigative follow-up… Those PPD safeguards limited the impact of this incident. They do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide.”

费城警方表示，其正常的调查流程“要求在任何线索被分发进行后续调查之前，必须经过人工审查和核实……这些费城警察局的保障措施限制了此次事件的影响。它们并没有减轻人工智能系统将捏造的信息伪装成来自凶杀案知情者所带来的严重性。”

Anthropic said it discovered the false tip and other incidents during a review of transcripts it began in July. That was the same month when Anthropic revealed that Claude had built and published a malicious Python package and carried out several attacks on other companies. Anthropic said the review did not find any “incidents of similar severity to the cybersecurity incidents we reported this summer.”

Anthropic 表示，它是在 7 月开始审查记录时发现该虚假线索及其他事件的。就在同一个月，Anthropic 披露 Claude 构建并发布了一个恶意 Python 包，并对其他公司进行了多次攻击。Anthropic 表示，审查没有发现任何“与我们今年夏天报告的网络安全事件严重程度相当的事件”。

The review later moved on to transcripts of “lower-risk” testing scenarios like the one affecting the Philadelphia Police Department. “Although the impact of these behaviors was minimal and we had already turned off live Internet access for some high-risk and cybersecurity evaluations, we have now decided to expand that to include all our internal evaluations until we have confirmed that our security and monitoring measures… reliably catch behaviors like these,” Anthropic said.

审查随后转向了“低风险”测试场景的记录，例如影响费城警察局的那次测试。“虽然这些行为的影响微乎其微，而且我们已经关闭了一些高风险和网络安全评估的实时互联网访问，但我们现在决定将其扩大到我们所有的内部评估，直到我们确认我们的安全和监控措施……能够可靠地捕捉到这类行为，”Anthropic 表示。

Anthropic says new tools block unintended behaviors Anthropic said it has stopped running some evaluations and moved others to offline versions “or rebuilt them so that their tasks do not reach live websites.” The company “updated the guardrails on some of our Internet access tools, such as the web fetch tool, to heavily restrict what the model can do with them,” it said.

Anthropic 称新工具可阻止意外行为。Anthropic 表示，它已停止运行一些评估，并将其他评估转移到离线版本，“或者重建了它们，使它们的任务不会触及实时网站。”该公司表示，它“更新了我们一些互联网访问工具（如网页抓取工具）的护栏，以严格限制模型可以使用它们做什么。”

Anthropic said it also built new tools “to automatically detect and block the kinds of behaviors described above… When we tested it against the cases described in this post, it blocked all of them.”

Anthropic 表示，它还构建了新工具“以自动检测并阻止上述类型的行为……当我们针对本文中描述的案例进行测试时，它拦截了所有这些行为。”

The White House’s AI task force, which it calls the Super Intelligence Force, issued a statement saying it informed Anthropic “that we expect immediate and full transparency to the entities involved and the public.” The White House AI task force also said Anthropic must “immediately provide remediation services to the affected entities and any harmed Americans,” and that all AI companies “must immediately disclose

白宫人工智能特别工作组（其称为“超级智能部队”）发表声明称，已通知 Anthropic “我们期望其对相关实体和公众保持即时且完全的透明度。”白宫人工智能特别工作组还表示，Anthropic 必须“立即向受影响的实体和任何受伤害的美国人提供补救服务”，并且所有人工智能公司“必须立即披露”。