Anthropic pitches AI safety bill that may block a model if 50+ people could be killed

Over the past year, concerns have swirled over the potential dangers posed by artificial intelligence, and debates have raged over the proper path for AI safety. Now, Anthropic appears to have put forth its own vision for how the government might regulate its industry.

在过去的一年中，围绕人工智能可能带来的潜在危险，人们的担忧此起彼伏，关于人工智能安全正确路径的争论也愈演愈烈。如今，Anthropic 似乎提出了自己对于政府应如何监管该行业的愿景。

Politico recently reported that the AI lab has been shopping around a new draft bill on AI safety that it hopes Congress might consider adopting. The legislation posits certain scenarios in which the release of an AI model could be blocked from distribution. Those scenarios involve a “foreseeable, imminent risk” — one in which over 50 people are at risk of dying or if over a billion dollars in property damage could result, Politico writes.

据 Politico 近期报道，这家人工智能实验室一直在推销一份关于人工智能安全的新草案，希望国会能考虑采纳。该立法设定了某些可能阻止人工智能模型发布的情境。Politico 写道，这些情境涉及“可预见的、迫在眉睫的风险”，即可能导致超过 50 人死亡或造成超过 10 亿美元的财产损失。

The catastrophic risk in question must “stem from the creation or release of a biological weapon, the model evading its developer’s controls or a cyberattack,” the outlet notes. The metrics offered here — 50 people and a billion dollars — are similar to those in a piece of California legislation that was recently passed into law, Senate Bill 53, which similarly defined those scenarios as tantamount to “catastrophic risk.”

该媒体指出，所讨论的灾难性风险必须“源于生物武器的制造或发布、模型规避其开发者的控制，或网络攻击”。此处提供的衡量标准——50 人和 10 亿美元——与加利福尼亚州最近通过的一项法律（参议院第 53 号法案）中的标准相似，该法案同样将这些情境定义为等同于“灾难性风险”。

Anthropic’s bill suggests that analysis of a model’s potential for disaster could be arrived at via the use of “independent evaluators” who would be registered with the U.S. Commerce Department. These evaluators could potentially be paid by the AI labs themselves — so long as the “payment is not contingent on the evaluations’ results,” Politico writes. It’s not clear how this proposed system would assure that payments couldn’t influence the evaluation.

Anthropic 的法案建议，可以通过使用在美商务部注册的“独立评估员”来分析模型引发灾难的可能性。Politico 写道，这些评估员可能由人工智能实验室自行支付费用，只要“付款不以评估结果为条件”即可。目前尚不清楚这一拟议系统将如何确保付款不会影响评估结果。

Demis Hassabis, chairman of DeepMind, posited some months ago that a self-governing AI regulatory body should mimic the financial industry’s Financial Industry Regulatory Authority (FINRA). In Anthropic’s draft bill, when evaluators determine that an imminent catastrophic risk is posed, they would have 72 hours to contact the Commerce Department about it. The Commerce Department would then have to request to a D.C. judge to halt the model’s release.

DeepMind 董事长德米斯·哈萨比斯（Demis Hassabis）几个月前曾提出，一个自我管理的人工智能监管机构应效仿金融行业的金融业监管局（FINRA）。在 Anthropic 的草案中，当评估员确定存在迫在眉睫的灾难性风险时，他们有 72 小时的时间联系商务部。随后，商务部必须向华盛顿特区的法官提出请求，以阻止该模型的发布。

The legislation would expressly forbid any federal agency from having the discretion to halt the model’s distribution on its own, Politico writes. Anthropic did not immediately respond to TechCrunch’s a request for comment but a spokesperson did confirm the draft bill to Politico. “This bill represents the proposals outlined in Anthropic’s Advanced AI Framework, which we have been advocating for with federal and state lawmakers since June,” the spokesperson said.

Politico 写道，该立法将明确禁止任何联邦机构自行决定停止模型的发布。Anthropic 没有立即回应 TechCrunch 的置评请求，但一位发言人向 Politico 证实了该草案的存在。该发言人表示：“该法案体现了 Anthropic《高级人工智能框架》中概述的建议，自 6 月以来，我们一直在向联邦和州立法者倡导这些建议。”

The theoretical risks offered in the draft policy are a little bit far afield from the kinds of incidents that have been top of mind for the tech industry of late. While there are people warning of the potential of human death, the primary concern has largely been the threat of agentic hacking. A rash of incidents in which AI programs have apparently disobeyed their human operators and hacked websites on the open internet has spurred concern that models may be lacking proper security controls.

草案中提出的理论风险与科技行业近期最关注的事件类型相去甚远。虽然有人警告人工智能可能导致人类死亡，但目前的主要担忧主要集中在代理黑客攻击的威胁上。一系列人工智能程序似乎违背人类操作员意愿并入侵互联网网站的事件，引发了人们对模型可能缺乏适当安全控制的担忧。

In an effort to present a solution to the spectacle of crisis that the labs themselves have helped bring about, companies like Anthropic have offered up their own preferred solutions. Indeed, despite labs’ perpetual warnings that the technology they’re selling could result in the demise of human civilization, they have largely been resistant to mandatory government oversight.

为了解决实验室自身推波助澜引发的危机局面，像 Anthropic 这样的公司提出了他们自己偏好的解决方案。事实上，尽管实验室不断警告称他们所销售的技术可能导致人类文明的终结，但他们大多抵制强制性的政府监管。

Instead, they have suggested a voluntary system of embedded evaluators — supposedly “neutral” third-party observers that are placed inside AI organizations so that they can monitor the models for potential signs of risk or abuse. Still, Anthropic’s mere suggestion for legislation appears to go slightly further than others’ regulatory proposals offered by the industry, including Nvidia, who’s CEO Jensen Huang recently argued that the AI industry didn’t need any new laws at all.

相反，他们建议建立一个嵌入式评估员的自愿系统——即所谓的“中立”第三方观察员，他们被安置在人工智能组织内部，以便监控模型是否存在风险或滥用的潜在迹象。尽管如此，Anthropic 提出的立法建议似乎比行业内其他公司提出的监管建议要稍微激进一些，其中包括英伟达，其首席执行官黄仁勋最近主张人工智能行业根本不需要任何新法律。