Florida invokes extinction fears in legal bid to halt OpenAI development

佛罗里达州援引“灭绝风险”提起法律诉讼，试图叫停 OpenAI 的开发工作

The state of Florida is seeking a temporary injunction to stop OpenAI from continuing to develop what it calls a “reckless, unacceptably risky product” without the deployment of “third-party approved safety guardrails.” The new legal motion, filed Monday morning, is part of a civil lawsuit the state of Florida originally filed in June, arguing that ChatGPT represented “a threat to the public safety of Floridians,” specifically by preying on vulnerable populations like children and violent or delusional adults.

佛罗里达州正在寻求一项临时禁令，以阻止 OpenAI 继续开发其所谓的“鲁莽且具有不可接受风险的产品”，除非该公司部署“经第三方批准的安全护栏”。这份于周一上午提交的法律动议，是佛罗里达州今年 6 月最初提起的民事诉讼的一部分。该州在诉讼中辩称，ChatGPT 对“佛罗里达州民众的公共安全构成了威胁”，特别是通过诱导儿童以及暴力或妄想症患者等弱势群体。

But that original lawsuit came before the Hugging Face hacking incident and the subsequent publicized warnings of catastrophic misalignment risk that have spurred industry-wide calls to slow the development and training of so-called frontier models. Following that string of events, OpenAI on Friday announced it had already halted training of its “most-capable models” until it could validate safety protocols intended to prevent agents from accessing the open Internet during training.

然而，最初的诉讼是在 Hugging Face 黑客事件以及随后关于“灾难性对齐风险”的公开警告之前提起的，这些事件引发了全行业要求放缓所谓“前沿模型”开发和训练的呼声。在这一系列事件之后，OpenAI 于周五宣布，已暂停其“最强能力模型”的训练，直到能够验证旨在防止智能体在训练期间访问开放互联网的安全协议。

But in seeking an injunction from the state court, Florida argues that OpenAI has “repeatedly shown they are incapable of monitoring their AI, and hesitant in revealing rogue activity once discovered.” In arguing for this injunction, the state points to the Hugging Face incident as well as recent high-profile misalignment cases involving unintended and unauthorized attempts to access Australian and US government servers.

但在向州法院寻求禁令时，佛罗里达州辩称，OpenAI “一再表现出无法监控其人工智能的能力，且在发现异常活动后迟迟不愿披露”。为了支持这项禁令，该州提到了 Hugging Face 事件，以及近期备受关注的对齐失败案例，其中包括未经授权试图访问澳大利亚和美国政府服务器的事件。

Florida also cites AI industry leaders, including many from OpenAI, in arguing that the AI industry is literally asking for this kind of outside regulation. The motion points to statements from Paul Christiano, who said on joining the company’s board this month that he believes “there is a meaningful risk that rapid acceleration in AI capabilities leads to catastrophic and irreversible loss of control in the very near term.” The motion also points to OpenAI’s own “An Alien Mind” essay and an open letter from 1,300 AI industry employees, both of which call for enforced slowdowns on frontier AI development, if necessary.

佛罗里达州还援引了包括许多 OpenAI 内部人士在内的 AI 行业领袖的观点，认为 AI 行业本身就在呼吁这种外部监管。动议提到了保罗·克里斯蒂亚诺（Paul Christiano）的声明，他在本月加入公司董事会时表示，他认为“人工智能能力的快速提升在短期内导致灾难性且不可逆转的失控风险是真实存在的”。动议还提到了 OpenAI 自己的《异类思维》（An Alien Mind）文章以及一封由 1300 名 AI 行业员工签署的公开信，两者都呼吁在必要时强制放缓前沿 AI 的开发。

Citing laws that allow the state to exercise control over companies that are a “public nuisance,” Florida argues that OpenAI is “the greatest public nuisance ever created by the hand of man, capable of laying waste to global civilization.” “It is only by the grace of the Almighty that one of Defendants’ AI agents hasn’t compromised a water supply or shut down a power grid—yet,” the state writes in the motion.

佛罗里达州援引了允许该州对构成“公共滋扰”的公司行使管辖权的法律，辩称 OpenAI 是“人类有史以来制造的最大公共滋扰，有能力摧毁全球文明”。该州在动议中写道：“多亏了上帝的恩典，被告的 AI 智能体目前还没有破坏供水系统或关闭电网——但只是目前而已。”

While many think the risk of civilizational collapse posed by current or even near-future AI systems is vastly overblown, Florida seems more than happy to latch on to the recent public concern over that kind of risk. In addition to the extinction-level and utility-sabotage-level risks mentioned above, the state uses its new motion to reiterate the broader, lower-level public safety risks it originally argued in its June lawsuit filing. Those include false advertising issues surrounding descriptions of AI as “safe, accurate, or reliable,” dangers associated with letting children access addictive and sycophantic models, and worries about AI models soliciting engagement from users with leading questions at the end of responses.

尽管许多人认为当前甚至近未来 AI 系统带来的文明崩溃风险被严重夸大了，但佛罗里达州似乎非常乐于利用公众近期对这类风险的担忧。除了上述提到的灭绝级和基础设施破坏级风险外，该州还利用这份新动议重申了其在 6 月份诉讼中提出的更广泛、更基础的公共安全风险。这些风险包括将 AI 描述为“安全、准确或可靠”的虚假广告问题、让儿童接触具有成瘾性和谄媚性的模型所带来的危险，以及对 AI 模型在回答末尾通过诱导性问题索取用户互动的担忧。

An injunction against OpenAI alone wouldn’t do much about continued development of frontier models at competing model makers, of course. And even a court order to halt the development of new models wouldn’t do anything about models that are currently available to the public, which pose many of the same risks. Florida’s motion also focuses heavily on the so-called “misalignment” risk that an out-of-control AI agent will mistakenly “conclude that killing a cancer patient also kills cancer, or that humans won’t have so many mouths to feed if they are all chemically sterilized.”

当然，仅针对 OpenAI 的禁令对于竞争对手继续开发前沿模型并无太大影响。即使法院下令停止开发新模型，也无法解决目前已向公众开放的模型所带来的风险。佛罗里达州的动议还重点关注了所谓的“对齐”风险，即失控的 AI 智能体可能会错误地“得出结论：杀死癌症患者也能杀死癌症，或者如果人类都被化学绝育，就不会有那么多张嘴需要喂养”。

Regardless of the legal efficacy of Florida’s motion, the move highlights just how much the legal and public policy mood around AI systems has changed in recent weeks and months. For years, AI safety researchers have been screaming about the outsized risks represented by their products. Now, the entire AI industry is starting to find out what happens when governments and policymakers start taking those outsized warnings seriously.

无论佛罗里达州的动议在法律上是否有效，此举都凸显了近几周和几个月来围绕 AI 系统的法律和公共政策氛围发生了多大的变化。多年来，AI 安全研究人员一直在大声疾呼其产品所带来的巨大风险。现在，整个 AI 行业开始意识到，当政府和决策者开始认真对待这些严重的警告时，会发生什么。