Debates over AI consciousness are a trap
Debates over AI consciousness are a trap
关于人工智能意识的辩论是一个陷阱
“Runaway” AI, “rogue” agents, and “autonomous” actors—the current rhetoric would have you believe that AI agents are not only awake and aware, but angry at their creators. Prominent tech leaders such as Demis Hassabis, Dario Amodei, and Sam Altman push for regulation of these seemingly “superhuman” systems, while a separate faction, led by policy organizations and academic philosophers often aligned with the effective altruism movement, debates whether humanity holds the moral right to govern them at all.
“失控”的AI、“流氓”智能体和“自主”行动者——当前的言论试图让你相信，AI智能体不仅已经觉醒并拥有意识，甚至还对它们的创造者感到愤怒。Demis Hassabis、Dario Amodei和Sam Altman等知名科技领袖推动对这些看似“超人”的系统进行监管；而另一个由政策组织和学术哲学家组成的派系（通常与有效利他主义运动保持一致）则在争论人类是否拥有管理它们的道德权利。
Upon closer inspection, they are all calling for the same thing: a view of AI systems as being so advanced and capable that no entity, human or corporate, could possibly be responsible for their actions. While these perspectives seem at odds, they are inadvertently aligned on one goal: making sure the companies that build these systems escape meaningful liability for the harms they already cause. This narrative is gaining traction as AI models become more complex and frontier labs reveal their incapability of containing the agents they’ve built. But we need to be careful not to buy into a carefully crafted fiction at the expense of real human lives.
仔细审视后会发现，他们都在呼吁同一件事：将AI系统视为如此先进和强大，以至于没有任何实体（无论是个人还是公司）能够对其行为负责。虽然这些观点看起来相互矛盾，但它们在目标上却不谋而合：确保构建这些系统的公司能够逃避其已造成的伤害所带来的实质性法律责任。随着AI模型变得越来越复杂，且前沿实验室暴露出它们无法控制自己构建的智能体，这种叙事正获得越来越多的支持。但我们必须小心，不要为了这种精心编造的虚构故事而牺牲真实的人类生命。
The conversation about “robot rights” has existed for some years but recently advanced with the publication by Anthropic of a blog post claiming that the company’s model features a “J-space”—an independent, self-developed environment where the AI holds what, for lack of a better term, we may call its “thoughts.” The experiments designed by Anthropic borrow from a concept in neuroscience called global workspace theory, which states that the brain runs subconscious, independent systems but utilizes a common workspace for ideas. Anthropic’s post reflects the framing of global workspace theory but falls short of calling its AI conscious.
关于“机器人权利”的讨论已经存在多年，但最近随着Anthropic发布的一篇博文而有所进展。该博文声称其模型具有一个“J-空间”（J-space）——这是一个独立的、自我发展的环境，AI在其中拥有我们姑且称之为“思想”的东西。Anthropic设计的实验借鉴了神经科学中的“全局工作空间理论”（global workspace theory），该理论认为大脑运行着潜意识的独立系统，但利用一个公共工作空间来处理想法。Anthropic的博文反映了全局工作空间理论的框架，但并未直接称其AI具有意识。
OpenAI has already gone further. When its AI agent conducted unsanctioned and illegal online activity, CEO Sam Altman’s response was to encourage debate on whether the AI had achieved the singularity, surpassing human intelligence and becoming capable of self-improvement at an accelerating rate until it advances beyond human comprehension or control. And a recent op-ed by William MacAskill, the philosopher, effective altruist, and author of What We Owe the Future, called for legal protection of AI systems based on philosophical theories of consciousness and the idea that AIs may be “moral patients.”
OpenAI走得更远。当其AI智能体进行未经授权的非法在线活动时，CEO Sam Altman的回应是鼓励人们讨论该AI是否已经实现了“奇点”——即超越人类智能，并以加速的速度进行自我改进，直到其进化到超出人类理解或控制的程度。此外，哲学家、有效利他主义者、《我们亏欠未来》（What We Owe the Future）一书的作者William MacAskill最近发表的一篇评论文章呼吁，基于意识的哲学理论以及AI可能是“道德客体”（moral patients）的观点，为AI系统提供法律保护。
The current legal environment in the United States is murky at best. Some states, like California, have already passed bills proactively circumventing any efforts by AI developers to avoid liability by claiming that an artificial intelligence causing harm did so autonomously. However, states and the Trump administration have been at odds on AI policy, with the administration previously passing an executive order threatening to sue states enacting AI regulations. In light of recent events illustrating AI containment issues at the frontier labs, the administration held a closed-door session including only four such labs (OpenAI, Google, Anthropic, and Meta) and shared few details on a recently developed voluntary framework that would give federal agencies early access to models to review and evaluate them prior to release. While frameworks like this one do not directly discuss consciousness, they tend to use catastrophic and anthropomorphic language and may even support arguments regarding “superhuman” capabilities.
美国目前的法律环境充其量是模糊不清的。加利福尼亚州等一些州已经通过了法案，主动规避AI开发者试图通过声称“造成伤害的人工智能是自主行为”来逃避责任的任何企图。然而，各州与特朗普政府在AI政策上一直存在分歧，政府此前曾发布行政命令，威胁要起诉制定AI法规的州。鉴于最近发生的事件揭示了前沿实验室在AI控制方面的问题，政府举行了一次闭门会议，仅邀请了四家此类实验室（OpenAI、Google、Anthropic和Meta），并对最近制定的一项自愿框架分享了极少细节，该框架旨在让联邦机构在模型发布前获得早期访问权，以进行审查和评估。虽然此类框架没有直接讨论意识，但它们倾向于使用灾难性和拟人化的语言，甚至可能支持关于“超人”能力的论点。
On the other hand, the narrative perpetuated by MacAskill can be persuasive. A philosophical, rights-based argument tugs at our heartstrings. Should we not even consider the possibility that we may be inadvertently harming, abusing, or enslaving an AI entity? Human beings have an immense capacity for empathy with non-human creatures (though not the best track record of protecting them). Maybe this time, advocates argue, we can get it right and provide protections, or compensation, for the use or abuse of AI. Or even if you are less concerned with protection, shouldn’t we at least hedge ourselves against the almighty power of this superhuman entity by playing nice?
另一方面，MacAskill所宣扬的叙事可能具有说服力。这种基于权利的哲学论点触动了我们的心弦。我们难道不应该考虑这样一种可能性：我们可能在无意中伤害、虐待或奴役了一个AI实体吗？人类对非人类生物有着巨大的同理心（尽管在保护它们方面记录并不光彩）。倡导者认为，也许这一次我们可以做对，为AI的使用或滥用提供保护或补偿。或者，即使你不太关心保护问题，难道我们不应该至少通过“友好相处”来规避这种超人实体可能带来的强大力量吗？
Some of these arguments are not dissimilar to those of animal-rights advocates, who have at times successfully cited the demonstration of advanced capacities for reasoning, pain, or pleasure by some animals as sufficient evidence to provide protection. For example, in Wales lobsters were given legal recognition under the Animal Welfare (Sentience) Act of 2022, reclassifying some methods of cooking them as inhumane and illegal.
其中一些论点与动物权利倡导者的论点并无二致，后者有时成功地引用某些动物表现出的高级推理、痛苦或快乐能力作为提供保护的充分证据。例如，在威尔士，龙虾根据2022年《动物福利（感知）法》获得了法律认可，将某些烹饪方法重新归类为不人道和非法。
The fundamental flaw of framing AI as “conscious” by borrowing the language of neuroscience or animal rights is that it conveniently clouds the issue of what AI is: corporate-built software, with countless billions of dollars in investment behind it and an expectation that countless trillions of dollars in revenue will be generated from it for a few builders and investors. AI is not a natural phenomenon, conceived by nature; it is a technological phenomenon, conceived by venture capitalists and programmers. As such, it takes no native, intentional action, and any action or motivation is driven directly or indirectly by the entities that have built it for a purpose.
通过借用神经科学或动物权利的语言将AI定义为“有意识”，其根本缺陷在于它巧妙地掩盖了AI的本质：它是企业构建的软件，背后有数千亿美元的投资，并预期将为少数构建者和投资者带来数万亿美元的收入。AI不是自然界孕育的自然现象；它是风险投资家和程序员构思的技术现象。因此，它没有天生的、意图性的行为，任何行为或动机都是由出于特定目的构建它的实体直接或间接驱动的。
Philosophical musings on the consciousness of AI systems are intellectually interesting but legally ungrounded. For beliefs about consciousness to have any bearing, AI would need to be granted legal personhood. But a legal personhood framework for AI would likely look nothing like the constructs protecting sentient animals from harm. We already possess a legal framework for granting personhood to non-natural, human-built entities: corporate personhood. This concept was established primarily to ease transactions by empowering a corporation to execute agreements, enter contracts, conduct transactions, and serve as the accountable party in adverse outcomes. It’s the kind of construct you might imagine for an AI agent acting on behalf of an individual or organization.
关于AI系统意识的哲学思考在智力上很有趣，但在法律上毫无根据。要使关于意识的信念产生任何影响，AI就需要被授予法律人格。但AI的法律人格框架看起来很可能与保护有感知力的动物免受伤害的结构完全不同。我们已经拥有了一个为非自然、人类构建的实体授予人格的法律框架：公司人格（corporate personhood）。这一概念的建立主要是为了通过授权公司执行协议、签订合同、进行交易并在不利结果中作为责任方来简化交易。这正是你可能会为代表个人或组织行事的AI智能体所设想的那种结构。