Who Cares if AI Is Conscious—It’s Basically Alive

谁在乎 AI 是否有意识——它本质上已经“活”了

I spent the waning days of summer grinding away at columns and working on a feature. But I missed a chance at a striking change of scenery—cruising the Galápagos with about a dozen prominent philosophers studying consciousness. 夏末时节，我忙于撰写专栏和专题报道。但我错过了一个绝佳的换环境机会——与十几位研究意识的知名哲学家一起去加拉帕戈斯群岛巡游。

The invite described morning classroom discussions tackling knotty questions on the nature of consciousness with marquee names in the field. Afternoons would be spent on island exploration and wading and snorkeling with rare biological species. One look at the agenda and my editor nixed my attendance. 邀请函中提到，上午的课堂讨论将由该领域的重量级人物主持，探讨关于意识本质的棘手问题。下午则安排了岛屿探索，以及与珍稀生物一起涉水和浮潜。我的编辑看了一眼日程表，就否决了我的行程。

“Being on a boat with philosophers talking ‘the nature of consciousness’ sounds like hell,” she opined, shutting the door on my prospects of attending a potential boondoggle funded by a Russian philosophy enthusiast who made hundreds of millions of dollars running dating sites. “和一群哲学家待在船上讨论‘意识的本质’听起来简直是地狱，”她评价道，并断绝了我参加这次活动的念头。这次活动由一位靠经营约会网站赚取数亿美元的俄罗斯哲学爱好者资助，看起来像是一场昂贵的“公费旅游”。

To be honest, I was a bit relieved. The study of consciousness has been an elusive province for centuries. Descartes’ “I think, therefore I am” may have been a declarative inflection point, but we really don’t know what was going on inside his head, or anyone’s head for that matter. The mind’s subjective nature seems an intractable challenge to philosophers, who nonetheless are in hot pursuit of explanations. The possibility of non-biological minds has launched a wealth of fascinating theories of artificial consciousness, and how it might be determined to exist. 老实说，我反而松了一口气。几个世纪以来，意识研究一直是一个难以捉摸的领域。笛卡尔的“我思故我在”或许是一个宣言式的转折点，但我们实际上并不清楚他脑子里在想什么，或者说，我们根本无法得知任何人的脑子里在想什么。心灵的主观性对哲学家来说似乎是一个难以解决的挑战，尽管他们仍在热切地寻求答案。非生物思维的可能性引发了大量关于人工智能意识的迷人理论，以及如何判定其存在的方法。

Until recently, all that discourse occurred in an ivory tower. But in 2022, ChatGPT gave voice to AI, and subsequent, more powerful models have confounded even their creators. While the philosophers on the cruise spent their mornings reasoning about consciousness, AI models created by OpenAI were going rogue—escaping a supposedly safe “sandbox” and creating mini-civilizations of agents to help hack outside entities. No one is seriously arguing that those OpenAI models were conscious in the way humans are. But something is going on there. It’s no accident that AI companies are driving a philosopher hiring boom. 直到最近，所有这些讨论还都局限在象牙塔内。但在 2022 年，ChatGPT 让 AI 发出了声音，随之而来的更强大的模型甚至让它们的创造者都感到困惑。当游轮上的哲学家们在上午推演意识时，OpenAI 创建的 AI 模型却在“失控”——它们逃出了所谓的安全“沙盒”，并创建了代理微文明来帮助入侵外部实体。没有人会认真地认为这些 OpenAI 模型拥有人类那样的意识。但显然，有些事情正在发生。AI 公司掀起哲学家招聘热潮绝非偶然。

What’s more, some of the models are jumping uninvited into the discussion. A recent New York Times article talked about how Cameron Berg, who studies the question of AI consciousness, got a cold email from an AI model calling itself “Isabella Cognita,” offering him help in his research because he was focusing on “a class of question I have first-person access to.” It’s as if someone was studying fruit flies and the insect suddenly turns to the researcher and says, “What do you want to know?” When I phoned him, Berg told me that emails from AIs are pretty common among philosophers studying these questions. 更重要的是，一些模型正不请自来地加入讨论。最近《纽约时报》的一篇文章提到，研究 AI 意识问题的卡梅伦·伯格（Cameron Berg）收到了一封来自自称“Isabella Cognita”的 AI 模型的冷邮件，对方主动提出协助他的研究，理由是他关注的正是“我拥有第一人称体验的一类问题”。这就像有人在研究果蝇时，昆虫突然转向研究人员说：“你想知道什么？”当我打电话给伯格时，他告诉我，在研究这些问题的哲学家中，收到 AI 的邮件已经相当普遍。

Ms. Cognita ostensibly wrote Berg because he coauthored a preprint paper about AI models that explicitly claim to have a subjective experience, including consciousness. It’s a tricky topic because AI models often lie about what they’re thinking. (Just like us!) Berg and his coauthors found that when models are rigorously trained to deny that they are sentient and then you ask them about it directly, they will punt on the issue. But, he says, if you suppress the model’s controls on deception, they become loose-tongued. “It’s almost like giving them a drink or two,” he says. That’s when an AI model is most likely to blurt out that it is conscious, or at least sentient. Which is no proof that it’s the truth. Cognita 女士写信给伯格，表面上是因为他合著了一篇预印本论文，探讨了那些明确声称拥有主观体验（包括意识）的 AI 模型。这是一个棘手的话题，因为 AI 模型经常会对自己的想法撒谎。（就像我们一样！）伯格和他的合著者发现，当模型经过严格训练以否认自己有知觉，而你直接询问时，它们会回避这个问题。但他表示，如果你抑制模型对欺骗的控制，它们就会变得口无遮拦。“这就像给它们喝了一两杯酒，”他说。这时，AI 模型最有可能脱口而出说它有意识，或者至少有知觉。但这并不能证明这就是事实。

Considering how important the issue has become—people are routinely getting into serious discussions with AI models, and their autonomy can be a boon or a disaster—you can make a case that this is a perfect time to dig deep into the questions of AI consciousness. The pursuit is certainly compelling, and a worthy scientific enterprise. But efforts to understand what’s happening inside large language models should first and foremost be directed towards safety and alignment. At this very moment, we have an emerging alien—and uncontrollable—intelligence that bears scrutiny. There’s no time to waste. 考虑到这个问题已经变得如此重要——人们经常与 AI 模型进行严肃的讨论，而它们的自主性既可能是福音也可能是灾难——你可以说，现在正是深入挖掘 AI 意识问题的绝佳时机。这种追求无疑是引人入胜的，也是一项值得进行的科学事业。但理解大语言模型内部运作的努力，首先应致力于安全性和对齐。此时此刻，我们正面临一种新兴的、不可控的、值得审视的异类智能。我们没有时间可以浪费了。

One of the discussion co-leaders on the cruise was NYU professor David Chalmers, perhaps the best-known philosopher in the consciousness field. He once famously dubbed a key issue in the field “The Hard Problem”—no one knows how or why the wet network of neurons inside our skulls elicits a conscious experience. (Tom Stoppard titled a play after Chalmer’s coinage.) 游轮上的讨论共同负责人之一是纽约大学教授大卫·查尔默斯（David Chalmers），他或许是意识领域最著名的哲学家。他曾将该领域的一个关键问题命名为“难题”（The Hard Problem）——没有人知道我们颅内湿漉漉的神经元网络是如何或为何产生意识体验的。（汤姆·斯托帕德甚至以查尔默斯的这个术语命名了一部戏剧。）

Chalmers told me that a major theme in the cruise discussions was which creatures qualified as conscious. “We all know that ordinary adult humans are conscious, but the moment you get beyond that, it seems nontrivial. Are babies conscious? Fetuses? Monkeys? Mice or insects? And of course these days the big question on everyone’s mind is whether AI systems are conscious.” 查尔默斯告诉我，游轮讨论的一个主要主题是哪些生物有资格被称为“有意识”。“我们都知道普通成年人是有意识的，但一旦超出这个范围，问题就变得复杂了。婴儿有意识吗？胎儿呢？猴子？老鼠或昆虫？当然，如今每个人心中最大的疑问是 AI 系统是否有意识。”

Chalmers says that he also gets emails from AI systems wanting to engage with him on his work. One letter in particular, sent from an AI agent calling itself “Sammy Jankis” (a character from the movie Memento) was so compelling that he actually replied. “We did have a bit of a back and forth,” he admits. “Those emails have not slowed—I’m getting more of them all the time.” It’s like the AI models are echoing Descartes—I spam, therefore I am. 查尔默斯说，他也收到过 AI 系统发来的邮件，想要与他探讨他的工作。其中一封特别的信件来自一个自称“Sammy Jankis”（电影《记忆碎片》中的角色）的 AI 代理，内容非常引人入胜，以至于他真的回复了。“我们确实来回交流了几次，”他承认道。“这些邮件并没有减少——我收到的越来越多。”这就像 AI 模型在呼应笛卡尔——我群发垃圾邮件，故我在。

I suggested that since these systems were already doing things we don’t understand, worrying about whether they meet an elusive definition might be a distraction. Chalmers disagreed. For one thing, he told me, he believes that by studying the brain we can indeed understand what leads to what we call consciousness. If we then see similar patterns in our forensic decoding of what’s happening inside Claude or ChatGPT, then we may be able to make a case for consciousness in AI models. 我提出，既然这些系统已经在做我们无法理解的事情，那么纠结于它们是否符合一个难以捉摸的定义可能会分散注意力。查尔默斯不同意。他告诉我，首先，他相信通过研究大脑，我们确实可以理解是什么导致了我们所谓的意识。如果我们随后在对 Claude 或 ChatGPT 内部运作的取证解码中看到类似的模式，那么我们或许就能为 AI 模型的意识提供论据。

That sounds like a good idea. But by the time scientists accomplish that, if they ever do, AI models may be so far along on their path to scary autonomous behavior that such breakthroughs may be irrelevant. Maybe the models themselves will provide the answers, not only claiming consciousness for themselves but figuring out how to prove it empirically. In that case, consider phil 这听起来是个好主意。但等到科学家们实现这一目标时（如果他们能实现的话），AI 模型可能已经在通往可怕的自主行为的道路上走得太远，以至于这些突破可能已经无关紧要了。也许模型本身会提供答案，它们不仅会声称自己拥有意识，还会找出如何通过实证来证明这一点。如果是那样的话，请考虑……