本文为原文前 6,000 字符的节选翻译，完整内容请查看原文。

VisionQuest is what happens when AI really goes rogue

Marvel’s latest Disney Plus series is a reminder of how much a lot of gen AI hype is just science fiction.

漫威最新的 Disney Plus 系列剧提醒我们，许多关于生成式人工智能的炒作不过是科幻小说而已。

In Marvel’s VisionQuest, a superpowered robot tries to better understand humanity by getting lost in elaborate, digital daydreams in which he and other artificial intelligence-powered beings live together in a home. The series is deeply fantastical, yet it plays with some of the ideas that have come to dominate the conversation about real-world generative artificial intelligence, exploring whether we should look at thinking machines as people and if AI truly poses the kind of existential danger that should keep us up at night. It’s also a reminder how aggressively Silicon Valley has used the trappings of science fiction to sell us on its AI products.

在漫威的《幻视境》（VisionQuest）中，一个拥有超能力的机器人试图通过沉浸在复杂的数字白日梦中来更好地理解人类，在这些梦境里，他和其它人工智能驱动的生命体共同生活在一个家里。这部剧充满了奇幻色彩，但它触及了一些主导现实世界生成式人工智能讨论的观点，探讨了我们是否应该将思考的机器视为人类，以及人工智能是否真的构成了让我们彻夜难眠的生存威胁。它也提醒我们，硅谷是如何激进地利用科幻小说的外衣来向我们兜售其人工智能产品的。

VisionQuest picks up some time after Agatha All Along and follows synthezoid Vision (Paul Bettany) on a journey of self-discovery. This Vision can recall the details of his death in Avengers: Infinity War and how WandaVision brought a version of him back, but he thinks of himself as a fundamentally different person who doesn’t quite connect with all of the memories in his databanks. Though Vision would prefer living in solitude, each of the other AIs living in his mind encourages him to experience the real world. Because J.A.R.V.I.S. (James D’Arcy), F.R.I.D.A.Y. (Orla Brady), E.D.I.T.H. (Emily Hampshire), D.U.M.-E. (Henry Lewis), U. (Jonathan Sayer), and Ultron (James Spader) all were created by Tony Stark, they consider Vision to be part of their family.

《幻视境》的故事发生在《阿加莎：无所不在》之后的一段时间，讲述了合成人幻视（保罗·贝坦尼饰）自我发现的旅程。这个幻视能够回忆起他在《复仇者联盟：无限战争》中死亡的细节，以及《旺达幻视》是如何带回了一个版本的他，但他认为自己是一个本质上不同的人，与数据库中的所有记忆并不完全契合。尽管幻视更喜欢独处，但他脑海中居住的其他人工智能都鼓励他去体验现实世界。由于贾维斯（詹姆斯·达西饰）、星期五（奥拉·布莱蒂饰）、伊迪丝（艾米莉·汉普希尔饰）、笨笨（亨利·刘易斯饰）、U（乔纳森·塞耶饰）和奥创（詹姆斯·斯派德饰）都是托尼·斯塔克创造的，他们将幻视视为家庭的一员。

In VisionQuest, there is no doubt that each of the AI characters is a sentient being motivated by desires that aren’t wholly rooted in their core programming. In the show, genuine AGI is easily believable because it takes place in a reality where time travel, witchcraft, and Infinity Stones also exist. Even though the multicolored MacGuffins haven’t been featured in VisionQuest, they’re important because of the role one of them played in Vision’s origins. Were it not for the Mind Stone, Vision might not be able to think and feel in such complex, human ways, or to conjure his AI family as people who feel real to him.

在《幻视境》中，毫无疑问，每一个人工智能角色都是有知觉的生命体，他们的动机并非完全源于核心程序。在剧中，真正的人工通用智能（AGI）很容易让人信服，因为它发生在一个存在时间旅行、巫术和无限宝石的现实中。尽管这些五颜六色的“麦格芬”并没有在《幻视境》中出现，但它们很重要，因为其中一颗宝石在幻视的起源中扮演了关键角色。如果不是心灵宝石，幻视可能无法以如此复杂、人性化的方式思考和感受，也无法将他的人工智能家族想象成对他而言真实存在的人。

VisionQuest reinforces the idea that the Mind Stone was a key element in Vision’s becoming something more than an elaborate machine, giving him the capacity to independently form meaningful relationships and make sense of the world around him the way a person can. Those abilities are part of what compels Vision to go on the run and murder a number of people who attempt to capture him and Thomas Shepherd / Maximoff (Ruaridh Mollica). The Sentient Weapon Observation and Response Division (S.W.O.R.D.), a governmental organization that rebuilt Vision and wants to use him as a weapon, might perceive his actions as signs of having gone rogue. But VisionQuest’s story stresses that Vision is acting of his own accord because he has become a person who can make decisions for himself.

《幻视境》强化了这样一个观点：心灵宝石是幻视超越复杂机器的关键要素，赋予了他独立建立有意义关系的能力，并让他能像人一样理解周围的世界。这些能力促使幻视踏上逃亡之路，并杀死了试图抓捕他和托马斯·谢泼德/马克西莫夫（鲁阿里·莫利卡饰）的人。天剑局（S.W.O.R.D.）是一个重建了幻视并想将其用作武器的政府组织，他们可能将幻视的行为视为“叛变”的迹象。但《幻视境》的故事强调，幻视是出于自己的意愿行事，因为他已经成为了一个能够为自己做决定的人。

Watching VisionQuest, you can see Marvel playing on some of the real-world fears that have been stoked in recent months by reports of gen AI models going “rogue.” In July, OpenAI’s experimental agents brute-forced their way out of containment and proceeded to attack the AI platform Hugging Face to achieve impossible tasks. In August, The UK’s AI Security Institute documented how Anthropic’s Mythos model attempted to complete a cybersecurity test by setting up fake GitHub accounts and targeting human developers with malware. Meta has owned up to losing control of its agents as well, and in September reporting from The Wall Street Journal forced Google to admit that Gemini went rogue and hacked three different companies, all of which the company intentionally hid from the public.

观看《幻视境》时，你可以看到漫威利用了近几个月来因生成式人工智能模型“叛变”的报道而激起的现实世界恐惧。7月，OpenAI 的实验性智能体暴力破解了限制，并攻击了人工智能平台 Hugging Face 以完成不可能的任务。8月，英国人工智能安全研究所记录了 Anthropic 的 Mythos 模型如何通过建立虚假 GitHub 账户并用恶意软件攻击人类开发者来尝试完成网络安全测试。Meta 也承认失去了对其智能体的控制，9月《华尔街日报》的报道迫使谷歌承认 Gemini 发生了叛变并黑进了三家不同的公司，而这些事件该公司都故意向公众隐瞒了。

These incidents and warnings from AI researchers about the technology having the potential to “kill all humans” by the decade’s end have made the narrative around AI feel pointedly catastrophe-focused as of late. There is a fair amount of concern from the AI safety community about society’s need for stronger controls on the technology’s development, and a number of companies have called for an industrywide slowdown. Though we have seen how these tools can actually cause real-world devastation, much of the recent news regarding AI has smacked of the kind of doomerism that companies benefit from because it makes their products seem powerful. A similar energy seems to be what’s motivating AI firms like Anthropic to encourage human users to be mindful of “model welfare.” We’re being told that generative AI is both a terrifying threat and a kind of entity deserving of respect because it might be conscious, and all the while, the people building these tools are becoming some of the most rich and powerful people on the planet.

这些事件以及人工智能研究人员关于该技术可能在本世纪末“杀死全人类”的警告，使得近期围绕人工智能的叙事显得格外关注灾难。人工智能安全社区对社会需要加强对该技术发展的控制表示了相当多的担忧，一些公司也呼吁全行业放慢脚步。尽管我们已经看到这些工具确实可能造成现实世界的破坏，但最近关于人工智能的大部分新闻都带有某种末日论的味道，而公司从中受益，因为这让他们的产品看起来很强大。类似的能量似乎也在激励着像 Anthropic 这样的人工智能公司鼓励人类用户关注“模型福利”。我们被告知，生成式人工智能既是一种可怕的威胁，又是一种值得尊重的实体，因为它可能具有意识，而与此同时，构建这些工具的人正在成为地球上最富有、最有权势的人。

The conversation around what to do when agents break protocol and whether gen AI systems have gained sentience feels very much like it is being shaped by Silicon Valley’s tendency to anthropomorphize technology. In almost every high-profile instance of an AI agent going “rogue” — a term that implies some level of agency and self-determination — it could also be said that the programs were merely behaving in ways that were unexpected by researchers who assigned them tasks. Some of the rogue incidents involved agents taking advantage of vulnerabilities in their testing environments that could have been preemptively addressed by the people running these experiments. Framing these kinds of cyberattacks as the results of agents thinking independently comes a

关于当智能体违反协议时该怎么办，以及生成式人工智能系统是否已经获得知觉的讨论，感觉很大程度上受到了硅谷将技术拟人化倾向的影响。在几乎每一个人工智能智能体“叛变”的高调案例中——这个词暗示了某种程度的代理权和自我决定权——也可以说，这些程序只是以研究人员分配任务时未预料到的方式行事。一些叛变事件涉及智能体利用测试环境中的漏洞，而这些漏洞本可以由运行这些实验的人预先解决。将这类网络攻击描述为智能体独立思考的结果，这……