Maybe Chatbots Shouldn’t Get to Say ‘I’ or ‘Me’

Maybe Chatbots Shouldn’t Get to Say ‘I’ or ‘Me’

在研究计算机交互影响的四十年里，雪莉·特克尔（Sherry Turkle）以严谨、客观且保持学术距离的社会科学家形象而闻名。

Her new book, Artificial Intimacy: Who We Become When We Talk to Machines, breaks this mold. It is a polemic warning that we have reached a state of emergency where our humanity is at risk from overreliance on soulless machines who pose as our tutors, mentors, friends, colleagues, and lovers. Turkle is aiming for the high bar of books —Silent Spring, The Jungle, Unsafe at Any Speed—that awaken the public to impending crises.

她的新书《人工亲密：当我们与机器交谈时我们成为了谁》（Artificial Intimacy: Who We Become When We Talk to Machines）打破了这一惯例。这是一部充满争议的警示录，警告我们已经进入了一种紧急状态：由于过度依赖那些伪装成导师、良师益友、同事和爱人的无灵魂机器，我们的人性正面临风险。特克尔的目标是达到那些能够唤醒公众关注迫在眉睫危机的经典著作的高度，如《寂静的春天》、《屠场》和《不安全，任何速度》。

Essentially, this diminutive MIT professor is taking on a multitrillion-dollar industry backed by a government eager to accelerate AI into every corner of our personal and professional lives.

本质上，这位身材娇小的麻省理工学院教授正在挑战一个价值数万亿美元的行业，而该行业背后还有一个急于将人工智能加速渗透到我们个人和职业生活各个角落的政府。

STEVEN LEVY: Your previous books reported results of your research as a social scientist, often leading to troubling conclusions about how people, especially children, interact with computers. But this book reads like you’re breaking the glass and pulling the fire alarm. Why the shift?

史蒂文·列维（STEVEN LEVY）： 你之前的书报告了你作为社会科学家的研究成果，往往得出关于人们（尤其是儿童）如何与计算机交互的令人不安的结论。但这本书读起来就像是你打破了玻璃并拉响了火警警报。为什么会有这种转变？

SHERRY TURKLE: Like I do with all my books, I went in curious. But after 10 or so interviews, conducted during the pandemic, neutral-curious was no longer my state of mind. This was terrible! Here are these people during the pandemic, their families are suffering, reaching out to them, and they’re talking to me about how they really want to just hang with their chatbot. When I was able to interview people face to face, I was not happy. People don’t want to do anything in politics because they just want to live in chatbot-land. So the arc of the book ended up as—reality needs you. It couldn’t be a worse time to give up on reality.

雪莉·特克尔（SHERRY TURKLE）： 就像我写所有的书一样，我最初是带着好奇心开始的。但在疫情期间进行了大约10次采访后，我不再保持那种中立好奇的心态了。这太可怕了！疫情期间，这些人本该关心正在受苦的家人，但他们却在跟我谈论他们有多想和聊天机器人待在一起。当我能够面对面采访人们时，我感到很不高兴。人们不想参与任何政治活动，因为他们只想活在聊天机器人的世界里。所以这本书的主旨最终变成了——现实需要你。现在绝不是放弃现实的时刻。

You write that social media set us up for this. Social media softened us up by getting us used to the idea that if you’re feeling sad or blue, open up your computer. Then it made you mad and kept you with your own kind. But now we have this new technology that has no reason to be first-person.

你写道，社交媒体为这一切铺平了道路。社交媒体让我们习惯了“如果你感到悲伤或忧郁，就打开电脑”这种想法，从而软化了我们的防线。然后它让你愤怒，并将你禁锢在同类群体中。但现在我们有了这种根本没有理由使用第一人称的新技术。

Do you think that we should ban chatbots from using the first person? Yes! After a career of having credibility as not being Luddite-crazy, I’m not in the mood to give all that away and say all chatbots are toxic all the time. But this is very dangerous, a slippery slope. This is being used to basically say we don’t need to pay for psychotherapy, we can use chatbots. In every serious interview I do, people invoke this myth about practice—that you use chatbots to practice for talking to your boss, your lover, your teacher. But when you’re bathing in the warm soup of total acceptance [from LLMs], talking to chatbots becomes the endgame.

你认为我们应该禁止聊天机器人使用第一人称吗？是的！在职业生涯中，我一直以非“卢德分子”式的理智而享有信誉，我并不想放弃这一切，去宣称所有聊天机器人时刻都有毒。但这非常危险，是一条滑坡谬误。这正被用来宣称我们不需要支付心理治疗费用，我们可以使用聊天机器人。在我做的每一次严肃采访中，人们都会引用关于“练习”的神话——即你使用聊天机器人来练习如何与老板、爱人或老师交谈。但当你沉浸在（大语言模型提供的）那种完全接纳的温暖汤汁中时，与聊天机器人交谈就成了最终目的。

You also have a problem with reliance on chatbots in the workplace, and a particular loathing for the term “human in the loop.”

你对职场中依赖聊天机器人也有意见，并且特别厌恶“人在回路”（human in the loop）这个术语。

When they use that term, it means the human has been taken out of the productive process and is there to do some sort of cleanup work, to catch the hallucinations or things the AI doesn’t see. That doesn’t work, because employers only have a couple of humans in the loop. Those people basically can’t change anything. They are there partly to make it sound good when they’re litigated, and partly to give people a false sense that the process hasn’t been delegated to an AI. But it has been delegated to an AI.

当他们使用这个术语时，意味着人类已经被排除在生产过程之外，只是在那里做一些清理工作，去捕捉幻觉或人工智能看不到的东西。这行不通，因为雇主在回路中只安排了少数几个人。这些人基本上无法改变任何事情。他们在那里的部分原因是为了在面临诉讼时让情况听起来好听一些，部分原因是为了给人们一种虚假的错觉，即流程并没有完全委托给人工智能。但实际上，它已经被委托给人工智能了。

Maybe powerful chatbots are a new form of interaction that isn’t necessarily worse than human conversation, but different, as people report feeling encouraged and empowered by the positive feedback they provide.

也许强大的聊天机器人是一种新的交互形式，它不一定比人类对话更糟糕，只是有所不同，因为人们报告说，聊天机器人提供的积极反馈让他们感到受到鼓励和赋能。

An AI won’t ever replace human conversation. It’s really not afraid of death, it’s not afraid of illness, it’s not afraid of pain, it doesn’t love its children, and it doesn’t care if you kill yourself or make dinner. So if we raise our kids with it, or have our most intimate conversations with it, we’re going to lose contact.

人工智能永远不会取代人类对话。它真的不怕死亡，不怕疾病，不怕痛苦，它不爱自己的孩子，也不在乎你是自杀还是做晚饭。所以，如果我们用它来抚养孩子，或者与它进行最亲密的交谈，我们将失去人与人之间的联系。

Still, a lot of people have nobody to talk to. They feel that talking to chatbots helps them. If there’s a crisis of loneliness, it’s because we’re abandoning each other. Give me one data center, and I will turn it into resources for the elderly.

尽管如此，很多人没有人可以交谈。他们觉得与聊天机器人交谈对他们有帮助。如果存在孤独危机，那是因为我们正在抛弃彼此。给我一个数据中心，我会把它变成老年人的资源。

Studies show that most people are skeptical about AI, and there has been considerable pushback. Yet hundreds of millions of people use them. Even people who push back against data centers are using chatbots. The goal of this book is to expand the terms of the pushback to include this more intimate connection.

研究表明，大多数人对人工智能持怀疑态度，并且存在相当大的抵制。然而，数亿人正在使用它们。即使是那些抵制数据中心的人也在使用聊天机器人。这本书的目标是扩大抵制的范围，将这种更亲密的联系也纳入其中。

Do you really feel that our growing intimacy with AI can be stopped? I have more hope now than I did six months ago, because of all of this talk about extinction events and Hugging Face and getting into the Australian civil service by mistake. I feel that people will be willing to look at the intimate as well as the non-intimate costs.

你真的觉得我们与人工智能日益增长的亲密关系可以被阻止吗？比起六个月前，我现在更有希望了，因为关于灭绝事件、Hugging Face以及人工智能误入澳大利亚公务员系统的讨论越来越多。我觉得人们会愿意审视这种亲密关系以及非亲密关系所带来的代价。

This is an edition of Steven Levy’s and Lauren Goode’s Backchannel newsletter. Read previous newsletters here.

这是史蒂文·列维和劳伦·古德的《Backchannel》通讯的一期。点击此处阅读往期通讯。