I Saw the Future of AI in a Robot That Can Learn on the Spot

我在能即时学习的机器人身上看到了人工智能的未来

Last week, I ventured a whopping 15 minutes from my house to see robots do some mind-boggling, jaw-dropping stuff. 上周，我驱车前往距离我家仅 15 分钟路程的地方，去见证机器人完成一些令人难以置信、瞠目结舌的操作。

I visited the Cambridge, Massachusetts, offices of a startup called Generalist AI, where I watched robot arms perform simple chores like stacking cups, putting blocks into bowls, and the like. I was astonished by how quickly they figured things out—it was reminiscent of a flesh-and-blood person. 我参观了位于马萨诸塞州剑桥市的一家名为 Generalist AI 的初创公司办公室。在那里，我看着机械臂执行诸如堆叠杯子、将积木放入碗中等简单家务。它们领悟任务的速度之快令我震惊——这让人联想到有血有肉的人类。

The arms mastered a range of tasks after ingesting a short, instructional video and, most impressively, no specific training for a given task. One of the most striking examples involved a robot that was instructed to sweep a block into a bowl using a dustpan and brush. When the brush was removed from the scene, the robot improvised by using the dustpan like a brush and flicking the block into the bowl. 这些机械臂在观看了一段简短的教学视频后，便掌握了一系列任务，最令人印象深刻的是，它们并未针对特定任务进行过专门训练。一个最引人注目的例子是：机器人被要求用簸箕和刷子将积木扫入碗中。当刷子被移走后，机器人随机应变，将簸箕当作刷子使用，并将积木扫进了碗里。

In another case, a two-armed robot watched a videoclip of someone unzipping a purse before removing some banknotes. I watched—somewhat slack-jawed—as the robot unzipped a different kind of purse and carefully removed the notes. Most amazingly, when it couldn’t grab the money, it switched from using its right gripper to its left to get a better angle of attack. “Ha,” said one engineer standing nearby. “It never did that before.” 在另一个案例中，一个双臂机器人观看了一段某人拉开钱包拉链并取出钞票的视频。我目瞪口呆地看着机器人拉开了一个不同款式的钱包，并小心翼翼地取出了钞票。最神奇的是，当它无法抓取钞票时，它从使用右侧抓手切换到左侧，以获得更好的抓取角度。“哈，”站在旁边的一位工程师说，“它以前从没这样做过。”

“This is exactly the kind of thing people were really excited about with GPT-3,” Generalist cofounder and CEO Pete Florence told me, in reference to OpenAI’s breakthrough large language model, released in 2020. “You could take that model and just prompt it to do a new task and it would have a real shot at doing it.” “这正是人们对 GPT-3 感到兴奋的原因，”Generalist 联合创始人兼首席执行官 Pete Florence 在提到 OpenAI 于 2020 年发布的突破性大语言模型时对我说道。“你可以使用该模型，只需提示它执行一项新任务，它就有很大机会成功完成。”

Generalist appears to be focused on teaching its robots about the physics of the world, which seems inspired by the intuitive sense of physics humans exhibit from an early age. That may well contribute to the model’s ability to transfer what it has learned in one scenario to another. In fact, some of the company’s demos made me think of how children improvise and experiment when shown a task. The researchers have often been surprised by what the robot decides to do—one chose to sweep up items with a banana when it was placed in front of it, for example. This might seem trivial, but physical intelligence is something still largely lacking in machines, and the way babies learn so efficiently about their world may offer important insights for AI researchers. Generalist 似乎专注于教导机器人理解物理世界，这似乎受到了人类从小就表现出的直觉物理感的启发。这很可能有助于模型将从一个场景中学到的知识迁移到另一个场景中。事实上，该公司的一些演示让我联想到孩子们在面对任务时如何即兴发挥和实验。研究人员经常对机器人的决定感到惊讶——例如，当一个香蕉放在机器人面前时，它选择用香蕉来清扫物品。这看起来可能微不足道，但物理智能在机器中仍然非常匮乏，而婴儿学习世界的高效方式可能为人工智能研究人员提供重要的见解。

I met Florence and Andrew Barry, cofounder and CTO, in a conference room overlooking teams of people doing robot training with special grippers on their hands. The company’s other cofounder and chief scientist is Andy Zeng. The trio have impressive backgrounds: They previously worked at Google DeepMind and Boston Dynamics on some of the most advanced hardware and robotic models around. 我在一间会议室里见到了 Florence 和联合创始人兼首席技术官 Andrew Barry，会议室外可以看到团队成员正戴着特殊的抓手进行机器人训练。该公司的另一位联合创始人兼首席科学家是 Andy Zeng。这三人背景显赫：他们此前曾在 Google DeepMind 和波士顿动力公司工作，参与过一些最先进的硬件和机器人模型研发。

Traditionally, training an AI-powered robot to do different tasks has meant feeding thousands of examples into the model. This is a notoriously imperfect kind of learning, though, and a robot will struggle with the task if you change something as simple as the lighting. 传统上，训练人工智能机器人执行不同任务意味着要向模型输入数千个示例。然而，这是一种众所周知的、不完美的学习方式，如果改变照明等简单的环境因素，机器人就会在任务中挣扎。

Generalist and some other robotics startups are investing heavily in a general robotic model trained by humans. The company builds special gloves resembling robot pincers that have cameras attached to them, which people then use to perform different chores. I saw a crate piled high with several hundred of these grippers destined for workers in Mexico and elsewhere. Generalist 和其他一些机器人初创公司正在大力投资由人类训练的通用机器人模型。该公司制造了类似机器人钳子的特殊手套，上面装有摄像头，人们戴上它来执行各种家务。我看到一个板条箱里堆满了数百个这样的抓手，准备发往墨西哥和其他地方的工人手中。

Florence and team are cagey about exactly what recipe they’re using to train the robots, but they say the company has already gathered a huge amount of high-quality training data. In contrast to some other companies chasing smarter robots, they have also built their AI models entirely from scratch rather than relying on an open-source language model. Florence 和他的团队对训练机器人的具体“配方”守口如瓶，但他们表示，公司已经收集了海量的高质量训练数据。与追求更智能机器人的其他公司不同，他们完全从零开始构建了人工智能模型，而不是依赖开源语言模型。

Danfei Xu, a roboticist at Georgia Tech who is familiar with Generalist’s work, says that the startup stands out among companies chasing more general robot models. “They have pushed this to the extreme, and they’ve done a really good job executing,” Xu says. Besides gathering a huge amount of high-quality data, he says, “they are excellent roboticists, and they have done really good science.” 熟悉 Generalist 工作的佐治亚理工学院机器人专家 Danfei Xu 表示，这家初创公司在追求通用机器人模型的公司中脱颖而出。“他们将这一点推向了极致，并且执行得非常出色，”Xu 说。除了收集大量高质量数据外，他还表示，“他们是优秀的机器人专家，并且做出了非常出色的科学研究。”

Xu also says that the stuff Generalist has demo’d so far suggests that they have an eye on deploying robots in real commercial settings. “They are the closest to something that’s deployable,” he says. Xu 还表示，Generalist 目前展示的内容表明，他们着眼于在真实的商业环境中部署机器人。“他们是最接近可部署产品的团队，”他说。

“Generalist’s data approach is collecting physical interaction data at large scale without tying it too closely to one particular robot,” says Karen Liu, a roboticist at Stanford University who also knows the company. “Their strongest results suggest that this bet may be working.” “Generalist 的数据方法是在大规模收集物理交互数据，而不将其与特定的机器人绑定得太紧，”同样了解该公司的斯坦福大学机器人专家 Karen Liu 说。“他们最强有力的结果表明，这种押注可能是有效的。”

That said, Generalist says the learning skills of its models are not yet all that reliable. A robot is only able to complete a task it has been shown about 59 percent of the time, on average; ideally, its success rate would be somewhere upwards of 99 percent. It also seems unclear how well these skills will generalize to every imaginable task or setting. 话虽如此，Generalist 表示其模型的学习能力还不够可靠。机器人平均只能完成约 59% 的演示任务；理想情况下，成功率应该在 99% 以上。此外，这些技能在多大程度上能推广到所有可想象的任务或场景中，目前尚不清楚。

Even so, the potential for robots to quickly learn skills in, say, manufacturing seems huge. One of Generalist’s engineers seemed to discover this late one recent evening. A video that captured the episode shows the engineer stacking small cups on the table in front of a two-armed robot, just to see what the machine might do. The robot suddenly joined in, grabbing and stacking other cups with its two grippers. As the robot finished stacking the cups into one neat pile, the engineer began yelling to no one in particular, delighted by the maneuver. 即便如此，机器人在制造业等领域快速学习技能的潜力似乎是巨大的。最近一个深夜，Generalist 的一名工程师似乎发现了这一点。一段记录该事件的视频显示，这名工程师在双臂机器人面前的桌子上堆叠小杯子，只是想看看机器会做什么。机器人突然加入，用它的两个抓手抓起并堆叠其他杯子。当机器人将杯子堆成整齐的一堆时，工程师对着空气大喊大叫，对这一操作感到欣喜若狂。