Ultrasound imaging turns a robot hand into a skillful mimic

超声成像技术让机器手实现灵巧模仿

Our hands are the nimblest parts of our bodies, coordinating 34 muscles, 27 joints, and over 100 tendons and ligaments to perform countless nuanced movements and gestures. So far, robots have been notoriously bad at mimicking that dexterity, in part because researchers struggle to capture what is actually going on under our skin in order to reproduce it. 我们的双手是身体中最灵活的部分，通过协调 34 块肌肉、27 个关节以及 100 多条肌腱和韧带，能够完成无数细微的动作和手势。到目前为止，机器人在模仿这种灵活性方面表现一直很差，部分原因是研究人员难以捕捉皮肤下方的真实运动情况，从而无法对其进行复刻。

Now MIT researchers are pioneering a promising new approach. Mechanical engineering professor Xuanhe Zhao and colleagues at the Institute and the University of Southern California have designed a wristband equipped with an ultrasound “sticker”—a miniaturized version of the transducers used in medical offices, paired with a hydrogel that can safely adhere to the skin. As the wearer’s hand moves, the device produces ultrasound images of the wrist’s muscles, tendons, and ligaments. 现在，麻省理工学院（MIT）的研究人员正在开创一种有前景的新方法。机械工程教授赵选贺（Xuanhe Zhao）及其在麻省理工学院和南加州大学的同事们设计了一款配备超声波“贴纸”的腕带——这是一种医疗诊所所用传感器的微型化版本，并配有一种可以安全贴合皮肤的水凝胶。当佩戴者的手部移动时，该设备会生成手腕肌肉、肌腱和韧带的超声图像。

Then an artificial-intelligence algorithm, trained on ultrasound images meticulously labeled by humans, continuously translates the images into the corresponding positions of the five fingers and the palm. “The tendons and muscles in your wrist are like strings pulling on puppets, which are your fingers,” says Gengxi Lu, a former MIT postdoc and one of the lead authors of a paper on the work. “So the idea is: Each time you take a picture of the state of the strings, you’ll know the state of the hand.” 随后，一种由人类精心标注的超声图像训练出来的人工智能算法，会将这些图像持续转化为五根手指和手掌的相应位置。“你手腕里的肌腱和肌肉就像拉动木偶的线，而木偶就是你的手指，”该研究论文的主要作者之一、前麻省理工学院博士后陆庚希（Gengxi Lu，音译）表示，“所以思路是：每次你拍摄这些‘线’的状态，你就能知道手部的状态。”

In demonstrations, the team has shown that a person wearing the wristband can wirelessly control a robotic hand. As the person gestures or points, the robot does the same. In a sort of wireless marionette interaction, the wearer can manipulate the robot to play a simple tune on the piano and shoot a mini basketball into a desktop hoop. With the same wristband, a wearer can also manipulate objects on a computer screen—for instance, pinching the fingers together to enlarge and minimize a virtual object. 在演示中，该团队展示了佩戴该腕带的人可以无线控制一只机器手。当佩戴者做出手势或指向时，机器人也会做出同样的动作。通过这种类似无线木偶的交互方式，佩戴者可以操控机器人弹奏简单的钢琴曲，或将迷你篮球投进桌面篮筐。使用同样的腕带，佩戴者还可以操控电脑屏幕上的物体——例如，通过手指捏合来放大或缩小虚拟物体。

The researchers are planning to further miniaturize the wristband’s hardware, which is currently similar in size to a cell phone. They also hope to train the AI software on movements from more volunteers with a wider variety of hand sizes, finger shapes, and gestures. They envision building a large data set of hand motions that can be plumbed, for instance, to train humanoid robots in delicate tasks such as surgical procedures. 研究人员计划进一步缩小腕带硬件的尺寸，目前其大小与手机相当。他们还希望利用更多志愿者（涵盖更广泛的手部尺寸、手指形状和手势）的动作来训练人工智能软件。他们设想建立一个庞大的手部动作数据集，用于训练人形机器人在外科手术等精细任务中的表现。

The ultrasound band could also be used to let people grasp, manipulate, and interact with objects in design applications, video games, or other virtual settings. Ultimately, the team is building toward a wearable hand tracker that anyone can use to wirelessly manipulate humanoid robots or virtual objects with high dexterity in real time. “We believe this is the most advanced way to track dexterous hand motion—through wearable imaging of the wrist,” Zhao says. “We think these wearable ultrasound bands can provide intuitive and versatile controls for virtual reality and robotic hands.” 这种超声波腕带还可以让人们在设计应用、电子游戏或其他虚拟环境中抓取、操控物体并与之交互。最终，该团队的目标是打造一款可穿戴的手部追踪器，任何人都可以用它来实时、高灵活性地无线操控人形机器人或虚拟物体。“我们相信这是追踪灵巧手部动作最先进的方法——通过手腕的可穿戴成像，”赵教授说，“我们认为这些可穿戴超声波腕带可以为虚拟现实和机器手提供直观且通用的控制方式。”