From the Hugging Face Hub to robot hardware with Strands Agents and LeRobot

查看原文 / View Original 编辑 / Edit

From the Hugging Face Hub to robot hardware with Strands Agents and LeRobot

从 Hugging Face Hub 到机器人硬件：Strands Agents 与 LeRobot 的结合

A walkthrough of the LeRobot integration in Strands Robots - one agent loop, from a Hub dataset to a physical robot, with sim-to-real datasets in the same on-disk format and policies you swap with a string. 本文将带您了解 Strands Robots 中 LeRobot 的集成——通过一个智能体循环（agent loop），实现从 Hub 数据集到物理机器人的全流程，并确保仿真与现实（sim-to-real）的数据集采用相同的磁盘格式，且策略可以通过字符串轻松切换。

You have a robot, a folder of demonstration data on the Hugging Face Hub, and a new task you want it to learn. Today that takes five separate tools: one to record new demonstrations, another to train, a third to test in simulation, custom code to deploy on hardware, and yet another to coordinate when you have more than one robot. The pieces work on their own. They don’t talk to each other. 假设你拥有一台机器人、Hugging Face Hub 上的一文件夹演示数据，以及一个想要它学习的新任务。在今天，这通常需要五个独立的工具：一个用于录制新演示，一个用于训练，第三个用于仿真测试，编写自定义代码部署到硬件，以及另一个用于在拥有多台机器人时进行协调。这些组件各自为政，互不通信。

Strands Robots is an open source SDK from AWS (Apache 2.0) that exposes robot abstractions, simulation, and the LeRobot stack as AgentTools that you compose into a single Strands agent. The integration is deliberately thin: LeRobot’s own scripts handle hardware recording and calibration, and the Strands AgentTools come in for the parts an agent actually orchestrates. Strands Robots 是 AWS 推出的开源 SDK（采用 Apache 2.0 协议），它将机器人抽象、仿真和 LeRobot 堆栈封装为 AgentTools，你可以将其组合成一个单一的 Strands 智能体。这种集成设计得非常轻量：LeRobot 自身的脚本负责处理硬件录制和校准，而 Strands AgentTools 则介入智能体实际需要编排的部分。

The simulation tool records LeRobotDatasets in the same format LeRobot writes on hardware. GR00T and LerobotLocal serve policy inference behind a common interface, and MolmoAct2 checkpoints run through the LerobotLocal path. A peer mesh fans the agent out to remote robots. The dataset format stays exactly as LeRobot wrote it; the agent loop is the glue. 仿真工具以 LeRobot 在硬件上写入的相同格式记录 LeRobotDatasets。GR00T 和 LerobotLocal 在统一接口下提供策略推理，MolmoAct2 检查点则通过 LerobotLocal 路径运行。对等网络（peer mesh）将智能体扩展到远程机器人。数据集格式与 LeRobot 原始格式完全一致；智能体循环起到了粘合剂的作用。

This post walks you through five steps inside a single agent: build the agent over the LeRobot AgentTools, record a demonstration as a LeRobotDataset in simulation, run a policy on the same robot, deploy the same agent code to a physical SO-101 with one keyword argument change, and broadcast commands across a fleet over the Zenoh mesh. 本文将引导您完成单个智能体内的五个步骤：基于 LeRobot AgentTools 构建智能体，在仿真中将演示录制为 LeRobotDataset，在同一机器人上运行策略，通过更改一个关键字参数将相同的智能体代码部署到物理 SO-101 机器人，并通过 Zenoh 网络向整个机器人集群广播指令。

At the end, you can clone the working sample application from GitHub and run it on your laptop in simulation. No hardware, no GPU, no Hugging Face credentials needed for the default path. The runnable companion to this post lives at examples/lerobot/hub_to_hardware.py and hub_to_hardware.ipynb. The notebook is sim-only and Mock-policy by default. 最后，您可以从 GitHub 克隆示例应用程序，并在笔记本电脑上进行仿真运行。默认路径无需硬件、GPU 或 Hugging Face 凭据。本文配套的可运行代码位于 examples/lerobot/hub_to_hardware.py 和 hub_to_hardware.ipynb。该 Notebook 默认仅限仿真，并使用 Mock 策略。

What you’ll build

你将构建什么

The Strands Robots SDK exposes the LeRobot stack as AgentTools that you compose into one Strands agent. The example agent in this post does four things: record new demonstrations in simulation, push the result to the Hub as a LeRobotDataset, run a policy in simulation against that same format, and deploy the same agent code to a physical robot with one keyword argument change. When you have more than one robot, the agent can coordinate the whole fleet through a built-in peer mesh. Strands Robots SDK 将 LeRobot 堆栈公开为 AgentTools，你可以将其组合成一个 Strands 智能体。本文的示例智能体执行四项任务：在仿真中录制新演示，将结果作为 LeRobotDataset 推送到 Hub，在仿真中针对相同格式运行策略，并通过更改一个关键字参数将相同的智能体代码部署到物理机器人。当拥有多台机器人时，智能体可以通过内置的对等网络协调整个集群。

For hardware recording and calibration, LeRobot’s own CLIs (lerobot-record, lerobot-calibrate) handle the bring-up; the agent picks up from there. 对于硬件录制和校准，LeRobot 自身的 CLI 工具（lerobot-record, lerobot-calibrate）负责启动；智能体则从后续步骤接管。

Figure 1. Robot(“so100”) defaults to a MuJoCo-backed simulation; mode=“real” returns a hardware robot driven by LeRobot. Both modes share the same DatasetRecorder and the same policy providers, so a dataset captured in sim and a dataset captured on hardware use the same on-disk LeRobotDataset format. 图 1. Robot("so100") 默认指向基于 MuJoCo 的仿真；mode="real" 则返回由 LeRobot 驱动的物理机器人。两种模式共享相同的 DatasetRecorder 和策略提供程序，因此在仿真中捕获的数据集与在硬件上捕获的数据集使用相同的磁盘 LeRobotDataset 格式。

Two design choices make this work. First, Robot(“so100”) returns a simulation by default (no hardware, no risk), and mode=“real” returns a hardware-backed robot driven by LeRobot. The agent code is identical across both modes. Second, the DatasetRecorder that writes a LeRobotDataset is shared between the simulation path and LeRobot’s own hardware recording, so a dataset captured in MuJoCo and one captured from a physical SO-101 are in the same format. 两个设计选择实现了这一点。首先，Robot("so100") 默认返回仿真（无硬件，无风险），而 mode="real" 返回由 LeRobot 驱动的硬件机器人。两种模式下的智能体代码完全相同。其次，写入 LeRobotDataset 的 DatasetRecorder 在仿真路径和 LeRobot 自身的硬件录制之间共享，因此在 MuJoCo 中捕获的数据集与从物理 SO-101 捕获的数据集格式相同。

The whole workflow in five lines of Python: 整个工作流程仅需五行 Python 代码：

from strands_robots import Robot
from strands import Agent

arm = Robot("so100") # mode="sim" (default - safe, no hardware)
agent = Agent(tools=[arm])
agent("Pick up the red cube")

What follows is what’s actually happening inside that call, step by step. 接下来将逐步解析该调用内部实际发生的过程。

Prerequisites

前置条件

Minimal (default simulation path) 最小化要求（默认仿真路径）

  • Python 3.12+, on Linux or macOS (Apple Silicon supported for the MuJoCo backend).
  • Python 3.12+，运行于 Linux 或 macOS（MuJoCo 后端支持 Apple Silicon）。
  • A Strands-compatible model provider for the agent’s reasoning. Amazon Bedrock with AWS credentials, the Anthropic API, OpenAI, or Ollama running locally.
  • 一个与 Strands 兼容的模型提供商，用于智能体的推理。可以是带有 AWS 凭据的 Amazon Bedrock、Anthropic API、OpenAI 或本地运行的 Ollama。
  • Strands Robots installed with the install extras: uv pip install "strands-robots[sim-mujoco,lerobot,mesh]"
  • 安装带有额外组件的 Strands Robots：uv pip install "strands-robots[sim-mujoco,lerobot,mesh]"

That’s it. The example in this post runs end-to-end on a laptop with these three. 就是这样。本文中的示例仅需这三项即可在笔记本电脑上端到端运行。

Advanced (hardware deployment, real policies, Hub push) 进阶要求（硬件部署、真实策略、Hub 推送）

  • A Hugging Face account and token with write permission, for pushing datasets and pulling policy checkpoints from the Hub.
  • 一个拥有写入权限的 Hugging Face 账户和 Token，用于从 Hub 推送数据集和拉取策略检查点。
  • For the hardware path: an SO-101 follower and leader pair, or any other LeRobot-supported robot. Both devices need calibration files under ~/.cache/huggingface/lerobot/calibration/.
  • 硬件路径要求：一对 SO-101 跟随者和领导者，或任何其他 LeRobot 支持的机器人。两个设备都需要在 ~/.cache/huggingface/lerobot/calibration/ 下存放校准文件。
  • For local GR00T inference: an NVIDIA GPU with at least 16 GB of video memory and Docker installed. The post uses the gr00t_inference tool’s lifecycle="full" action, which pulls the image, downloads a checkpoint, and starts the container in one call.
  • 本地 GR00T 推理要求：配备至少 16GB 显存的 NVIDIA GPU 并安装 Docker。本文使用了 gr00t_inference 工具的 lifecycle="full" 操作，该操作可一键拉取镜像、下载检查点并启动容器。

Step 1 - Set up the example

第一步 - 设置示例

Install Strands Robots and get the example files: 安装 Strands Robots 并获取示例文件：

uv pip install "strands-robots[sim-mujoco,lerobot,mesh]"
git clone https://github.com/strands-labs/robots.git
cd robots

Export your Hugging Face token if you want the agent to push datasets or pull policies from the Hub. This is optional for the default simulation path in this post; the example runs end-to-end with the Mock policy and writes the dataset to your local cache without needing Hub access. 如果您希望智能体从 Hub 推送数据集或拉取策略，请导出您的 Hugging Face Token。对于本文的默认仿真路径，这是可选的；示例可以使用 Mock 策略端到端运行，并将数据集写入本地缓存，无需访问 Hub。

export HF_TOKEN=hf_...

The runnable example lives at examples/lerobot/hub_to_hardware.py (Python script) and hub_to_hardware.ipynb (notebook), in the strands-labs/robots repository alongside the MuJoCo and LIBERO examples. The notebook is the recommended starting point: open it in JupyterLab and run cells top-to-bottom in simulation mode without any hardware connected. 可运行的示例位于 strands-labs/robots 仓库中的 examples/lerobot/hub_to_hardware.py（Python 脚本）和 hub_to_hardware.ipynb（Notebook），与 MuJoCo 和 LIBERO 示例并列。推荐从 Notebook 开始：在 JupyterLab 中打开它，并在未连接任何硬件的情况下以仿真模式从上到下运行单元格。

Step 2 - Record demonstrations and push to the Hub

第二步 - 录制演示并推送到 Hub

The simulation tool records LeRobotDatasets in the same format LeRob… 仿真工具以 LeRobot 写入的相同格式记录 LeRobotDatasets…

← 返回新闻列表 / Back to news list