The State of Simulation for Physical AI: An Overview

物理人工智能仿真现状：概览

The primary challenge in building physical AI systems is data availability. Large language models (LLMs) and vision-language models (VLMs) can be trained on internet-scale datasets, but robotics and physical AI systems do not have the same advantage. To train a physical AI system, a robot must learn the consequences of interacting with the physical world. For example, it needs to understand what happens when a cup slips, a cable bends, or a gripper contacts an object at the wrong angle. Collecting this kind of data in the real world is slow, expensive, risky, and sometimes impractical due to the destructive nature of the tasks.

构建物理人工智能（Physical AI）系统的主要挑战在于数据获取。大语言模型（LLMs）和视觉语言模型（VLMs）可以在互联网规模的数据集上进行训练，但机器人和物理人工智能系统并不具备同样的优势。为了训练物理人工智能系统，机器人必须学习与物理世界交互所产生的后果。例如，它需要理解当杯子滑落、电缆弯曲或机械爪以错误角度接触物体时会发生什么。在现实世界中收集此类数据既缓慢、昂贵、充满风险，且由于任务的破坏性，有时甚至是不切实际的。

Simulation provides a bridge by enabling developers to generate large amounts of photorealistic, physically grounded data. By teleoperating robots in simulation and scaling data collection through GPU parallelism, developers can generate thousands of hours of robot experience at a fraction of the cost of real-world collection.

仿真技术通过使开发者能够生成大量具有照片级真实感且符合物理规律的数据，架起了一座桥梁。通过在仿真环境中远程操作机器人，并利用 GPU 并行计算扩展数据收集，开发者可以以极低的成本生成数千小时的机器人经验数据，这仅相当于现实世界收集成本的一小部分。

Earlier robotics simulators were often used primarily to debug geometry, test controllers, or visualize robot motion. Today, simulation has become part of the model development loop. Teams use it to generate perception datasets, train reinforcement learning policies, collect demonstrations, augment real-world data, benchmark models, and test policies against rare or adversarial scenarios. This shift is why industrial research labs and academic groups are increasingly contributing to, or developing, simulation engines that can meet these new requirements.

早期的机器人仿真器通常主要用于调试几何结构、测试控制器或可视化机器人运动。如今，仿真已成为模型开发循环的一部分。团队利用它来生成感知数据集、训练强化学习策略、收集演示数据、增强现实世界数据、对模型进行基准测试，并在罕见或对抗性场景中测试策略。这种转变正是工业研究实验室和学术团体越来越多地参与贡献或开发能够满足这些新需求的仿真引擎的原因。

These requirements can be understood through a three-computer paradigm:

Training computer: A large GPU cluster used to process data and train foundation AI models.
Simulation computer: A GPU workstation or cluster using GPU-accelerated physics and RTX rendering to generate robot experience, sensor data, and simulated interactions.
On-robot computer: An edge device, such as a NVIDIA Jetson AGX Thor-class system, that runs the trained policy or model during deployment.

这些需求可以通过“三机范式”来理解：

训练计算机：用于处理数据和训练人工智能基础模型的大型 GPU 集群。
仿真计算机：使用 GPU 加速物理引擎和 RTX 渲染的 GPU 工作站或集群，用于生成机器人经验、传感器数据和模拟交互。
机载计算机：部署在机器人上的边缘设备（如 NVIDIA Jetson AGX Thor 级系统），用于在实际部署期间运行训练好的策略或模型。

Many simulation engines are available today, each targeting different use cases such as humanoid robots, aerial vehicles, ground robots, autonomous vehicles, quadrupeds, and dexterous manipulators. These engines also differ in their support for reinforcement learning, batched simulation, contact-rich physics, photorealistic rendering, and sensor simulation.

目前市面上有许多仿真引擎，每种引擎都针对不同的应用场景，例如人形机器人、飞行器、地面机器人、自动驾驶汽车、四足机器人和灵巧机械手。这些引擎在对强化学习、批量仿真、接触丰富的物理模拟、照片级渲染和传感器仿真的支持上也各不相同。

Which Simulation Engine Should I Use?

我应该使用哪种仿真引擎？

Developers often struggle with which simulation engine to use for the use case they are tackling. Oftentimes, this falls on a few fundamental questions, such as: Do I need scalable synthetic data generation workflows? Do I need to do reinforcement learning? What kind of sensor support does a given simulation engine have? What kind of 3D asset formats does the simulation engine support? What environmental fidelity and scale are required?

开发者在面对特定应用场景时，往往难以抉择该使用哪种仿真引擎。这通常取决于几个基本问题，例如：我是否需要可扩展的合成数据生成工作流？我是否需要进行强化学习？特定的仿真引擎支持什么样的传感器？该仿真引擎支持哪些 3D 资产格式？所需的仿真环境保真度和规模是多少？

NVIDIA Isaac Sim and Isaac Lab, MuJoCo, PyBullet, Drake, and Genesis are some of the most popular simulation engines for humanoids and two-arm dexterous manipulation. Each of these simulation engines is unique to specific use cases such as scaling reinforcement learning, generating synthetic data, testing algorithms and/or models in simulation.

NVIDIA Isaac Sim 和 Isaac Lab、MuJoCo、PyBullet、Drake 以及 Genesis 是目前用于人形机器人和双臂灵巧操作最流行的仿真引擎。这些仿真引擎各具特色，分别适用于扩展强化学习、生成合成数据以及在仿真中测试算法和/或模型等特定用例。

MuJoCo

MuJoCo, short for Multi-Joint dynamics with Contact, is a fast, accurate, open-source physics engine built mainly for robotics, biomechanics, reinforcement learning, control, and simulation of articulated systems. Unlike engines focused mostly on games or visual realism, MuJoCo is designed around precise dynamics, contact-rich motion, and model-based optimization. MuJoCo emphasizes speed, accuracy, and modeling power for systems where physical correctness matters.

MuJoCo

MuJoCo（Multi-Joint dynamics with Contact 的缩写）是一个快速、精确的开源物理引擎，主要为机器人技术、生物力学、强化学习、控制和关节系统仿真而构建。与主要侧重于游戏或视觉真实感的引擎不同，MuJoCo 的设计核心是精确的动力学、接触丰富的运动和基于模型的优化。对于物理正确性至关重要的系统，MuJoCo 强调速度、精度和建模能力。

MuJoCo Warp

MuJoCo Warp, also called MJWarp, is a GPU-accelerated implementation of MuJoCo written in NVIDIA Warp, a Python framework for writing high-performance CUDA-accelerated differentiable kernels - tailored for robot learning and physics simulation.

MuJoCo Warp

MuJoCo Warp（也称为 MJWarp）是 MuJoCo 的 GPU 加速实现，使用 NVIDIA Warp 编写。NVIDIA Warp 是一个用于编写高性能 CUDA 加速可微内核的 Python 框架，专为机器人学习和物理仿真而量身定制。