A Synthetic-Driven Vision System for Assembly Step Recognition

一种用于装配步骤识别的合成驱动视觉系统

Abstract: Quality control in industrial assembly is essential, and real-time monitoring of the assembly process is crucial for preventing costly defects and ensuring production reliability. Vision-based automated inspection offers a powerful solution for such real-time monitoring. However, due to the specialized industrial components and processes, training these models typically relies on task-specific real-world data, which is costly and labor-intensive to collect and annotate.

摘要： 工业装配中的质量控制至关重要，而对装配过程进行实时监控对于防止昂贵的缺陷并确保生产可靠性至关重要。基于视觉的自动化检测为这种实时监控提供了一种强有力的解决方案。然而，由于工业组件和工艺的专业性，训练这些模型通常依赖于特定任务的真实世界数据，而这些数据的收集和标注既昂贵又耗费人力。

In this paper, we propose a system that automatically generates realistic assembly sequences and further trains real-time inspection models using the synthetic data. It can be efficiently applied to a given task within an hour, requiring only CAD models and simple step descriptions.

在本文中，我们提出了一种能够自动生成逼真装配序列，并利用合成数据进一步训练实时检测模型的系统。该系统仅需 CAD 模型和简单的步骤描述，即可在不到一小时内高效应用于特定任务。

Focusing on practical challenges, our system integrates a physics-based motion generation module to capture the variance of different human assembly, designs domain-randomized rendering to deal with the environmental complexity and variation, and employs an object-detection-based step recognition module for robust sim-to-real transfer, leading to 92.4% accuracy on a real-world assembly case with 46.7%, 15.8% and 61.2% performance improvement, respectively.

针对实际挑战，我们的系统集成了一个基于物理的运动生成模块以捕捉不同人类装配方式的差异，设计了域随机化渲染来应对环境的复杂性和变化，并采用基于目标检测的步骤识别模块以实现稳健的“仿真到现实”（sim-to-real）迁移。在真实世界的装配案例中，该系统达到了 92.4% 的准确率，性能分别提升了 46.7%、15.8% 和 61.2%。

Overall, our system provides a practical solution for industrial assembly inspection without requiring expensive real-world data collection and annotation, with the effectiveness validated on real industrial assembly tasks.

总而言之，我们的系统为工业装配检测提供了一种实用的解决方案，无需昂贵的真实世界数据收集和标注，且其有效性已在真实的工业装配任务中得到了验证。