Real-Time Frame- and Event-based Object Detection with Spiking Neural Networks on Edge Neuromorphic Hardware: Design, Deployment and Benchmark

基于边缘神经形态硬件的脉冲神经网络实时帧与事件驱动目标检测：设计、部署与基准测试

Abstract: Real-time object detection on energy-constrained platforms is critical for applications such as UAV-based inspection, autonomous navigation, and mobile robotics. Spiking neural networks (SNNs) on neuromorphic hardware are believed to be significantly more energy-efficient than conventional artificial neural networks (ANNs).

摘要： 在能源受限的平台上进行实时目标检测，对于无人机巡检、自主导航和移动机器人等应用至关重要。人们普遍认为，在神经形态硬件上运行的脉冲神经网络（SNN）比传统的人工神经网络（ANN）具有更高的能效。

In this work, we present a comprehensive methodology for designing general SNN detection architectures targeting neuromorphic platforms, along with the engineering adaptations required to deploy them on the state-of-the-art Neuromorphic processor, Intel Loihi 2.

在这项工作中，我们提出了一套全面的方法论，用于设计针对神经形态平台的通用 SNN 检测架构，并介绍了将其部署到最先进的神经形态处理器 Intel Loihi 2 上所需的工程适配方案。

We benchmark SNN-based object detection on Loihi 2 using both frame-based and event-based datasets, comparing performance with ANN-based detection on the NVIDIA Jetson Orin Nano, NVIDIA Jetson Nano B01, and the Apple M2 CPU.

我们使用基于帧和基于事件的数据集，对 Loihi 2 上的 SNN 目标检测进行了基准测试，并将其性能与 NVIDIA Jetson Orin Nano、NVIDIA Jetson Nano B01 以及 Apple M2 CPU 上的 ANN 检测性能进行了对比。

Our results show that SNNs on Loihi 2 can perform real-time detection while achieving the lowest per-inference dynamic energy among all platforms. Also, Loihi 2 outperforms the other platforms in terms of power consumption, though ANNs on Jetson Orin Nano achieve higher inference rates.

研究结果表明，Loihi 2 上的 SNN 能够实现实时检测，同时在所有测试平台中实现了最低的单次推理动态能耗。此外，尽管 Jetson Orin Nano 上的 ANN 实现了更高的推理速率，但 Loihi 2 在功耗方面优于其他平台。

Furthermore, our ANN-to-SNN distillation-aware training enables SNNs to recover 87-100% of the detection accuracy of their ANN counterparts while maintaining lower inference latency; without distillation, SNNs exhibit an 11-27% accuracy drop. These results highlight the potential of neuromorphic systems for energy-efficient, real-time object detection at the edge.

此外，我们提出的“ANN 转 SNN 蒸馏感知训练”方法，使 SNN 在保持较低推理延迟的同时，能够恢复其 ANN 对应模型 87-100% 的检测精度；若不使用蒸馏技术，SNN 的精度会下降 11-27%。这些结果凸显了神经形态系统在边缘侧实现高能效、实时目标检测的巨大潜力。