TriSAR-3D and PaintFusion: A Novel Trimodal LiDAR-RGB-Thermal Dataset and Method for 3D Person Detection in Search and Rescue

查看原文 / View Original 编辑 / Edit

Computer Science > Computer Vision and Pattern Recognition arXiv:2610.22509 (cs) [Submitted on 15 Oct 2026] Title: TriSAR-3D and PaintFusion: A Novel Trimodal LiDAR-RGB-Thermal Dataset and Method for 3D Person Detection in Search and Rescue Authors: Markus Kramer, Jan Wachenfeld, Oskar von Stryk.

计算机科学 > 计算机视觉与模式识别 arXiv:2610.22509 (cs) [提交于 2026 年 10 月 15 日] 标题：TriSAR-3D 和 PaintFusion：一种用于搜救中 3D 人员检测的新型三模态激光雷达-RGB-热成像数据集与方法作者：Markus Kramer, Jan Wachenfeld, Oskar von Stryk。

Abstract: Reliable 3D person detection is important for rescue robotics to give first responders situational awareness. Yet most multimodal 3D detection research targets automotive scenarios in which pedestrians mostly appear in a narrow range of upright, near-road poses. It is therefore unclear how much of the reported performance transfers to rescue settings, as a person may be lying, partially occluded, or in unstructured indoor and outdoor environments.

摘要：可靠的 3D 人员检测对于救援机器人向急救人员提供态势感知至关重要。然而，大多数多模态 3D 检测研究针对的是汽车场景，其中行人大多以直立、靠近道路的姿态出现。因此，尚不清楚所报告的性能在多大程度上能迁移到救援环境中，因为救援场景中的人员可能是躺着的、部分被遮挡的，或者处于非结构化的室内外环境中。

To investigate this gap, we introduce TriSAR-3D, a multimodal person detection dataset recorded with a high-resolution 128-channel LiDAR, an RGB camera, and a thermal camera. It captures varied poses in indoor as well as outdoor locations at different times of day to study their influence on 3D object detector performance.

为了研究这一差距，我们引入了 TriSAR-3D，这是一个使用高分辨率 128 线激光雷达、RGB 相机和热成像相机记录的多模态人员检测数据集。它捕捉了室内外不同地点和不同时间段的各种姿态，以研究它们对 3D 目标检测器性能的影响。

Furthermore, we propose a novel trimodal 3D detection pipeline, PaintFusion, that fuses PointPainting-based evidence using a simple MLP. We show that the derived method reaches 65.4 AP@50 and thereby outperforms strong LiDAR-only (CenterPoint, 53.4), RGB+LiDAR (BEVFusion, 60.5), and adapted trimodal (Tri-BEVFusion, 60.7) baselines on the dataset.

此外，我们提出了一种新型三模态 3D 检测流水线 PaintFusion，它使用简单的多层感知机（MLP）融合基于 PointPainting 的证据。我们证明该方法达到了 65.4 的 AP@50，从而在该数据集上优于强大的仅激光雷达（CenterPoint, 53.4）、RGB+激光雷达（BEVFusion, 60.5）和改进的三模态（Tri-BEVFusion, 60.7）基准。

At the same time, it is substantially faster than other fusion-based methods such as BEVFusion, with a per-frame latency of 51.6 ms compared to 101 ms. Additionally, we test how the performance of different 3D object detectors generalizes to unseen pose categories. Removing lying instances from training data collapses the lying-pose AP@50 to near zero across all evaluated detectors. The results indicate that dataset diversity exerts a large effect on 3D object detection performance.

同时，它比 BEVFusion 等其他基于融合的方法快得多，单帧延迟为 51.6 毫秒，而后者为 101 毫秒。此外，我们测试了不同 3D 目标检测器的性能如何泛化到未见过的姿态类别。从训练数据中移除躺卧实例会导致所有评估检测器的躺卧姿态 AP@50 几乎降至零。结果表明，数据集的多样性对 3D 目标检测性能有很大影响。