Applying Deep Learning for cockpit segmentation in the context of mixed reality

在混合现实背景下应用深度学习进行驾驶舱分割

Abstract: Computer vision is an area that has been growing continuously. With the advance of technologies with a first-person view, new development opportunities have emerged inside the area. Mixed reality promotes virtual environments with objects from the physical world shown in real time. For that, it’s necessary to be concerned with the immersion of the user in this simulated environment, increasingly seeking to bring it closer to a possible desired reality.

摘要： 计算机视觉是一个持续发展的领域。随着第一人称视角技术的进步，该领域内涌现出了新的发展机遇。混合现实技术通过实时呈现物理世界中的物体来构建虚拟环境。因此，必须关注用户在模拟环境中的沉浸感，并不断寻求使其更接近理想的现实。

This paper proposes the development of image processing in order to perform the segmentation of images to identify what is foreground and background in order to facilitate the union of virtual and real images. Thus, the present work obtain real images of the user using the off-highway truck simulator CAT793F, through a camera, to be able to perform the segmentation of such images with artificial intelligence.

本文提出了一种图像处理开发方案，旨在通过图像分割来识别前景和背景，从而促进虚拟图像与真实图像的融合。为此，本研究通过摄像头获取用户使用 CAT793F 非公路卡车模拟器时的真实图像，并利用人工智能对这些图像进行分割。

The convolutional neural network architectures “U-net” and “DeepLabV3+” are applied to perform image segmentation. As a result, metrics with around 90% accuracy were presented and the best model was determined.

研究应用了“U-net”和“DeepLabV3+”卷积神经网络架构来进行图像分割。结果显示，模型准确率达到了 90% 左右，并据此确定了最佳模型。

---

Paper Details:

• Authors: Alexandre Leles Sousa, Pedro de Oliveira Nielson, Erick Oliveira Rodrigues, Rafael Francisco dos Santos, Giovani Bernardes Vitor
• arXiv ID: 2606.06520
• Subjects: Computer Vision and Pattern Recognition (cs.CV); Graphics (cs.GR)
• Submission Date: 2 Jun 2026

论文详情：

• 作者： Alexandre Leles Sousa, Pedro de Oliveira Nielson, Erick Oliveira Rodrigues, Rafael Francisco dos Santos, Giovani Bernardes Vitor
• arXiv ID: 2606.06520
• 学科分类： 计算机视觉与模式识别 (cs.CV)；图形学 (cs.GR)
• 提交日期： 2026年6月2日