Learning and Reasoning with Geometry: How Generation Helps Spatial Understanding

Title: Learning and Reasoning with Geometry: How Generation Helps Spatial Understanding

标题：学习与几何推理：生成模型如何助力空间理解

Image and video generators can produce spatially consistent visual content, but how to use this capability to improve spatial understanding remains underexplored. We study this problem from two perspectives: geometric supervision during training and explicit geometric evidence during inference.

图像和视频生成器能够产生空间一致的视觉内容，但如何利用这一能力来提升空间理解能力仍未得到充分探索。我们从两个角度研究了这一问题：训练过程中的几何监督以及推理过程中的显式几何证据。

To this end, we introduce a unified geometric understanding and generation model (GeoUG) that integrates depth and multi-view generation with spatial understanding. First, we design a multi-scale depth codec that enables metric-depth generation using a pretrained visual autoencoder, and show that joint training yields task-dependent gains in spatial understanding.

为此，我们引入了一个统一的几何理解与生成模型（GeoUG），它将深度和多视图生成与空间理解相结合。首先，我们设计了一种多尺度深度编解码器，利用预训练的视觉自动编码器实现度量深度生成，并证明了联合训练在空间理解方面带来了任务相关的增益。

Second, we introduce 3DCueBench to control geometric difficulty while balancing selected 2D cues, and find that generated depth provides larger reasoning gains on questions requiring fine-grained geometric discrimination. Building on this finding, we develop adaptive interleaved reasoning, in which GeoUG first drafts an answer and then decides whether to generate geometric evidence to confirm or revise it.

其次，我们引入了 3DCueBench 来控制几何难度，同时平衡选定的二维线索，并发现生成的深度信息在需要细粒度几何辨别的任务中提供了更大的推理增益。基于这一发现，我们开发了自适应交错推理，其中 GeoUG 首先起草一个答案，然后决定是否生成几何证据来确认或修正该答案。

We train this decision through reinforcement learning, balancing accuracy gains over retaining the draft against generation cost. GeoUG achieves circular accuracies of 97.9%, 90.8%, and 94.2% on CV-Bench-3D, 3DCueBench, and DA-2K with fewer generation calls than always generating.

我们通过强化学习来训练这一决策过程，在保持草稿的准确性增益与生成成本之间取得平衡。GeoUG 在 CV-Bench-3D、3DCueBench 和 DA-2K 上分别达到了 97.9%、90.8% 和 94.2% 的循环准确率，且其生成调用次数少于始终进行生成的模式。