Coarse3D: Controllable and Faithful Sparse Structure Generation for Image-to-3D

查看原文 / View Original 编辑 / Edit

Computer Science > Computer Vision and Pattern Recognition arXiv:2610.41231 (cs) [Submitted on 30 Oct 2026] Title: Coarse3D: Controllable and Faithful Sparse Structure Generation for Image-to-3D Authors: Gaige Wang, Bowen Pan, Hao Ren, Jiangjing Lyu, Chengfei Lyu.

计算机科学 > 计算机视觉与模式识别 arXiv:2610.41231 (cs) [提交于 2026 年 10 月 30 日] 标题：Coarse3D：用于图像生成 3D 的可控且忠实的稀疏结构生成作者：Gaige Wang, Bowen Pan, Hao Ren, Jiangjing Lyu, Chengfei Lyu。

Abstract: Coarse-to-fine 3D-native generation relies heavily on the first-stage sparse structure, which determines the global orientation and overall geometry for subsequent refinement. However, existing coarse generators suffer from two key limitations: their orientation is difficult to control due to ambiguous semantic canonicalization, and their structures are often inconsistent with input images due to weak 2D-3D grounding.

摘要：从粗到细的 3D 原生生成在很大程度上依赖于第一阶段的稀疏结构，该结构决定了后续细化的全局方向和整体几何形状。然而，现有的粗糙生成器存在两个主要局限性：由于语义规范化模糊，其方向难以控制；且由于 2D-3D 接地较弱，其结构往往与输入图像不一致。

We propose Coarse3D, a sparse-structure generation model that produces orientation-controllable and structurally faithful voxel geometry from arbitrary unposed images. Coarse3D introduces three designs: a first-frame anchored canonical space that aligns generation with the first input view while preserving object-centric normalization; a dual-tower DiT that couples generation with auxiliary feed-forward reconstruction to learn spatially grounded features; and spatial-prior guided attention that regularizes cross-attention with pixel-to-voxel correspondence.

我们提出了 Coarse3D，这是一种稀疏结构生成模型，可以从任意未标注姿态的图像中生成方向可控且结构忠实的体素几何体。Coarse3D 引入了三种设计：一种以第一帧为锚点的规范空间，在保持以对象为中心的归一化的同时，将生成与第一个输入视图对齐；一种双塔 DiT，将生成与辅助前馈重建相结合以学习空间接地特征；以及空间先验引导的注意力机制，通过像素到体素的对应关系来规范交叉注意力。

Coarse3D can directly replace the coarse stage of existing pipelines. Experiments show that it substantially improves sparse structure quality and consistently enhances final 3D generation, achieving state-of-the-art performance.

Coarse3D 可以直接替换现有流程中的粗糙阶段。实验表明，它显著提高了稀疏结构的质量，并持续增强了最终的 3D 生成效果，达到了最先进的性能水平。