The neural implicit representation has shown its effectiveness in novel view synthesis and high-quality 3D reconstruction from multi-view images. However, most approaches focus on holistic scene representation yet ignore individual objects inside it, thus limiting potential downstream applications. In order to learn object-compositional representation, a few works incorporate the 2D semantic map as a cue in training to grasp the difference between objects. But they neglect the strong connections between object geometry and instance semantic information, which leads to inaccurate modeling of individual instance. This paper proposes a novel framework, ObjectSDF, to build an object-compositional neural implicit representation with high fidelity in 3D reconstruction and object representation. Observing the ambiguity of conventional volume rendering pipelines, we model the scene by combining the Signed Distance Functions (SDF) of individual object to exert explicit surface constraint. The key in distinguishing different instances is to revisit the strong association between an individual object's SDF and semantic label. Particularly, we convert the semantic information to a function of object SDF and develop a unified and compact representation for scene and objects. Experimental results show the superiority of ObjectSDF framework in representing both the holistic object-compositional scene and the individual instances. Code can be found at https://qianyiwu.github.io/objectsdf/
翻译:神经隐含的表示方式在新颖的观点综合和高品质的多视图图像3D重建中显示了其有效性。然而,大多数方法侧重于整体场景代表,却忽视了其中的个别物体,从而限制了潜在的下游应用。为了学习对象-组合代表,一些作品将 2D 语义地图纳入到培训中,以了解对象之间的差异。但是它们忽略了物体几何和实例语义信息之间的紧密联系,从而导致个人实例的不准确建模。本文件提议了一个新颖的框架,即OcsSDF,以构建一个在 3D 重建与对象代表中高度忠实的物体-组合神经隐含代表;观察常规体积输送管道的模糊性,我们通过将个体物体的已签远程功能(SDF)合并来模拟场景,以施加明显的表面约束。区分不同情况的关键是重新审视单个物体的SDF和语义标签之间的牢固联系。特别是,我们将语义信息转换为对象SDF的函数,并为场景和物体形成一个统一和紧凑的表。实验性结果显示物体/DFDF框架的优越性。在代表整个目标/ADFDFDF/C/C/As/ apractimual