单体: 单体 3D 语义场景补全 (MonoScene: Monocular 3D Semantic Scene Completion)

MonoScene proposes a 3D Semantic Scene Completion (SSC) framework, where the dense geometry and semantics of a scene are inferred from a single monocular RGB image. Different from the SSC literature, relying on 2.5 or 3D input, we solve the complex problem of 2D to 3D scene reconstruction while jointly inferring its semantics. Our framework relies on successive 2D and 3D UNets bridged by a novel 2D-3D features projection inspiring from optics and introduces a 3D context relation prior to enforce spatio-semantic consistency. Along with architectural contributions, we introduce novel global scene and local frustums losses. Experiments show we outperform the literature on all metrics and datasets while hallucinating plausible scenery even beyond the camera field of view. Our code and trained models are available at https://github.com/cv-rits/MonoScene

翻译：MonoScene 提出了一个 3D 语义场景补全框架( SSC) 3D 。在该框架中,一个场景的密度几何学和语义从一个单眼 RGB 图像中推断出来。与 SSC 文献不同, 我们依靠2.5 或 3D 输入, 解决了 2D 至 3D 场景重建的复杂问题, 同时共同推断出其语义。我们的框架依靠相继的 2D 和 3D UNets 相继连接的2D 和 3D UNets 。我们的框架依靠来自光学的小说 2D-3D 特征投影, 并引入了 3D 上下文, 在强制执行 spotio- semanic 一致性之前。除了建筑贡献外, 我们还引入了新的全球场景和局部条形体损失。实验显示我们超越了所有仪表和数据集的文献, 而在摄影场外。我们的代码和训练有素的模型可以在 https://github. com/ cv- rits/ Mon- trits/ MonSene 。