We propose a novel method applicable in many scene understanding problems that adapts the Monte Carlo Tree Search (MCTS) algorithm, originally designed to learn to play games of high-state complexity. From a generated pool of proposals, our method jointly selects and optimizes proposals that minimize the objective term. In our first application for floor plan reconstruction from point clouds, our method selects and refines the room proposals, modelled as 2D polygons, by optimizing on an objective function combining the fitness as predicted by a deep network and regularizing terms on the room shapes. We also introduce a novel differentiable method for rendering the polygonal shapes of these proposals. Our evaluations on the recent and challenging Structured3D and Floor-SP datasets show significant improvements over the state-of-the-art, without imposing hard constraints nor assumptions on the floor plan configurations. In our second application, we extend our approach to reconstruct general 3D room layouts from a color image and obtain accurate room layouts. We also show that our differentiable renderer can easily be extended for rendering 3D planar polygons and polygon embeddings. Our method shows high performance on the Matterport3D-Layout dataset, without introducing hard constraints on room layout configurations.
翻译:我们提出适用于许多场景理解问题的新方法,以适应蒙特卡洛树搜索(MCTS)算法,该算法最初旨在学习玩高状态复杂的游戏。我们的方法从生成的一组建议中,共同选择和优化了尽量减少目标术语的建议。在我们第一次从点云中进行地面计划重建的应用程序中,我们的方法选择和完善了以 2D 多边形为模范的房间建议,优化了将深层次网络预测的健身和室内形状的定时条件结合起来的客观功能。我们还引入了一种创新的不同方法来构建这些提议的多边形形。我们对最近和具有挑战性的结构3D和地面-SP数据集的评估显示,在不强加硬性限制或假设地面计划配置的情况下,对现状进行了重大改进。在我们的第二个应用中,我们扩展了从彩色图像中重建一般的 3D 房间布局的方法,并获得了准确的室内布局。我们还表明,我们不同的造型模型可以很容易被扩展,用于绘制3D 平面多边形和多边嵌嵌式。我们的方法显示的是,在不设硬式的图像上展示了高性配置。我们的方法显示的是,在MON3PROport3D3的图像上的高级配置。