Our method studies the complex task of object-centric 3D understanding from a single RGB-D observation. As it is an ill-posed problem, existing methods suffer from low performance for both 3D shape and 6D pose and size estimation in complex multi-object scenarios with occlusions. We present ShAPO, a method for joint multi-object detection, 3D textured reconstruction, 6D object pose and size estimation. Key to ShAPO is a single-shot pipeline to regress shape, appearance and pose latent codes along with the masks of each object instance, which is then further refined in a sparse-to-dense fashion. A novel disentangled shape and appearance database of priors is first learned to embed objects in their respective shape and appearance space. We also propose a novel, octree-based differentiable optimization step, allowing us to further improve object shape, pose and appearance simultaneously under the learned latent space, in an analysis-by-synthesis fashion. Our novel joint implicit textured object representation allows us to accurately identify and reconstruct novel unseen objects without having access to their 3D meshes. Through extensive experiments, we show that our method, trained on simulated indoor scenes, accurately regresses the shape, appearance and pose of novel objects in the real-world with minimal fine-tuning. Our method significantly out-performs all baselines on the NOCS dataset with an 8% absolute improvement in mAP for 6D pose estimation. Project page: https://zubair-irshad.github.io/projects/ShAPO.html
翻译:我们的方法是从一个 RGB- D 观测中研究以物体为中心的 3D 理解的复杂任务。 由于这是一个不正确的问题, 现有方法在复杂的多目标假设中, 3D 形状和 6D 形状的性能和大小估计都存在低效问题。 我们提出 ShAPO, 一种联合多球探测、 3D 纹理重建、 6D 对象构成和大小估计的方法。 ShAPO 的密钥是一条单一的管道, 以回归形状、 外观和每个对象的面罩来显示潜在的代码, 然后以稀薄至感敏的方式进一步完善。 一个新的变形形状和外观的外观数据库首先学会将物体嵌入各自的形状和外观空间。 我们还提出一个新颖的、 以树本为基础的不同优化步骤, 使我们能够进一步改进物体的形状, 并同时在所学的潜层空间下, 以分析同步的方式 。 我们的新隐含的物体估计面图解的表达方式使我们能够精确地识别和重新重建新的、 3D- D- AS 样的图像的精确地展示我们的模型。 通过广泛的实验, 我们的模拟的模拟的模型的模拟的模型的模型的模型的模型的模型, 展示, 展示, 我们展示所有的模型的模拟的模型的模型的模型的模拟的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型的模型。