Retrieving accurate 3D reconstructions of objects from the way they reflect light is a very challenging task in computer vision. Despite more than four decades since the definition of the Photometric Stereo problem, most of the literature has had limited success when global illumination effects such as cast shadows, self-reflections and ambient light come into play, especially for specular surfaces. Recent approaches have leveraged the power of deep learning in conjunction with computer graphics in order to cope with the need of a vast number of training data in order to invert the image irradiance equation and retrieve the geometry of the object. However, rendering global illumination effects is a slow process which can limit the amount of training data that can be generated. In this work we propose a novel pixel-wise training procedure for normal prediction by replacing the training data (observation maps) of globally rendered images with independent per-pixel generated data. We show that global physical effects can be approximated on the observation map domain and this simplifies and speeds up the data creation procedure. Our network, PX-NET, achieves the state-of-the-art performance compared to other pixelwise methods on synthetic datasets, as well as the Diligent real dataset on both dense and sparse light settings.
翻译:在计算机视野中,从反映光的方式中获取精确的三维天体的重建是极具挑战性的任务。尽管自光度立体问题定义以来已有40多年了,但大多数文献在投影阴影、自我反射和环境光等全球照明效应开始起作用时,特别是在光谱表面,成效有限。最近的方法利用了与计算机图形相结合的深度学习能力,以满足大量培训数据的需求,从而改变图像辐照方程式和检索天体的几何学。然而,全球照明效应是一个缓慢的过程,可以限制可生成的培训数据的数量。在这项工作中,我们提出了一个新的像素智能培训程序,用于正常预测,用独立的每像素生成数据取代全球图像的培训数据(观察地图)。我们表明,全球物理效应可以与观测地图域相近,从而简化和加速数据创建程序。我们的网络、PX-NET和智能数据模型的快速度运行方式,可以作为合成数据的最精确度的精确度,作为其他数据的最精确度的模型。