Recent advances in machine learning have created increasing interest in solving visual computing problems using a class of coordinate-based neural networks that parametrize physical properties of scenes or objects across space and time. These methods, which we call neural fields, have seen successful application in the synthesis of 3D shapes and image, animation of human bodies, 3D reconstruction, and pose estimation. However, due to rapid progress in a short time, many papers exist but a comprehensive review and formulation of the problem has not yet emerged. In this report, we address this limitation by providing context, mathematical grounding, and an extensive review of literature on neural fields. This report covers research along two dimensions. In Part I, we focus on techniques in neural fields by identifying common components of neural field methods, including different representations, architectures, forward mapping, and generalization methods. In Part II, we focus on applications of neural fields to different problems in visual computing, and beyond (e.g., robotics, audio). Our review shows the breadth of topics already covered in visual computing, both historically and in current incarnations, demonstrating the improved quality, flexibility, and capability brought by neural fields methods. Finally, we present a companion website that contributes a living version of this review that can be continually updated by the community.
翻译:在机器学习方面最近的进展使人们越来越有兴趣利用一组协调的神经网络来解决视觉计算问题,这种网络可以将场景或物体的物理特性在时空间和时间上进行分辨。这些方法被称为神经领域,在3D形状和图像的合成、人体身体的动画、3D重建以及估计方面都成功地应用了这些方法。然而,由于在很短的时间内取得了迅速的进展,许多文件都存在,但是还没有对问题进行全面的审查和阐述。在本报告中,我们通过提供背景、数学基础和对神经领域的文献进行广泛审查来解决这一局限性。本报告涵盖了两个方面的研究。在第一部分,我们侧重于神经领域的技术,确定了神经领域方法的共同组成部分,包括不同的表现、结构、前方绘图和一般化方法。在第二部分,我们侧重于神经领域在视觉计算和范围以外(例如机器人、音响)的不同问题的应用。我们的审查显示了视觉计算、历史和当代的文献中已经涵盖的课题的广度。我们侧重于神经领域的技术,展示了改进的质量、灵活性和能力。最后,我们注重神经领域方法的共同组成部分的共同组成部分,从而不断更新网站。