In this paper, we aim to create generalizable and controllable neural signed distance fields (SDFs) that represent clothed humans from monocular depth observations. Recent advances in deep learning, especially neural implicit representations, have enabled human shape reconstruction and controllable avatar generation from different sensor inputs. However, to generate realistic cloth deformations from novel input poses, watertight meshes or dense full-body scans are usually needed as inputs. Furthermore, due to the difficulty of effectively modeling pose-dependent cloth deformations for diverse body shapes and cloth types, existing approaches resort to per-subject/cloth-type optimization from scratch, which is computationally expensive. In contrast, we propose an approach that can quickly generate realistic clothed human avatars, represented as controllable neural SDFs, given only monocular depth images. We achieve this by using meta-learning to learn an initialization of a hypernetwork that predicts the parameters of neural SDFs. The hypernetwork is conditioned on human poses and represents a clothed neural avatar that deforms non-rigidly according to the input poses. Meanwhile, it is meta-learned to effectively incorporate priors of diverse body shapes and cloth types and thus can be much faster to fine-tune, compared to models trained from scratch. We qualitatively and quantitatively show that our approach outperforms state-of-the-art approaches that require complete meshes as inputs while our approach requires only depth frames as inputs and runs orders of magnitudes faster. Furthermore, we demonstrate that our meta-learned hypernetwork is very robust, being the first to generate avatars with realistic dynamic cloth deformations given as few as 8 monocular depth frames.
翻译:在本文中,我们的目标是创建可笼统且可控的神经标志的距离场(SDFs),它们代表着单心深度观测的人类。最近深层学习的进展,特别是神经隐含的表达方式,使得人类形状的重建和可控的阿凡达体生成了不同感官输入物。然而,为了产生现实的布质变形,我们通常需要用新输入物来生成,水密胶片或密集的全体扫描通常需要作为输入物。此外,由于难以有效模拟不同身体形状和布型的成形自制布质变形,现有方法从擦入开始就采用每个部位/布型的优化,这是计算成本昂贵的。相比之下,我们建议的一种方法可以快速生成现实的衣质变的人类腹形变形变形变形变形变形变形,因为只有单心深度的图像,我们通过元化学习来学习一种超网络的初始化,可以预测神经变形变形变形变形变形变形的参数。超级网络只能以人体变形变形变形和变形变形的内脏方法为条件,我们首先需要穿的神经变形变形变形变形变形变形的神经的神经变形,而要将变形变形变形变形变形变形变形变形变形变形的内变形变形变形,同时要显示和变形变形变形的变形的变形变形变形的变形的变形的变形的变形的变形变形变形变形的变形变形变形变形变形变形变形变形变形的变形的变形的变形的变形的变形变形变形的变形变形变形的变形变形变形变形变形变形变形变形体,要的变形变形变形的变形体,要的变形和变形变形的变形和变形的变形的变形的变形变形变形的变形的变形变形的变形变形的变形的变形的变形和变形的变形的变形的变形的变形变形变形变形变形变形的变形的变形变形变形的变形体,