Learning to reconstruct 3D garments is important for dressing 3D human bodies of different shapes in different poses. Previous works typically rely on 2D images as input, which however suffer from the scale and pose ambiguities. To circumvent the problems caused by 2D images, we propose a principled framework, Garment4D, that uses 3D point cloud sequences of dressed humans for garment reconstruction. Garment4D has three dedicated steps: sequential garments registration, canonical garment estimation, and posed garment reconstruction. The main challenges are two-fold: 1) effective 3D feature learning for fine details, and 2) capture of garment dynamics caused by the interaction between garments and the human body, especially for loose garments like skirts. To unravel these problems, we introduce a novel Proposal-Guided Hierarchical Feature Network and Iterative Graph Convolution Network, which integrate both high-level semantic features and low-level geometric features for fine details reconstruction. Furthermore, we propose a Temporal Transformer for smooth garment motions capture. Unlike non-parametric methods, the reconstructed garment meshes by our method are separable from the human body and have strong interpretability, which is desirable for downstream tasks. As the first attempt at this task, high-quality reconstruction results are qualitatively and quantitatively illustrated through extensive experiments. Codes are available at https://github.com/hongfz16/Garment4D.
翻译:重建3D服装对于以不同形状的3D人的身体穿戴不同形状的3D服装十分重要。 以前的作品通常依赖2D图像作为投入, 但这些图像受到规模的影响, 并造成模糊不清。 为了绕过2D图像造成的问题, 我们提议了一个原则框架, 配色4D, 将穿衣人的3D点云序列用于服装重建。 配色4D有三个专门步骤: 制衣登记、 制衣估计和服装重建。 主要挑战有两重:(1) 3D特征的精细细节有效学习;和(2) 捕捉服装与人体互动导致的服装动态, 特别是服装与裙子等松散的服装。 为了克服这些问题, 我们提出了一个新的建议- 指导性高等级地貌网络和动态图象变异网络, 将高层次的语义特征和低层次的地理测量特征结合起来进行细细细细节重建。 此外, 我们提议为光滑的服装动作捕捉取一个温度变形器。 与非分辨方法不同, 我们的方法对服装的重塑成型的成型的服装结构, 具有高层次的尝试。