We present SCANimate, an end-to-end trainable framework that takes raw 3D scans of a clothed human and turns them into an animatable avatar. These avatars are driven by pose parameters and have realistic clothing that moves and deforms naturally. SCANimate does not rely on a customized mesh template or surface mesh registration. We observe that fitting a parametric 3D body model, like SMPL, to a clothed human scan is tractable while surface registration of the body topology to the scan is often not, because clothing can deviate significantly from the body shape. We also observe that articulated transformations are invertible, resulting in geometric cycle consistency in the posed and unposed shapes. These observations lead us to a weakly supervised learning method that aligns scans into a canonical pose by disentangling articulated deformations without template-based surface registration. Furthermore, to complete missing regions in the aligned scans while modeling pose-dependent deformations, we introduce a locally pose-aware implicit function that learns to complete and model geometry with learned pose correctives. In contrast to commonly used global pose embeddings, our local pose conditioning significantly reduces long-range spurious correlations and improves generalization to unseen poses, especially when training data is limited. Our method can be applied to pose-aware appearance modeling to generate a fully textured avatar. We demonstrate our approach on various clothing types with different amounts of training data, outperforming existing solutions and other variants in terms of fidelity and generality in every setting. The code is available at https://scanimate.is.tue.mpg.de.
翻译:我们展示了Scanimate, 是一个从端到端的可训练框架, 将布衣人的原始 3D 扫描成3D 扫描, 将其转换成可想象的变异体。 这些变异体是由表面参数驱动的, 并有自然移动和变形的现实服装。 Scanimate 并不依赖定制的网格模板或表面网格登记。 我们观察到, 像 SMPL 那样, 将3D 体形模型与布衣人扫描相匹配是可移植到布衣人扫描的, 而将体型结构向扫描的表面方法进行表面登记往往不是, 因为服装与身体形状的形状有显著的变异。 我们还观察到, 表达的变异的变异性变化是不可变异的变异性变异性, 表达的变异性变的变异性变异性, 显示清晰的变异性变异性变异性变形是不可变异性变异性, 显示清晰的变异性变异性变异性变异性变异性变形, 的变异性变异性变异性变异性变形是无法变异性变形。 我们的变异性变异性变变变变异性变异性变异性变变变变变变形变变变变变变形变形变形变形的变变变变变变变变变变变变变变变异性变形的变的变变变形,,,, 变变变变的变的变变变变型的变异性变性变异性变变变变变变变变变变变变异性变变变变变异性变变变变变变变变变变变变变的变的变变变变变变变变变变变变变变变变变变变变变的变的变变变变的变变的变的变的变变变变变的变的变的变的变的变的变的变的变的变的变的变的变的变的变变的变的变的变的变的变的变的变的变的变的变的变的变的变形,,,,,我们变变变变变变变的