We present PyMAF-X, a regression-based approach to recovering a full-body parametric model from a single image. This task is very challenging since minor parametric deviation may lead to noticeable misalignment between the estimated mesh and the input image. Moreover, when integrating part-specific estimations to the full-body model, existing solutions tend to either degrade the alignment or produce unnatural wrist poses. To address these issues, we propose a Pyramidal Mesh Alignment Feedback (PyMAF) loop in our regression network for well-aligned human mesh recovery and extend it as PyMAF-X for the recovery of expressive full-body models. The core idea of PyMAF is to leverage a feature pyramid and rectify the predicted parameters explicitly based on the mesh-image alignment status. Specifically, given the currently predicted parameters, mesh-aligned evidence will be extracted from finer-resolution features accordingly and fed back for parameter rectification. To enhance the alignment perception, an auxiliary dense supervision is employed to provide mesh-image correspondence guidance while spatial alignment attention is introduced to enable the awareness of the global contexts for our network. When extending PyMAF for full-body mesh recovery, an adaptive integration strategy is proposed in PyMAF-X to produce natural wrist poses while maintaining the well-aligned performance of the part-specific estimations. The efficacy of our approach is validated on several benchmark datasets for body-only and full-body mesh recovery, where PyMAF and PyMAF-X effectively improve the mesh-image alignment and achieve new state-of-the-art results. The project page with code and video results can be found at https://www.liuyebin.com/pymaf-x.
翻译:我们提出一个基于回归法的PyMAF-X,这是从单一图像中恢复全体模拟模型的一种基于回归法的方法。这一任务非常具有挑战性,因为轻微的参数偏差可能导致估计网格和输入图像之间明显不匹配。此外,如果将部分特定估算与全体模型相结合,现有的解决方案往往会降低校正或产生非自然手腕构成。为了解决这些问题,我们提议在我们的回归网中采用一个Pyramidal Mesh匹配反馈(PyMAF)环绕,用于完善的人类网格恢复,并将其作为恢复全体机型模型的PyMAF-X。PyMA的核心想法是利用一个功能金字塔,并纠正基于整体图像调整状态的预测参数。具体地说,根据目前预测的参数,网形分辨率校正证据将相应地提取,并反馈参数校正的校正。为了加强校正感,我们采用了一个辅助性精密的内嵌式对正对等的校正方向,同时引入空间调整关注,以使全球范围内对全体调整的认识,同时将实现网络的图像校正校正结果。