Predicting natural and diverse 3D hand gestures from the upper body dynamics is a practical yet challenging task in virtual avatar creation. Previous works usually overlook the asymmetric motions between two hands and generate two hands in a holistic manner, leading to unnatural results. In this work, we introduce a novel bilateral hand disentanglement based two-stage 3D hand generation method to achieve natural and diverse 3D hand prediction from body dynamics. In the first stage, we intend to generate natural hand gestures by two hand-disentanglement branches. Considering the asymmetric gestures and motions of two hands, we introduce a Spatial-Residual Memory (SRM) module to model spatial interaction between the body and each hand by residual learning. To enhance the coordination of two hand motions wrt. body dynamics holistically, we then present a Temporal-Motion Memory (TMM) module. TMM can effectively model the temporal association between body dynamics and two hand motions. The second stage is built upon the insight that 3D hand predictions should be non-deterministic given the sequential body postures. Thus, we further diversify our 3D hand predictions based on the initial output from the stage one. Concretely, we propose a Prototypical-Memory Sampling Strategy (PSS) to generate the non-deterministic hand gestures by gradient-based Markov Chain Monte Carlo (MCMC) sampling. Extensive experiments demonstrate that our method outperforms the state-of-the-art models on the B2H dataset and our newly collected TED Hands dataset.
翻译:从上体动态中预测自然和不同的自然 3D 手势是一个实际但具有挑战性的任务。 之前的工作通常忽略双手之间不对称的动作, 通常会以整体的方式产生两只手之间的不对称动作, 从而导致非自然的结果。 在此工作中, 我们引入了一个新的双手分解方法, 以两阶段 3D 手工生成方法为基础, 从身体动态中实现自然和多样化的 3D 手势。 在第一阶段, 我们打算用两个手分解的分支产生自然手势。 考虑到两只手的不对称动作和动作, 我们引入了一个空间- 恢复记忆模块, 以模拟两只手之间的空间互动, 以整体的方式生成非非非非非非自然的学习。 为了加强两只手动作的协调, 我们然后从整体上提出一个运动- 3D 手动作的动作动作动作动作, 以便从最初的 IMF 开始, 将我们基于 IMF 的手序 的手序预测方法, 展示我们从最初的 IMF 方向到最后的 IMF IMF 。</s>