Mixed reality applications require tracking the user's full-body motion to enable an immersive experience. However, typical head-mounted devices can only track head and hand movements, leading to a limited reconstruction of full-body motion due to variability in lower body configurations. We propose BoDiffusion -- a generative diffusion model for motion synthesis to tackle this under-constrained reconstruction problem. We present a time and space conditioning scheme that allows BoDiffusion to leverage sparse tracking inputs while generating smooth and realistic full-body motion sequences. To the best of our knowledge, this is the first approach that uses the reverse diffusion process to model full-body tracking as a conditional sequence generation task. We conduct experiments on the large-scale motion-capture dataset AMASS and show that our approach outperforms the state-of-the-art approaches by a significant margin in terms of full-body motion realism and joint reconstruction error.
翻译:BoDiffusion:利用扩散方法合成全身人体运动的稀疏观测
混合现实应用需要追踪用户的全身运动,以实现沉浸式体验。但是,典型的头戴式设备只能跟踪头部和手部动作,由于较低身体位置配置的可变性,导致无法完整重建全身运动。我们提出了 BoDiffusion –- 一种用于运动合成的生成扩散模型来解决这个欠约束的重建问题。我们提供了一种时间和空间调节方案,使 BoDiffusion 可以利用稀疏跟踪输入,同时生成平滑而逼真的全身运动序列。据我们所知,这是第一种使用反向扩散过程将全身跟踪建模为条件序列生成任务的方法。我们在大规模动作捕捉数据集 AMASS 上进行实验证明,相对于现有最先进的方法,我们的方法在全身运动逼真度和关节重建误差方面取得了显着的优势。