Generative models for audio-conditioned dance motion synthesis map music features to dance movements. Models are trained to associate motion patterns to audio patterns, usually without an explicit knowledge of the human body. This approach relies on a few assumptions: strong music-dance correlation, controlled motion data and relatively simple poses and movements. These characteristics are found in all existing datasets for dance motion synthesis, and indeed recent methods can achieve good results.We introduce a new dataset aiming to challenge these common assumptions, compiling a set of dynamic dance sequences displaying complex human poses. We focus on breakdancing which features acrobatic moves and tangled postures. We source our data from the Red Bull BC One competition videos. Estimating human keypoints from these videos is difficult due to the complexity of the dance, as well as the multiple moving cameras recording setup. We adopt a hybrid labelling pipeline leveraging deep estimation models as well as manual annotations to obtain good quality keypoint sequences at a reduced cost. Our efforts produced the BRACE dataset, which contains over 3 hours and 30 minutes of densely annotated poses. We test state-of-the-art methods on BRACE, showing their limitations when evaluated on complex sequences. Our dataset can readily foster advance in dance motion synthesis. With intricate poses and swift movements, models are forced to go beyond learning a mapping between modalities and reason more effectively about body structure and movements.
翻译:用于舞蹈运动合成和舞蹈运动的有声调的舞蹈动画综合地图音乐特征的生成模型。 模型经过培训, 将运动模式与音态模式联系起来, 通常没有对人类身体的明确了解。 这种方法依赖于几个假设: 音乐- 音乐- 舞蹈的紧密关联、 受控运动数据以及相对简单的外形和运动。 这些特征可见于舞蹈动作合成的所有现有数据集中, 以及最近的方法可以取得良好结果。 我们引入了一个新的数据集, 旨在挑战这些共同假设, 汇编一套显示复杂人姿势的动态舞蹈序列。 我们侧重于以具有精密的手动和缠绕姿势的断裂式舞蹈模式。 我们从红牛BC 1 竞赛视频中获取数据。 这些视频中的人类关键点很难估算,因为舞蹈的复杂性, 以及多个移动摄像机的设置。 我们采用了混合标签管道, 利用深度估算模型和手动说明, 以降低成本获得高质量的关键点序列。 我们的努力产生了BRACE数据集, 包含3小时和30分钟的密集姿势。 我们测试了“ ”状态- 和“ 快速的舞蹈运动结构”,, 能够有效地显示“ 快速的“ ” 快速的“ ” 结构” 结构”, 以及“, 快速的“, 和“快速” 结构”,,, 展示“快速” 结构” 以及“动态”,,,,,, 以及“快速的“动态” 和“动态结构”, 以及“快速” 结构” 结构”,,,,,,,,, 能够展示着“快速的“快速的“快速的“动态” 和“动态”, ”,, ”, 和“动态”, 结构” ” ”,,, 和“动态”, 结构”,,,,, 和“快速”, 和“,, 和“ 结构”,,, 和“,,,, 和“ 和“ ” ” ” ” ”,,,, 和“动态”