Given a series of natural language descriptions, our task is to generate 3D human motions that correspond semantically to the text, and follow the temporal order of the instructions. In particular, our goal is to enable the synthesis of a series of actions, which we refer to as temporal action composition. The current state of the art in text-conditioned motion synthesis only takes a single action or a single sentence as input. This is partially due to lack of suitable training data containing action sequences, but also due to the computational complexity of their non-autoregressive model formulation, which does not scale well to long sequences. In this work, we address both issues. First, we exploit the recent BABEL motion-text collection, which has a wide range of labeled actions, many of which occur in a sequence with transitions between them. Next, we design a Transformer-based approach that operates non-autoregressively within an action, but autoregressively within the sequence of actions. This hierarchical formulation proves effective in our experiments when compared with multiple baselines. Our approach, called TEACH for "TEmporal Action Compositions for Human motions", produces realistic human motions for a wide variety of actions and temporal compositions from language descriptions. To encourage work on this new task, we make our code available for research purposes at our $\href{teach.is.tue.mpg.de}{\text{website}}$.
翻译:根据一系列自然语言描述,我们的任务是生成3D人类动作,这些动作与文本的音义相对应,并遵循指令的时间顺序。特别是,我们的目标是使一系列行动能够合成,我们称之为时间行动构成。目前,文本调整的动作合成状态只采取单一行动或单句作为输入。这部分是由于缺乏包含行动序列的适当培训数据,但也是由于其非航空模型的编制方法的计算复杂性,这种非航空模型的编制方法不适宜于长顺序。在这个工作中,我们处理这两个问题。首先,我们利用最近的BABEL动议文本收集方法,它有各种各样的标记行动,其中许多是在它们之间的转换顺序上发生的。接下来,我们设计了一个基于变动器的方法,在行动序列内操作不偏向,但在行动序列内自动递增。这种等级化的编制方法在我们的实验中证明是有效的,与多个基线相比,我们的方法是“人类动作的时间动作构成”,我们从人类运动的角度,我们利用BABELL运动的动作-文字收集,在它们之间的交替顺序上可以产生现实的人类动作。我们的任务中,我们设计出一个现实的模型,用于各种动作的模型的模型的模型,我们的工作是用来进行。