We introduce a novel approach for gait transfer from unconstrained videos in-the-wild. In contrast to motion transfer, the objective here is not to imitate the source's motions by the target, but rather to replace the walking source with the target, while transferring the target's typical gait. Our approach can be trained only once with multiple sources and is able to transfer the gait of the target from unseen sources, eliminating the need for retraining for each new source independently. Furthermore, we propose a novel metrics for gait transfer based on gait recognition models that enable to quantify the quality of the transferred gait, and show that existing techniques yield a discrepancy that can be easily detected. We introduce Cycle Transformers GAN (CTrGAN), that consist of a decoder and encoder, both Transformers, where the attention is on the temporal domain between complete images rather than the spatial domain between patches. Using a widely-used gait recognition dataset, we demonstrate that our approach is capable of producing over an order of magnitude more realistic personalized gaits than existing methods, even when used with sources that were not available during training. As part of our solution, we present a detector that determines whether a video is real or generated by our model.
翻译:我们引入了一种新颖的方法,从不受限制的视频中从全天候上进行传输。 与运动传输不同, 我们的目的不是模仿源码的动作, 而是在转移目标的典型动作的同时用目标替换行走源。 我们的方法只能用多种来源来培训一次, 并且能够将目标的步态从无形来源中转移出来, 独立地消除了对每个新来源进行再培训的需要。 此外, 我们提议了一种基于步数识别模型的行数传输的新指标, 以便能够量化转让的格子的质量, 并表明现有的技术会产生很容易检测到的差异。 我们引入了由解码器和编码器组成的循环变换码器 GAN(CTrGAN ), 由解码器和编码器组成, 两者都是变换器, 注意的时间范围是完整图像之间的时间范围, 而不是间隔之间的空间域。 我们使用广泛使用的格数识别数据集, 证明我们的方法能够产生比现有方法更符合现实的大小的个性化阵列, 即使是在培训过程中使用的源中无法使用的。 作为我们所制作到的图像的一部分, 。