We introduce CharacterGAN, a generative model that can be trained on only a few samples (8 - 15) of a given character. Our model generates novel poses based on keypoint locations, which can be modified in real time while providing interactive feedback, allowing for intuitive reposing and animation. Since we only have very limited training samples, one of the key challenges lies in how to address (dis)occlusions, e.g. when a hand moves behind or in front of a body. To address this, we introduce a novel layering approach which explicitly splits the input keypoints into different layers which are processed independently. These layers represent different parts of the character and provide a strong implicit bias that helps to obtain realistic results even with strong (dis)occlusions. To combine the features of individual layers we use an adaptive scaling approach conditioned on all keypoints. Finally, we introduce a mask connectivity constraint to reduce distortion artifacts that occur with extreme out-of-distribution poses at test time. We show that our approach outperforms recent baselines and creates realistic animations for diverse characters. We also show that our model can handle discrete state changes, for example a profile facing left or right, that the different layers do indeed learn features specific for the respective keypoints in those layers, and that our model scales to larger datasets when more data is available.
翻译:我们引入了“ 字符GAN ”, 这是一种只针对某个特性的几个样本( 8 - 15 ) 的基因模型。 我们的模型产生基于关键点位置的新面貌, 可以实时修改, 同时提供互动反馈, 允许直观的重新定位和动画。 由于我们只有非常有限的培训样本, 关键的挑战之一是如何解决( 分化), 例如当一个手在某个机构后面或前面移动时。 为了解决这个问题, 我们引入了一种新的分层方法, 将输入关键点明确分成不同层次, 独立处理。 这些层次代表着该特性的不同部分, 并且提供了强烈的隐含偏差, 帮助获得现实的结果, 即使是强( 分解) 的分解和动。 为了将各个层次的特性结合起来, 我们使用适应性缩放方法, 取决于所有关键点。 最后, 我们引入了一种隐蔽的连接限制, 以减少在测试时以极端分流的配置方式出现的扭曲性作品。 为了解决这个问题, 我们展示我们的方法超越了最近的基线, 并为不同的字符创造现实的动画。 我们还显示我们的模型可以处理离层的特性, 在不同的层次上, 不同的层次上, 不同的层次上, 能够学习更具体的层次 不同的数据, 不同的层次 。