Modern generative models achieve excellent quality in a variety of tasks including image or text generation and chemical molecule modeling. However, existing methods often lack the essential ability to generate examples with requested properties, such as the age of the person in the photo or the weight of the generated molecule. Incorporating such additional conditioning factors would require rebuilding the entire architecture and optimizing the parameters from scratch. Moreover, it is difficult to disentangle selected attributes so that to perform edits of only one attribute while leaving the others unchanged. To overcome these limitations we propose PluGeN (Plugin Generative Network), a simple yet effective generative technique that can be used as a plugin to pre-trained generative models. The idea behind our approach is to transform the entangled latent representation using a flow-based module into a multi-dimensional space where the values of each attribute are modeled as an independent one-dimensional distribution. In consequence, PluGeN can generate new samples with desired attributes as well as manipulate labeled attributes of existing examples. Due to the disentangling of the latent representation, we are even able to generate samples with rare or unseen combinations of attributes in the dataset, such as a young person with gray hair, men with make-up, or women with beards. We combined PluGeN with GAN and VAE models and applied it to conditional generation and manipulation of images and chemical molecule modeling. Experiments demonstrate that PluGeN preserves the quality of backbone models while adding the ability to control the values of labeled attributes.
翻译:现代基因模型在包括图像或文本生成和化学分子模型在内的各种任务中达到极佳质量。 但是,现有的方法往往缺乏以所要求的特性生成示例的基本能力,例如照片中的人的年龄或所生成分子的重量。 纳入这些额外的调制因素需要重建整个结构,并从零开始优化参数。 此外,很难分解选定的属性,以便只对一个属性进行编辑,而使其他属性保持不变。 为了克服这些局限性,我们提议了PluGeN(插头生成网络),这是一种简单而有效的基因化技术,可以用作预先训练的基因化模型的插件。我们的方法背后的想法是利用一个基于流的模块将纠缠绕的潜在代表转换成一个多维空间,使每个属性的值建成独立的一维分布模型。因此, PluGeN可以生成带有理想属性的新样本,并操纵现有示例的模型。由于潜在代表的脱色,我们甚至能够用罕见或不可见的PI值组合模型制作样本,同时用A型模型和G型号模型来显示女性的灰度。