Generative adversarial networks have proven to be a powerful tool for learning complex and high-dimensional data distributions, but issues such as mode collapse have been shown to make it difficult to train them. This is an even harder problem when the data is decentralized over several clients in a federated learning setup, as problems such as client drift and non-iid data make it hard for federated averaging to converge. In this work, we study the task of how to learn a data distribution when training data is heterogeneously decentralized over clients and cannot be shared. Our goal is to sample from this distribution centrally, while the data never leaves the clients. We show using standard benchmark image datasets that existing approaches fail in this setting, experiencing so-called client drift when the local number of epochs becomes to large. We thus propose a novel approach we call EFFGAN: Ensembles of fine-tuned federated GANs. Being an ensemble of local expert generators, EFFGAN is able to learn the data distribution over all clients and mitigate client drift. It is able to train with a large number of local epochs, making it more communication efficient than previous works.
翻译:生成的对抗性网络已证明是学习复杂和高维数据分布的有力工具,但模式崩溃等问题表明难以对其进行培训。当数据在一个联合学习结构中分散到几个客户时,这是一个更困难的问题,因为客户流和非二面数据等问题使得联盟平均数据难以趋同。在这项工作中,我们研究如何在培训数据分散到客户和无法共享时学习数据分布。我们的目标是从这种分布中采集数据样本,而数据从未离开客户。我们用标准基准图像数据集显示,在这种环境下,现有方法失败,当当地不同时代的数量变得庞大时,经历所谓的客户漂移。因此我们建议一种新颖的方法,我们称之为 EFFGAN: 精细调整的federerect GANs 组合。 EFFGAN 是一个本地专家发电机的组合, 能够学习所有客户的数据分布, 并减少客户的漂流。我们能够用大量本地用户进行训练, 使其通信效率高于先前的工作。