Federated Learning (FL) deals with learning a central model (i.e. the server) in privacy-constrained scenarios, where data are stored on multiple devices (i.e. the clients). The central model has no direct access to the data, but only to the updates of the parameters computed locally by each client. This raises a problem, known as statistical heterogeneity, because the clients may have different data distributions (i.e. domains). This is only partly alleviated by clustering the clients. Clustering may reduce heterogeneity by identifying the domains, but it deprives each cluster model of the data and supervision of others. Here we propose a novel Cluster-driven Graph Federated Learning (FedCG). In FedCG, clustering serves to address statistical heterogeneity, while Graph Convolutional Networks (GCNs) enable sharing knowledge across them. FedCG: i) identifies the domains via an FL-compliant clustering and instantiates domain-specific modules (residual branches) for each domain; ii) connects the domain-specific modules through a GCN at training to learn the interactions among domains and share knowledge; and iii) learns to cluster unsupervised via teacher-student classifier-training iterations and to address novel unseen test domains via their domain soft-assignment scores. Thanks to the unique interplay of GCN over clusters, FedCG achieves the state-of-the-art on multiple FL benchmarks.
翻译:联邦学习联合会(FL) 处理在受隐私限制的情景中学习中央模型(即服务器)的问题,该模型的数据储存在多种设备(即客户)上。中央模型无法直接访问数据,只能直接访问每个客户在当地计算参数的更新。这产生了一个问题,即统计差异性,因为客户可能拥有不同的数据分布(即域),这仅通过组合客户而部分缓解。组合通过确定域,可能减少不同类型(即服务器),但会剥夺其他数据和监督的每个组群模式。在这里,我们建议采用一个新的由群组驱动的联邦学习联合会(FedCGG) 。在FedCGGG中,分组有助于解决统计差异性,而图表革命网络(GCN)则有助于它们之间分享知识。 FedCGG:i) 通过符合FL组合的组合和即时效软化域模块(分支)确定每个域域域的域域;二) 通过GCN在培训中将特定域模块连接到通过GCN的域域际互动学习,通过G-C的域际学习它们之间的高级学习,并分享知识。