Federated Learning (FL) enables the multiple participating devices to collaboratively contribute to a global neural network model while keeping the training data locally. Unlike the centralized training setting, the non-IID and imbalanced (statistical heterogeneity) training data of FL is distributed in the federated network, which will increase the divergences between the local models and global model and further degrade the performance. In this paper, we propose a novel federated learning framework FedGroup based on a similarity-based clustering strategy, in which we 1) group the training of clients based on the similarities between the clients' optimize directions; 2) reduce the complexity of high-dimension low-sample size (HDLSS) parameter updates data clustering by decomposing the direction vectors to derive the ternary cosine similarity. FedGroup can achieve improvements by dividing joint optimization into groups of sub-optimization, and can be combined with FedProx, the state-of-the-art federated optimization algorithm. We evaluate FedGroup and FedGrouProx (combined with FedProx) on several open datasets. The experimental results show that our proposed frameworks significantly improving absolute test accuracy by +14.7% on FEMNIST compared to FedAvg, +5.4% on Sentiment140 compared to FedProx.
翻译:联邦学习联合会(FL)使多种参与设备能够合作为全球神经网络模式作出贡献,同时在当地保持培训数据。与集中培训环境不同,FL的非二二维和不平衡(统计异质性)培训数据在联邦网络中分布,这将增加地方模型和全球模型之间的差异,进一步降低绩效。在本文中,我们提议基于类似基于集群战略的FedGroup(我们1)将客户培训基于客户优化方向的相似性进行分组。2 降低高二维度低抽样(HDLSS)参数的复杂性,通过分解方向矢量以得出长期共生相似性来更新数据组合。FedGroup(FedGroup)可以通过将联合优化分为次优化组来实现改进,并与FedProx(最新组合组合)和FedGroup(与FedProx结合)组合。我们用一些开放数据基数对FDI+SENA的绝对性测试来大幅改进我们提议的FDI+%的实验结果。