Federated Learning aims at training a global model from multiple decentralized devices (i.e. clients) without exchanging their private local data. A key challenge is the handling of non-i.i.d. (independent identically distributed) data across multiple clients that may induce disparities of their local features. We introduce the Hyperspherical Federated Learning (SphereFed) framework to address the non-i.i.d. issue by constraining learned representations of data points to be on a unit hypersphere shared by clients. Specifically, all clients learn their local representations by minimizing the loss with respect to a fixed classifier whose weights span the unit hypersphere. After federated training in improving the global model, this classifier is further calibrated with a closed-form solution by minimizing a mean squared loss. We show that the calibration solution can be computed efficiently and distributedly without direct access of local data. Extensive experiments indicate that our SphereFed approach is able to improve the accuracy of multiple existing federated learning algorithms by a considerable margin (up to 6% on challenging datasets) with enhanced computation and communication efficiency across datasets and model architectures.
翻译:联邦学习联盟(Federal Learning)旨在从多个分散装置(即客户)中培训一个全球模型,而不必交换其私人当地数据。一个关键的挑战是如何处理多个客户之间可能造成本地特征差异的非i.i.d.(单独分布的)数据。我们引入了超球联邦学习框架,以解决非i.i.d.问题,方法是限制在客户共享的单位超视距上显示所学的数据点。具体地说,所有客户都通过尽量减少其重量跨越单位超视距的固定分类器的损失来了解他们在当地的表现。在对改进全球模型进行联合培训后,该分类器通过尽可能减少平均平方损失来进一步校准封闭式解决方案。我们表明,校准解决方案可以在不直接访问本地数据的情况下进行高效的计算和分布。广泛的实验表明,我们的Sphoereferfed 方法能够提高多种现有联合学习算法的准确性,通过相当大的幅度(在具有挑战性的数据设置方面高达6%),通过强化的计算和通信模型结构提高计算和通信效率。