Along with the popularity of Artificial Intelligence (AI) and Internet-of-Things (IoT), Federated Learning (FL) has attracted steadily increasing attentions as a promising distributed machine learning paradigm, which enables the training of a central model on for numerous decentralized devices without exposing their privacy. However, due to the biased data distributions on involved devices, FL inherently suffers from low classification accuracy in non-IID scenarios. Although various device grouping method have been proposed to address this problem, most of them neglect both i) distinct data distribution characteristics of heterogeneous devices, and ii) contributions and hazards of local models, which are extremely important in determining the quality of global model aggregation. In this paper, we present an effective FL method named FedEntropy with a novel dynamic device grouping scheme, which makes full use of the above two factors based on our proposed maximum entropy judgement heuristic.Unlike existing FL methods that directly aggregate local models returned from all the selected devices, in one FL round FedEntropy firstly makes a judgement based on the pre-collected soft labels of selected devices and then only aggregates the local models that can maximize the overall entropy of these soft labels. Without collecting local models that are harmful for aggregation, FedEntropy can effectively improve global model accuracy while reducing the overall communication overhead. Comprehensive experimental results on well-known benchmarks show that, FedEntropy not only outperforms state-of-the-art FL methods in terms of model accuracy and communication overhead, but also can be integrated into them to enhance their classification performance.
翻译:随着人工智能(AI)和互联网电话(IoT)的普及,联邦学习联合会(FL)作为一个有希望的分布式机器学习模式,吸引了越来越多的注意力,作为一个很有希望的分布式机器学习模式,从而能够对许多分散装置的中央模型进行培训,而不会暴露其隐私;然而,由于有关装置的数据分布有偏差,FL在非IID情景中本来就存在分类准确性低的问题。虽然提出了各种设备分组方法来解决这一问题,但大多数都忽视了各种装置的不同数据分布特点,以及地方模型的贡献和危害,而这些模型对于确定全球模型集成的质量极为重要。 在本文件中,我们提出了一个名为FedEntropy的有效FL方法,这个方法充分利用了以上两个因素,其依据是我们拟议的最高温性判断超常。 与现有的FL方法一样,直接综合从所有选定装置返回的当地模型,在FedEntro Pripretopi 中首先根据所选的软性标签进行判断,而随后只有软性成本化的当地模型才能收集。