Client selection strategies are widely adopted to handle the communication-efficient problem in recent studies of Federated Learning (FL). However, due to the large variance of the selected subset's update, prior selection approaches with a limited sampling ratio cannot perform well on convergence and accuracy in heterogeneous FL. To address this problem, in this paper, we propose a novel stratified client selection scheme to reduce the variance for the pursuit of better convergence and higher accuracy. Specifically, to mitigate the impact of heterogeneity, we develop stratification based on clients' local data distribution to derive approximate homogeneous strata for better selection in each stratum. Concentrating on a limited sampling ratio scenario, we next present an optimized sample size allocation scheme by considering the diversity of stratum's variability, with the promise of further variance reduction. Theoretically, we elaborate the explicit relation among different selection schemes with regard to variance, under heterogeneous settings, we demonstrate the effectiveness of our selection scheme. Experimental results confirm that our approach not only allows for better performance relative to state-of-the-art methods but also is compatible with prevalent FL algorithms.
翻译:最近对联邦学习联合会(FL)的研究广泛采用客户选择战略,处理沟通效率问题,不过,由于选定子集更新情况差异很大,抽样比例有限的事先选择方法在异质FL的趋同性和准确性方面不能产生良好效果。 为了解决这个问题,我们在本文件中提出一个新的分层客户选择方案,以减少差异,以寻求更好的趋同和更高的准确性。具体地说,为了减轻异质性的影响,我们根据客户的当地数据分布发展分层,以得出大致相同的层次,以便在每个区进行更好的选择。我们以有限的抽样比率方案为中心,接下来提出一个优化的抽样规模分配方案,方法是考虑分层变异性的多样性,并有可能进一步减少差异。理论上,我们详细阐述不同选择方案之间在差异方面的明确关系,在不同的环境下,我们展示我们的选择方案的有效性。实验结果证实,我们的方法不仅允许与最先进的方法相比,而且与流行的FL算法相容。