Federated learning allows clients to collaboratively train models on datasets that are acquired in different locations and that cannot be exchanged because of their size or regulations. Such collected data is increasingly non-independent and non-identically distributed (non-IID), negatively affecting training accuracy. Previous works tried to mitigate the effects of non-IID datasets on training accuracy, focusing mainly on non-IID labels, however practical datasets often also contain non-IID features. To address both non-IID labels and features, we propose FedGMCC, a novel framework where a central server aggregates client models that it can cluster together. FedGMCC clustering relies on a Monte Carlo procedure that samples the output space of client models, infers their position in the weight space on a loss manifold and computes their geometric connection via an affine curve parametrization. FedGMCC aggregates connected models along their path connectivity to produce a richer global model, incorporating knowledge of all connected client models. FedGMCC outperforms FedAvg and FedProx in terms of convergence rates on the EMNIST62 and a genomic sequence classification datasets (by up to +63%). FedGMCC yields an improved accuracy (+4%) on the genomic dataset with respect to CFL, in high non-IID feature space settings and label incongruency.
翻译:联邦学习使客户能够合作培训在不同地点获得的、因其规模或规章而不能交换的数据集模型,这种收集的数据越来越不独立,而且非身份分布(非IID),对培训准确性产生了不利影响。以前的工作试图减轻非IID数据集对培训准确性的影响,主要侧重于非IID标签,但实用数据集往往也包含非IID特征。为了处理非IID标签和特点,我们提议FedGMCC,这是一个新的框架,中央服务器将客户模型集中在一起。联邦GMCC的集群依赖蒙特卡洛程序,该程序将客户模型的产出空间取样到损失的方位上,并用折合曲线的曲线对模型进行几何联系。联邦GMCC将连接的模型汇总起来,以产生一个更丰富的全球模型,吸收所有连接客户模型的知识。FedGMCC将FAvg和FedPROx组合成一个可以组合的客户模型。FDGMIST62和FMGMGMQ4的不精确度数据序列,以CMIC+GMASetroc 的更高比例为标准。