Federated learning (FL) is an emerging promising paradigm of privacy-preserving machine learning (ML). An important type of FL is cross-silo FL, which enables a small scale of organizations to cooperatively train a shared model by keeping confidential data locally and aggregating weights on a central parameter server. However, the central server may be vulnerable to malicious attacks or software failures in practice. To address this issue, in this paper, we propose DeFL, a novel decentralized weight aggregation framework for cross-silo FL. DeFL eliminates the central server by aggregating weights on each participating node and weights of only the current training round are maintained and synchronized among all nodes. We use Multi-Krum to enable aggregating correct weights from honest nodes and use HotStuff to ensure the consistency of the training round number and weights among all nodes. Besides, we theoretically analyze the Byzantine fault tolerance, convergence, and complexity of DeFL. We conduct extensive experiments over two widely-adopted public datasets, i.e. CIFAR-10 and Sentiment140, to evaluate the performance of DeFL. Results show that DeFL defends against common threat models with minimal accuracy loss, and achieves up to 100x reduction in storage overhead and up to 12x reduction in network overhead, compared to state-of-the-art decentralized FL approaches.
翻译:联邦学习(FL)是保护隐私机器学习(ML)的一个新兴有希望的新模式。一个重要的FL是跨SIlo FL,它使小规模组织能够通过在当地保留保密数据并在中央参数服务器上汇总权重,合作培训一个共享模式;然而,中央服务器可能易受恶意攻击或软件实际失灵的影响。为了解决这一问题,我们在本文件中提议DFL为跨silo FL(ML)提供一个新的分散权重汇总框架。 DeFL通过将每个参与节点的权重和当前一轮培训的权重汇总起来,在所有节点之间保持和同步。我们使用多Krum来使诚实节点的权重汇总,并使用热力来确保所有节点之间培训轮数和权重的一致性。此外,我们从理论上分析Byzantine过失容忍度、趋同性和复杂性。我们对两个广泛采用的公共数据集进行了广泛的实验,即CIFAR-10和Sentiment140,以评价DeFL的运行情况。结果显示,DeFL-FL在降低普通的间接费用方面,在降低第12号主机压模型方面,在降低第12号。