Ensemble learning combines results from multiple machine learning models in order to provide a better and optimised predictive model with reduced bias, variance and improved predictions. However, in federated learning it is not feasible to apply centralised ensemble learning directly due to privacy concerns. Hence, a mechanism is required to combine results of local models to produce a global model. Most distributed consensus algorithms, such as Byzantine fault tolerance (BFT), do not normally perform well in such applications. This is because, in such methods predictions of some of the peers are disregarded, so a majority of peers can win without even considering other peers' decisions. Additionally, the confidence score of the result of each peer is not normally taken into account, although it is an important feature to consider for ensemble learning. Moreover, the problem of a tie event is often left un-addressed by methods such as BFT. To fill these research gaps, we propose PoSw (Proof of Swarm), a novel distributed consensus algorithm for ensemble learning in a federated setting, which was inspired by particle swarm based algorithms for solving optimisation problems. The proposed algorithm is theoretically proved to always converge in a relatively small number of steps and has mechanisms to resolve tie events while trying to achieve sub-optimum solutions. We experimentally validated the performance of the proposed algorithm using ECG classification as an example application in healthcare, showing that the ensemble learning model outperformed all local models and even the FL-based global model. To the best of our knowledge, the proposed algorithm is the first attempt to make consensus over the output results of distributed models trained using federated learning.
翻译:合并学习是多种机器学习模型的结果,以便提供更好和最优化的预测模型,减少偏差、差异和改进预测。然而,在联谊学习中,由于隐私问题直接应用集中的全套学习是不可行的。因此,需要一种机制将本地模型的结果结合起来,以产生一个全球模型。大多数分布式的共识算法,如Byzantine过失容忍(BFT),通常在这类应用中表现不好。这是因为,在这种方法中,一些同侪的预测被忽略,因此,大多数同侪可以不考虑其他同侪的决定而赢得最佳的预测模型。此外,每个同侪结果的信任分数通常不会被考虑在内,尽管这是考虑共同学习的一个重要特点。此外,连接事件的问题往往没有被BFT等方法所处理。为了填补这些研究差距,我们建议Posw(Swarm Program) 一种新式的共识算法,用于在联合环境下学习最优的混合计算结果,这种算法甚至由粒子的Swarm 模型所启发, 将所有基于精度的计算方法的精选的精选的精度算方法, 也一直用来模拟地进行模拟的精细分析, 以便比较地研磨地研磨的研磨的研算方法的研磨。