Vertical federated learning (VFL) is an emerging paradigm that allows different parties (e.g., organizations or enterprises) to collaboratively build machine learning models with privacy protection. In the training phase, VFL only exchanges the intermediate statistics, i.e., forward activations and backward derivatives, across parties to compute model gradients. Nevertheless, due to its geo-distributed nature, VFL training usually suffers from the low WAN bandwidth. In this paper, we introduce CELU-VFL, a novel and efficient VFL training framework that exploits the local update technique to reduce the cross-party communication rounds. CELU-VFL caches the stale statistics and reuses them to estimate model gradients without exchanging the ad hoc statistics. Significant techniques are proposed to improve the convergence performance. First, to handle the stochastic variance problem, we propose a uniform sampling strategy to fairly choose the stale statistics for local updates. Second, to harness the errors brought by the staleness, we devise an instance weighting mechanism that measures the reliability of the estimated gradients. Theoretical analysis proves that CELU-VFL achieves a similar sub-linear convergence rate as vanilla VFL training but requires much fewer communication rounds. Empirical results on both public and real-world workloads validate that CELU-VFL can be up to six times faster than the existing works.
翻译:纵向联盟学习(VFL)是一个新兴范例,使不同当事方(例如组织或企业)能够合作建立具有隐私保护的机器学习模式。在培训阶段,VFL只交换中间统计数据,即前置激活和后向衍生衍生物,供各方计算模型梯度。然而,由于其地理分布性质,VFL培训通常受到低广域网带宽的限制。在本文件中,我们引入了CELU-VFL,这是一个新颖而有效的VFL培训框架,它利用当地更新技术减少跨党派交流回合。CELU-VFL隐藏了滚动统计数据,再利用它们来估计模型梯度,而无需交换临时统计数据。提出了改进趋同性业绩的重要技术。首先,为了处理随机差异问题,我们提出了一种统一的抽样战略,以公平选择本地更新所用的标准统计数据。第二,为了利用较快的粘误,我们设计了一个实例加权机制,以测量估计的梯度的可靠性。CELU-VFL值统计需要比V-FLV的快速的进度,但比V-FL的进度要低得多的进度,因为CL-FL-FL的进度要求在真实的进度上取得比V-FLV-FLV-FLV-C-FLV的进度的进度的进度的进度,而要低的进度的进度是比V-FLV-FLV-FL-C-FLV-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-FL-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-C-