Cross-device Federated Learning is an increasingly popular machine learning setting to train a model by leveraging a large population of client devices with high privacy and security guarantees. However, communication efficiency remains a major bottleneck when scaling federated learning to production environments, particularly due to bandwidth constraints during uplink communication. In this paper, we formalize and address the problem of compressing client-to-server model updates under the Secure Aggregation primitive, a core component of Federated Learning pipelines that allows the server to aggregate the client updates without accessing them individually. In particular, we adapt standard scalar quantization and pruning methods to Secure Aggregation and propose Secure Indexing, a variant of Secure Aggregation that supports quantization for extreme compression. We establish state-of-the-art results on LEAF benchmarks in a secure Federated Learning setup with up to 40$\times$ compression in uplink communication with no meaningful loss in utility compared to uncompressed baselines.
翻译:跨联邦学习是一个日益流行的机器学习环境,通过利用大量具有高度隐私和安全保障的客户设备来培训模型;然而,通信效率在将联合学习推广到生产环境方面仍然是一个重大瓶颈,特别是由于在上行通信过程中带宽的限制;在本文件中,我们正式确定并解决了在安全聚合原始系统下压缩客户对服务器模型更新的问题,这是联邦学习管道的核心组成部分,使服务器能够在不单独访问的情况下汇总客户更新内容。特别是,我们调整标准缩放定量和调整方法,以保障聚合,并提出安全索引,这是安全聚合的变异,支持极端压缩的量化。我们在安全的联邦学习设置中,对LEAF基准设定了最先进的结果,在连接通信中压缩了40美元,与未压缩的基线相比,没有造成任何有意义的效用损失。