视觉变异器与提高精密度、通信效率和数据隐私的分离学习的CutMix相匹配 (Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learning)

from arxiv, won the Best Student Paper Award at International Workshop on Trustworthy Federated Learning in Conjunction with IJCAI 2022 (FL-IJCAI'22), Vienna, Austria

This article seeks for a distributed learning solution for the visual transformer (ViT) architectures. Compared to convolutional neural network (CNN) architectures, ViTs often have larger model sizes, and are computationally expensive, making federated learning (FL) ill-suited. Split learning (SL) can detour this problem by splitting a model and communicating the hidden representations at the split-layer, also known as smashed data. Notwithstanding, the smashed data of ViT are as large as and as similar as the input data, negating the communication efficiency of SL while violating data privacy. To resolve these issues, we propose a new form of CutSmashed data by randomly punching and compressing the original smashed data. Leveraging this, we develop a novel SL framework for ViT, coined CutMixSL, communicating CutSmashed data. CutMixSL not only reduces communication costs and privacy leakage, but also inherently involves the CutMix data augmentation, improving accuracy and scalability. Simulations corroborate that CutMixSL outperforms baselines such as parallelized SL and SplitFed that integrates FL with SL.

翻译：文章寻求视觉变压器( VIT) 结构的分布式学习解决方案。与进化神经网络( CNN) 结构相比, Vits 通常具有较大的模型大小, 并且计算成本很高, 使得联合学习( FL) 不合适。拆分学习( SL) 可以通过拆分模型和传递拆解层的隐藏表达形式( 也称为粉碎数据 ) 来回避这一问题。尽管如此, Vit 的碎碎版数据与输入数据一样大, 并类似, 否定 SL 的通信效率, 同时侵犯数据隐私。为了解决这些问题, 我们建议了一种新型的剪切成数据, 通过随机拳击和压缩原始折叠数据。利用此选项, 我们为 VIT 开发了一个新的 SL 框架, 硬化的 CutMixSL SixSL, 传递剪切数据。剪切MixSL 不仅降低通信成本和隐私泄漏, 也必然涉及 CutMix 数据增强、精度和可缩缩缩缩。。。模拟校正校正校正校正校正校正校正校正校正校正校正校正校正校正校对 Cut MSL 和校正和校正校正校正校正校正校正校正校正校正校正校正校正校正校正校正校正校正校正校正校正校正校正校对校对校对校对校对校对校对。