论文标题
Visual Transformer符合CutMix，以提高精确性，沟通效率和分裂学习中的数据隐私
Visual Transformer Meets CutMix for Improved Accuracy, Communication Efficiency, and Data Privacy in Split Learning
论文作者
论文摘要
本文为视觉变压器（VIT）体系结构提供了分布式学习解决方案。与卷积神经网络（CNN）架构相比，VIT通常具有较大的模型尺寸，并且计算上的昂贵，从而使联合学习（FL）不适合使用。拆分学习（SL）可以通过拆分模型并在拆分层上传达隐藏的表示形式（也称为粉碎的数据）来避开此问题。尽管如此，VIT的粉碎数据与输入数据一样大，在违反数据隐私时否定了SL的通信效率。为了解决这些问题，我们通过随机打孔和压缩原始粉碎的数据来提出一种新形式的切割数据。利用这一点，我们为VIT，CUTMIXSL开发了一个新颖的SL框架，并传达了切割的数据。 cutmixsl不仅降低了通信成本和隐私泄漏，而且固有地涉及cutmix数据的增强，从而提高了准确性和可扩展性。模拟证实了cutmixsl的表现优于平行的SL等基线，并拆分将FL与SL集成在一起。
This article seeks for a distributed learning solution for the visual transformer (ViT) architectures. Compared to convolutional neural network (CNN) architectures, ViTs often have larger model sizes, and are computationally expensive, making federated learning (FL) ill-suited. Split learning (SL) can detour this problem by splitting a model and communicating the hidden representations at the split-layer, also known as smashed data. Notwithstanding, the smashed data of ViT are as large as and as similar as the input data, negating the communication efficiency of SL while violating data privacy. To resolve these issues, we propose a new form of CutSmashed data by randomly punching and compressing the original smashed data. Leveraging this, we develop a novel SL framework for ViT, coined CutMixSL, communicating CutSmashed data. CutMixSL not only reduces communication costs and privacy leakage, but also inherently involves the CutMix data augmentation, improving accuracy and scalability. Simulations corroborate that CutMixSL outperforms baselines such as parallelized SL and SplitFed that integrates FL with SL.
