论文标题
联合进行性稀疏（清除，合并，调子）+
Federated Progressive Sparsification (Purge, Merge, Tune)+
论文作者
论文摘要
为了改善神经网络的联合培训，我们开发了FedSparsify，这是一种基于渐进的体重幅度修剪的稀疏策略。我们的方法有几个好处。首先，由于网络的规模越来越小，因此培训期间的计算和通信成本降低。其次，将模型逐渐限制为较小的参数集，这有助于局部模型的对齐/合并，并以高稀疏率提高学习绩效。第三，最终的稀疏模型明显较小，这提高了推理效率并优化了加密通信期间的操作延迟。我们通过实验表明，FedSparsify学习了高稀疏性和学习绩效的子网。与现有的修剪和非修剪基线相比，我们的稀疏模型可以达到原始模型的十分之一。
To improve federated training of neural networks, we develop FedSparsify, a sparsification strategy based on progressive weight magnitude pruning. Our method has several benefits. First, since the size of the network becomes increasingly smaller, computation and communication costs during training are reduced. Second, the models are incrementally constrained to a smaller set of parameters, which facilitates alignment/merging of the local models and improved learning performance at high sparsification rates. Third, the final sparsified model is significantly smaller, which improves inference efficiency and optimizes operations latency during encrypted communication. We show experimentally that FedSparsify learns a subnetwork of both high sparsity and learning performance. Our sparse models can reach a tenth of the size of the original model with the same or better accuracy compared to existing pruning and nonpruning baselines.
