论文标题
即时修剪：一种无需微调的可回收修剪方法
Pruning On-the-Fly: A Recoverable Pruning Method without Fine-tuning
论文作者
论文摘要
大多数现有的修剪作品都是资源密集型的，需要对修剪模型进行重新调整或微调以保持准确性。我们提出了一种基于超球体学习和损失惩罚条款的无经度修剪方法。提出的损失罚款项将一些模型权重远非零，而其余的权重值则在零接近零，可以安全地修剪，而无需重新训练和准确的准确度下降。此外，我们提出的方法可以通过用其平均值替换修剪值来立即恢复修剪模型的准确性。我们的方法获得了最新的最先进的结果，可导致无重新修剪，并在Imainet-18/50和Mobilenetv2上使用Imagenet数据集进行评估。一个人可以轻松获得50 \％修剪的RESNET18模型，其精度下降了0.47 \％。通过微调，实验结果表明，与现有作品相比，我们的方法可以显着提高修剪模型的准确性。例如，70 \％修剪（除了第一卷积层除外）MobilenetV2模型的精度仅下降3.5 \％，远小于7 \％$ \ sim $ 10 \％10 \％的精度下降，并使用常规方法。
Most existing pruning works are resource-intensive, requiring retraining or fine-tuning of the pruned models for accuracy. We propose a retraining-free pruning method based on hyperspherical learning and loss penalty terms. The proposed loss penalty term pushes some of the model weights far from zero, while the rest weight values are pushed near zero and can be safely pruned with no need for retraining and a negligible accuracy drop. In addition, our proposed method can instantly recover the accuracy of a pruned model by replacing the pruned values with their mean value. Our method obtains state-of-the-art results in retraining-free pruning and is evaluated on ResNet-18/50 and MobileNetV2 with ImageNet dataset. One can easily get a 50\% pruned ResNet18 model with a 0.47\% accuracy drop. With fine-tuning, the experiment results show that our method can significantly boost the accuracy of the pruned models compared with existing works. For example, the accuracy of a 70\% pruned (except the first convolutional layer) MobileNetV2 model only drops 3.5\%, much less than the 7\% $\sim$ 10\% accuracy drop with conventional methods.
