论文标题
使用结构化的稀疏卷积来推动效率极限
Pushing the Efficiency Limit Using Structured Sparse Convolutions
论文作者
论文摘要
减轻修剪是压缩深层卷积神经网络的最受欢迎的方法之一。最近的工作表明，在随机初始化的深神经网络中，存在稀疏的子网，可实现与原始网络相当的性能。不幸的是，找到这些子网涉及训练和修剪的迭代阶段，这在计算上可能很昂贵。我们提出了结构化的稀疏卷积（SSC），该卷积（SSC）利用图像中的固有结构来减少卷积过滤器中的参数。与在初始化时执行修剪的现有方法相比，这导致卷积架构的效率提高。我们表明，SSC是``有效的体系结构''中常用层（深度，群集和点卷积）的概括。对众所周知的CNN模型和数据集进行了广泛的实验，显示了该方法的有效性。与CIFAR-10，CIFAR-100，Tiny-Imagenet和Imagenet分类基准的基线相比，基于SSC的体系结构实现了最先进的性能。
Weight pruning is among the most popular approaches for compressing deep convolutional neural networks. Recent work suggests that in a randomly initialized deep neural network, there exist sparse subnetworks that achieve performance comparable to the original network. Unfortunately, finding these subnetworks involves iterative stages of training and pruning, which can be computationally expensive. We propose Structured Sparse Convolution (SSC), which leverages the inherent structure in images to reduce the parameters in the convolutional filter. This leads to improved efficiency of convolutional architectures compared to existing methods that perform pruning at initialization. We show that SSC is a generalization of commonly used layers (depthwise, groupwise and pointwise convolution) in ``efficient architectures.'' Extensive experiments on well-known CNN models and datasets show the effectiveness of the proposed method. Architectures based on SSC achieve state-of-the-art performance compared to baselines on CIFAR-10, CIFAR-100, Tiny-ImageNet, and ImageNet classification benchmarks.
