论文标题
自组卷积神经网络
Self-grouping Convolutional Neural Networks
论文作者
论文摘要
尽管小组卷积运算符越来越多地用于深卷积神经网络中，以提高计算效率并减少参数数量，但大多数现有方法通过将每个卷积层的过滤器的预定层分配到多个常规过滤组中，构建其组卷积体系结构，并具有相同的空间组大小和数据独立性，从而预见了其潜力的全部利用，从而产生了全面的利用。为了解决这个问题，我们提出了一种新型的方法，即设计自组的卷积神经网络，称为SG-CNN，其中每个卷积层组本身的过滤器基于其重要性向量的相似性。具体地说，对于每个过滤器，我们首先评估其输入通道的重要性值以识别重要性向量，然后通过群集进行分组。使用所得\ emph {数据依赖性}质心，我们修剪较不重要的连接，该连接不太重要，该连接隐范了修剪的准确性损失，从而产生了一组\ emph {viverse}组卷积过滤器。随后，我们开发了两个微调方案，即（1）本地和全球微调以及（2）全球唯一的微调，它们可以在实验上提供可比较的结果，以恢复修剪过的网络的识别能力。在CIFAR-10/100和Imagenet数据集上进行的全面实验表明，我们的自我组卷积方法适应了各种最新的CNN体系结构，例如ResNet和Densenet，并且在压缩比，加速和识别准确性方面可提供卓越的性能。我们证明了SG-CNN通过转移学习概括的能力，包括域的适应和对象检测，显示了竞争结果。我们的源代码可在https://github.com/qingbeiguo/sg-cnn.git上找到。
Although group convolution operators are increasingly used in deep convolutional neural networks to improve the computational efficiency and to reduce the number of parameters, most existing methods construct their group convolution architectures by a predefined partitioning of the filters of each convolutional layer into multiple regular filter groups with an equal spatial group size and data-independence, which prevents a full exploitation of their potential. To tackle this issue, we propose a novel method of designing self-grouping convolutional neural networks, called SG-CNN, in which the filters of each convolutional layer group themselves based on the similarity of their importance vectors. Concretely, for each filter, we first evaluate the importance value of their input channels to identify the importance vectors, and then group these vectors by clustering. Using the resulting \emph{data-dependent} centroids, we prune the less important connections, which implicitly minimizes the accuracy loss of the pruning, thus yielding a set of \emph{diverse} group convolution filters. Subsequently, we develop two fine-tuning schemes, i.e. (1) both local and global fine-tuning and (2) global only fine-tuning, which experimentally deliver comparable results, to recover the recognition capacity of the pruned network. Comprehensive experiments carried out on the CIFAR-10/100 and ImageNet datasets demonstrate that our self-grouping convolution method adapts to various state-of-the-art CNN architectures, such as ResNet and DenseNet, and delivers superior performance in terms of compression ratio, speedup and recognition accuracy. We demonstrate the ability of SG-CNN to generalise by transfer learning, including domain adaption and object detection, showing competitive results. Our source code is available at https://github.com/QingbeiGuo/SG-CNN.git.
