论文标题
Dropcluster：卷积网络的结构化辍学
DropCluster: A structured dropout for convolutional networks
论文作者
论文摘要
与完全连接的层相比，辍学作为防止过度拟合的常规器在卷积层中的有效性较小。这是因为辍学掉落的特征随机，而无需考虑局部结构。当特征与空间相关时，例如在卷积层的情况下，掉落特征的信息仍然可以通过相邻特征传播到后续层。为了解决这个问题，已经提出了辍学的结构化形式。这些方法的缺点是它们不适合数据。在这项工作中，我们利用卷积层输出中的结构，并引入一种新型的结构化正则化方法，名为Dropcluster。我们的方法簇在卷积层中的特征，并在训练迭代期间随机丢弃所得的簇。关于CIFAR-10/100，SVHN和APPA现实数据集的实验表明，我们的方法是有效的，并且比其他方法更好地拟合。
Dropout as a common regularizer to prevent overfitting in deep neural networks has been less effective in convolutional layers than in fully connected layers. This is because Dropout drops features randomly, without considering local structure. When features are spatially correlated, as in the case of convolutional layers, information from the dropped features can still propagate to subsequent layers via neighboring features. To address this problem, structured forms of Dropout have been proposed. A drawback of these methods is that they do not adapt to the data. In this work, we leverage the structure in the outputs of convolutional layers and introduce a novel structured regularization method named DropCluster. Our approach clusters features in convolutional layers, and drops the resulting clusters randomly during training iterations. Experiments on CIFAR-10/100, SVHN, and APPA-REAL datasets demonstrate that our approach is effective and controls overfitting better than other approaches.
