论文标题
步枪：通过重新定位完全连接层的深度转移学习深度传播
RIFLE: Backpropagation in Depth for Deep Transfer Learning through Re-Initializing the Fully-connected LayEr
论文作者
论文摘要
使用预训练的模型对深度卷积神经网络（CNN）进行微调有助于将知识从较大的数据集传递到目标任务。尽管即使训练数据集很小，也可以在很大程度上提高准确性，但转移学习结果通常受到CNN重量紧密的预训练模型的限制（Liu等，2019），因为这里的反向传播会给更深的CNN层带来较小的更新。在这项工作中，我们提出了步枪 - 一种简单而有效的策略，通过在微调程序中定期重新定位完全连接的层来加深转移学习环境中的反向传播。步枪为深CNN层的重量带来了有意义的更新，并改善了低级功能学习，而随机化的效果可以在整个整体学习过程中很容易收敛。实验表明，步枪的使用显着提高了广泛的数据集上的深层传递学习准确性，在类似目的的情况下，在同一设置中，在同一设置下，在同一设置下，辍学，dropconnect，dropconnect，stochasticdeppth，dropconnect，stochasticdeppth，drowsconnect，intchasticdeppth，drowsconnect，intchasticDeptth，dropconnect，stochasticdepth和循环学习率都具有0.5％-2％的测试准确性。经验案例和消融研究进一步表明步枪为深度CNN层带来了有意义的更新，并改善了精度。
Fine-tuning the deep convolution neural network(CNN) using a pre-trained model helps transfer knowledge learned from larger datasets to the target task. While the accuracy could be largely improved even when the training dataset is small, the transfer learning outcome is usually constrained by the pre-trained model with close CNN weights (Liu et al., 2019), as the backpropagation here brings smaller updates to deeper CNN layers. In this work, we propose RIFLE - a simple yet effective strategy that deepens backpropagation in transfer learning settings, through periodically Re-Initializing the Fully-connected LayEr with random scratch during the fine-tuning procedure. RIFLE brings meaningful updates to the weights of deep CNN layers and improves low-level feature learning, while the effects of randomization can be easily converged throughout the overall learning procedure. The experiments show that the use of RIFLE significantly improves deep transfer learning accuracy on a wide range of datasets, out-performing known tricks for the similar purpose, such as Dropout, DropConnect, StochasticDepth, Disturb Label and Cyclic Learning Rate, under the same settings with 0.5% -2% higher testing accuracy. Empirical cases and ablation studies further indicate RIFLE brings meaningful updates to deep CNN layers with accuracy improved.
