论文标题
QuickNets：节省培训并防止早期神经体系结构过度自信
QuickNets: Saving Training and Preventing Overconfidence in Early-Exit Neural Architectures
论文作者
论文摘要
深度神经网络有很长的培训和处理时间。添加到神经网络中的早期出口使网络可以在时间敏感应用程序中使用网络中的中间激活进行早期预测。但是，提前出口增加了神经网络的训练时间。我们介绍了QuickNet：一种新颖的级联培训算法，用于更快的神经网络培训。 QuickNets以层次的方式进行训练，以便每个连续的层仅在无法正确分类的样品上训练。我们证明，与标准反向传播相比，QuickNets可以动态分发学习，并降低培训成本和推理成本。此外，我们介绍了承诺层，通过确定过度自信的预测并证明其成功，从而显着改善了早期退出。
Deep neural networks have long training and processing times. Early exits added to neural networks allow the network to make early predictions using intermediate activations in the network in time-sensitive applications. However, early exits increase the training time of the neural networks. We introduce QuickNets: a novel cascaded training algorithm for faster training of neural networks. QuickNets are trained in a layer-wise manner such that each successive layer is only trained on samples that could not be correctly classified by the previous layers. We demonstrate that QuickNets can dynamically distribute learning and have a reduced training cost and inference cost compared to standard Backpropagation. Additionally, we introduce commitment layers that significantly improve the early exits by identifying for over-confident predictions and demonstrate its success.
