论文标题
内核调制：一种训练卷积神经网络的参数效率方法
Kernel Modulation: A Parameter-Efficient Method for Training Convolutional Neural Networks
论文作者
论文摘要
深度神经网络，尤其是卷积神经网络（Convnets），在许多视觉任务中取得了令人难以置信的成功，但是它们通常需要数百万参数才能获得良好的准确性性能。随着使用Convnet的增加应用程序，在内存，带宽和能量方面，在嵌入式设备上更新数百个网络以进行嵌入式设备上的多个任务可能会很昂贵。降低此成本的方法包括模型压缩和参数有效的模型，可为每个新任务调整网络层的子集。这项工作提出了一种新型的参数有效核调制（KM）方法，该方法适用基本网络的所有参数而不是层的子集。 KM使用轻巧的任务特异性内核调制器，仅需要额外的1.4％的基本网络参数。有了多个任务，只有任务专题化的KM权重传达并存储在最终用户设备上。我们将此方法应用于培训探报中，以转移学习和元学习场景。我们的结果表明，与传输学习基准的其他参数有效方法相比，KM的精度高达9％。
Deep Neural Networks, particularly Convolutional Neural Networks (ConvNets), have achieved incredible success in many vision tasks, but they usually require millions of parameters for good accuracy performance. With increasing applications that use ConvNets, updating hundreds of networks for multiple tasks on an embedded device can be costly in terms of memory, bandwidth, and energy. Approaches to reduce this cost include model compression and parameter-efficient models that adapt a subset of network layers for each new task. This work proposes a novel parameter-efficient kernel modulation (KM) method that adapts all parameters of a base network instead of a subset of layers. KM uses lightweight task-specialized kernel modulators that require only an additional 1.4% of the base network parameters. With multiple tasks, only the task-specialized KM weights are communicated and stored on the end-user device. We applied this method in training ConvNets for Transfer Learning and Meta-Learning scenarios. Our results show that KM delivers up to 9% higher accuracy than other parameter-efficient methods on the Transfer Learning benchmark.
