论文标题
元门票：在随机初始化的神经网络中寻找最佳子网
Meta-ticket: Finding optimal subnetworks for few-shot learning within randomly initialized neural networks
论文作者
论文摘要
神经网络（NNS）的几乎没有学习是一个重要的问题，旨在培训使用一些数据的NN。主要的挑战是如何避免过度拟合，因为过度参数化的NN可以很容易地将其过度拟合到如此小的数据集中。以前的工作（例如Finn等人2017的MAML）通过元学习来应对这一挑战，该学习如何通过使用各种任务来学习如何从一些数据中学习。另一方面，一种避免过度拟合的常规方法是通过赋予稀疏的NN结构（例如计算机视觉中的卷积层）来限制假设空间。但是，尽管这种手动设计的稀疏结构对于足够大的数据集的样品有效，但它们仍然不足以进行几次学习。然后，自然出现以下问题：（1）我们可以发现通过元学习有效学习有效的稀疏结构吗？ （2）它将在元将军中带来什么好处？在这项工作中，我们提出了一种新型的元学习方法，称为Meta-Ticket，以在随机初始化的NN中找到最佳的稀疏子网，以进行几次学习。我们在经验上证明了元门票成功地发现了可以为每个给定任务学习专门功能的稀疏子网。由于这种任务适应能力，与基于MAML的方法相比，元计算机可以实现优越的元化将军，尤其是大型NNS。该代码可在以下网址找到：https：//github.com/dchiji-ntt/meta-ticket
Few-shot learning for neural networks (NNs) is an important problem that aims to train NNs with a few data. The main challenge is how to avoid overfitting since over-parameterized NNs can easily overfit to such small dataset. Previous work (e.g. MAML by Finn et al. 2017) tackles this challenge by meta-learning, which learns how to learn from a few data by using various tasks. On the other hand, one conventional approach to avoid overfitting is restricting hypothesis spaces by endowing sparse NN structures like convolution layers in computer vision. However, although such manually-designed sparse structures are sample-efficient for sufficiently large datasets, they are still insufficient for few-shot learning. Then the following questions naturally arise: (1) Can we find sparse structures effective for few-shot learning by meta-learning? (2) What benefits will it bring in terms of meta-generalization? In this work, we propose a novel meta-learning approach, called Meta-ticket, to find optimal sparse subnetworks for few-shot learning within randomly initialized NNs. We empirically validated that Meta-ticket successfully discover sparse subnetworks that can learn specialized features for each given task. Due to this task-wise adaptation ability, Meta-ticket achieves superior meta-generalization compared to MAML-based methods especially with large NNs. The code is available at: https://github.com/dchiji-ntt/meta-ticket
