论文标题
持续学习中知识转移的理论
A Theory for Knowledge Transfer in Continual Learning
论文作者
论文摘要
持续学习一系列任务是深层神经网络中的活跃领域。研究的主要挑战是灾难性遗忘或干扰新获得的知识的现象，从以前的任务中的知识。最近的工作调查了向新任务的远期知识转移。向后转移以改善以前的任务中获得的知识的关注要少得多。通常，人们对知识转移如何有助于不断学习的任务有限。我们提出了一个持续监督学习中知识转移的理论，该理论考虑了前进和向后转移。我们旨在了解它们对越来越多知识的学习者的影响。我们得出这些转移机制的每一种。这些界限对特定实现（例如深神经网络）不可知。我们证明，对于观察相关任务的持续学习者而言，前进和向后转移都可以随着观察到更多的任务而提高性能。
Continual learning of a stream of tasks is an active area in deep neural networks. The main challenge investigated has been the phenomenon of catastrophic forgetting or interference of newly acquired knowledge with knowledge from previous tasks. Recent work has investigated forward knowledge transfer to new tasks. Backward transfer for improving knowledge gained during previous tasks has received much less attention. There is in general limited understanding of how knowledge transfer could aid tasks learned continually. We present a theory for knowledge transfer in continual supervised learning, which considers both forward and backward transfer. We aim at understanding their impact for increasingly knowledgeable learners. We derive error bounds for each of these transfer mechanisms. These bounds are agnostic to specific implementations (e.g. deep neural networks). We demonstrate that, for a continual learner that observes related tasks, both forward and backward transfer can contribute to an increasing performance as more tasks are observed.
