论文标题
学习用于无任务持续学习的进化混合模型
Learning an evolved mixture model for task-free continual learning
论文作者
论文摘要
最近，持续学习（CL）引起了巨大的兴趣，因为它使深度学习模型能够获取新知识，而无需忘记以前学习的信息。但是，大多数现有作品都需要知道任务身份和边界，这在实际情况下是不现实的。在本文中，我们在CL中解决了一个更具挑战性和更现实的环境，即无任务的持续学习（TFCL），其中模型在没有明确任务信息的非平稳数据流上培训。为了解决TFCL，我们介绍了一个进化的混合模型，该模型的网络体系结构动态扩展以适应数据分布变化。我们通过评估使用Hilbert Schmidt独立标准（HSIC）来评估每个混合模型组件中存储的知识与当前存储器缓冲区的知识之间的概率距离来实现此扩展机制。我们进一步介绍了两种简单的辍学机制，以选择性地删除存储的示例，以避免存储器超负荷，同时保持内存多样性。经验结果表明，所提出的方法实现了出色的性能。
Recently, continual learning (CL) has gained significant interest because it enables deep learning models to acquire new knowledge without forgetting previously learnt information. However, most existing works require knowing the task identities and boundaries, which is not realistic in a real context. In this paper, we address a more challenging and realistic setting in CL, namely the Task-Free Continual Learning (TFCL) in which a model is trained on non-stationary data streams with no explicit task information. To address TFCL, we introduce an evolved mixture model whose network architecture is dynamically expanded to adapt to the data distribution shift. We implement this expansion mechanism by evaluating the probability distance between the knowledge stored in each mixture model component and the current memory buffer using the Hilbert Schmidt Independence Criterion (HSIC). We further introduce two simple dropout mechanisms to selectively remove stored examples in order to avoid memory overload while preserving memory diversity. Empirical results demonstrate that the proposed approach achieves excellent performance.
