论文标题
在班级增量学习的动态扩展体系结构中解决任务混乱
Resolving Task Confusion in Dynamic Expansion Architectures for Class Incremental Learning
论文作者
论文摘要
动态扩展体系结构在班级学习中变得越来越流行，这主要是由于它在减轻灾难性遗忘方面的优势。但是，在此框架内无法很好地评估任务混乱，例如，不同任务类之间的差异并没有得到充分学习（即任务间混乱，ITC），并且仍然对最新类别的批处理（即旧的纽约混乱，ONC，ONC）进行了某些优先级。我们从经验上验证了两种混乱的副作用。同时，提出了一种称为任务相关的增量学习（TCIL）的新颖解决方案，以鼓励跨任务的歧视性和公平功能利用。 TCIL执行多层知识蒸馏，以传播从旧任务到新任务的知识。它在功能和logit级别上建立信息流路径，从而使学习能够意识到旧课程。此外，注意力机制和分类器重新评分还用于生成更公平的分类得分。我们在CIFAR100和ImagEnet100数据集上进行了广泛的实验。结果表明，TCIL始终达到最先进的准确性。它可以减轻ITC和ONC，同时在与灾难性忘记的战斗中表现出优势，即使没有排练记忆也没有保留。
The dynamic expansion architecture is becoming popular in class incremental learning, mainly due to its advantages in alleviating catastrophic forgetting. However, task confusion is not well assessed within this framework, e.g., the discrepancy between classes of different tasks is not well learned (i.e., inter-task confusion, ITC), and certain priority is still given to the latest class batch (i.e., old-new confusion, ONC). We empirically validate the side effects of the two types of confusion. Meanwhile, a novel solution called Task Correlated Incremental Learning (TCIL) is proposed to encourage discriminative and fair feature utilization across tasks. TCIL performs a multi-level knowledge distillation to propagate knowledge learned from old tasks to the new one. It establishes information flow paths at both feature and logit levels, enabling the learning to be aware of old classes. Besides, attention mechanism and classifier re-scoring are applied to generate more fair classification scores. We conduct extensive experiments on CIFAR100 and ImageNet100 datasets. The results demonstrate that TCIL consistently achieves state-of-the-art accuracy. It mitigates both ITC and ONC, while showing advantages in battle with catastrophic forgetting even no rehearsal memory is reserved.
