论文标题
关于深度学习与系统概括之间的内置冲突
On a Built-in Conflict between Deep Learning and Systematic Generalization
论文作者
论文摘要
在本文中，我们假设内部功能共享是削弱O.O.D.的原因之一。或对分类任务进行深度学习的系统概括。在同等预测下，模型将输入空间分配为由边界隔开的多个部分。共享的功能更喜欢重复使用边界，从而导致新输出的零件较少，这与系统的概括冲突。我们在标准深度学习模型中显示了这种现象，例如完全连接，卷积，残留网络，LSTM和（视觉）变压器。我们希望这项研究能够为系统概括提供新的见解，并构成新的研究方向的基础。
In this paper, we hypothesize that internal function sharing is one of the reasons to weaken o.o.d. or systematic generalization in deep learning for classification tasks. Under equivalent prediction, a model partitions an input space into multiple parts separated by boundaries. The function sharing prefers to reuse boundaries, leading to fewer parts for new outputs, which conflicts with systematic generalization. We show such phenomena in standard deep learning models, such as fully connected, convolutional, residual networks, LSTMs, and (Vision) Transformers. We hope this study provides novel insights into systematic generalization and forms a basis for new research directions.
