论文标题
深度学习中计算语言学的不可阻挡的兴起
The Unstoppable Rise of Computational Linguistics in Deep Learning
论文作者
论文摘要
在本文中，我们追踪了应用于自然语言理解任务的神经网络的历史，并确定语言性质对神经网络体系结构的发展所做的关键贡献。我们专注于可变结合及其在基于注意力的模型中的实例化的重要性，并认为变压器不是序列模型，而是诱导结构模型。这种观点可以预测对自然语言理解深度学习体系结构中研究所面临的挑战。
In this paper, we trace the history of neural networks applied to natural language understanding tasks, and identify key contributions which the nature of language has made to the development of neural network architectures. We focus on the importance of variable binding and its instantiation in attention-based models, and argue that Transformer is not a sequence model but an induced-structure model. This perspective leads to predictions of the challenges facing research in deep learning architectures for natural language understanding.
