论文标题
回到语言的经常性模型
Circling Back to Recurrent Models of Language
论文作者
论文摘要
仅仅因为某些纯粹的经常性模型遭受了很难优化和在当今硬件上效率低下的效率，因此它们不一定是语言的糟糕模型。我们通过稍微更好的复发细胞，体系结构，客观以及优化的组合可以改善这些模型的程度来证明这一点。在此过程中，我们在小型数据集和ENWIK8上建立了一种新的艺术状态，并通过动态评估建立了最新技术。
Just because some purely recurrent models suffer from being hard to optimize and inefficient on today's hardware, they are not necessarily bad models of language. We demonstrate this by the extent to which these models can still be improved by a combination of a slightly better recurrent cell, architecture, objective, as well as optimization. In the process, we establish a new state of the art for language modelling on small datasets and on Enwik8 with dynamic evaluation.
