论文标题
抽象不是记忆：伯特和英文文章系统
Abstraction not Memory: BERT and the English Article System
论文作者
论文摘要
文章预测是一项长期以来一直无视精确语言描述的任务。因此，这项任务非常适合评估模型模拟本人说话者直觉的能力。为此，我们将英语英语的人和预先培训的模型的性能与文章预测的任务进行比较，以三道选择（a/an，the，零）。我们对伯特（Bert）的实验表明，伯特（Bert）在所有文章中都超越了人类。特别是，伯特（Bert）在检测零文章时远远优于人类，这可能是因为我们使用深层神经模型可以轻松拾取的规则插入它们。更有趣的是，我们发现，当通道间协议较高时，伯特倾向于与注释者同意，而不是与语料库一致，但是随着通知者协议下降，与语料库的同意更多。我们认为，尽管接受了语料库的培训，但与注释者的这种对齐方式表明，伯特没有记住文章的使用，而是捕获了对文章的高级概括，类似于人类的直觉。
Article prediction is a task that has long defied accurate linguistic description. As such, this task is ideally suited to evaluate models on their ability to emulate native-speaker intuition. To this end, we compare the performance of native English speakers and pre-trained models on the task of article prediction set up as a three way choice (a/an, the, zero). Our experiments with BERT show that BERT outperforms humans on this task across all articles. In particular, BERT is far superior to humans at detecting the zero article, possibly because we insert them using rules that the deep neural model can easily pick up. More interestingly, we find that BERT tends to agree more with annotators than with the corpus when inter-annotator agreement is high but switches to agreeing more with the corpus as inter-annotator agreement drops. We contend that this alignment with annotators, despite being trained on the corpus, suggests that BERT is not memorising article use, but captures a high level generalisation of article use akin to human intuition.
