论文标题
零射击实体与有效的远程序列建模链接
Zero-shot Entity Linking with Efficient Long Range Sequence Modeling
论文作者
论文摘要
本文考虑了零射击实体链接的问题，其中测试时间中可能不存在培训中的链接。遵循主要的基于BERT的研究工作，我们发现一种简单而有效的方法是扩展远程序列建模。与以前的许多方法不同，我们的方法不需要较长的位置嵌入BERT昂贵的BERT培训。取而代之的是，我们提出了一种称为“嵌入重复”的有效位置嵌入初始化方法，该方法基于BERT碱基初始化了较大的位置嵌入。在Wikia的零照片EL数据集上，我们的方法将SOTA从76.06％提高到79.08％，对于长期数据，相应的改进从74.57％到82.14％。我们的实验表明，在不重述BERT模型的情况下，远程序列建模的有效性。
This paper considers the problem of zero-shot entity linking, in which a link in the test time may not present in training. Following the prevailing BERT-based research efforts, we find a simple yet effective way is to expand the long-range sequence modeling. Unlike many previous methods, our method does not require expensive pre-training of BERT with long position embedding. Instead, we propose an efficient position embeddings initialization method called Embedding-repeat, which initializes larger position embeddings based on BERT-Base. On Wikia's zero-shot EL dataset, our method improves the SOTA from 76.06% to 79.08%, and for its long data, the corresponding improvement is from 74.57% to 82.14%. Our experiments suggest the effectiveness of long-range sequence modeling without retraining the BERT model.
