论文标题
重新思考神经模型的概括：指定的实体识别案例研究
Rethinking Generalization of Neural Models: A Named Entity Recognition Case Study
论文作者
论文摘要
尽管基于神经网络的模型在大量的NLP任务上取得了令人印象深刻的性能，但不同模型的概括行为仍然很熟悉：这种出色的性能是否暗示着一个完美的概括模型，还是仍然存在一些局限性？在本文中，我们将NER任务作为测试床，从不同的角度分析现有模型的概括行为，并通过我们提出的措施的镜头来表征其概括能力的差异，这使我们指导我们更好地设计模型和培训方法。深入分析的实验在分解性能分析，注释错误，数据集偏见和类别关系方面诊断了现有神经NER模型的瓶颈，这暗示了改进的方向。我们已经发布了数据集：（reconlll，ploner）在我们的项目页面上进行未来研究：http：//pfliu.com/interpretner/。作为本文的副产品，我们开源了一个项目，该项目涉及最近的NER论文的全面摘要，并将其分类为不同的研究主题：https：//github.com/pfliu-nlp/named-entity-entity-recognition-nementition-recognition-nerpapers。
While neural network-based models have achieved impressive performance on a large body of NLP tasks, the generalization behavior of different models remains poorly understood: Does this excellent performance imply a perfect generalization model, or are there still some limitations? In this paper, we take the NER task as a testbed to analyze the generalization behavior of existing models from different perspectives and characterize the differences of their generalization abilities through the lens of our proposed measures, which guides us to better design models and training methods. Experiments with in-depth analyses diagnose the bottleneck of existing neural NER models in terms of breakdown performance analysis, annotation errors, dataset bias, and category relationships, which suggest directions for improvement. We have released the datasets: (ReCoNLL, PLONER) for the future research at our project page: http://pfliu.com/InterpretNER/. As a by-product of this paper, we have open-sourced a project that involves a comprehensive summary of recent NER papers and classifies them into different research topics: https://github.com/pfliu-nlp/Named-Entity-Recognition-NER-Papers.
