论文标题
机器阅读理解中的核心推理
Coreference Reasoning in Machine Reading Comprehension
论文作者
论文摘要
核心解决方案对于自然语言理解至关重要，并且长期以来在NLP中进行了研究。近年来，随着问题回答的格式（QA）成为机器阅读理解的标准（MRC），例如Dasigi等人进行了数据收集工作。 （2019年），尝试评估MRC模型推理有关核心的能力的尝试。但是，正如我们所表明的那样，MRC中的核心推理比以前的想法更大。 MRC数据集不能反映自然分布，因此，核心推理的挑战。具体而言，这些数据集的成功并不能反映模型在核心推理方面的熟练程度。我们提出了一种创建MRC数据集的方法，该方法可以更好地反映核心推理的挑战，并使用它来创建样本评估集。我们数据集中的结果表明，最新的模型仍然在这些现象上遇到困难。此外，在训练MRC模型时，我们开发了一种有效的方法来利用现有核心分辨率数据集中自然发生的核心现象。这使我们能够展示最先进模型的核心推理能力的改善。代码和结果数据集可在https://github.com/ukplab/coref-reasoning-in-qa上找到。
Coreference resolution is essential for natural language understanding and has been long studied in NLP. In recent years, as the format of Question Answering (QA) became a standard for machine reading comprehension (MRC), there have been data collection efforts, e.g., Dasigi et al. (2019), that attempt to evaluate the ability of MRC models to reason about coreference. However, as we show, coreference reasoning in MRC is a greater challenge than earlier thought; MRC datasets do not reflect the natural distribution and, consequently, the challenges of coreference reasoning. Specifically, success on these datasets does not reflect a model's proficiency in coreference reasoning. We propose a methodology for creating MRC datasets that better reflect the challenges of coreference reasoning and use it to create a sample evaluation set. The results on our dataset show that state-of-the-art models still struggle with these phenomena. Furthermore, we develop an effective way to use naturally occurring coreference phenomena from existing coreference resolution datasets when training MRC models. This allows us to show an improvement in the coreference reasoning abilities of state-of-the-art models. The code and the resulting dataset are available at https://github.com/UKPLab/coref-reasoning-in-qa.
