论文标题
部分可观测时空混沌系统的无模型预测
Momentum Contrastive Pre-training for Question Answering
论文作者
论文摘要
现有的提取问题回答（QA）的预训练方法会产生与语法结构中的自然问题不同的披肩状查询，该查询可能会过度拟合预训练的模型到简单的关键字匹配。为了解决这个问题，我们提出了一种用于提出质量质量质量测量方法的新型动量对比预训练（MCROSS）方法。具体而言，MCROSS引入了动量对比学习框架，以使类似粘合的和自然查询的样本对之间的答案概率对齐。因此，预先训练的模型可以更好地将类似披肩样本的知识转移到回答自然问题中。三个基准测试质量检查数据集的实验结果表明，与监督和零弹性场景中的所有基准相比，我们的方法都取得了明显的改进。
Existing pre-training methods for extractive Question Answering (QA) generate cloze-like queries different from natural questions in syntax structure, which could overfit pre-trained models to simple keyword matching. In order to address this problem, we propose a novel Momentum Contrastive pRe-training fOr queStion anSwering (MCROSS) method for extractive QA. Specifically, MCROSS introduces a momentum contrastive learning framework to align the answer probability between cloze-like and natural query-passage sample pairs. Hence, the pre-trained models can better transfer the knowledge learned in cloze-like samples to answering natural questions. Experimental results on three benchmarking QA datasets show that our method achieves noticeable improvement compared with all baselines in both supervised and zero-shot scenarios.
