论文标题
Scai-QRECC在对话问题回答上共享任务
SCAI-QReCC Shared Task on Conversational Question Answering
论文作者
论文摘要
面向搜索的对话AI（SCAI）是一个既定的场所，定期将焦点放在最近的工作中，以推动对话搜索领域。 Scai'21是作为独立的在线活动组织的，并在对话中的答案上进行了共同的任务。由于所有参与者团队都对此任务进行了答案生成模型，因此我们确定对这种情况下的答案正确性的评估是主要的挑战和当前的研究差距。除了自动评估外，我们进行了两个众包实验，以收集注释，以解决答案的合理性和忠诚。由于这项共享任务，用于评估的原始对话质量检查数据集进一步扩展了参与者系统产生的替代正确答案。
Search-Oriented Conversational AI (SCAI) is an established venue that regularly puts a spotlight upon the recent work advancing the field of conversational search. SCAI'21 was organised as an independent on-line event and featured a shared task on conversational question answering. Since all of the participant teams experimented with answer generation models for this task, we identified evaluation of answer correctness in this settings as the major challenge and a current research gap. Alongside the automatic evaluation, we conducted two crowdsourcing experiments to collect annotations for answer plausibility and faithfulness. As a result of this shared task, the original conversational QA dataset used for evaluation was further extended with alternative correct answers produced by the participant systems.
