论文标题
从人类判断中重新考虑机器翻译的单词级质量估计
Rethink about the Word-level Quality Estimation for Machine Translation from Human Judgement
论文作者
论文摘要
机器翻译（MT）的单词级质量估计（QE）旨在在不参考的情况下找出翻译句子中的潜在翻译错误。通常，关于文字级别量化宽松的传统作品旨在根据文章编辑工作来预测翻译质量，其中通过比较MT句子和通过翻译误差率（TER）工具的MT句子和后编辑的句子之间的单词来自动生成。虽然可以使用后编辑的工作来在某种程度上测量翻译质量，但我们发现它通常与人类关于该单词是否良好或翻译不佳的判断相抵触。为了克服限制，我们首先创建了一个金色基准数据集，即\ emph {hjqe}（人类对质量估计的判断），专家翻译直接注释了对其判断的不良翻译单词。此外，为了进一步利用平行语料库，我们提出了使用两个标签校正策略的自我监督的预训练，即标记改进策略和基于树的注释策略，以使基于TER的基于TER的人工量化QE copus更接近\ emph {hjqe}。我们根据公开可用的WMT EN-DE和EN-ZH Corpora进行了实质性实验。结果不仅表明我们提出的数据集与人类判断更一致，而且还确认了拟议的标签纠正策略的有效性。
Word-level Quality Estimation (QE) of Machine Translation (MT) aims to find out potential translation errors in the translated sentence without reference. Typically, conventional works on word-level QE are designed to predict the translation quality in terms of the post-editing effort, where the word labels ("OK" and "BAD") are automatically generated by comparing words between MT sentences and the post-edited sentences through a Translation Error Rate (TER) toolkit. While the post-editing effort can be used to measure the translation quality to some extent, we find it usually conflicts with the human judgement on whether the word is well or poorly translated. To overcome the limitation, we first create a golden benchmark dataset, namely \emph{HJQE} (Human Judgement on Quality Estimation), where the expert translators directly annotate the poorly translated words on their judgements. Additionally, to further make use of the parallel corpus, we propose the self-supervised pre-training with two tag correcting strategies, namely tag refinement strategy and tree-based annotation strategy, to make the TER-based artificial QE corpus closer to \emph{HJQE}. We conduct substantial experiments based on the publicly available WMT En-De and En-Zh corpora. The results not only show our proposed dataset is more consistent with human judgment but also confirm the effectiveness of the proposed tag correcting strategies.\footnote{The data can be found at \url{https://github.com/ZhenYangIACAS/HJQE}.}
