论文标题
科比：基于知识的机器翻译评估
KoBE: Knowledge-Based Machine Translation Evaluation
论文作者
论文摘要
我们提出了一种简单有效的机器翻译评估方法，该方法不需要参考翻译。我们的方法是基于（1）基于每个源句子中发现的实体提到的，并针对大规模的多语言知识库进行了翻译，以及（2）测量候选人中发现的扎根实体与在源中发现的实体的召回。我们的方法与从WMT19基准中的18个语言对中的9对中的9对中的人类判断达到了最高的相关性，无需参考，这是该任务上单个评估方法的最大胜利。在4个语言对上，我们还与BLEU实现了与人类判断更高的相关性。为了促进进一步的研究，我们发布了一个数据集，其中包含来自WMT19指标数据的18个语言对的180万个基础实体提及的数据集。
We propose a simple and effective method for machine translation evaluation which does not require reference translations. Our approach is based on (1) grounding the entity mentions found in each source sentence and candidate translation against a large-scale multilingual knowledge base, and (2) measuring the recall of the grounded entities found in the candidate vs. those found in the source. Our approach achieves the highest correlation with human judgements on 9 out of the 18 language pairs from the WMT19 benchmark for evaluation without references, which is the largest number of wins for a single evaluation method on this task. On 4 language pairs, we also achieve higher correlation with human judgements than BLEU. To foster further research, we release a dataset containing 1.8 million grounded entity mentions across 18 language pairs from the WMT19 metrics track data.
