论文标题
神经机器翻译带有错误校正
Neural Machine Translation with Error Correction
论文作者
论文摘要
神经机器翻译（NMT）生成了作为输入的下一个目标令牌，在训练过程中，前一个地面真相目标令牌，而先前在推理过程中生成的目标令牌，这会导致训练和推理之间的差异以及误差传播，并影响翻译精度。在本文中，我们向NMT引入了错误校正机制，该机制纠正了以前生成的令牌中的错误信息，以更好地预测下一代币。具体而言，我们将XLNET的两流自我发作引入到NMT解码器中，在该解码器中，查询流用于预测下一代令牌，同时使用内容流来纠正先前预测的令牌中的错误信息。我们利用计划的抽样来模拟培训期间的预测错误。在三个IWSLT转换数据集和两个WMT转换数据集上进行的实验表明，我们的方法对变压器基线和计划采样的改进进行了改进。进一步的实验分析还验证了我们提出的误差校正机制的有效性，以提高翻译质量。
Neural machine translation (NMT) generates the next target token given as input the previous ground truth target tokens during training while the previous generated target tokens during inference, which causes discrepancy between training and inference as well as error propagation, and affects the translation accuracy. In this paper, we introduce an error correction mechanism into NMT, which corrects the error information in the previous generated tokens to better predict the next token. Specifically, we introduce two-stream self-attention from XLNet into NMT decoder, where the query stream is used to predict the next token, and meanwhile the content stream is used to correct the error information from the previous predicted tokens. We leverage scheduled sampling to simulate the prediction errors during training. Experiments on three IWSLT translation datasets and two WMT translation datasets demonstrate that our method achieves improvements over Transformer baseline and scheduled sampling. Further experimental analyses also verify the effectiveness of our proposed error correction mechanism to improve the translation quality.
