论文标题
序列标记及以后的无监督的跨语言适应
Unsupervised Cross-lingual Adaptation for Sequence Tagging and Beyond
论文作者
论文摘要
多语言预训练的语言模型（MPTLMS）的跨语性适应性主要由两行作品组成：零击方法和基于翻译的方法，这些方法已在序列级任务上进行了广泛的研究。我们通过评估它们在更细颗粒的序列标记任务上的性能来进一步验证这些跨语性适应方法的功效。在重新审查了他们的优势和缺点之后，我们提出了一个新颖的框架，以巩固零拍的方法和基于翻译的方法以更好地适应性能。我们不是简单地通过机器翻译数据来增强源数据，而是根据一些翻译数据估算的梯度快速更新MPTLMS，而不是通过机器翻译数据来快速更新MPTLM。然后，将适应方法应用于精制参数，并以温暖的方式进行跨语性转移。九种目标语言的实验结果表明，我们的方法对各种序列标记任务的跨语性适应有益。
Cross-lingual adaptation with multilingual pre-trained language models (mPTLMs) mainly consists of two lines of works: zero-shot approach and translation-based approach, which have been studied extensively on the sequence-level tasks. We further verify the efficacy of these cross-lingual adaptation approaches by evaluating their performances on more fine-grained sequence tagging tasks. After re-examining their strengths and drawbacks, we propose a novel framework to consolidate the zero-shot approach and the translation-based approach for better adaptation performance. Instead of simply augmenting the source data with the machine-translated data, we tailor-make a warm-up mechanism to quickly update the mPTLMs with the gradients estimated on a few translated data. Then, the adaptation approach is applied to the refined parameters and the cross-lingual transfer is performed in a warm-start way. The experimental results on nine target languages demonstrate that our method is beneficial to the cross-lingual adaptation of various sequence tagging tasks.
