论文标题
对中文单词细分的元学习预训练
Pre-training with Meta Learning for Chinese Word Segmentation
论文作者
论文摘要
最近的研究表明，预训练的模型（PTMS）对中文单词分割（CWS）有益。但是，以前作品中使用的PTM通常采用语言建模作为训练前任务，缺乏特定于任务的先前分割知识，而忽略了训练任务和下游CWS任务之间的差异。在本文中，我们提出了一种特定于CWS的预训练模型Metaseg，该模型采用统一体系结构，并将元学习算法纳入多准则的预训练任务。经验结果表明，Metaseg可以从不同的现有标准中利用常见的先前分割知识，并减轻预训练模型和下游CWS任务之间的差异。此外，Metaseg可以在十二个广泛使用的CWS数据集上实现新的最先进的性能，并显着提高低资源设置中的模型性能。
Recent researches show that pre-trained models (PTMs) are beneficial to Chinese Word Segmentation (CWS). However, PTMs used in previous works usually adopt language modeling as pre-training tasks, lacking task-specific prior segmentation knowledge and ignoring the discrepancy between pre-training tasks and downstream CWS tasks. In this paper, we propose a CWS-specific pre-trained model METASEG, which employs a unified architecture and incorporates meta learning algorithm into a multi-criteria pre-training task. Empirical results show that METASEG could utilize common prior segmentation knowledge from different existing criteria and alleviate the discrepancy between pre-trained models and downstream CWS tasks. Besides, METASEG can achieve new state-of-the-art performance on twelve widely-used CWS datasets and significantly improve model performance in low-resource settings.
