论文标题
基于修订的微调机制的生物医学关系提取的BERT模型研究
Investigation of BERT Model on Biomedical Relation Extraction Based on Revised Fine-tuning Mechanism
论文作者
论文摘要
随着生物医学文献的爆炸性增长，设计自动工具以从文献中提取信息在生物医学研究中具有重要意义。最近，适合生物医学领域的基于变压器的BERT模型产生了领先的结果。但是，所有现有的BERT模型用于关系分类仅利用上一层的部分知识。在本文中，我们将研究在BERT模型的微调过程中利用整个层的方法。据我们所知，我们是第一个探索这种方法的人。实验结果表明，我们的方法改善了BERT模型性能，并优于三个基准数据集上的最先进方法，用于不同的关系提取任务。此外，进一步的分析表明，可以从BERT模型的最后一层中学到有关关系的关键知识。
With the explosive growth of biomedical literature, designing automatic tools to extract information from the literature has great significance in biomedical research. Recently, transformer-based BERT models adapted to the biomedical domain have produced leading results. However, all the existing BERT models for relation classification only utilize partial knowledge from the last layer. In this paper, we will investigate the method of utilizing the entire layer in the fine-tuning process of BERT model. To the best of our knowledge, we are the first to explore this method. The experimental results illustrate that our method improves the BERT model performance and outperforms the state-of-the-art methods on three benchmark datasets for different relation extraction tasks. In addition, further analysis indicates that the key knowledge about the relations can be learned from the last layer of BERT model.
