论文标题
平滑航行：通过表示平滑度分析，改善预训练的语言模型的积极学习
Smooth Sailing: Improving Active Learning for Pre-trained Language Models with Representation Smoothness Analysis
论文作者
论文摘要
积极学习（AL）方法旨在减轻监督学习中的标签复杂性。尽管最近的工作证明了将AL与大型预训练的语言模型（PLM）结合使用的好处，但它经常忽略了阻碍AL有效性的实际挑战。我们通过利用表示平滑度分析来确保AL是可行的，即有效和可行的，我们应对这些挑战。首先，我们提出了一种早期停止技术，该技术不需要验证集（在现实情况下通常无法使用），并观察到对多个数据集和AL方法的随机采样的显着改进。此外，我们发现任务适应可以改善AL，而AL的标准简短微调并不能改善随机抽样。我们的工作证明了表示平滑度分析对Al的有用性，并引入了AL停止标准，可降低标签复杂性。
Developed to alleviate prohibitive labeling costs, active learning (AL) methods aim to reduce label complexity in supervised learning. While recent work has demonstrated the benefit of using AL in combination with large pre-trained language models (PLMs), it has often overlooked the practical challenges that hinder the effectiveness of AL. We address these challenges by leveraging representation smoothness analysis to ensure AL is feasible, that is, both effective and practicable. Firstly, we propose an early stopping technique that does not require a validation set -- often unavailable in realistic AL conditions -- and observe significant improvements over random sampling across multiple datasets and AL methods. Further, we find that task adaptation improves AL, whereas standard short fine-tuning in AL does not provide improvements over random sampling. Our work demonstrates the usefulness of representation smoothness analysis for AL and introduces an AL stopping criterion that reduces label complexity.
