论文标题
原型微调：在不同的数据大小下进行稳健性能
Prototypical Fine-tuning: Towards Robust Performance Under Varying Data Sizes
论文作者
论文摘要
在本文中，我们朝着将大型参数模型与非参数原型网络相结合。我们提出了典型的微调，这是一种用于微调审计语言模型（LM）的新型典型框架，该框架自动学习偏见以提高不同数据尺寸的预测性能，尤其是低资源设置。我们的原型微调方法可以根据数据点的数量和模型的固有属性自动调整模型容量。此外，我们提出了四个针对最佳解决方案进行有效原型的原理。各个数据集的实验结果表明，我们的工作在各种低资源设置以及在高资源场景中的可比性且通常更好的表现下实现了显着的性能提高。
In this paper, we move towards combining large parametric models with non-parametric prototypical networks. We propose prototypical fine-tuning, a novel prototypical framework for fine-tuning pretrained language models (LM), which automatically learns a bias to improve predictive performance for varying data sizes, especially low-resource settings. Our prototypical fine-tuning approach can automatically adjust the model capacity according to the number of data points and the model's inherent attributes. Moreover, we propose four principles for effective prototype fine-tuning towards the optimal solution. Experimental results across various datasets show that our work achieves significant performance improvements under various low-resource settings, as well as comparable and usually better performances in high-resource scenarios.
