论文标题
基于及时的模型是无知的吗？
Are Prompt-based Models Clueless?
论文作者
论文摘要
具有特定于任务的头脑的大型预训练的大型预训练的语言模型已在许多自然语言理解基准的基准方面提高了最新。但是，具有特定于任务的头部的模型需要大量的培训数据，使其容易受到学习和利用数据集特定的表面提示，这些提示不会推广到其他数据集。提示通过重复语言模型头并格式化任务输入以匹配预训练目标，从而减少了数据要求。因此，预计很少有基于迅速的模型不会利用浅表提示。本文介绍了对少数基于迅速的模型是否还利用表面提示的经验研究。分析有关MNLI，SNLI，HAN和COPA的几个基于迅速的模型，揭示了基于及时的模型还利用了表面提示。尽管这些模型在具有表面提示的实例上表现良好，但它们在没有表面提示的情况下通常表现不佳或仅略高于随机精度。
Finetuning large pre-trained language models with a task-specific head has advanced the state-of-the-art on many natural language understanding benchmarks. However, models with a task-specific head require a lot of training data, making them susceptible to learning and exploiting dataset-specific superficial cues that do not generalize to other datasets. Prompting has reduced the data requirement by reusing the language model head and formatting the task input to match the pre-training objective. Therefore, it is expected that few-shot prompt-based models do not exploit superficial cues. This paper presents an empirical examination of whether few-shot prompt-based models also exploit superficial cues. Analyzing few-shot prompt-based models on MNLI, SNLI, HANS, and COPA has revealed that prompt-based models also exploit superficial cues. While the models perform well on instances with superficial cues, they often underperform or only marginally outperform random accuracy on instances without superficial cues.
