论文标题
矢量定位的输入反映软提示，以了解自然语言理解
Vector-Quantized Input-Contextualized Soft Prompts for Natural Language Understanding
论文作者
论文摘要
迅速调整在很大程度上是成功的方法，它是一种调节大规模预训练的语言模型以执行下游任务的方法。到目前为止，软提示调整会学习一组固定的特定任务连续向量，即在整个任务样本中保持静态的柔软令牌。但是，固定的提示可能无法很好地推广到任务所包含的各种输入。为了解决这个问题，我们提出了矢量量化的输入 - 封闭式提示（VIP），以扩展到软提示调谐框架。 VIP特别关注两个方面 - 上下文提示，通过小规模的句子编码来学习软提示令牌的输入特定上下文化，并量化了通过上下文的提示将上下文提示映射到通过向量量化网络的一组可学习的代码书向量的提示。在各种语言理解任务（例如Superglue，QA，关系分类，NER和NLI）的任务上，VIP的表现使软提示调谐（PT）基线的平均利润率平均为1.19％。此外，我们的概括研究表明，VIP在室外质量检查和NLI任务上分别学习了更强大的及时表示，超过了PT的0.6％-5.3％，并且在跨越12个领域的4个任务上的多任务设置上的余量为0.75％。
Prompt Tuning has been largely successful as a parameter-efficient method of conditioning large-scale pre-trained language models to perform downstream tasks. Thus far, soft prompt tuning learns a fixed set of task-specific continuous vectors, i.e., soft tokens that remain static across the task samples. A fixed prompt, however, may not generalize well to the diverse kinds of inputs the task comprises. In order to address this, we propose Vector-quantized Input-contextualized Prompts (VIP) as an extension to the soft prompt tuning framework. VIP particularly focuses on two aspects -- contextual prompts that learns input-specific contextualization of the soft prompt tokens through a small-scale sentence encoder and quantized prompts that maps the contextualized prompts to a set of learnable codebook vectors through a Vector quantization network. On various language understanding tasks like SuperGLUE, QA, Relation classification, NER and NLI, VIP outperforms the soft prompt tuning (PT) baseline by an average margin of 1.19%. Further, our generalization studies show that VIP learns more robust prompt representations, surpassing PT by a margin of 0.6% - 5.3% on Out-of-domain QA and NLI tasks respectively, and by 0.75% on Multi-Task setup over 4 tasks spanning across 12 domains.
