论文标题
演示搜索预测：为知识密集型NLP编写检索和语言模型
Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP
论文作者
论文摘要
通过检索启动的内在学习已成为一种强大的方法，用于使用冷冻语言模型（LM）和检索模型（RM）来解决知识密集型任务。现有的工作将它们结合在一起，简单地“检索到阅读”管道中，其中RM检索插入LM提示的段落。为了开始充分实现冷冻LM和RMS的潜力，我们提出了演示搜索预测（DSP），该框架依赖于LM和RM之间的复杂管道中的自然语言文本。 DSP可以表达高级程序，以引导管道示范，寻找相关段落并产生扎根的预测，系统地将问题分解为LM和RM可以更可靠地处理的小型转换。 We have written novel DSP programs for answering questions in open-domain, multi-hop, and conversational settings, establishing in early evaluations new state-of-the-art in-context learning results and delivering 37-120%, 8-39%, and 80-290% relative gains against the vanilla LM (GPT-3.5), a standard retrieve-then-read pipeline, and a contemporaneous self-ask pipeline, respectively.我们在https://github.com/stanfordnlp/dsp上发布DSP
Retrieval-augmented in-context learning has emerged as a powerful approach for addressing knowledge-intensive tasks using frozen language models (LM) and retrieval models (RM). Existing work has combined these in simple "retrieve-then-read" pipelines in which the RM retrieves passages that are inserted into the LM prompt. To begin to fully realize the potential of frozen LMs and RMs, we propose Demonstrate-Search-Predict (DSP), a framework that relies on passing natural language texts in sophisticated pipelines between an LM and an RM. DSP can express high-level programs that bootstrap pipeline-aware demonstrations, search for relevant passages, and generate grounded predictions, systematically breaking down problems into small transformations that the LM and RM can handle more reliably. We have written novel DSP programs for answering questions in open-domain, multi-hop, and conversational settings, establishing in early evaluations new state-of-the-art in-context learning results and delivering 37-120%, 8-39%, and 80-290% relative gains against the vanilla LM (GPT-3.5), a standard retrieve-then-read pipeline, and a contemporaneous self-ask pipeline, respectively. We release DSP at https://github.com/stanfordnlp/dsp
