论文标题
可以从理想语言模型中提取元素语义
Entailment Semantics Can Be Extracted from an Ideal Language Model
论文作者
论文摘要
语言模型通常仅在文本上进行培训，而无需其他基础。关于从这种过程中可以推断出多少自然语言语义的争论。我们证明，假设训练句子是由格里斯·特工（Gricean Agents）产生的，即遵循实用语言学语言学理论的基本原则，可以从理想的语言模型中提取句子之间的判断，该句子之间的判断是从理想的语言模型中提取的，该模型已经完美地学习了其目标分布。我们还显示，可以从对这种Gricean数据训练的语言模型的预测中解码需要判断。我们的结果揭示了一种理解未标记的语言数据编码的语义信息的途径，以及从语言模型中提取语义的潜在框架。
Language models are often trained on text alone, without additional grounding. There is debate as to how much of natural language semantics can be inferred from such a procedure. We prove that entailment judgments between sentences can be extracted from an ideal language model that has perfectly learned its target distribution, assuming the training sentences are generated by Gricean agents, i.e., agents who follow fundamental principles of communication from the linguistic theory of pragmatics. We also show entailment judgments can be decoded from the predictions of a language model trained on such Gricean data. Our results reveal a pathway for understanding the semantic information encoded in unlabeled linguistic data and a potential framework for extracting semantics from language models.
