论文标题
通过自我监督的跳过培训的数学推理
Mathematical Reasoning via Self-supervised Skip-tree Training
论文作者
论文摘要
我们检查了应用于数学公式的自我监督语言建模是否可以实现逻辑推理。我们建议一些逻辑推理任务，可用于评估接受正式数学语句训练的语言模型，例如类型推理，建议缺失假设并完成平等性。为了培训语言模型以进行正式数学，我们提出了一项新颖的跳过任务。我们发现，在跳过的TREE任务上训练的模型表现出令人惊讶的强大数学推理能力，并且超过了对标准跳过序列任务的训练的模型。我们还通过衡量预测在其他证明中可证明和有用的频率来分析模型制定新猜想的能力。
We examine whether self-supervised language modeling applied to mathematical formulas enables logical reasoning. We suggest several logical reasoning tasks that can be used to evaluate language models trained on formal mathematical statements, such as type inference, suggesting missing assumptions and completing equalities. To train language models for formal mathematics, we propose a novel skip-tree task. We find that models trained on the skip-tree task show surprisingly strong mathematical reasoning abilities, and outperform models trained on standard skip-sequence tasks. We also analyze the models' ability to formulate new conjectures by measuring how often the predictions are provable and useful in other proofs.
