论文标题
重新思考符号回归数据集和基准进行科学发现
Rethinking Symbolic Regression Datasets and Benchmarks for Scientific Discovery
论文作者
论文摘要
本文重新审视了符号回归（SR）的数据集和评估标准，特别关注其科学发现的潜力。专注于基于Feynman物理学讲座的现有数据集中使用的一组公式，我们重新创建了120个数据集，以讨论科学发现符号回归的性能（SRSD）。对于120个SRSD数据集中的每个数据集，我们仔细检查了公式及其变量的属性，以设计值的值得逼真的采样范围，以便可以使用我们的新SRSD数据集来评估SRSD的潜力，例如SR方法是否可以（re）从此类数据集中发现物理定律。我们还创建了另一个包含虚拟变量的120个数据集，以检查SR方法是否只能选择必要的变量。此外，我们建议在预测方程和真实方程树之间使用归一化的编辑距离（NED），以解决一个关键问题，即现有的SR指标是目标值与给定输入的SR模型之间的二进制或错误。我们使用各种代表性SR方法在新的SRSD数据集上进行基准实验。实验结果表明，我们提供了更现实的绩效评估，我们的用户研究表明，NED与人类法官的相关性明显超过现有的SR度量。我们发布代码的存储库和240个SRSD数据集。
This paper revisits datasets and evaluation criteria for Symbolic Regression (SR), specifically focused on its potential for scientific discovery. Focused on a set of formulas used in the existing datasets based on Feynman Lectures on Physics, we recreate 120 datasets to discuss the performance of symbolic regression for scientific discovery (SRSD). For each of the 120 SRSD datasets, we carefully review the properties of the formula and its variables to design reasonably realistic sampling ranges of values so that our new SRSD datasets can be used for evaluating the potential of SRSD such as whether or not an SR method can (re)discover physical laws from such datasets. We also create another 120 datasets that contain dummy variables to examine whether SR methods can choose necessary variables only. Besides, we propose to use normalized edit distances (NED) between a predicted equation and the true equation trees for addressing a critical issue that existing SR metrics are either binary or errors between the target values and an SR model's predicted values for a given input. We conduct benchmark experiments on our new SRSD datasets using various representative SR methods. The experimental results show that we provide a more realistic performance evaluation, and our user study shows that the NED correlates with human judges significantly more than an existing SR metric. We publish repositories of our code and 240 SRSD datasets.
