论文标题
适应和评估梯度提高决策树的影响力估计方法
Adapting and Evaluating Influence-Estimation Methods for Gradient-Boosted Decision Trees
论文作者
论文摘要
影响估计分析训练数据的变化如何导致不同的模型预测；这种分析可以帮助我们更好地理解这些预测，制定这些预测的模型以及对其培训的数据集。但是，大多数影响估计技术都是为具有连续参数的深度学习模型而设计的。梯度提高决策树（GBDTS）是一种强大且广泛使用的模型。但是，这些模型是带有不透明决策过程的黑匣子。为了更好地理解GBDT预测并通常改进这些模型，我们适应了为GBDT设计的最新和流行的影响估计方法。具体而言，我们调整了代表点方法和曲霉蛋白，分别表示我们的新方法Trex和Boostin；源代码可在https://github.com/jjbrophy47/tree_influence获得。我们将这些方法与22个实际数据集的5种不同评估措施与4个流行的GBDT实现进行了比较，并使用5种不同的评估措施进行了比较。这些实验为我们提供了有关如何影响GBDT模型中估计工作的不同方法的全面概述。我们发现BOOSTIN是GBDT的有效影响估计方法，其性能比现有工作同样好或更好，同时又快四个数量级。我们的评估还表明，保留的（LOO）再培训的金标准方法始终识别出最大的影响力训练示例，但在为给定目标预测找到最有影响力的训练示例方面表现较差。
Influence estimation analyzes how changes to the training data can lead to different model predictions; this analysis can help us better understand these predictions, the models making those predictions, and the data sets they're trained on. However, most influence-estimation techniques are designed for deep learning models with continuous parameters. Gradient-boosted decision trees (GBDTs) are a powerful and widely-used class of models; however, these models are black boxes with opaque decision-making processes. In the pursuit of better understanding GBDT predictions and generally improving these models, we adapt recent and popular influence-estimation methods designed for deep learning models to GBDTs. Specifically, we adapt representer-point methods and TracIn, denoting our new methods TREX and BoostIn, respectively; source code is available at https://github.com/jjbrophy47/tree_influence. We compare these methods to LeafInfluence and other baselines using 5 different evaluation measures on 22 real-world data sets with 4 popular GBDT implementations. These experiments give us a comprehensive overview of how different approaches to influence estimation work in GBDT models. We find BoostIn is an efficient influence-estimation method for GBDTs that performs equally well or better than existing work while being four orders of magnitude faster. Our evaluation also suggests the gold-standard approach of leave-one-out (LOO) retraining consistently identifies the single-most influential training example but performs poorly at finding the most influential set of training examples for a given target prediction.
