论文标题
利润不足以解释梯度提升
Margins are Insufficient for Explaining Gradient Boosting
论文作者
论文摘要
提升是机器学习中最成功的想法之一，在几乎没有微调的情况下实现了出色的实践表现。增强分类器的成功通常归因于利润率的改善。 Schapire等人在开创性的工作中率先提出了对边缘解释的重点。 （1998年），并以Gao and Zhou（2013）约束的$ K $'TH边际概括达到顶峰，最近证明这对于某些数据分布而言几乎是紧张的（Gronlund等人，2019年）。在这项工作中，我们首先证明了$ k $'的边界界限不足以解释最先进的梯度助推器的性能。然后，我们解释了$ k $'第三保证金的简短启动，并证明了对增强分类器的强大，更精致的基于保证金的概括，这确实成功地解释了现代渐变助推器的性能。最后，我们改善了Grønlund等人最近的概括下限。 （2019）。
Boosting is one of the most successful ideas in machine learning, achieving great practical performance with little fine-tuning. The success of boosted classifiers is most often attributed to improvements in margins. The focus on margin explanations was pioneered in the seminal work by Schapire et al. (1998) and has culminated in the $k$'th margin generalization bound by Gao and Zhou (2013), which was recently proved to be near-tight for some data distributions (Gronlund et al. 2019). In this work, we first demonstrate that the $k$'th margin bound is inadequate in explaining the performance of state-of-the-art gradient boosters. We then explain the short comings of the $k$'th margin bound and prove a stronger and more refined margin-based generalization bound for boosted classifiers that indeed succeeds in explaining the performance of modern gradient boosters. Finally, we improve upon the recent generalization lower bound by Grønlund et al. (2019).
