论文标题
关于过度拟合和不足的信息理论观点
An Information-Theoretic Perspective on Overfitting and Underfitting
论文作者
论文摘要
我们提出了一个信息理论框架，以理解机器学习中过度拟合和不足的信息，并证明正式的不确定性确定任意分类算法是否会过度贴上数据集。通过从数据集传输到模型的信息测量算法容量，我们考虑算法容量和数据集之间的不匹配，以提供签名，以适用于模型可以过度拟合或低于数据集的何时。我们介绍了上限算法能力，在用于机器学习的算法搜索框架中建立了与数量的关系，并将我们的工作与最近的信息理论方法联系起来。
We present an information-theoretic framework for understanding overfitting and underfitting in machine learning and prove the formal undecidability of determining whether an arbitrary classification algorithm will overfit a dataset. Measuring algorithm capacity via the information transferred from datasets to models, we consider mismatches between algorithm capacities and datasets to provide a signature for when a model can overfit or underfit a dataset. We present results upper-bounding algorithm capacity, establish its relationship to quantities in the algorithmic search framework for machine learning, and relate our work to recent information-theoretic approaches to generalization.
