论文标题
关于一对一的有条件互相信息，以进行概括
On Leave-One-Out Conditional Mutual Information For Generalization
论文作者
论文摘要
我们根据一项新的有条件互信息（LOO-CMI）的新量度来得出有关监督学习算法的理论概括范围。与其他CMI界面相反，这些界限是不利用问题结构并且可能很难在实践中评估的黑框界限，我们的oo-CMI界限可以很容易地计算，并且可以与其他概念（例如经典的一对跨跨验证，优化算法的稳定性，优化算法的稳定性）进行解释。它既适用于训练算法的输出及其预测。我们通过评估其在深度学习的情况下评估其预测的概括差距来验证界限的质量。特别是，我们的界限在大规模的图像分类任务上是无效的。
We derive information theoretic generalization bounds for supervised learning algorithms based on a new measure of leave-one-out conditional mutual information (loo-CMI). Contrary to other CMI bounds, which are black-box bounds that do not exploit the structure of the problem and may be hard to evaluate in practice, our loo-CMI bounds can be computed easily and can be interpreted in connection to other notions such as classical leave-one-out cross-validation, stability of the optimization algorithm, and the geometry of the loss-landscape. It applies both to the output of training algorithms as well as their predictions. We empirically validate the quality of the bound by evaluating its predicted generalization gap in scenarios for deep learning. In particular, our bounds are non-vacuous on large-scale image-classification tasks.
