论文标题
多层感知的Pac-Bayesian概括界
PAC-Bayesian Generalization Bounds for MultiLayer Perceptrons
论文作者
论文摘要
我们研究了多层感知器（MLP）的Pac-Bayesian泛化界限，并具有横向熵损失。最重要的是，我们在两个方面介绍了MLP的概率解释：（i）MLP制定了吉布斯分布的家族，并且（ii）最大程度地减少MLP的跨透明副本损失等同于贝叶斯变异推断，这与研究Pac-bayesian在MLPS上的固定概率基础。此外，根据证据下限（ELBO），我们证明具有跨熵损失的MLP固有地保证了Pac Bayesian的概括界限，而最小化MLP的Pac-Bayesian泛化范围等效于最大化ELBO。最后，我们验证了在基准数据集上绑定的拟议的pac-bayesian泛化。
We study PAC-Bayesian generalization bounds for Multilayer Perceptrons (MLPs) with the cross entropy loss. Above all, we introduce probabilistic explanations for MLPs in two aspects: (i) MLPs formulate a family of Gibbs distributions, and (ii) minimizing the cross-entropy loss for MLPs is equivalent to Bayesian variational inference, which establish a solid probabilistic foundation for studying PAC-Bayesian bounds on MLPs. Furthermore, based on the Evidence Lower Bound (ELBO), we prove that MLPs with the cross entropy loss inherently guarantee PAC- Bayesian generalization bounds, and minimizing PAC-Bayesian generalization bounds for MLPs is equivalent to maximizing the ELBO. Finally, we validate the proposed PAC-Bayesian generalization bound on benchmark datasets.
