论文标题
Pac-Bayes分析超出了通常的界限
PAC-Bayes Analysis Beyond the Usual Bounds
论文作者
论文摘要
我们专注于随机学习模型，学习者观察一组有限的培训示例，而学习过程的输出是在假设空间上的数据依赖性分布。然后，学习的数据依赖性分布用于做出随机预测，此处介绍的高级主题是保证在训练过程中未见的示例中预测的质量，即概括。在这种情况下，未知量的关注量是数据依赖性随机预测因子的预期风险，可以通过Pac-Bayes分析得出上限，从而导致Pac-Bayes界限。 具体而言，我们提出了随机核的基本Pac-bayes不等式，从中可能会导致各种已知的Pac-Bayes边界以及新型边界的扩展。我们阐明了固定“无数据”先验，有限损失和I.I.D.的要求的作用。数据。我们强调说，这些要求用于上限一个指数矩术语，而基本的pac-bayes定理在没有这些限制的情况下仍然有效。我们提出了三个范围，以说明使用数据依赖性先验的使用，其中包括一个无限的正方形损耗。
We focus on a stochastic learning model where the learner observes a finite set of training examples and the output of the learning process is a data-dependent distribution over a space of hypotheses. The learned data-dependent distribution is then used to make randomized predictions, and the high-level theme addressed here is guaranteeing the quality of predictions on examples that were not seen during training, i.e. generalization. In this setting the unknown quantity of interest is the expected risk of the data-dependent randomized predictor, for which upper bounds can be derived via a PAC-Bayes analysis, leading to PAC-Bayes bounds. Specifically, we present a basic PAC-Bayes inequality for stochastic kernels, from which one may derive extensions of various known PAC-Bayes bounds as well as novel bounds. We clarify the role of the requirements of fixed 'data-free' priors, bounded losses, and i.i.d. data. We highlight that those requirements were used to upper-bound an exponential moment term, while the basic PAC-Bayes theorem remains valid without those restrictions. We present three bounds that illustrate the use of data-dependent priors, including one for the unbounded square loss.
