论文标题
均值野生型在均衡状态中的功能近似：熵正则化和受控的麦凯恩·维拉索夫动力学
Function approximation by neural nets in the mean-field regime: Entropic regularization and controlled McKean-Vlasov dynamics
论文作者
论文摘要
我们考虑了两层神经网的功能近似问题，这些神经网的随机权重从Kullback-Leibler Divergence的意义上讲是几乎“高斯”。我们的设置是平均场限制，其中隐藏层中神经元的有限种群被连续的合奏所取代。我们表明，问题可以用作概率在（有限长度）路径的空间上的全球最小化，而在权重概率度量上。该功能性将终端度量的$ l^2 $近似风险与相对于各向同性布朗尼运动的kl差异的风险进行。我们表征了独特的全局最小化器，并检查了可以实现它的权重的概率措施的动态。特别是，我们表明，最佳路径空间度量对应于Föllmer漂移，这是McKean-Vlasov最佳控制问题的解决方案，与经典的Schrödinger桥接问题密切相关。虽然通常无法以封闭形式获得föllmer漂移，从而限制了其潜在算法效用，但我们说明了在熵正则化的各种条件下，平均场范围兰格文鸟扩散作为有限的时间近似。具体而言，我们表明，当正则化的密度最小化为对数孔时，它会密切跟踪föllmer漂移。
We consider the problem of function approximation by two-layer neural nets with random weights that are "nearly Gaussian" in the sense of Kullback-Leibler divergence. Our setting is the mean-field limit, where the finite population of neurons in the hidden layer is replaced by a continuous ensemble. We show that the problem can be phrased as global minimization of a free energy functional on the space of (finite-length) paths over probability measures on the weights. This functional trades off the $L^2$ approximation risk of the terminal measure against the KL divergence of the path with respect to an isotropic Brownian motion prior. We characterize the unique global minimizer and examine the dynamics in the space of probability measures over weights that can achieve it. In particular, we show that the optimal path-space measure corresponds to the Föllmer drift, the solution to a McKean-Vlasov optimal control problem closely related to the classic Schrödinger bridge problem. While the Föllmer drift cannot in general be obtained in closed form, thus limiting its potential algorithmic utility, we illustrate the viability of the mean-field Langevin diffusion as a finite-time approximation under various conditions on entropic regularization. Specifically, we show that it closely tracks the Föllmer drift when the regularization is such that the minimizing density is log-concave.
