论文标题
具有大都市采样的自适应随机傅立叶特征
Adaptive random Fourier features with Metropolis sampling
论文作者
论文摘要
确定神经网络近似的监督学习问题$ \ mathbb {r}^d \ ni x \ mapsto \ sum_ {k = 1}^k \hatβ_ke^{\ mathrm {i}ω__k\ cdot x} $，带有一个隐藏的层，其中一个隐藏的层是一个随机的funier algorithm。傅立叶功能，即使用自适应大都市采样器对\ Mathbb {r}^d $中的频率$ω_k\ in \ Mathbb {r}^d $进行采样。 Metropolis测试接受提案频率$ω_k'$，具有相应的振幅$ \hatβ_k'$，具有概率$ \ min \ min \ big \ big \ big \ {1，（| \hatβ_K'|/| \hatβ_k|）^γ\ big \} $，以某些正面参数为单位$ uligation comminitation comminitation comminitation comminitation $ giantim $ giantim $γ$γ$γ$γ$γ$γ$γ。这种自适应，非参数随机方法渐近地导致$ k \ to \ infty $，以均衡振幅$ | \hatβ_k| $，类似于确定性的自适应算法的不同方程。在随机傅立叶特征方法中，等均分配幅度渐近地与独立样品的最佳密度相对应。提供数值证据以证明所提出算法的近似特性和效率。该算法在合成数据和现实世界高维基准测试上进行了测试。
The supervised learning problem to determine a neural network approximation $\mathbb{R}^d\ni x\mapsto\sum_{k=1}^K\hatβ_k e^{\mathrm{i}ω_k\cdot x}$ with one hidden layer is studied as a random Fourier features algorithm. The Fourier features, i.e., the frequencies $ω_k\in\mathbb{R}^d$, are sampled using an adaptive Metropolis sampler. The Metropolis test accepts proposal frequencies $ω_k'$, having corresponding amplitudes $\hatβ_k'$, with the probability $\min\big\{1, (|\hatβ_k'|/|\hatβ_k|)^γ\big\}$, for a certain positive parameter $γ$, determined by minimizing the approximation error for given computational work. This adaptive, non-parametric stochastic method leads asymptotically, as $K\to\infty$, to equidistributed amplitudes $|\hatβ_k|$, analogous to deterministic adaptive algorithms for differential equations. The equidistributed amplitudes are shown to asymptotically correspond to the optimal density for independent samples in random Fourier features methods. Numerical evidence is provided in order to demonstrate the approximation properties and efficiency of the proposed algorithm. The algorithm is tested both on synthetic data and a real-world high-dimensional benchmark.
