论文标题
STEIN的分散分布的变异推断
Stein Variational Inference for Discrete Distributions
论文作者
论文摘要
基于梯度的近似推理方法，例如Stein变化梯度下降（SVGD），为可区分的连续分布提供了简单和通用的推理引擎。但是，现有的SVGD形式不能直接应用于离散分布。在这项工作中，我们通过提出一个简单而通用的框架来填补这一空白，该框架将离散分布转换为等效的分段连续分布，将无梯度的SVGD应用于执行有效的近似推断。经验结果表明，我们的方法优于传统算法，例如Gibbs采样和不连续的Hamiltonian Monte Carlo在各种挑战性的离散图形模型上。我们证明了我们的方法为学习二进制神经网络（BNN）的学习提供了有前途的工具，在CIFAR-10数据集中学习二元化的Alexnet方面的其他广泛使用的集合方法。此外，这种转换可以直接用于无梯度的二键式Stein差异，以对离散分布进行拟合优度（GOF）测试。我们提出的方法的表现优于现有的GOF测试方法，用于棘手的离散分布。
Gradient-based approximate inference methods, such as Stein variational gradient descent (SVGD), provide simple and general-purpose inference engines for differentiable continuous distributions. However, existing forms of SVGD cannot be directly applied to discrete distributions. In this work, we fill this gap by proposing a simple yet general framework that transforms discrete distributions to equivalent piecewise continuous distributions, on which the gradient-free SVGD is applied to perform efficient approximate inference. The empirical results show that our method outperforms traditional algorithms such as Gibbs sampling and discontinuous Hamiltonian Monte Carlo on various challenging benchmarks of discrete graphical models. We demonstrate that our method provides a promising tool for learning ensembles of binarized neural network (BNN), outperforming other widely used ensemble methods on learning binarized AlexNet on CIFAR-10 dataset. In addition, such transform can be straightforwardly employed in gradient-free kernelized Stein discrepancy to perform goodness-of-fit (GOF) test on discrete distributions. Our proposed method outperforms existing GOF test methods for intractable discrete distributions.
