论文标题
梯度应保持路径：反向和前进kl差异的更好估计量以归一流的流量
Gradients should stay on Path: Better Estimators of the Reverse- and Forward KL Divergence for Normalizing Flows
论文作者
论文摘要
我们提出了一种算法，以估计反向和前向kullback-leibler差异的路径梯度，以明显可逆地归一流。与标准的总梯度估计量相比，所得的路径梯度估计器可直接实施，具有较低的差异，不仅导致训练的速度更快，而且导致总体近似结果更好。我们还证明，路径梯度训练不太容易受到模式折叠的影响。鉴于我们的结果，我们期望路径梯度估计器将成为训练归一化流量的新标准方法。
We propose an algorithm to estimate the path-gradient of both the reverse and forward Kullback-Leibler divergence for an arbitrary manifestly invertible normalizing flow. The resulting path-gradient estimators are straightforward to implement, have lower variance, and lead not only to faster convergence of training but also to better overall approximation results compared to standard total gradient estimators. We also demonstrate that path-gradient training is less susceptible to mode-collapse. In light of our results, we expect that path-gradient estimators will become the new standard method to train normalizing flows for variational inference.
