论文标题
在研究基于得分的生成模型的保守性质
On Investigating the Conservative Property of Score-Based Generative Models
论文作者
论文摘要
现有基于分数的模型（SBM）可以根据其参数化方法分类为受约束的SBMS（CSBM）或无约束的SBMS（USBMS）。 CSBM模型概率密度充当玻尔兹曼分布，并将其预测作为某些标量值能量函数的负梯度。另一方面，USBM采用了能够直接估算得分的灵活体系结构，而无需明确建模能量功能。在本文中，我们证明了CSBM的架构约束可能会限制其建模能力。此外，我们表明USBMS无法保护保守性的财产可能会导致实践中的绩效下降。为了解决上述问题，我们提出了基于准保守分数的模型（QCSBMS），以保持CSBM和USBM的优势。我们的理论推导表明，通过利用Hutchinson的痕量估计器，可以将QCSBMS的训练目标有效地整合到训练过程中。此外，我们对CIFAR-10，CIFAR-100，ImageNet和SVHN数据集的实验结果验证了QCSBMS的有效性。最后，我们使用单层自动编码器的示例证明QCSBMS的优势是合理的。
Existing Score-Based Models (SBMs) can be categorized into constrained SBMs (CSBMs) or unconstrained SBMs (USBMs) according to their parameterization approaches. CSBMs model probability density functions as Boltzmann distributions, and assign their predictions as the negative gradients of some scalar-valued energy functions. On the other hand, USBMs employ flexible architectures capable of directly estimating scores without the need to explicitly model energy functions. In this paper, we demonstrate that the architectural constraints of CSBMs may limit their modeling ability. In addition, we show that USBMs' inability to preserve the property of conservativeness may lead to degraded performance in practice. To address the above issues, we propose Quasi-Conservative Score-Based Models (QCSBMs) for keeping the advantages of both CSBMs and USBMs. Our theoretical derivations demonstrate that the training objective of QCSBMs can be efficiently integrated into the training processes by leveraging the Hutchinson's trace estimator. In addition, our experimental results on the CIFAR-10, CIFAR-100, ImageNet, and SVHN datasets validate the effectiveness of QCSBMs. Finally, we justify the advantage of QCSBMs using an example of a one-layered autoencoder.
