论文标题
从点到函数：扩散模型中的无限维度表示
From Points to Functions: Infinite-dimensional Representations in Diffusion Models
论文作者
论文摘要
基于扩散的生成模型学会将非结构化噪声转移到复杂的目标分布中，而不是生成的对抗网络（GAN）或变分自动编码器（VAE）的解码器（VAE），这些解码器（VAE）在单个步骤中从目标分布中产生样品。因此，在扩散模型中，每个样品自然连接到随机轨迹，该轨迹是对学习的随机微分方程（SDE）的解决方案。生成模型仅与此轨迹的最终状态有关，该轨迹从所需的分布中传达样品。 Abstreiter等。 Al表明，这些随机轨迹可以看作是连续过滤器，这些过滤器沿途清洗了信息。因此，合理询问是否存在中间时间步骤，保留信息对于给定的下游任务是最佳的。在这项工作中，我们表明，来自不同时间步骤的信息内容组合为下游任务提供了更好的更好表示。我们介绍了一个基于注意力和复发的模块，该模块``学习混合了''的各个时步的信息内容，从而导致下游任务中的表现卓越。
Diffusion-based generative models learn to iteratively transfer unstructured noise to a complex target distribution as opposed to Generative Adversarial Networks (GANs) or the decoder of Variational Autoencoders (VAEs) which produce samples from the target distribution in a single step. Thus, in diffusion models every sample is naturally connected to a random trajectory which is a solution to a learned stochastic differential equation (SDE). Generative models are only concerned with the final state of this trajectory that delivers samples from the desired distribution. Abstreiter et. al showed that these stochastic trajectories can be seen as continuous filters that wash out information along the way. Consequently, it is reasonable to ask if there is an intermediate time step at which the preserved information is optimal for a given downstream task. In this work, we show that a combination of information content from different time steps gives a strictly better representation for the downstream task. We introduce an attention and recurrence based modules that ``learn to mix'' information content of various time-steps such that the resultant representation leads to superior performance in downstream tasks.
