论文标题
具有随机关注的神经过程：更多地关注上下文数据集
Neural Processes with Stochastic Attention: Paying more attention to the context dataset
论文作者
论文摘要
神经过程（NP）旨在基于给定上下文数据集随机完成看不见的数据点。 NP基本上利用给定数据集作为上下文表示，以为新任务提供合适的标识符。为了提高预测准确性，许多NP的变体都研究了上下文嵌入方法，这些方法通常设计新型的网络体系结构和聚集功能，使置换不变。在这项工作中，我们为NP提出了一种随机关注机制，以捕获适当的上下文信息。从信息理论的角度来看，我们证明了所提出的方法鼓励上下文嵌入与目标数据集区别开来，从而允许NP考虑目标数据集中的功能，并独立嵌入到上下文中。我们观察到，所提出的方法即使在嘈杂的数据集和受限制的任务分布下，也可以适当地捕获上下文嵌入，而典型的NP遭受了缺乏上下文嵌入的影响。我们从经验上表明，我们的方法通过一维回归，捕食者捕集模型和图像完成大大优于各个领域中常规的NP。此外，提出的方法还通过Movielens-10k数据集验证，这是一个现实世界中的问题。
Neural processes (NPs) aim to stochastically complete unseen data points based on a given context dataset. NPs essentially leverage a given dataset as a context representation to derive a suitable identifier for a novel task. To improve the prediction accuracy, many variants of NPs have investigated context embedding approaches that generally design novel network architectures and aggregation functions satisfying permutation invariant. In this work, we propose a stochastic attention mechanism for NPs to capture appropriate context information. From the perspective of information theory, we demonstrate that the proposed method encourages context embedding to be differentiated from a target dataset, allowing NPs to consider features in a target dataset and context embedding independently. We observe that the proposed method can appropriately capture context embedding even under noisy data sets and restricted task distributions, where typical NPs suffer from a lack of context embeddings. We empirically show that our approach substantially outperforms conventional NPs in various domains through 1D regression, predator-prey model, and image completion. Moreover, the proposed method is also validated by MovieLens-10k dataset, a real-world problem.
