论文标题
卷积神经网络的结构重量先验
Structured Weight Priors for Convolutional Neural Networks
论文作者
论文摘要
选择适合一项任务的架构先验良好（例如图像数据的卷积）对于深神经网络（NNS）的成功至关重要。相反，这些体系结构中的重量先验通常是含糊不清的，例如〜独立的高斯分布，这导致了有关贝叶斯深度学习实用程序的辩论。本文探讨了将结构添加到体重先验中的好处。它最初考虑了卷积NN的一层过滤器，根据随机Gabor过滤器设计了先验。其次，它考虑通过估计每个隐藏特征与每个类别的关系，从而在最终权重的先前添加结构。经验结果表明，这些结构化的权重先验会导致图像数据更有意义的功能先验。这有助于对体重先验的重要性进行持续的讨论。
Selection of an architectural prior well suited to a task (e.g. convolutions for image data) is crucial to the success of deep neural networks (NNs). Conversely, the weight priors within these architectures are typically left vague, e.g.~independent Gaussian distributions, which has led to debate over the utility of Bayesian deep learning. This paper explores the benefits of adding structure to weight priors. It initially considers first-layer filters of a convolutional NN, designing a prior based on random Gabor filters. Second, it considers adding structure to the prior of final-layer weights by estimating how each hidden feature relates to each class. Empirical results suggest that these structured weight priors lead to more meaningful functional priors for image data. This contributes to the ongoing discussion on the importance of weight priors.
