论文标题
深喂食神经网络中特征提取的局部几何解释
A Local Geometric Interpretation of Feature Extraction in Deep Feedforward Neural Networks
论文作者
论文摘要
在本文中，我们提出了局部几何分析，以解释深度喂养神经网络如何从高维数据中提取低维特征。我们的研究表明，在局部几何区域中，在神经网络的一层中的最佳重量以及上一层产生的最佳特征包括由该层的贝叶斯动作确定的矩阵的低级别近似值。该结果成立（i）用于分析神经网络的输出层和隐藏层，以及（ii）具有非变化梯度的神经元激活函数。我们使用两个有监督的学习问题来说明我们的结果：基于神经网络的最大似然分类（即SoftMax回归）和基于神经网络的最小平均均方估计。这些理论结果的实验验证将在我们的未来工作中进行。
In this paper, we present a local geometric analysis to interpret how deep feedforward neural networks extract low-dimensional features from high-dimensional data. Our study shows that, in a local geometric region, the optimal weight in one layer of the neural network and the optimal feature generated by the previous layer comprise a low-rank approximation of a matrix that is determined by the Bayes action of this layer. This result holds (i) for analyzing both the output layer and the hidden layers of the neural network, and (ii) for neuron activation functions with non-vanishing gradients. We use two supervised learning problems to illustrate our results: neural network based maximum likelihood classification (i.e., softmax regression) and neural network based minimum mean square estimation. Experimental validation of these theoretical results will be conducted in our future work.
