论文标题
通过链规范化规则和假设训练测试来了解神经网络的体重相似性
Understanding Weight Similarity of Neural Networks via Chain Normalization Rule and Hypothesis-Training-Testing
论文作者
论文摘要
我们提出了一种可以量化非凸神经网络的权重相似性的重量相似度度量方法。为了了解不同训练的模型的重量相似性，我们建议从神经网络的重量中提取特征表示。我们首先通过引入链条归一化规则来使神经网络的权重标准化，该规范用于体重表示学习和权重相似度度量。我们将传统的假设测试方法扩展到假设训练测试统计推断方法，以验证神经网络的体重相似性的假设。 With the chain normalization rule and the new statistical inference, we study the weight similarity measure on Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN), and find that the weights of an identical neural network optimized with the Stochastic Gradient Descent (SGD) algorithm converge to a similar local solution in a metric space.重量相似性度量为神经网络的局部解决方案提供了更多的了解。在几个数据集上的实验始终验证了体重相似度度量的假设。
We present a weight similarity measure method that can quantify the weight similarity of non-convex neural networks. To understand the weight similarity of different trained models, we propose to extract the feature representation from the weights of neural networks. We first normalize the weights of neural networks by introducing a chain normalization rule, which is used for weight representation learning and weight similarity measure. We extend the traditional hypothesis-testing method to a hypothesis-training-testing statistical inference method to validate the hypothesis on the weight similarity of neural networks. With the chain normalization rule and the new statistical inference, we study the weight similarity measure on Multi-Layer Perceptron (MLP), Convolutional Neural Network (CNN), and Recurrent Neural Network (RNN), and find that the weights of an identical neural network optimized with the Stochastic Gradient Descent (SGD) algorithm converge to a similar local solution in a metric space. The weight similarity measure provides more insight into the local solutions of neural networks. Experiments on several datasets consistently validate the hypothesis of weight similarity measure.
