论文标题
通过Kronecker多层体系结构，深度学习的重量矩阵维度降低
Weight Matrix Dimensionality Reduction in Deep Learning via Kronecker Multi-layer Architectures
论文作者
论文摘要
使用神经网络的深度学习是生成复杂数据模型的有效技术。但是，当网络具有大量层和节点产生的大型模型容量时，培训可能会很昂贵。为了在这种计算性过高的制度中进行培训，降低性降低技术减轻了计算负担，并允许实施更健壮的网络。我们通过基于Kronecker产品分解的快速矩阵乘法的新深度学习结构提出了一种新型的这种维度降低类型。特别是我们的网络构建可以看作是Kronecker产品引起的“扩展”完全连接网络的稀疏性。分析和实际示例表明，这种体系结构允许对神经网络进行训练和实施，并大大减少计算时间和资源，同时与传统的前馈神经网络相比达到了类似的错误水平。
Deep learning using neural networks is an effective technique for generating models of complex data. However, training such models can be expensive when networks have large model capacity resulting from a large number of layers and nodes. For training in such a computationally prohibitive regime, dimensionality reduction techniques ease the computational burden, and allow implementations of more robust networks. We propose a novel type of such dimensionality reduction via a new deep learning architecture based on fast matrix multiplication of a Kronecker product decomposition; in particular our network construction can be viewed as a Kronecker product-induced sparsification of an "extended" fully connected network. Analysis and practical examples show that this architecture allows a neural network to be trained and implemented with a significant reduction in computational time and resources, while achieving a similar error level compared to a traditional feedforward neural network.
