论文标题
建筑不可知论神经网络
Architecture Agnostic Neural Networks
论文作者
论文摘要
在本文中，我们探索了一种由大脑随机突触修剪启发的替代方法，用于合成神经网络体系结构。在一个人的一生中，许多不同的神经元结构负责执行相同的任务。这表明生物神经网络在某种程度上是建筑不可知的。但是，人造网络依靠其微调的重量和手工制作的体系结构来表现出色。这种对比提出了一个问题：我们可以建立人工体系结构不可知的神经网络吗？为了使这项研究基础，我们利用了与大脑电路相似的稀疏二元神经网络。在这个稀疏的二进制范式中，我们采样了许多二元架构，以创建未通过反向传播训练的建筑不可知神经网络家庭。这些高性能的网络家族具有相同的稀疏性，二进制权重的分布以及在静态和动态任务中取得成功。总而言之，我们创建一个体系结构搜索程序，以发现家庭或架构不可知的神经网络。
In this paper, we explore an alternate method for synthesizing neural network architectures, inspired by the brain's stochastic synaptic pruning. During a person's lifetime, numerous distinct neuronal architectures are responsible for performing the same tasks. This indicates that biological neural networks are, to some degree, architecture agnostic. However, artificial networks rely on their fine-tuned weights and hand-crafted architectures for their remarkable performance. This contrast begs the question: Can we build artificial architecture agnostic neural networks? To ground this study we utilize sparse, binary neural networks that parallel the brain's circuits. Within this sparse, binary paradigm we sample many binary architectures to create families of architecture agnostic neural networks not trained via backpropagation. These high-performing network families share the same sparsity, distribution of binary weights, and succeed in both static and dynamic tasks. In summation, we create an architecture manifold search procedure to discover families or architecture agnostic neural networks.
