论文标题
有限深度的机器：朝着神经网络的形式化
Machines of finite depth: towards a formalization of neural networks
论文作者
论文摘要
我们提供一个统一的框架，其中人造神经网络及其架构可以正式描述为一般数学构造的特殊情况 - 有限深度的机器。与神经网络不同，机器具有精确的定义，几个属性自然而然地遵循。有限深度的机器是模块化的（可以合并），可有效地计算且可区分。机器的向后通过再次是一台计算机，可以使用与前向通行相同的过程计算而无需开销。我们通过统一的实现在理论上和实际上证明了这一说法，该实施概括了具有丰富的快捷方式结构的几种经典体系结构，即密集，卷积和经常性的神经网络，以及它们各自的背部传播规则。
We provide a unifying framework where artificial neural networks and their architectures can be formally described as particular cases of a general mathematical construction--machines of finite depth. Unlike neural networks, machines have a precise definition, from which several properties follow naturally. Machines of finite depth are modular (they can be combined), efficiently computable and differentiable. The backward pass of a machine is again a machine and can be computed without overhead using the same procedure as the forward pass. We prove this statement theoretically and practically, via a unified implementation that generalizes several classical architectures--dense, convolutional, and recurrent neural networks with a rich shortcut structure--and their respective backpropagation rules.
