论文标题
Relu神经网络线性区域的组合结构的算法测定
Algorithmic Determination of the Combinatorial Structure of the Linear Regions of ReLU Neural Networks
论文作者
论文摘要
我们从算法上确定了规范多面体复合物的所有维度的区域和方面，这是relu网络分解其输入空间的通用对象。我们表明，规范多面体复合物的顶点的位置以及它们相对于层地图的迹象决定了所有维度的整个刻面结构。我们提出了一种计算完整的组合结构的算法，利用了我们的定理，该定理对规范多面体复合物是立方体，并且具有与其相结构兼容的乘法。所得算法在中间神经元的数量上是数值稳定的多项式时间，并且在所有维度上都获得了准确的信息。这使我们能够获得，例如，具有低维输入的网络的决策边界的真实拓扑。我们在初始化时在此类网络上运行经验，发现单独的宽度不会增加观察到的拓扑结构，而是在深度存在下的宽度。我们的算法的源代码可在https://github.com/mmasden/caronicalpoly上在线访问。
We algorithmically determine the regions and facets of all dimensions of the canonical polyhedral complex, the universal object into which a ReLU network decomposes its input space. We show that the locations of the vertices of the canonical polyhedral complex along with their signs with respect to layer maps determine the full facet structure across all dimensions. We present an algorithm which calculates this full combinatorial structure, making use of our theorems that the dual complex to the canonical polyhedral complex is cubical and it possesses a multiplication compatible with its facet structure. The resulting algorithm is numerically stable, polynomial time in the number of intermediate neurons, and obtains accurate information across all dimensions. This permits us to obtain, for example, the true topology of the decision boundaries of networks with low-dimensional inputs. We run empirics on such networks at initialization, finding that width alone does not increase observed topology, but width in the presence of depth does. Source code for our algorithms is accessible online at https://github.com/mmasden/canonicalpoly.
