论文标题
在前馈神经网络中表达线性平等约束
Expressing linear equality constraints in feedforward neural networks
论文作者
论文摘要
我们寻求在馈电神经网络中施加线性，平等约束。由于顶层预测指标通常是非线性的，因此，如果我们寻求部署标准凸优化方法和强二重性，这是一项艰巨的任务。为了克服这一点，我们引入了一个新的鞍点拉格朗日，并带有辅助预测变量，在该变量上施加了约束。消除辅助变量会导致引入的拉格朗日乘数上的双重最小化问题，以满足线性约束。这个最小化问题与重量矩阵上的标准学习问题相结合。从这一理论发展线上，我们可以获得对拉格朗日参数的令人惊讶的解释，即额外的倒数第二层隐藏单位，其固定权重来自约束。因此，尽管包含拉格朗日参数，但仍可以使用标准的最小化方法 - 这是一个非常令人满意的，尽管是意外的，但发现。将来设想从多标签分类到受约束的自动编码器的示例。该代码已在https://github.com/anandrajan0/smartalec上提供
We seek to impose linear, equality constraints in feedforward neural networks. As top layer predictors are usually nonlinear, this is a difficult task if we seek to deploy standard convex optimization methods and strong duality. To overcome this, we introduce a new saddle-point Lagrangian with auxiliary predictor variables on which constraints are imposed. Elimination of the auxiliary variables leads to a dual minimization problem on the Lagrange multipliers introduced to satisfy the linear constraints. This minimization problem is combined with the standard learning problem on the weight matrices. From this theoretical line of development, we obtain the surprising interpretation of Lagrange parameters as additional, penultimate layer hidden units with fixed weights stemming from the constraints. Consequently, standard minimization approaches can be used despite the inclusion of Lagrange parameters -- a very satisfying, albeit unexpected, discovery. Examples ranging from multi-label classification to constrained autoencoders are envisaged in the future. The code has been made available at https://github.com/anandrajan0/smartalec
