论文标题
改进的确定性L2鲁棒性的技术
Improved techniques for deterministic l2 robustness
论文作者
论文摘要
在$ l_ {2} $ norm下具有严格1- lipschitz约束的培训卷积神经网络（CNNS）对于对抗性鲁棒性，可解释的梯度和稳定的培训非常有用。 1-lipschitz CNN通常是通过强制执行每一层具有正交雅各布矩阵（用于所有输入）来设计的，以防止梯度在反向传播过程中消失。但是，它们的性能通常显着落后于启发式方法，以实施Lipschitz的约束，而所得的CNN不是\ textit {proved} 1- lipschitz。 In this work, we reduce this gap by introducing (a) a procedure to certify robustness of 1-Lipschitz CNNs by replacing the last linear layer with a 1-hidden layer MLP that significantly improves their performance for both standard and provably robust accuracy, (b) a method to significantly reduce the training time per epoch for Skew Orthogonal Convolution (SOC) layers (>30\% reduction for deeper networks) and （c）使用$ l_ {2} $输入到歧管的距离的数学属性的一类合并层是1-lipschitz。使用这些方法，我们在CIFAR-10上（+1.79 \％和+3.82 \％）上的标准且可证明的鲁棒精度可显着提高所有网络（+3.78 \％\％和+4.75 \％）的标准和可证明的鲁棒精度。代码可在\ url {https://github.com/singlasahil14/impreved_l2_robustness}获得。
Training convolutional neural networks (CNNs) with a strict 1-Lipschitz constraint under the $l_{2}$ norm is useful for adversarial robustness, interpretable gradients and stable training. 1-Lipschitz CNNs are usually designed by enforcing each layer to have an orthogonal Jacobian matrix (for all inputs) to prevent the gradients from vanishing during backpropagation. However, their performance often significantly lags behind that of heuristic methods to enforce Lipschitz constraints where the resulting CNN is not \textit{provably} 1-Lipschitz. In this work, we reduce this gap by introducing (a) a procedure to certify robustness of 1-Lipschitz CNNs by replacing the last linear layer with a 1-hidden layer MLP that significantly improves their performance for both standard and provably robust accuracy, (b) a method to significantly reduce the training time per epoch for Skew Orthogonal Convolution (SOC) layers (>30\% reduction for deeper networks) and (c) a class of pooling layers using the mathematical property that the $l_{2}$ distance of an input to a manifold is 1-Lipschitz. Using these methods, we significantly advance the state-of-the-art for standard and provable robust accuracies on CIFAR-10 (gains of +1.79\% and +3.82\%) and similarly on CIFAR-100 (+3.78\% and +4.75\%) across all networks. Code is available at \url{https://github.com/singlasahil14/improved_l2_robustness}.
