论文标题
司额对抗训练算法
A Sublinear Adversarial Training Algorithm
论文作者
论文摘要
对抗训练是一种广泛使用的策略，可以使神经网络具有抵抗对抗性扰动的能力。对于宽度$ m $，$ n $输入培训数据的神经网络，在$ d $ dimension中，需要$ω（MND）$每次培训迭代的时间费用来进行前进和向后计算。在本文中，我们分析了具有转移的Relu激活的两层神经网络上对抗训练程序的收敛保证，并表明每次迭代的每个输入数据都只能激活$ O（M）$神经元。此外，通过应用半空间报告数据结构，我们开发了一种用于对抗培训的算法，以$ O（m n d）$ $ O（m n d）$。
Adversarial training is a widely used strategy for making neural networks resistant to adversarial perturbations. For a neural network of width $m$, $n$ input training data in $d$ dimension, it takes $Ω(mnd)$ time cost per training iteration for the forward and backward computation. In this paper we analyze the convergence guarantee of adversarial training procedure on a two-layer neural network with shifted ReLU activation, and shows that only $o(m)$ neurons will be activated for each input data per iteration. Furthermore, we develop an algorithm for adversarial training with time cost $o(m n d)$ per iteration by applying half-space reporting data structure.
