论文标题
一般激活的随机神经网络中的对抗示例
Adversarial Examples in Random Neural Networks with General Activations
论文作者
论文摘要
大量的经验工作记录了深度学习模型中缺乏鲁棒性来进行对抗例子。最近的理论工作证明，对抗性示例在具有亚指数宽度和relu或平滑激活的两层网络中无处不在，以及具有子指数宽度的多层relu网络。我们提出了相同类型的结果，对宽度和一般局部Lipschitz的连续激活无限制。 更准确地说，给定一个神经网络$ f（\，\ cdot \ ,,; {\boldsymbolθ}）$，带有随机重量$ {\boldsymbolθ} $，并功能vector $ {\ boldsymbol x} $ $ \ nabla _ {\ boldsymbol x} f（{\ boldsymbol x}; {\boldsymbolθ}）$。我们的证明是基于高斯调节技术。我们不是在$ {\ boldsymbol x} $的附近$ f $表示近似线性，而是表征$ f（{\ boldsymbol x}; {\boldsymbolθ}}）$和$ f（{\ boldsymbol x}'; {\ boldsymbol x}'; {\ boldsymbol tuld $ $ tobol {\ boldsymbol x} -s（{\ boldsymbol x}）\ nabla _ {\ boldsymbol x} f（{\ boldsymbol x}; {\ boldsymbolths; {\boldsymbolθ}）$。
A substantial body of empirical work documents the lack of robustness in deep learning models to adversarial examples. Recent theoretical work proved that adversarial examples are ubiquitous in two-layers networks with sub-exponential width and ReLU or smooth activations, and multi-layer ReLU networks with sub-exponential width. We present a result of the same type, with no restriction on width and for general locally Lipschitz continuous activations. More precisely, given a neural network $f(\,\cdot\,;{\boldsymbol θ})$ with random weights ${\boldsymbol θ}$, and feature vector ${\boldsymbol x}$, we show that an adversarial example ${\boldsymbol x}'$ can be found with high probability along the direction of the gradient $\nabla_{\boldsymbol x}f({\boldsymbol x};{\boldsymbol θ})$. Our proof is based on a Gaussian conditioning technique. Instead of proving that $f$ is approximately linear in a neighborhood of ${\boldsymbol x}$, we characterize the joint distribution of $f({\boldsymbol x};{\boldsymbol θ})$ and $f({\boldsymbol x}';{\boldsymbol θ})$ for ${\boldsymbol x}' = {\boldsymbol x}-s({\boldsymbol x})\nabla_{\boldsymbol x}f({\boldsymbol x};{\boldsymbol θ})$.
