论文标题
变压器的鲁棒性验证
Robustness Verification for Transformers
论文作者
论文摘要
旨在正式证明神经网络的预测行为的鲁棒性验证已成为理解模型行为并获得安全保证的重要工具。但是，以前的方法通常只能处理具有相对简单架构的神经网络。在本文中，我们考虑了变压器的鲁棒性验证问题。变形金刚具有复杂的自我发场层，对验证构成了许多挑战，包括跨非线性和交叉位置依赖性，这在先前的工作中尚未讨论过。我们解决了这些挑战，并为变压器开发了第一个鲁棒性验证算法。通过我们的方法计算出的认证鲁棒性界限比通过天真的间隔结合传播更紧。这些界限还阐明了解释变压器，因为它们始终反映了情感分析中不同单词的重要性。
Robustness verification that aims to formally certify the prediction behavior of neural networks has become an important tool for understanding model behavior and obtaining safety guarantees. However, previous methods can usually only handle neural networks with relatively simple architectures. In this paper, we consider the robustness verification problem for Transformers. Transformers have complex self-attention layers that pose many challenges for verification, including cross-nonlinearity and cross-position dependency, which have not been discussed in previous works. We resolve these challenges and develop the first robustness verification algorithm for Transformers. The certified robustness bounds computed by our method are significantly tighter than those by naive Interval Bound Propagation. These bounds also shed light on interpreting Transformers as they consistently reflect the importance of different words in sentiment analysis.
