论文标题
通过结构化先验，通过对抗训练改善分布的概括
Improving Out-of-Distribution Generalization by Adversarial Training with Structured Priors
论文作者
论文摘要
当数据分布与培训域中的数据不同时，深层模型通常无法在测试域中很好地概括。在解决这种分布（OOD）概括问题的众多方法中，人们对利用对抗性训练（AT）的兴趣越来越多，以提高OOD的性能。最近的工作表明，通过以样本进行样品获得的鲁棒模型也保留了对有偏见的测试域的可传递性。在本文中，我们从经验上表明，在OOD性能方面的样本方面的改善有限。具体而言，我们发现只能在较小的扰动范围内保持性能，而通用（UAT）对较大的扰动更为强大。这为我们提供了线索，即具有通用（低维）结构的对抗扰动可以增强对OOD场景中常见的大数据分布变化的鲁棒性。受此启发的启发，我们提出了两个具有低级别结构的变体，以训练OOD射击模型。在域基准测试基准上进行的广泛实验表明，我们提出的方法表现优于经验风险最小化（ERM）和样本。我们的代码可从https://github.com/novaglow646/nips22-mat-and-ldat-for-ood获得。
Deep models often fail to generalize well in test domains when the data distribution differs from that in the training domain. Among numerous approaches to address this Out-of-Distribution (OOD) generalization problem, there has been a growing surge of interest in exploiting Adversarial Training (AT) to improve OOD performance. Recent works have revealed that the robust model obtained by conducting sample-wise AT also retains transferability to biased test domains. In this paper, we empirically show that sample-wise AT has limited improvement on OOD performance. Specifically, we find that AT can only maintain performance at smaller scales of perturbation while Universal AT (UAT) is more robust to larger-scale perturbations. This provides us with clues that adversarial perturbations with universal (low dimensional) structures can enhance the robustness against large data distribution shifts that are common in OOD scenarios. Inspired by this, we propose two AT variants with low-rank structures to train OOD-robust models. Extensive experiments on DomainBed benchmark show that our proposed approaches outperform Empirical Risk Minimization (ERM) and sample-wise AT. Our code is available at https://github.com/NOVAglow646/NIPS22-MAT-and-LDAT-for-OOD.
