论文标题
走向领域 - 不足的对比学习
Towards Domain-Agnostic Contrastive Learning
论文作者
论文摘要
尽管最近取得了成功，但大多数对比度的自我监督学习方法都是特定于领域的，严重依赖于需要有关特定领域的知识的数据增强技术，例如图像裁剪和旋转。为了克服这种局限性，我们提出了一种新型的域 - 无知方法来进行对比学习，名为DACL，该方法适用于不太容易获得的势头，因此数据增强技术不易获得。我们方法的关键是使用混合噪声来创建相似和不同的示例，通过在输入或隐藏状态级别上以不同的方式混合数据样本。为了证明DACL的有效性，我们在各个领域（例如表格数据，图像和图形）进行实验。我们的结果表明，DACL不仅胜过其他域 - 不合时宜的no噪声方法，例如高斯 - 噪声，而且还可以与诸如SIMCLR之类的域特异性方法（例如SIMCLR）结合在一起，以改善自我监督的视觉表示学习。最后，我们理论上分析了我们的方法，并显示出比基于高斯噪声的对比学习方法的优势。
Despite recent success, most contrastive self-supervised learning methods are domain-specific, relying heavily on data augmentation techniques that require knowledge about a particular domain, such as image cropping and rotation. To overcome such limitation, we propose a novel domain-agnostic approach to contrastive learning, named DACL, that is applicable to domains where invariances, and thus, data augmentation techniques, are not readily available. Key to our approach is the use of Mixup noise to create similar and dissimilar examples by mixing data samples differently either at the input or hidden-state levels. To demonstrate the effectiveness of DACL, we conduct experiments across various domains such as tabular data, images, and graphs. Our results show that DACL not only outperforms other domain-agnostic noising methods, such as Gaussian-noise, but also combines well with domain-specific methods, such as SimCLR, to improve self-supervised visual representation learning. Finally, we theoretically analyze our method and show advantages over the Gaussian-noise based contrastive learning approach.
