论文标题
通过基于能量的模型：联合培训和预培训对领域 - 反应半监督学习的经验研究
An empirical study of domain-agnostic semi-supervised learning via energy-based models: joint-training and pre-training
论文作者
论文摘要
一类最近的半监督学习（SSL）方法在很大程度上依赖于特定领域的数据增强。相比之下，生成的SSL方法涉及基于生成模型的无监督学习或通过联合培训或预训练，并且从成为域 - 不可思议的角度来看，因为它们本质上不需要固有的数据增强，因此更具吸引力。联合培训估计观测值和标签的联合分布，而预训练仅因观测值而进行。最近，基于能量的模型（EBM）取得了生成建模的有希望的结果。通过EBM进行SSL的联合培训，已经探索了不同数据方式的令人鼓舞的结果。在本文中，我们做出了两个贡献。首先，我们通过EBM探索用于SSL的预训练，并将其与联合培训进行比较。其次，对图像分类和自然语言标签的域进行了一系列实验，以使基于EBM基于EBM的SSL方法的性能进行现实。发现联合培训EBM的表现优于预训练EBM，但几乎始终如一。
A class of recent semi-supervised learning (SSL) methods heavily rely on domain-specific data augmentations. In contrast, generative SSL methods involve unsupervised learning based on generative models by either joint-training or pre-training, and are more appealing from the perspective of being domain-agnostic, since they do not inherently require data augmentations. Joint-training estimates the joint distribution of observations and labels, while pre-training is taken over observations only. Recently, energy-based models (EBMs) have achieved promising results for generative modeling. Joint-training via EBMs for SSL has been explored with encouraging results across different data modalities. In this paper, we make two contributions. First, we explore pre-training via EBMs for SSL and compare it to joint-training. Second, a suite of experiments are conducted over domains of image classification and natural language labeling to give a realistic whole picture of the performances of EBM based SSL methods. It is found that joint-training EBMs outperform pre-training EBMs marginally but nearly consistently.
