论文标题
对违规图像到图像翻译的对比度学习
Contrastive Learning for Unpaired Image-to-Image Translation
论文作者
论文摘要
在图像到图像翻译中，输出中的每个贴片应反映输入中相应贴片的内容，而不是域。我们建议使用基于对比度学习的框架，提出了一种直接的方法 - 最大化两者之间的相互信息。该方法鼓励两个元素（相应的补丁）将相对于数据集中的其他元素（其他补丁）映射到学到的特征空间中的相似点，称为负面。我们探索了几种关键的设计选择，以使对比度学习在图像合成设置中有效。值得注意的是，我们使用一种基于补丁的方法，而不是在整个图像上操作。此外，我们从输入图像本身而不是从数据集中绘制负面因素。我们证明，我们的框架可以在未配对的图像到图像翻译设置中实现单方面的翻译，同时提高质量并减少训练时间。此外，我们的方法甚至可以扩展到每个“域”只是一个图像的训练设置。
In image-to-image translation, each patch in the output should reflect the content of the corresponding patch in the input, independent of domain. We propose a straightforward method for doing so -- maximizing mutual information between the two, using a framework based on contrastive learning. The method encourages two elements (corresponding patches) to map to a similar point in a learned feature space, relative to other elements (other patches) in the dataset, referred to as negatives. We explore several critical design choices for making contrastive learning effective in the image synthesis setting. Notably, we use a multilayer, patch-based approach, rather than operate on entire images. Furthermore, we draw negatives from within the input image itself, rather than from the rest of the dataset. We demonstrate that our framework enables one-sided translation in the unpaired image-to-image translation setting, while improving quality and reducing training time. In addition, our method can even be extended to the training setting where each "domain" is only a single image.
