论文标题
通过从2D模型转移的知识数据效率的3D学习者
Data Efficient 3D Learner via Knowledge Transferred from 2D Model
论文作者
论文摘要
收集和标记注册的3D点云是昂贵的。结果，与2D图像对应物相比，用于培训的3D资源通常受到限制。在这项工作中，我们通过通过RGB-D图像从强2D模型转移知识来应对3D任务的数据稀缺挑战。具体而言，我们利用一个强大且训练有素的语义分割模型对2D图像来增强使用伪标签的RGB-D图像。然后，增强数据集可用于预先训练3D模型。最后，通过简单地对一些标记的3D实例进行微调，我们的方法已经超过了针对3D标签效率而定制的现有最先进的方法。我们还表明，我们的预训练可以改善均值老师和熵最小化的结果，这表明转移的知识在半监督的环境中很有帮助。我们验证方法对两个流行的3D模型和三个不同任务的有效性。在扫描仪官方评估中，我们在数据效率的轨道上建立了新的最新语义细分结果。
Collecting and labeling the registered 3D point cloud is costly. As a result, 3D resources for training are typically limited in quantity compared to the 2D images counterpart. In this work, we deal with the data scarcity challenge of 3D tasks by transferring knowledge from strong 2D models via RGB-D images. Specifically, we utilize a strong and well-trained semantic segmentation model for 2D images to augment RGB-D images with pseudo-label. The augmented dataset can then be used to pre-train 3D models. Finally, by simply fine-tuning on a few labeled 3D instances, our method already outperforms existing state-of-the-art that is tailored for 3D label efficiency. We also show that the results of mean-teacher and entropy minimization can be improved by our pre-training, suggesting that the transferred knowledge is helpful in semi-supervised setting. We verify the effectiveness of our approach on two popular 3D models and three different tasks. On ScanNet official evaluation, we establish new state-of-the-art semantic segmentation results on the data-efficient track.
