论文标题
SWIN MAE：小数据集的蒙版自动编码器
Swin MAE: Masked Autoencoders for Small Datasets
论文作者
论文摘要
在医学图像分析中，深度学习模型的开发主要受到缺乏大型且通知的数据集的限制。无监督的学习不需要标签，更适合解决医学图像分析问题。但是，大多数当前无监督的学习方法都需要应用于大型数据集。为了使无监督的学习适用于小型数据集，我们提出了Swin Mae，它是Swin Transformer作为骨架的蒙版自动编码器。即使在只有几千张医学图像的数据集中，并且在不使用任何预训练的模型的情况下，Swin Mae仍然能够纯粹从图像中学习有用的语义功能。就下游任务的转移学习结果而言，它可以相等甚至略高于通过在Imagenet上训练的Swin Transformer获得的监督模型。该代码可在https://github.com/zian-xu/swin-mae上公开获取。
The development of deep learning models in medical image analysis is majorly limited by the lack of large-sized and well-annotated datasets. Unsupervised learning does not require labels and is more suitable for solving medical image analysis problems. However, most of the current unsupervised learning methods need to be applied to large datasets. To make unsupervised learning applicable to small datasets, we proposed Swin MAE, which is a masked autoencoder with Swin Transformer as its backbone. Even on a dataset of only a few thousand medical images and without using any pre-trained models, Swin MAE is still able to learn useful semantic features purely from images. It can equal or even slightly outperform the supervised model obtained by Swin Transformer trained on ImageNet in terms of the transfer learning results of downstream tasks. The code is publicly available at https://github.com/Zian-Xu/Swin-MAE.
