论文标题
FedIC：通过校准蒸馏进行非IID和长尾数据的联合学习
FEDIC: Federated Learning on Non-IID and Long-Tailed Data via Calibrated Distillation
论文作者
论文摘要
联合学习提供了一种隐私保证，可以通过不同类型的数据为分布式客户生成良好的深度学习模型。然而，处理非IID数据是联邦学习最具挑战性的问题之一。研究人员提出了各种方法来消除非IID的负面影响。但是，他们只专注于非IID数据，规定普遍的类别分布是平衡的。在许多实际应用中，通用类分布是长尾分配，这会导致模型严重偏见。因此，本文研究了联邦学习中非IID和长尾数据的联合问题，并提出了一种称为联邦合奏蒸馏的相应解决方案，并具有不平衡校准（FEDIC）。为了处理非IID数据，FedIC使用模型集合来利用对非IID数据训练的模型的多样性。然后，提出了一种具有logit调整和校准门控网络的新蒸馏方法，以有效地解决长尾问题。与联合学习和长尾学习的最新方法相比，我们在CIFAR-10-LT，CIFAR-100-LT和Imagenet-LT上评估了FedIC。我们的代码可在https://github.com/shangxinyi/fedic上找到。
Federated learning provides a privacy guarantee for generating good deep learning models on distributed clients with different kinds of data. Nevertheless, dealing with non-IID data is one of the most challenging problems for federated learning. Researchers have proposed a variety of methods to eliminate the negative influence of non-IIDness. However, they only focus on the non-IID data provided that the universal class distribution is balanced. In many real-world applications, the universal class distribution is long-tailed, which causes the model seriously biased. Therefore, this paper studies the joint problem of non-IID and long-tailed data in federated learning and proposes a corresponding solution called Federated Ensemble Distillation with Imbalance Calibration (FEDIC). To deal with non-IID data, FEDIC uses model ensemble to take advantage of the diversity of models trained on non-IID data. Then, a new distillation method with logit adjustment and calibration gating network is proposed to solve the long-tail problem effectively. We evaluate FEDIC on CIFAR-10-LT, CIFAR-100-LT, and ImageNet-LT with a highly non-IID experimental setting, in comparison with the state-of-the-art methods of federated learning and long-tail learning. Our code is available at https://github.com/shangxinyi/FEDIC.
