论文标题
CD $^2 $ -PFED：循环蒸馏引导的频道解耦，用于模型个性化
CD$^2$-pFed: Cyclic Distillation-guided Channel Decoupling for Model Personalization in Federated Learning
论文作者
论文摘要
联合学习（FL）是一个分布式学习范式，使多个客户能够协作学习共享的全球模型。尽管取得了最近的进展，但与异质数据客户端打交道仍然具有挑战性，因为差异数据分布通常会阻止全球模型为每个参与客户提供良好的概括能力。在本文中，我们提出了CD^2-PFED，这是一种新型的循环蒸馏引导的通道解耦框架，以在数据异质性的各种环境下个性化FL的全局模型。与以前建立层次个性化以克服不同客户端的非IID数据的工作不同，我们首次尝试通过渠道依次进行模型个性化分配，称为频道解耦。为了进一步促进私人和共享权重之间的协作，我们提出了一种新颖的环状蒸馏计划，以在联邦期间在本地和全球模型表示之间实施一致的正则化。在周期性蒸馏的指导下，我们的频道解耦框架可以为不同种类的异质性提供更准确和广泛的结果，例如特征偏斜，标签分布偏斜和概念转移。在包括自然图像和医学图像分析任务在内的四个基准测试的全面实验证明了我们方法对本地和外部验证的一致性。
Federated learning (FL) is a distributed learning paradigm that enables multiple clients to collaboratively learn a shared global model. Despite the recent progress, it remains challenging to deal with heterogeneous data clients, as the discrepant data distributions usually prevent the global model from delivering good generalization ability on each participating client. In this paper, we propose CD^2-pFed, a novel Cyclic Distillation-guided Channel Decoupling framework, to personalize the global model in FL, under various settings of data heterogeneity. Different from previous works which establish layer-wise personalization to overcome the non-IID data across different clients, we make the first attempt at channel-wise assignment for model personalization, referred to as channel decoupling. To further facilitate the collaboration between private and shared weights, we propose a novel cyclic distillation scheme to impose a consistent regularization between the local and global model representations during the federation. Guided by the cyclical distillation, our channel decoupling framework can deliver more accurate and generalized results for different kinds of heterogeneity, such as feature skew, label distribution skew, and concept shift. Comprehensive experiments on four benchmarks, including natural image and medical image analysis tasks, demonstrate the consistent effectiveness of our method on both local and external validations.
