论文标题
自适应联合优化
Adaptive Federated Optimization
论文作者
论文摘要
联合学习是一个分布式的机器学习范式，其中大量客户与中央服务器协调以学习模型而无需共享自己的培训数据。标准联合优化方法（例如联邦平均（FedAvg））通常很难调节并表现出不利的收敛行为。在未赋予的设置中，自适应优化方法在打击此类问题方面取得了显着成功。在这项工作中，我们提出了适应性优化器的联合版本，包括Adagrad，Adam和Yogi，并在存在异质数据的情况下分析其收敛性，以进行一般的非凸面设置。我们的结果突出了客户异质性和沟通效率之间的相互作用。我们还对这些方法进行了广泛的实验，并表明自适应优化器的使用可以显着提高联合学习的性能。
Federated learning is a distributed machine learning paradigm in which a large number of clients coordinate with a central server to learn a model without sharing their own training data. Standard federated optimization methods such as Federated Averaging (FedAvg) are often difficult to tune and exhibit unfavorable convergence behavior. In non-federated settings, adaptive optimization methods have had notable success in combating such issues. In this work, we propose federated versions of adaptive optimizers, including Adagrad, Adam, and Yogi, and analyze their convergence in the presence of heterogeneous data for general non-convex settings. Our results highlight the interplay between client heterogeneity and communication efficiency. We also perform extensive experiments on these methods and show that the use of adaptive optimizers can significantly improve the performance of federated learning.
