论文标题
梯度下降类型方法：背景和简单的统一收敛分析
Gradient Descent-Type Methods: Background and Simple Unified Convergence Analysis
论文作者
论文摘要
在本书章节中，我们简要描述构成梯度下降方法及其加速和随机变体的主要组成部分。我们旨在从数学角度来解释这些组件，包括理论和实际方面，但在基础上。我们将重点关注梯度下降方法的基本变体，然后将视图扩展到最近的变体，尤其是降低方差的随机梯度方案（SGD）。我们的方法依赖于揭示问题内部介绍的结构以及对目标函数的假设。我们的收敛分析统一了几个已知结果，并依赖于一般但基本的递归表达。我们已经对几种常见方案进行了分析。
In this book chapter, we briefly describe the main components that constitute the gradient descent method and its accelerated and stochastic variants. We aim at explaining these components from a mathematical point of view, including theoretical and practical aspects, but at an elementary level. We will focus on basic variants of the gradient descent method and then extend our view to recent variants, especially variance-reduced stochastic gradient schemes (SGD). Our approach relies on revealing the structures presented inside the problem and the assumptions imposed on the objective function. Our convergence analysis unifies several known results and relies on a general, but elementary recursive expression. We have illustrated this analysis on several common schemes.
