论文标题
移载：用于源分离网络的自适应梯度剪辑
AutoClip: Adaptive Gradient Clipping for Source Separation Networks
论文作者
论文摘要
剪切梯度是改善梯度下降的已知方法，但需要手工选择剪接阈值超级参数。我们提出了自动运输方法，这是一种根据训练期间观察到的梯度规范的历史，用于自动和自适应选择梯度剪辑阈值。实验结果表明，应用自载管的结果改善了音频源分离网络的概括性能。观察接受和不进行手术训练的分离网络的训练动力学表明，载体指导将优化到损失景观的一部分更平滑。载载非常易于实现，并且可以轻松地集成到跨多个域的各种应用程序中。
Clipping the gradient is a known approach to improving gradient descent, but requires hand selection of a clipping threshold hyperparameter. We present AutoClip, a simple method for automatically and adaptively choosing a gradient clipping threshold, based on the history of gradient norms observed during training. Experimental results show that applying AutoClip results in improved generalization performance for audio source separation networks. Observation of the training dynamics of a separation network trained with and without AutoClip show that AutoClip guides optimization into smoother parts of the loss landscape. AutoClip is very simple to implement and can be integrated readily into a variety of applications across multiple domains.
