论文标题
QREG：量化的正则效应
QReg: On Regularization Effects of Quantization
论文作者
论文摘要
在本文中，我们研究了DNN培训中量化的影响。我们假设体重量化是正规化的一种形式，正则化量与量化水平（精度）相关。我们通过提供分析研究和经验结果来证实我们的假设。通过将重量量化为重量噪声的一种形式，我们探讨了该噪声在训练时如何通过网络传播。然后，我们表明该噪声的大小与量化水平相关。为了确认我们的分析研究，我们在本文中进行了广泛的实验列表，在本文中，我们表明量化的正则化效果可以在各种视觉任务和模型中看到各种数据集。基于我们的研究，我们建议8位量化在不同视力任务和模型中提供了可靠的正则化形式。
In this paper we study the effects of quantization in DNN training. We hypothesize that weight quantization is a form of regularization and the amount of regularization is correlated with the quantization level (precision). We confirm our hypothesis by providing analytical study and empirical results. By modeling weight quantization as a form of additive noise to weights, we explore how this noise propagates through the network at training time. We then show that the magnitude of this noise is correlated with the level of quantization. To confirm our analytical study, we performed an extensive list of experiments summarized in this paper in which we show that the regularization effects of quantization can be seen in various vision tasks and models, over various datasets. Based on our study, we propose that 8-bit quantization provides a reliable form of regularization in different vision tasks and models.
