论文标题
TensorFlow重新分解中的非确定性
Non-Determinism in TensorFlow ResNets
论文作者
论文摘要
我们表明，训练重置中用于GPU的图像分类的随机性由GPU的非确定性主导，而不是由网络的权重和偏见的初始化或给定的小匹配的序列。测试集准确性的标准偏差为0.02，固定种子的标准偏差为0.027，而不同种子的标准偏差为0.027-近74％的重新网络模型的标准偏差是无确定性的。对于测试集损失，标准偏差的比率超过80 \％。这些结果要求对深度学习模型进行更强大的评估策略，因为跨运行结果的大量变化可能仅来自GPU随机性。
We show that the stochasticity in training ResNets for image classification on GPUs in TensorFlow is dominated by the non-determinism from GPUs, rather than by the initialisation of the weights and biases of the network or by the sequence of minibatches given. The standard deviation of test set accuracy is 0.02 with fixed seeds, compared to 0.027 with different seeds---nearly 74\% of the standard deviation of a ResNet model is non-deterministic. For test set loss the ratio of standard deviations is more than 80\%. These results call for more robust evaluation strategies of deep learning models, as a significant amount of the variation in results across runs can arise simply from GPU randomness.
