论文标题
改善神经网络概括的杂交损失功能
Hybridised Loss Functions for Improved Neural Network Generalisation
论文作者
论文摘要
损失功能在人工神经网络（ANN）的培训中起着重要作用，并且可以影响ANN模型的普遍化能力以及其他属性。具体而言，已经表明，交叉熵和总和平方误差损失函数会导致不同的训练动力学，并且具有彼此互补的不同特性。以前已经提出，熵和总和误差损失函数的混合物可以结合两个函数的优势，同时限制其缺点。在这项研究中研究了这种混合损失功能的有效性。结果表明，两个损失函数的杂交提高了ANN在所有考虑的问题上的概括能力。从总和平方误差损失函数开始训练的混合损耗函数，后来切换到跨熵误差损失函数的平均表现最佳，或者与所考虑的所有问题所测试的最佳损耗函数无显着差异。这项研究表明，通过切换到跨熵误差损失函数，可以进一步利用由总和误差损耗函数发现的最小值。因此可以得出结论，两种损失函数的杂交可能会导致ANN的表现更好。
Loss functions play an important role in the training of artificial neural networks (ANNs), and can affect the generalisation ability of the ANN model, among other properties. Specifically, it has been shown that the cross entropy and sum squared error loss functions result in different training dynamics, and exhibit different properties that are complementary to one another. It has previously been suggested that a hybrid of the entropy and sum squared error loss functions could combine the advantages of the two functions, while limiting their disadvantages. The effectiveness of such hybrid loss functions is investigated in this study. It is shown that hybridisation of the two loss functions improves the generalisation ability of the ANNs on all problems considered. The hybrid loss function that starts training with the sum squared error loss function and later switches to the cross entropy error loss function is shown to either perform the best on average, or to not be significantly different than the best loss function tested for all problems considered. This study shows that the minima discovered by the sum squared error loss function can be further exploited by switching to cross entropy error loss function. It can thus be concluded that hybridisation of the two loss functions could lead to better performance in ANNs.
