论文标题
CPMLHO：通过切割平面和混合级优化的高参数调整
CPMLHO:Hyperparameter Tuning via Cutting Plane and Mixed-Level Optimization
论文作者
论文摘要
神经网络的高参数优化可以表示为双重优化问题。双光线优化用于自动更新高参数，而高参数的梯度是基于最佳响应函数的近似梯度。找到最佳响应功能非常耗时。在本文中，我们提出了CPMLHO，一种使用切割平面方法和混合级别目标函数的新的高参数优化方法。将切割平面添加到内层中，以限制响应函数的空间。为了获得更准确的高度级别，混合级别可以使用训练集和验证集的损失来灵活调整损失函数。与现有方法相比，实验结果表明，我们的方法可以自动更新训练过程中的超参数，并且可以找到具有更高准确性和更快收敛的更优质的超参数。
The hyperparameter optimization of neural network can be expressed as a bilevel optimization problem. The bilevel optimization is used to automatically update the hyperparameter, and the gradient of the hyperparameter is the approximate gradient based on the best response function. Finding the best response function is very time consuming. In this paper we propose CPMLHO, a new hyperparameter optimization method using cutting plane method and mixed-level objective function.The cutting plane is added to the inner layer to constrain the space of the response function. To obtain more accurate hypergradient,the mixed-level can flexibly adjust the loss function by using the loss of the training set and the verification set. Compared to existing methods, the experimental results show that our method can automatically update the hyperparameters in the training process, and can find more superior hyperparameters with higher accuracy and faster convergence.
