论文标题
深度神经网络的集合工作流程从超参数搜索到推理GPU群集的推理
A Deep Neural Networks ensemble workflow from hyperparameter search to inference leveraging GPU clusters
论文作者
论文摘要
结合（或带有结合）的自动化机器学习试图自动构建深度神经网络（DNNS）的合奏，以实现定性的预测。众所周知，DNN的合奏避免了过度拟合，但它们是记忆和耗时的方法。因此，理想的汽车将在一次运行时间内产生有关准确性和推理速度的不同集合。尽管以前的Automl专注于搜索最佳模型以最大化其概括能力，但我们宁愿提出一个新的Automl，以构建一个较大的精确和多样化的单个模型的库，以构建合奏。首先，我们的广泛基准显示出异步超频带是一种有效且可靠的方法，可以构建大量不同的模型来组合它们。然后，提出了一种基于多目标贪婪算法的新合奏选择方法，以通过控制其计算成本来生成准确的合奏。最后，我们提出了一种基于分配优化的GPU群集中DNNS集合的推断，以优化DNNS集合的推断。使用集合方法产生的AutoML在训练阶段和推理阶段都使用有效的GPU簇在两个数据集上显示出强大的结果。
Automated Machine Learning with ensembling (or AutoML with ensembling) seeks to automatically build ensembles of Deep Neural Networks (DNNs) to achieve qualitative predictions. Ensemble of DNNs are well known to avoid over-fitting but they are memory and time consuming approaches. Therefore, an ideal AutoML would produce in one single run time different ensembles regarding accuracy and inference speed. While previous works on AutoML focus to search for the best model to maximize its generalization ability, we rather propose a new AutoML to build a larger library of accurate and diverse individual models to then construct ensembles. First, our extensive benchmarks show asynchronous Hyperband is an efficient and robust way to build a large number of diverse models to combine them. Then, a new ensemble selection method based on a multi-objective greedy algorithm is proposed to generate accurate ensembles by controlling their computing cost. Finally, we propose a novel algorithm to optimize the inference of the DNNs ensemble in a GPU cluster based on allocation optimization. The produced AutoML with ensemble method shows robust results on two datasets using efficiently GPU clusters during both the training phase and the inference phase.
