论文标题
NAS：预测辅助进化神经结构搜索
PRE-NAS: Predictor-assisted Evolutionary Neural Architecture Search
论文作者
论文摘要
神经架构搜索（NAS）旨在在神经网络中自动化建筑工程。这通常需要一个高的计算开销来评估搜索过程中搜索空间中所有可能网络的集合。通过减轻评估每个候选网络的需求，对网络性能的预测可以减轻这一高度计算开销。开发这样的预测变量通常需要大量的评估架构，这些架构可能难以获得。我们通过提出一种新型基于进化的NAS策略，预测辅助的E-NAS（NAS）来应对这一挑战，即使在评估的架构中，它也可以很好地表现。 NAS前利用新的进化搜索策略，并整合了几代人的高保真权重继承。与一声策略不同，由于体重分担而可能遭受评估偏见，NAS的后代候选人在拓扑上是同质的，这会避免偏见并导致更准确的预测。 NAS-Bench-201和DARTS搜索空间的广泛实验表明，NAS可以超越最先进的NAS方法。只有单个GPU搜索0.6天，NAS可以找到竞争性架构，该体系结构可在CIFAR-10和Imagenet上分别达到2.40％和24％的测试错误率。
Neural architecture search (NAS) aims to automate architecture engineering in neural networks. This often requires a high computational overhead to evaluate a number of candidate networks from the set of all possible networks in the search space during the search. Prediction of the networks' performance can alleviate this high computational overhead by mitigating the need for evaluating every candidate network. Developing such a predictor typically requires a large number of evaluated architectures which may be difficult to obtain. We address this challenge by proposing a novel evolutionary-based NAS strategy, Predictor-assisted E-NAS (PRE-NAS), which can perform well even with an extremely small number of evaluated architectures. PRE-NAS leverages new evolutionary search strategies and integrates high-fidelity weight inheritance over generations. Unlike one-shot strategies, which may suffer from bias in the evaluation due to weight sharing, offspring candidates in PRE-NAS are topologically homogeneous, which circumvents bias and leads to more accurate predictions. Extensive experiments on NAS-Bench-201 and DARTS search spaces show that PRE-NAS can outperform state-of-the-art NAS methods. With only a single GPU searching for 0.6 days, competitive architecture can be found by PRE-NAS which achieves 2.40% and 24% test error rates on CIFAR-10 and ImageNet respectively.
