论文标题
超级网络的进化神经级联搜索
Evolutionary Neural Cascade Search across Supernetworks
论文作者
论文摘要
为了通过现代神经网络实现出色的性能，拥有正确的网络体系结构很重要。神经体系结构搜索（NAS）涉及自动发现特定于任务的网络体系结构。现代NAS接近利用子网编码候选神经网络体系结构的超级网络。这些子网络可以同时训练，从而消除了从头开始训练每个网络的需求，从而提高了NAS的效率。一种称为神经结构转移（NAT）的最新方法进一步提高了NAS对计算机视觉任务的效率，它通过使用多目标进化算法找到了在Imagenet上预处理的超级网络的高质量子网络。在NAT的基础上，我们引入了Encas-进化神经级联搜索。 Encas可用于搜索多个预审预告片的超级网，以实现不同神经网络体系结构级联的权衡方面，从而最大程度地提高了精度，同时最大程度地减少了拖鞋的数量。我们在常见的计算机视觉基准测试（CIFAR-10，CIFAR-100，Imagenet）上测试Encas，并在先前最先前的NAS模型上实现帕累托优势。此外，在518个公开的成像网分类器池中应用Encas会导致所有计算方案中的帕累托优势，并将最大准确性从88.6％提高到89.0％，伴随着362 GFLOPS的计算工作减少18％。我们的代码可从https://github.com/awesomelemon/encas获得
To achieve excellent performance with modern neural networks, having the right network architecture is important. Neural Architecture Search (NAS) concerns the automatic discovery of task-specific network architectures. Modern NAS approaches leverage supernetworks whose subnetworks encode candidate neural network architectures. These subnetworks can be trained simultaneously, removing the need to train each network from scratch, thereby increasing the efficiency of NAS. A recent method called Neural Architecture Transfer (NAT) further improves the efficiency of NAS for computer vision tasks by using a multi-objective evolutionary algorithm to find high-quality subnetworks of a supernetwork pretrained on ImageNet. Building upon NAT, we introduce ENCAS - Evolutionary Neural Cascade Search. ENCAS can be used to search over multiple pretrained supernetworks to achieve a trade-off front of cascades of different neural network architectures, maximizing accuracy while minimizing FLOPs count. We test ENCAS on common computer vision benchmarks (CIFAR-10, CIFAR-100, ImageNet) and achieve Pareto dominance over previous state-of-the-art NAS models up to 1.5 GFLOPs. Additionally, applying ENCAS to a pool of 518 publicly available ImageNet classifiers leads to Pareto dominance in all computation regimes and to increasing the maximum accuracy from 88.6% to 89.0%, accompanied by an 18\% decrease in computation effort from 362 to 296 GFLOPs. Our code is available at https://github.com/AwesomeLemon/ENCAS
