论文标题
设计网络设计空间
Designing Network Design Spaces
论文作者
论文摘要
在这项工作中，我们提出了一个新的网络设计范式。我们的目标是帮助促进对网络设计的理解，并发现跨设置概括的设计原则。我们设计的网络设计空间不是专注于设计各个网络实例，该空间参数化网络群体。总体过程类似于网络的经典手动设计，但升至设计空间水平。使用我们的方法，我们探讨了网络设计的结构方面，并到达了一个低维的设计空间，该设计空间由我们称为Regnet的简单常规网络组成。 Regnet参数化的核心见解非常简单：可以通过量化的线性函数来解释良好网络的宽度和深度。我们分析了Regnet设计空间，并得出了有趣的发现，这些发现与当前的网络设计实践不符。 Regnet设计空间提供了简单快速的网络，这些网络在各种失败制度中都可以正常运行。在可比的训练设置和拖船下，Regnet模型在GPU上的速度快5倍，超过了流行的EfficityNet模型。
In this work, we present a new network design paradigm. Our goal is to help advance the understanding of network design and discover design principles that generalize across settings. Instead of focusing on designing individual network instances, we design network design spaces that parametrize populations of networks. The overall process is analogous to classic manual design of networks, but elevated to the design space level. Using our methodology we explore the structure aspect of network design and arrive at a low-dimensional design space consisting of simple, regular networks that we call RegNet. The core insight of the RegNet parametrization is surprisingly simple: widths and depths of good networks can be explained by a quantized linear function. We analyze the RegNet design space and arrive at interesting findings that do not match the current practice of network design. The RegNet design space provides simple and fast networks that work well across a wide range of flop regimes. Under comparable training settings and flops, the RegNet models outperform the popular EfficientNet models while being up to 5x faster on GPUs.
