论文标题
重新思考空间自适应的归一化
Rethinking Spatially-Adaptive Normalization
论文作者
论文摘要
最近在有条件的语义图像合成中，空间自适应的归一化取得了非常成功的范围，该综合方法通过从语义布局中学到的空间变化的转换来调节标准化的激活，以保护语义信息免于被冲走。尽管表现令人印象深刻，但仍然需要对框内的真实优势有更深入的了解，以帮助减少这些新结构引入的重要计算和参数开销。在本文中，从回报的角度来看，我们对Spade的有效性进行了深入的分析，并观察到其优势实际上主要来自其语义意识，而不是空间适应性。受这一点的启发，我们提出了类自适应归一化（进化枝），这是一种不适合空间位置或布局的轻量级变体。从这种设计中受益，进化枝大大降低了计算成本，同时仍然能够保留一代人的语义信息。在多个挑战性数据集上进行了广泛的实验表明，尽管由此产生的保真度与Spade相当，但其开销比Spade便宜得多。以ADE20K数据集的生成器为例，进化枝介绍的额外参数和计算成本仅为4.57％和0.07％，而Spade的发电机成本分别为39.21％和234.73％。
Spatially-adaptive normalization is remarkably successful recently in conditional semantic image synthesis, which modulates the normalized activation with spatially-varying transformations learned from semantic layouts, to preserve the semantic information from being washed away. Despite its impressive performance, a more thorough understanding of the true advantages inside the box is still highly demanded, to help reduce the significant computation and parameter overheads introduced by these new structures. In this paper, from a return-on-investment point of view, we present a deep analysis of the effectiveness of SPADE and observe that its advantages actually come mainly from its semantic-awareness rather than the spatial-adaptiveness. Inspired by this point, we propose class-adaptive normalization (CLADE), a lightweight variant that is not adaptive to spatial positions or layouts. Benefited from this design, CLADE greatly reduces the computation cost while still being able to preserve the semantic information during the generation. Extensive experiments on multiple challenging datasets demonstrate that while the resulting fidelity is on par with SPADE, its overhead is much cheaper than SPADE. Take the generator for ADE20k dataset as an example, the extra parameter and computation cost introduced by CLADE are only 4.57% and 0.07% while that of SPADE are 39.21% and 234.73% respectively.
