论文标题
用于内容感知神经视频传递的有效元调整
Efficient Meta-Tuning for Content-aware Neural Video Delivery
论文作者
论文摘要
最近，深层神经网络（DNN）用于减少带宽并提高互联网视频传递的质量。现有的方法训练服务器上每个视频块的相应内容超级分辨率（SR）模型，并将低分辨率（LR）视频块以及SR模型一起流到客户端。尽管他们取得了令人鼓舞的结果，但网络培训的巨大计算成本限制了其实际应用。在本文中，我们提出了一种名为“有效元调节”（EMT）的方法，以降低计算成本。 EMT没有从头开始训练，而是将一个元学习的模型改编成输入视频的第一部分。至于以下块，它通过以前的改编模型的梯度掩盖选择了部分参数。为了实现EMT的进一步加速，我们提出了一种新颖的抽样策略，以从视频帧中提取最具挑战性的补丁。提出的策略高效，带来了可忽略的额外成本。我们的方法大大降低了计算成本并取得更好的性能，为将神经视频传递技术应用于实际应用铺平了道路。我们基于各种有效的SR架构进行了广泛的实验，包括ESPCN，SRCNN，FSRCNN和EDSR-1，证明了我们工作的概括能力。该代码通过\ url {https://github.com/neural-video-delivery/emt-pytorch-eccv2022}发布。
Recently, Deep Neural Networks (DNNs) are utilized to reduce the bandwidth and improve the quality of Internet video delivery. Existing methods train corresponding content-aware super-resolution (SR) model for each video chunk on the server, and stream low-resolution (LR) video chunks along with SR models to the client. Although they achieve promising results, the huge computational cost of network training limits their practical applications. In this paper, we present a method named Efficient Meta-Tuning (EMT) to reduce the computational cost. Instead of training from scratch, EMT adapts a meta-learned model to the first chunk of the input video. As for the following chunks, it fine-tunes the partial parameters selected by gradient masking of previous adapted model. In order to achieve further speedup for EMT, we propose a novel sampling strategy to extract the most challenging patches from video frames. The proposed strategy is highly efficient and brings negligible additional cost. Our method significantly reduces the computational cost and achieves even better performance, paving the way for applying neural video delivery techniques to practical applications. We conduct extensive experiments based on various efficient SR architectures, including ESPCN, SRCNN, FSRCNN and EDSR-1, demonstrating the generalization ability of our work. The code is released at \url{https://github.com/Neural-video-delivery/EMT-Pytorch-ECCV2022}.
