论文标题
单眼动态视图综合：现实检查
Monocular Dynamic View Synthesis: A Reality Check
论文作者
论文摘要
我们研究了单眼视频的最新动态视图合成（DVS）进展。尽管现有的方法表现出了令人印象深刻的结果，但我们显示了实际捕获过程与现有实验协议之间的差异，这些协议在培训过程中有效地泄漏了多视图信号。我们定义有效的多视图因子（EMF），以量化基于相对摄像机运动的输入捕获序列中存在的多视图信号的量。我们介绍了两个新的指标：共同可见性掩盖图像指标和对应的准确性，它们克服了现有协议中的问题。我们还提出了一个新的iPhone数据集，其中包括更多样化的现实变形序列。使用我们提出的实验方案，我们表明，在对复杂运动进行建模时，最新的方法在没有多视图线索和4-5 dB下降的情况下观察到蒙版PSNR的1-2 dB下降。代码和数据可以在https://hangg7.com/dycheck上找到。
We study the recent progress on dynamic view synthesis (DVS) from monocular video. Though existing approaches have demonstrated impressive results, we show a discrepancy between the practical capture process and the existing experimental protocols, which effectively leaks in multi-view signals during training. We define effective multi-view factors (EMFs) to quantify the amount of multi-view signal present in the input capture sequence based on the relative camera-scene motion. We introduce two new metrics: co-visibility masked image metrics and correspondence accuracy, which overcome the issue in existing protocols. We also propose a new iPhone dataset that includes more diverse real-life deformation sequences. Using our proposed experimental protocol, we show that the state-of-the-art approaches observe a 1-2 dB drop in masked PSNR in the absence of multi-view cues and 4-5 dB drop when modeling complex motion. Code and data can be found at https://hangg7.com/dycheck.
