论文标题
auto3d：通过难以理解的变分观点和全局3D表示，新颖的视图综合
AUTO3D: Novel view synthesis through unsupervisely learned variational viewpoint and global 3D representation
论文作者
论文摘要
本文针对基于学习的新视图从单个或有限的2D图像中进行构成，而无需姿势监督。在以观看者为中心的坐标中，我们构建了一个端到端的可训练条件变异框架，以消除不可忽视的相对姿势/旋转和隐式全局3D表示（形状，纹理和以观看者为中心的坐标等的起源等）。 3D对象的全局外观是通过从任何数量的观点拍摄的几个外观描述的图像给出的。我们的空间相关模块以排列不变的方式从外观描述的图像中提取全局3D表示。我们的系统可以在不明确3D重建的情况下获得隐式的3D理解。借助毫无疑问的以观看者为中心的相对档案/旋转代码，解码器可以通过在先前的分布中采样相对订单来连续地幻觉新的视图。在各种应用中，我们证明我们的模型可以比姿势/3D模型监督学习的新型视图合成（NVS）方法获得可比较甚至更好的结果，并具有任何数量的输入视图。
This paper targets on learning-based novel view synthesis from a single or limited 2D images without the pose supervision. In the viewer-centered coordinates, we construct an end-to-end trainable conditional variational framework to disentangle the unsupervisely learned relative-pose/rotation and implicit global 3D representation (shape, texture and the origin of viewer-centered coordinates, etc.). The global appearance of the 3D object is given by several appearance-describing images taken from any number of viewpoints. Our spatial correlation module extracts a global 3D representation from the appearance-describing images in a permutation invariant manner. Our system can achieve implicitly 3D understanding without explicitly 3D reconstruction. With an unsupervisely learned viewer-centered relative-pose/rotation code, the decoder can hallucinate the novel view continuously by sampling the relative-pose in a prior distribution. In various applications, we demonstrate that our model can achieve comparable or even better results than pose/3D model-supervised learning-based novel view synthesis (NVS) methods with any number of input views.
