论文标题
混合现实中视频自我头像的实时以自我为中心的细分
Real Time Egocentric Segmentation for Video-self Avatar in Mixed Reality
论文作者
论文摘要
在这项工作中，我们介绍了我们的实时自我分割算法。由于我们在Thundernet的架构中启发的浅网络，我们的算法对于640x480的输入分辨率达到了66 fps的帧速率。此外，我们非常重视培训数据的可变性。更具体地说，我们描述了我们的中心体（Egobodies）数据集的创建过程，该数据集由来自三个数据集的近10,000张图像组成，这些图像既来自综合方法又创建。我们进行实验以了解各个数据集的贡献；比较用自我生物体训练的雷网模型，以更简单，更复杂的先前方法进行比较，并在分段质量和推理时间上以现实生活设置进行了相应的性能。所描述的经过训练的语义分割算法已经集成到混合现实的端到端系统中，使用户有可能在沉浸在MR场景中的同时看到自己的身体。
In this work we present our real-time egocentric body segmentation algorithm. Our algorithm achieves a frame rate of 66 fps for an input resolution of 640x480, thanks to our shallow network inspired in Thundernet's architecture. Besides, we put a strong emphasis on the variability of the training data. More concretely, we describe the creation process of our Egocentric Bodies (EgoBodies) dataset, composed of almost 10,000 images from three datasets, created both from synthetic methods and real capturing. We conduct experiments to understand the contribution of the individual datasets; compare Thundernet model trained with EgoBodies with simpler and more complex previous approaches and discuss their corresponding performance in a real-life setup in terms of segmentation quality and inference times. The described trained semantic segmentation algorithm is already integrated in an end-to-end system for Mixed Reality (MR), making it possible for users to see his/her own body while being immersed in a MR scene.
