论文标题
学习机器人操纵的3D动态场景表示
Learning 3D Dynamic Scene Representations for Robot Manipulation
论文作者
论文摘要
机器人操纵的3D场景表示应捕获三个关键对象属性：永久性 - 随着时间的流逝而被阻塞的对象继续存在； Amodal完整性 - 即使只有部分观察值，对象具有3D占有率；时空连续性 - 每个物体的运动在空间和时间上是连续的。在本文中，我们介绍了3D动态场景表示（DSR），这是一个3D体积的场景表示形式，同时发现，跟踪，重建对象并预测其动态，同时捕获所有三个属性。我们进一步提出了DSR-NET，该数据学会在多个相互作用上汇总视觉观察，以逐渐构建和完善DSR。我们的模型在模拟和真实数据上都使用DSR建模3D场景动力学实现了最新的性能。结合模型预测性控制，DSR-NET可以在下游机器人操纵任务（例如平面推动）中进行准确的计划。视频可从https://youtu.be/gqjyg3nqj80获得。
3D scene representation for robot manipulation should capture three key object properties: permanency -- objects that become occluded over time continue to exist; amodal completeness -- objects have 3D occupancy, even if only partial observations are available; spatiotemporal continuity -- the movement of each object is continuous over space and time. In this paper, we introduce 3D Dynamic Scene Representation (DSR), a 3D volumetric scene representation that simultaneously discovers, tracks, reconstructs objects, and predicts their dynamics while capturing all three properties. We further propose DSR-Net, which learns to aggregate visual observations over multiple interactions to gradually build and refine DSR. Our model achieves state-of-the-art performance in modeling 3D scene dynamics with DSR on both simulated and real data. Combined with model predictive control, DSR-Net enables accurate planning in downstream robotic manipulation tasks such as planar pushing. Video is available at https://youtu.be/GQjYG3nQJ80.
