论文标题
模仿sindhorn距离学习
Imitation Learning with Sinkhorn Distances
论文作者
论文摘要
模仿学习算法已被解释为差异最小化问题的变体。比较专家和学习者之间的入住措施的能力对于他们从演示中学习的有效性至关重要。在本文中，我们通过将模仿学习作为最小化占用度量之间的距离距离来介绍可进行的解决方案。该公式结合了最佳运输指标的宝贵特性，将非重叠分布与在对手学习的特征空间中定义的余弦距离成本进行比较。这导致了高度歧视性的评论家网络和最佳运输计划，随后指导模仿学习。我们使用奖励度量标准和sindhorn距离度量标准评估了所提出的方法。有关实现和复制结果，请参考以下存储库https://github.com/gpapagiannis/sinkhorn-imitation。
Imitation learning algorithms have been interpreted as variants of divergence minimization problems. The ability to compare occupancy measures between experts and learners is crucial in their effectiveness in learning from demonstrations. In this paper, we present tractable solutions by formulating imitation learning as minimization of the Sinkhorn distance between occupancy measures. The formulation combines the valuable properties of optimal transport metrics in comparing non-overlapping distributions with a cosine distance cost defined in an adversarially learned feature space. This leads to a highly discriminative critic network and optimal transport plan that subsequently guide imitation learning. We evaluate the proposed approach using both the reward metric and the Sinkhorn distance metric on a number of MuJoCo experiments. For the implementation and reproducing results please refer to the following repository https://github.com/gpapagiannis/sinkhorn-imitation.
