论文标题
在深度强化学习中塑造动作空间
Action Space Shaping in Deep Reinforcement Learning
论文作者
论文摘要
强化学习（RL）在包括视频游戏在内的各种学习环境中的培训代理方面已经成功。但是，此类工作会从游戏的原始作品中修改并缩小动作空间。这是为了避免尝试“毫无意义”的动作并简化实施。目前，这主要是基于直觉完成的，很少有系统的研究支持设计决策。在这项工作中，我们旨在通过在视频游戏环境中进行广泛的实验来了解这些动作空间修改。我们的结果表明，特定于领域的动作的去除和连续行动的离散化对于成功学习至关重要。有了这些见解，我们希望通过澄清哪些动作空间易于学习，以减轻在新环境中的使用。
Reinforcement learning (RL) has been successful in training agents in various learning environments, including video-games. However, such work modifies and shrinks the action space from the game's original. This is to avoid trying "pointless" actions and to ease the implementation. Currently, this is mostly done based on intuition, with little systematic research supporting the design decisions. In this work, we aim to gain insight on these action space modifications by conducting extensive experiments in video-game environments. Our results show how domain-specific removal of actions and discretization of continuous actions can be crucial for successful learning. With these insights, we hope to ease the use of RL in new environments, by clarifying what action-spaces are easy to learn.
