论文标题
因果关系坎贝尔·戈德哈特（Campbell-Goodhart）的法律和强化学习
Causal Campbell-Goodhart's law and Reinforcement Learning
论文作者
论文摘要
坎贝尔·古德哈特（Campbell-Goodhart）的定律与因果推断错误有关，决策代理的目的是影响与目标目标相关但并不能可靠地引起它的变量。这是经济学和政治学上的众所周知的错误，但在人工智能研究中没有广泛标记。通过一个简单的例子，我们展示了现成的深度加固学习（RL）算法不一定不受此认知错误的影响。非政策学习方法是被欺骗的，而在上政策的方法却没有。实际的含义是，将RL幼稚地应用于复杂的现实生活问题可能会导致人类犯的相同类型的政策错误。应该仔细考虑理解基于增强学习的解决方案的因果模型。
Campbell-Goodhart's law relates to the causal inference error whereby decision-making agents aim to influence variables which are correlated to their goal objective but do not reliably cause it. This is a well known error in Economics and Political Science but not widely labelled in Artificial Intelligence research. Through a simple example, we show how off-the-shelf deep Reinforcement Learning (RL) algorithms are not necessarily immune to this cognitive error. The off-policy learning method is tricked, whilst the on-policy method is not. The practical implication is that naive application of RL to complex real life problems can result in the same types of policy errors that humans make. Great care should be taken around understanding the causal model that underpins a solution derived from Reinforcement Learning.
