论文标题
在有自我寻求人类的情况下重新思考安全的控制
Rethinking Safe Control in the Presence of Self-Seeking Humans
论文作者
论文摘要
即使在最严重的人类不确定性中，安全的控制方法通常旨在安全地行事。但是，人类可能会利用这样的安全优先系统，从而为每个人带来更大的风险。尽管具有重要意义，但先前的工作尚未调查并解释了安全控制中的这些因素。在本文中，我们利用了从游戏理论到建模人类短视，寻求自我寻求行为的基于交互的回报结构，以及人类如何根据先前的经验改变对机器的策略。我们将这种战略性的人类行为纳入安全控制架构。结果，与确定性最差的安全控制技术和基于平衡的随机方法相比，我们的方法可以实现更好的安全性和性能权衡。我们的发现表明，迫切需要重新考虑在人类技术互动中使用的安全控制框架，以追求所有人的更大安全。
Safe control methods are often intended to behave safely even in worst-case human uncertainties. However, humans may exploit such safety-first systems, which results in greater risk for everyone. Despite their significance, no prior work has investigated and accounted for such factors in safe control. In this paper, we leverage an interaction-based payoff structure from game theory to model humans' short-sighted, self-seeking behaviors and how humans change their strategies toward machines based on prior experience. We integrate such strategic human behaviors into a safe control architecture. As a result, our approach achieves better safety and performance trade-offs when compared to both deterministic worst-case safe control techniques and equilibrium-based stochastic methods. Our findings suggest an urgent need to fundamentally rethink the safe control framework used in human-technology interaction in pursuit of greater safety for all.
