论文标题

深入强化学习,以防止智能电网保护,以防止协调的多阶段传输线攻击

Deep Reinforcement Learning for Smart Grid Protection Against Coordinated Multistage Transmission Line Attacks

论文作者

Yu, Liang, Gao, Zhen, Qin, Shuqi, Zhang, Meng, Shen, Chao, Guan, Xiaohong, Yue, Dong

论文摘要

随着电网连通性的提高,输电线路的故障可能会触发级联故障,这可能会导致大量的经济损失和严重的负面社会影响。因此,在各种类型的攻击下识别可能引发级联失败并部署国防资源以保护它们的关键路线非常重要。由于与单阶段攻击或无协调的多阶段攻击相比,协调的多阶段线攻击可能会导致更大的负面影响,因此本文打算在协调的多阶段攻击下识别可能引发级联失败并最佳部署有限的防御资源的关键线。为此,我们首先考虑了多个攻击者和多个阶段的总生成损失最大化问题。由于解决方案空间的尺寸很大,解决该法式问题非常具有挑战性。为了克服挑战,我们将问题重新制定为马尔可夫游戏,并设计其组成部分,例如状态,行动和奖励。接下来,我们提出了一种可扩展的算法,以基于多代理的深入强化学习和优先的体验重播来解决马尔可夫游戏,这可以确定最佳的攻击线序列。然后,我们设计了一种防御策略来决定最佳防御线。广泛的仿真结果表明了拟议算法和设计的防御策略的有效性。

With the increase of connectivity in power grid, a cascading failure may be triggered by the failure of a transmission line, which can lead to substantial economic losses and serious negative social impacts. Therefore, it is very important to identify the critical lines under various types of attacks that may initiate a cascading failure and deploy defense resources to protect them. Since coordinated multistage line attacks can lead to larger negative impacts compared with a single-stage attack or a multistage attack without coordination, this paper intends to identify the critical lines under coordinated multistage attacks that may initiate a cascading failure and deploy limited defense resources optimally. To this end, we first formulate a total generation loss maximization problem with the consideration of multiple attackers and multiple stages. Due to the large size of solution space, it is very challenging to solve the formulated problem. To overcome the challenge, we reformulate the problem as a Markov game and design its components, e.g., state, action, and reward. Next, we propose a scalable algorithm to solve the Markov game based on multi-agent deep reinforcement learning and prioritized experience replay, which can determine the optimal attacking line sequences. Then, we design a defense strategy to decide the optimal defense line set. Extensive simulation results show the effectiveness of the proposed algorithm and the designed defense strategy.

扫码加入交流群

加入微信交流群

微信交流群二维码

扫码加入学术交流群,获取更多资源