基于深度强化学习的电网拓扑优化及潮流控制

doi:10.16183/j.cnki.jsjtu.2021.S2.002

上海交通大学学报 ›› 2021, Vol. 55 ›› Issue (S2): 7-14.doi: 10.16183/j.cnki.jsjtu.2021.S2.002

基于深度强化学习的电网拓扑优化及潮流控制

周毅, 周良才(), 丁佳立, 高佳宁

国家电网有限公司华东分部,上海 200120

收稿日期:2021-10-28 出版日期:2021-12-28 发布日期:2022-01-24
通讯作者: 周良才 E-mail:liangcaizhou@163.com
作者简介:周毅(1982-),男,上海市人,高级工程师,主要从事电网调度、电力系统自动化研究.

Power Network Topology Optimization and Power Flow Control Based on Deep Reinforcement Learning

ZHOU Yi, ZHOU Liangcai(), DING Jiali, GAO Jianing

East Branch of State Grid Corporation of China, Shanghai 200120, China

Received:2021-10-28 Online:2021-12-28 Published:2022-01-24
Contact: ZHOU Liangcai E-mail:liangcaizhou@163.com

摘要/Abstract

摘要：

在追求碳中和的过程中,电源侧和负荷侧的巨大变化对电网运行和调度人员提出了新的要求和挑战.实时电网拓扑优化控制是一种成本低且有效的系统级缓解措施,然而,除了最简单的线路切换,由于电网网络拓扑优化问题的组合和非线性特性,常规的优化方法难以在短时间内完成求解.提出了一种新的基于人工智能的方法,通过考虑各种实际约束的电网拓扑优化控制来最大化系统可用传输容量.首先,利用模仿学习为智能体提供良好的初始策略.然后,通过深度强化学习引导智能体进行探索和学习,显著提升智能体的训练效率.最后,设计了一种预警机制来帮助智能体在长时间的运行决策过程中找到良好的拓扑控制策略,协助智能体找到最佳动作时间,从而有效地提高了方法的容错性和鲁棒性.该方法在IEEE 14节点开源数据集上进行了测试,测试结果验证了提出方法的有效性.

关键词: 拓扑优化控制, 可用传输容量, 模仿学习, 深度强化学习

Abstract:

In the pursuit of carbon neutrality, huge changes on the power supply side and the load side have brought forward new requirements and challenges for grid operation and dispatchers. A low-cost and effective measure is real-time power grid network topology optimization and control (NTOC). However, except for the simplest action of line switching, the combinatorial and non-linear nature of the NTOC problem has made existing approaches infeasible for grids of reasonable scales. This paper proposes a novel artificial intelligence (AI) based approach for maximizing available transfer capabilities (ATCs) via network topology control considering various practical constraints and uncertainties. First, imitation learning is utilized to provide a good initial policy for the AI agent. Then, the agent is trained through deep reinforcement learning with a novel guided exploration technique, which significantly improves the training efficiency. Finally, an early warning mechanism is designed to help the agent identify a proper action time, which effectively improves the fault tolerance and robustness of the method. The effectiveness of the proposed approach is tested by using open-sourced data of the IEEE 14-note system.

Key words: topology optimization and control, available transfer capability, imitation learning, deep reinforcement learning

中图分类号:

TM711

周毅, 周良才, 丁佳立, 高佳宁. 基于深度强化学习的电网拓扑优化及潮流控制[J]. 上海交通大学学报, 2021, 55(S2): 7-14.

ZHOU Yi, ZHOU Liangcai, DING Jiali, GAO Jianing. Power Network Topology Optimization and Power Flow Control Based on Deep Reinforcement Learning[J]. Journal of Shanghai Jiao Tong University, 2021, 55(S2): 7-14.

图/表 9

图1

图2

图3

图4

图5

图6

图7

图8

表1

参考文献 18

[1]	周毅, 钱学东. 华东电网动态区域控制误差应用分析[J]. 电力系统自动化, 2010, 34(8): 106-110.
	ZHOU Yi, QIAN Xuedong. Applications analysis of dynamic ACE in East China power grid[J]. Automation of Electric Power Systems, 2010, 34(8): 106-110.
[2]	游广增, 汤翔鹰, 胡炎, 等. 基于典型运行场景聚类的电力系统灵活性评估方法[J]. 上海交通大学学报, 2021, 55(7): 802-813.
	YOU Guangzeng, TANG Xiangying, HU Yan, et al. Flexibility evaluation method for power system based on clustering of typical operating scenarios[J]. Journal of Shanghai Jiao Tong University, 2021, 55(7): 802-813.
[3]	DUAN J J, SHI D, DIAO R S, et al. Deep-reinforcement-learning-based autonomous voltage control for power grid operations[J]. IEEE Transactions on Power Systems, 2020, 35(1): 814-817. doi: 10.1109/TPWRS.59 URL
[4]	GLAVITSCH H. Switching as means of control in the power system[J]. International Journal of Electrical Power & Energy Systems, 1985, 7(2): 92-100. doi: 10.1016/0142-0615(85)90014-6 URL
[5]	MAZI A, WOLLENBERG B, HESSE M. Corrective control of power system flows by line and bus-bar switching[J]. IEEE Transactions on Power System, 1986, 1(3): 258-264. doi: 10.1109/TPWRS.1986.4334990 URL
[6]	FISHER E B, O’NEILL R P, FERRIS M C. Optimal transmission switching[J]. IEEE Transactions on Power Systems, 2008, 23(3): 1346-1355. doi: 10.1109/TPWRS.2008.922256 URL
[7]	KHODAEI A, SHAHIDEHPOUR M. Transmission switching in security-constrained unit commitment[J]. IEEE Transactions on Power Systems, 2010, 25(4): 1937-1945. doi: 10.1109/TPWRS.2010.2046344 URL
[8]	FULLER J D, RAMASRA R, CHA A. Fast heuristics for transmission-line switching[J]. IEEE Tran-sactions on Power Systems, 2012, 27(3): 1377-1386.
[9]	DEHGHANIAN P, WANG Y P, GURRALA G, et al. Flexible implementation of power system corrective topology control[J]. Electric Power Systems Research, 2015, 128:79-89. doi: 10.1016/j.epsr.2015.07.001 URL
[10]	ALHAZMI M, DEHGHANIAN P, WANG S Y, et al. Power grid optimal topology control considering correlations of system uncertainties[J]. IEEE Tran-sactions on Industry Applications, 2019, 55(6): 5594-5604.
[11]	王岩, 陈耀然, 韩兆龙, 等. 基于互信息理论与递归神经网络的短期风速预测模型[J]. 上海交通大学学报, 2021, 55(9): 1080-1086.
	WANG Yan, CHEN Yaoran, HAN Zhaolong, et al. Short-term wind speed forecasting model based on mutual information and recursive neural network[J]. Journal of Shanghai Jiao Tong University, 2021, 55(9): 1080-1086.
[12]	李鹏, 阮晓钢, 朱晓庆, 等. 基于深度强化学习的区域化视觉导航方法[J]. 上海交通大学学报, 2021, 55(5): 575-585.
	LI Peng, RUAN Xiaogang, ZHU Xiaoqing, et al. A regionalization vision navigation method based on deep reinforcement learning[J]. Journal of Shanghai Jiao Tong University, 2021, 55(5): 575-585.
[13]	SILVER D, SCHRITTWIESER J, SIMONYAN K, et al. Mastering the game of go without human knowledge[J]. Nature, 2017, 550(7676): 354-359. doi: 10.1038/nature24270 URL
[14]	SUTTON R, BARTO A. Reinforcement learning: An introduction[M]. Cambridge: The MIT Press, 2017.
[15]	HASSELT H, GUEZ A, SILVER D. Deep reinforcement learning with double q-learning[C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, Arizona, USA: Association for the Advancement of Artificial Intelligence, 2016.
[16]	WANG Z, SCHAUL T. Dueling network architectures for deep reinforcement learning[C]// Proceedings of the 33rd International Conference on Machine Learning, New York, USA: International Machine Learning Society, 2016.
[17]	SCHAUL T, QUAN J. Prioritized experience replay[C]// International Conference on Learning Representations, San Juan, Puerto Rico, USA: International Conference of Legal Regulators, 2016.
[18]	、 LEROUSSEAU M. A power network simulator with a reinforcement learning-focused usage[DB/OL]. (2021-05-06)[2021-08-20]. https://pypownet.readthedocs.io/en/latest/index.html.

智能体	失败次数	平均得分	平均得分 (排除失败场景)
不作为	91	2471.42	4534.72
仅使用模仿学习训练	198	38.210	3820.63
使用引导式训练	7	4269.63	442.49
使用预警机制λ=0.850	0	4253.40	4253.40
使用预警机制λ=0.875	1	4347.56	4369.41
使用预警机制λ=0.900	0	4396.77	4396.77
使用预警机制λ=0.925	0	4493.27	4493.27
使用预警机制λ=0.950	0	4492.89	4492.89
使用预警机制λ=0.975	2	4446.12	4491.03

基于深度强化学习的电网拓扑优化及潮流控制

Power Network Topology Optimization and Power Flow Control Based on Deep Reinforcement Learning

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 18

相关文章 1

编辑推荐

Metrics

本文评价