Efficiency Optimization Method for Data Sampling in Power Grid Topology Scheduling Simulation

doi:10.16182/j.issn1004731x.joss.23-0958

Abstract

Abstract:

To address the large simulation computational workload and low simulation speed caused by the scale and complexity of the new power system, a simulation acceleration method for topology scheduling based on the distributed and quantization mechanisms is proposed. The parallelization of topology scheduling models is used to increase the scale of data simulation sampling in unit time. The introduced quantization operators accelerate the computation speed of the topology scheduling model operators, reduces the time cost of the every single simulation. Case studies confirm the effectiveness of the topology simulation acceleration, in which the available transfer capacity of the simulated power system increases by approximately 2 times in unit time and achieves the simulation acceleration by 1.5~2.5 times in various-scale power grid scenarios.

Key words: topology scheduling simulation, DRL, sampling efficiency optimization, model quantization, power grid digital twin

CLC Number:

TP391

Zhao Yingying, Dong Pusen, Zhu Tianchen, Li Fan, Su Yun, Tai Zhenying, Sun Qingyun, Fan Hang. Efficiency Optimization Method for Data Sampling in Power Grid Topology Scheduling Simulation[J]. Journal of System Simulation, 2024, 36(2): 283-295.

Figures/Tables 13

Table 1

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Table 2

Table 3

Fig. 6

Table 4

Fig. 7

Fig. 8

Fig. 9

References 37

1	董伟杰, 刘科研, 王义龙, 等. 高比例分布式发电接入配电网自适应控制方法[J]. 系统仿真学报, 2020, 32(10): 2052-2058.
	Dong Weijie, Liu Keyan, Wang Yilong, et al. Adaptive Control Method of High Proportion Distributed Generation Connected to Distribution Network[J]. Journal of System Simulation, 2020, 32(10): 2052-2058.
2	Zhou Mike, Yan Jianfeng, Zhou Xiaoxin. Real-time Online Analysis of Power Grid[J]. CSEE Journal of Power and Energy Systems, 2020, 6(1): 236-238.
3	Song Xinya, Cai Hui, Kircheis Jan, et al. Application of Digital Twin Assistant-system in State Estimation for Inverter Dominated Grid[C]//2020 55th International Universities Power Engineering Conference (UPEC). Piscataway, NJ, USA: IEEE, 2020: 1-6.
4	陈强, 王意, 李康顺. 双重需求响应的虚拟电厂建模与调度研究[J]. 系统仿真学报, 2023, 35(4): 822-832.
	Chen Qiang, Wang Yi, Li Kangshun. Research on Modeling and Scheduling of Virtual Power Plant with Dual Demand Response[J]. Journal of System Simulation, 2023, 35(4): 822-832.
5	沈沉, 曹仟妮, 贾孟硕, 等. 电力系统数字孪生的概念、特点及应用展望[J]. 中国电机工程学报, 2022, 42(2): 487-498, 中插4.
	Shen Chen, Cao Qianni, Jia Mengshuo, et al. Concepts, Characteristics and Prospects of Application of Digital Twin in Power System[J]. Proceedings of the CSEE, 2022, 42(2): 487-498, 中插4.
6	冯昌森, 沈佳静, 赵崇娟, 等. 基于合作博弈的智慧能源社区协同运行策略[J]. 电力自动化设备, 2021, 41(4): 85-93.
	Feng Changsen, Shen Jiajing, Zhao Chongjuan, et al. Cooperative Game-based Coordinated Operation Strategy of Smart Energy Community[J]. Electric Power Automation Equipment, 2021, 41(4): 85-93.
7	王珂, 姚建国, 余佩遥, 等. 基于深度强化学习的电网前瞻调度智能决策架构及关键技术初探[J]. 中国电机工程学报, 2022, 42(15): 5430-5438, 中插4.
	Wang Ke, Yao Jianguo, Yu Peiyao, et al. Architecture and Key Technologies of Intelligent Decision-making of Power Grid Look-ahead Dispatch Based on Deep Reinforcement Learning[J]. Proceedings of the CSEE, 2022, 42(15): 5430-5438, 中插4.
8	Glavitsch H. Switching as Means of Control in the Power System[J]. International Journal of Electrical Power & Energy Systems, 1985, 7(2): 92-100.
9	Mazi A A, Wollenberg B F, Hesse M H. Corrective Control of Power System Flows by Line and Bus-bar Switching[J]. IEEE Transactions on Power Systems, 1986, 1(3): 258-264.
10	Fisher E B, O'Neill R P, Ferris M C. Optimal Transmission Switching[J]. IEEE Transactions on Power Systems, 2008, 23(3): 1346-1355.
11	Khodaei A, Shahidehpour M. Transmission Switching in Security-constrained Unit Commitment[J]. IEEE Transactions on Power Systems, 2010, 25(4): 1937-1945.
12	David Fuller J, Ramasra Raynier, Cha Amanda. Fast Heuristics for Transmission-line Switching[J]. IEEE Transactions on Power Systems, 2012, 27(3): 1377-1386.
13	Dehghanian P, Wang Yaping, Gurrala Gurunath, et al. Flexible Implementation of Power System Corrective Topology Control[J]. Electric Power Systems Research, 2015, 128: 79-89.
14	Alhazmi M, Dehghanian P, Wang Shiyuan, et al. Power Grid Optimal Topology Control Considering Correlations of System Uncertainties[J]. IEEE Transactions on Industry Applications, 2019, 55(6): 5594-5604.
15	魏利胜, 杨奔奔, 孙瑞霞. 基于新型BBO算法的微电网优化调度研究[J]. 系统仿真学报, 2023, 35(5): 1075-1085.
	Wei Lisheng, Yang Benben, Sun Ruixia. Research on Optimal Scheduling of Microgrid Based on NBBO Algorithm[J]. Journal of System Simulation, 2023, 35(5): 1075-1085.
16	Duan Jiajun, Shi Di, Diao Ruisheng, et al. Deep-reinforcement-learning-based Autonomous Voltage Control for Power Grid Operations[J]. IEEE Transactions on Power Systems, 2020, 35(1): 814-817.
17	Xi Lei, Yu Lu, Xu Yanchun, et al. A Novel Multi-agent DDQN-AD Method-based Distributed Strategy for Automatic Generation Control of Integrated Energy Systems[J]. IEEE Transactions on Sustainable Energy, 2020, 11(4): 2417-2426.
18	马苗苗, 董利鹏, 刘向杰. 基于Q-learning算法的多智能体微电网能量管理策略[J]. 系统仿真学报, 2023, 35(7): 1487-1496.
	Ma Miaomiao, Dong Lipeng, Liu Xiangjie. Energy Management Strategy of Multi-agent Microgrid Based on Q-learning Algorithm[J]. Journal of System Simulation, 2023, 35(7): 1487-1496.
19	王新迎, 赵琦, 赵黎媛, 等. 基于深度Q学习的电热综合能源系统能量管理[J]. 电力建设, 2021, 42(3): 10-18.
	Wang Xinying, Zhao Qi, Zhao Liyuan, et al. Energy Management Approach for Integrated Electricity-heat Energy System Based on Deep Q-learning Network[J]. Electric Power Construction, 2021, 42(3): 10-18.
20	李嘉文, 余涛, 张孝顺, 等. 基于改进深度确定性梯度算法的AGC发电功率指令分配方法[J]. 中国电机工程学报, 2021, 41(21): 7198-7211, 中插2.
	Li Jiawen, Yu Tao, Zhang Xiaoshun, et al. AGC Power Generation Command Allocation Method Based on Improved Deep Deterministic Policy Gradient Algorithm[J]. Proceedings of the CSEE, 2021, 41(21): 7198-7211, 中插2.
21	周毅, 周良才, 丁佳立, 等. 基于深度强化学习的电网拓扑优化及潮流控制[J]. 上海交通大学学报, 2021, 55(增2): 7-14.
	Zhou Yi, Zhou Liangcai, Ding Jiali, et al. Power Network Topology Optimization and Power Flow Control Based on Deep Reinforcement Learning[J]. Journal of Shanghai Jiaotong University, 2021, 55(S2): 7-14.
22	Marot Antoine, Donnot Benjamin, Dulac-Arnold Gabriel, et al. Learning to Run a Power Network Challenge: a Retrospective Analysis[C]//Proceedings of the NeurIPS 2020 Competition and Demonstration Track. Chia Laguna Resort, Sardinia, Italy: PMLR, 2021: 112-132.
23	李宏仲, 王磊, 林冬, 等. 多主体参与可再生能源消纳的Nash博弈模型及其迁移强化学习求解[J]. 中国电机工程学报, 2019, 39(14): 4135-4149.
	Li Hongzhong, Wang Lei, Lin Dong, et al. A Nash Game Model of Multi-agent Participation in Renewable Energy Consumption and the Solving Method Via Transfer Reinforcement Learning[J]. Proceedings of the CSEE, 2019, 39(14): 4135-4149.
24	Ahrarinouri Mehdi, Rastegar Mohammad, Karami K, et al. Distributed Reinforcement Learning Energy Management Approach in Multiple Residential Energy hubs[J]. Sustainable Energy, Grids and Networks, 2022, 32: 100795.
25	Qin Yude, Ke Ji, Wang Biao, et al. Energy Optimization for Regional Buildings Based on Distributed Reinforcement Learning[J]. Sustainable Cities and Society, 2022, 78: 103625.
26	林永君, 陈鑫, 杨凯, 等. 含多微网的主动配电网双层分布式优化调度[J]. 系统仿真学报, 2022, 34(11): 2323-2336.
	Lin Yongjun, Chen Xin, Yang Kai, et al. Bilevel Distributed Optimal Dispatch of Active Distribution Network with Multi-microgrids[J]. Journal of System Simulation, 2022, 34(11): 2323-2336.
27	Krishnan S, Lam M, Chitlangia S, et al. QuaRL: Quantization for Fast and Environmentally Sustainable Reinforcement Learning[EB/OL]. (2022-11-14) [2023-06-30]. .
28	Louizos Christos, Reisser Matthias, Blankevoort T, et al. Relaxed Quantization for Discretized Neural Networks[EB/OL]. (2018-10-03) [2023-06-30]. .
29	Nagel M, Fournarakis M, Amjad R A, et al. A White Paper on Neural Network Quantization[EB/OL]. (2021-06-15) [2023-06-30]. .
30	Krishnamoorthi R. Quantizing Deep Convolutional Networks for Efficient Inference: A Whitepaper[EB/OL]. (2018-06-21) [2023-06-30]. .
31	Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous Control with Deep Reinforcement Learning[EB/OL]. (2019-07-05) [2023-06-30]. .
32	Horgan D, Quan J, Budden D, et al. Distributed Prioritized Experience Replay[J]. (2018-03-02) [2023-06-30]. .
33	Zhang Shangtong, Sutton Richard S. A Deeper Look at Experience Replay[EB/OL]. (2018-04-30) [2023-06-30]. .
34	Sutton R S, Barto A G. Introduction to Reinforcement Learning[M]. Cambridge: MIT Press, 1998.
35	Wang Ziyu, Schaul T, Hessel M, et al. Dueling Network Architectures for Deep Reinforcement Learning[C]//Proceedings of the 33rd International Conference on Machine Learning. Chia Laguna Resort, Sardinia, Italy: PMLR, 2016: 1995-2003.
36	van Hasselt Hado, Guez A, Silver D. Deep Reinforcement Learning with Double Q-learning[C]//Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence. Palo Alto, CA, USA: AAAI Press, 2016: 2094-2100.
37	Alexandre dos Santos Mignon, Ricardo Luis de Azevedo da Rocha. An Adaptive Implementation of ε-greedy in Reinforcement Learning[J]. Procedia Computer Science, 2017, 109: 1146-1151.

特征名	大小	描述
时间	6	年、月、周、日、时、分
发电机功率	2×N_gen	每个发电机的有功和无功功率
负载功率	2×N_load	每个负载的有功功率和无功功率
电线上的功率	4×N_line	每条电线两端的有功功率和无功功率
电线容量	N_line	每条电线的容量
拓扑信息	N_gen+N_load+2×N_line	每个元件连接到了其变电站的哪个母线上
电线状态	N_line	每条电线的状态 (连接/断开)

电网算例	变电站数量	线路数量	发电机数量	负载数量	状态空间维度	是否允许调整发电值
IEEE 5	5	8	2	3	74	否
IEEE 14	14	20	6	11	194	是
IEEE 118	36	59	22	37	590	是

硬件	信息类型	信息
CPU	型号	Intel(R) Xeon(R) Platinum 8338C CPU
	架构	x86_64
	核数	128
GPU	型号	NVIDIA GeForce RTX 4090
GPU	显存/MB	24 564
内存	总大小/MB	515 415

电网环境	方法	1/4 T	2/4 T	3/4 T	T
IEEE 5	DQN	254	430	842	1 029
	PPO	423	734	524	630
	本文方法	653	932	1 420	1 924
IEEE 14	DQN	215	465	560	564
	PPO	372	410	522	600
	本文方法	217	463	564	783
IEEE 118	DQN	43	48	52	58
	PPO	16	52	70	86
	本文方法	52	72	93	120

[1]	Wang Xinpeng, Fu Huiqiao, Deng Guizhou, Tang Kaiqiang, Chen Chunlin, Liu Canghai. Research on Motion Planning of Hexapod Robot Based on DRL and Free Gait [J]. Journal of System Simulation, 2024, 36(2): 373-384.
[2]	Pan Hainan, Chen Bailiang, Huang Kaihong, Ren Junkai, Cheng Chuang, Lu Huimin, Zhang Hui. Flipper Control Method for Tracked Robot Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2024, 36(2): 405-414.
[3]	An Jing, Si Guangya, Zhang Lei. Strategy Optimization Method of Multi-dimension Projection Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2024, 36(1): 39-49.
[4]	Wang Yukun, Wang Ze, Dong Liwei, Li Ni. Research on Multi-aircraft Air Combat Behavior Modeling Based on Hierarchical Intelligent Modeling Methods [J]. Journal of System Simulation, 2023, 35(10): 2249-2261.
[5]	Hui Nie, Kangshun Li, Yang Su. A Quantization Training Algorithm of Adaptive Learning Quantization Scale Fators [J]. Journal of System Simulation, 2022, 34(7): 1639-1650.
[6]	Lü Hongli, Wang Renfang. Image Segmentation Based on Active Contour Derived by Signed Pressure Force Function [J]. Journal of System Simulation, 2016, 28(1): 174-182.