Research on Economic Dispatching Strategy of CHP Units Based on SRL

doi:10.16182/j.issn1004731x.joss.23-1472

Abstract

Abstract:

In addressing the challenge of the DRL algorithm in the optimization of combined heat and power (CHP) units, lacking safety and stability guarantees, a scheduling optimization method based on SRL is proposed. Utilizing Dymola platform, a district heating system model is constructed with the CHP unit as the heat source. A MDP model for the economic dispatching of CHP units is designed, incorporating control barrier functions (CBF) to guide safe exploration in DRL. Simulation results show that the CBF-DRL method, in complex and nonlinear district heating systems, not only accelerates the convergence of DRL algorithms but also efficiently utilizes the thermal inertia of heating pipelines to enhance the economic performance of CHP units. In particular, it has significant safety benefits.

Key words: combined heat and power(CHP), district heating system, safe reinforcement learning, control barrier function(CBF), economic dispatch, co-simulation

CLC Number:

TP391.9

Wang Xin, Cui Chenggang, Wang Xiangxiang, Zhu Ping. Research on Economic Dispatching Strategy of CHP Units Based on SRL[J]. Journal of System Simulation, 2025, 37(4): 968-981.

Figures/Tables 18

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Table 1

Parameters of the heating system

参数	数值
[h_t, p_t ]/MW	(0,10), (10,5), (70,35), (0,50)
a₀	58.908 2
a₁	274.899 6
$T m i n S$ /℃	70
$τ m a x S$ /℃	110
$m ˙ m a x$ /(m·s^-1)	3
τ^R_min/℃	45
L/m	5 000, 15 000
d/m	0.595 8

Table 1

Fig. 13

Fig. 14

Fig. 15

Fig. 16

Fig. 17

References 23

1	Franco Alessandro, Bellina Francesco. Methods for Optimized Design and Management of CHP Systems for District Heating Networks (DHN)[J]. Energy Conversion and Management, 2018, 172: 21-31.
2	Rezaie B, Rosen M A. District Heating and Cooling: Review of Technology and Potential Enhancements[J]. Applied Energy, 2012, 93: 2-10.
3	Vandermeulen Annelies, Bram van der Heijde, Helsen Lieve. Controlling District Heating and Cooling Networks to Unlock Flexibility: A Review[J]. Energy, 2018, 151: 103-115.
4	Buffa Simone, Mohammad Hossein Fouladfar, Franchini Giuseppe, et al. Advanced Control and Fault Detection Strategies for District Heating and Cooling Systems—a Review[J]. Applied Sciences, 2021, 11(1): 455.
5	Xu Fei, Hao Ling, Chen Lei, et al. Integrated Heat and Power Optimal Dispatch Method Considering the District Heating Networks Flow Rate Regulation for Wind Power Accommodation[J]. Energy, 2023, 263, Part A: 125656.
6	Verrilli Francesca, Srinivasan S, Gambino Giovanni, et al. Model Predictive Control-based Optimal Operations of District Heating System with Thermal Energy Storage and Flexible Loads[J]. IEEE Transactions on Automation Science and Engineering, 2017, 14(2): 547-557.
7	Grosswindhager S, Voigt A, Kozek M. Predictive Control of District Heating Network Using Fuzzy DMC[C]//2012 Proceedings of International Conference on Modelling, Identification and Control. Piscataway: IEEE, 2012: 241-246.
8	Yang Dong, Yan Siyun, Zhou Dengji, et al. Reinforcement Learning Methods on Optimization Problems of Natural Gas Pipeline Networks[C]//2020 4th International Conference on Smart Grid and Smart Cities (ICSGSC). Piscataway: IEEE, 2020: 29-34.
9	Zhang Bin, Hu Weihao, Cao Di, et al. Deep Reinforcement Learning-based Approach for Optimizing Energy Conversion in Integrated Electrical and Heating System with Renewable Energy[J]. Energy Conversion and Management, 2019, 202: 112199.
10	Claessens B J, Vanhoudt D, Desmedt J, et al. Model-free Control of Thermostatically Controlled Loads Connected to a District Heating Network[J]. Energy and Buildings, 2018, 159: 1-10.
11	Wan Yanni, Qin Jiahu, Ma Qichao, et al. Multi-agent DRL-based Data-driven Approach for PEVs Charging/Discharging Scheduling in Smart Grid[J]. Journal of the Franklin Institute, 2022, 359(4): 1747-1767.
12	Gangopadhyay Briti, Dasgupta Pallab, Dey Soumyajit. Safe and Stable RL (S²RL) Driving Policies Using Control Barrier and Control Lyapunov Functions[J]. IEEE Transactions on Intelligent Vehicles, 2023, 8(2): 1889-1899.
13	Cheng R, Orosz Gábor, Murray R M, et al. End-to-end Safe Reinforcement Learning Through Barrier Functions for Safety-critical Continuous Control Tasks[C]//Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty-First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educational Advances in Artificial Intelligence. Palo Alto: AAAI Press, 2019: 3387-3395.
14	Marvi Z, Kiumarsi B. Safe Reinforcement Learning: A Control Barrier Function Optimization Approach[J]. International Journal of Robust and Nonlinear Control, 2021, 31(6): 1923-1940.
15	Cohen M H, Belta C. Safe Exploration in Model-based Reinforcement Learning Using Control Barrier Functions[J]. Automatica, 2023, 147: 110684.
16	Gräber M, Kosowski K, Richter C, et al. Modelling of Heat Pumps with an Object-oriented Model Library for Thermodynamic Systems[J]. Mathematical and Computer Modelling of Dynamical Systems, 2010, 16(3): 195-209.
17	Leitner B, Widl E, Gawlik W, et al. A Method for Technical Assessment of Power-to-heat Use Cases to Couple Local District Heating and Electrical Distribution Grids[J]. Energy, 2019, 182: 729-738.
18	Wetter M, Zuo Wangda, Nouidui T S, et al. Modelica Buildings Library[J]. Journal of Building Performance Simulation, 2014, 7(4): 253-270.
19	van der Heijde B, Fuchs M, Ribas Tugores C, et al. Dynamic Equation-based Thermo-hydraulic Pipe Model for District Heating and Cooling Systems[J]. Energy Conversion and Management, 2017, 151: 158-169.
20	Zhou Suyang, Hu Zijian, Gu Wei, et al. Combined Heat and Power System Intelligent Economic Dispatch: A Deep Reinforcement Learning Approach[J]. International Journal of Electrical Power & Energy Systems, 2020, 120: 106016.
21	Haghrah A, Nazari-Heris M, Mohammadi-Ivatloo B. Solving Combined Heat and Power Economic Dispatch Problem Using Real Coded Genetic Algorithm with Improved Mühlenbein Mutation[J]. Applied Thermal Engineering, 2016, 99: 465-475.
22	庞力平, 梁其缘, 梁惠勋, 等. 基于Dymola平台的超超临界二次再热机组切除高压加热器动态仿真[J]. 热能动力工程, 2023, 38(1): 120-128.
	Pang Liping, Liang Qiyuan, Liang Huixun, et al. Dynamic Simulation of Ultra-supercritical Double Reheat Unit Based on Dymola Platform After HP Feedwater Heater Bypass[J]. Journal of Engineering for Thermal Energy and Power, 2023, 38(1): 120-128.
23	Li Zhigang, Wu Wenchuan, Shahidehpour M, et al. Combined Heat and Power Dispatch Considering Pipeline Energy Storage of District Heating Network[J]. IEEE Transactions on Sustainable Energy, 2016, 7(1): 12-22.

[1]	Ren Xuyang, Bu Xuhui, Yin Yanling, Liu Jinghua. Coordinated and Optimal Dispatching for Wind-photovoltaic-storage Systems Based on Multi-strategy Multi-objective Differential Evolution Algorithm [J]. Journal of System Simulation, 2025, 37(2): 450-461.
[2]	Yu Yang, Xia Yuxing, Lu Wentao, Liu Mai, Gao Shixuan, Chen Dongyang. Two-level Optimal Dispatch of Power System Based on Load-storage Carbon Flow Model [J]. Journal of System Simulation, 2024, 36(10): 2288-2299.
[3]	Zhao Yu, Yang Caijin, Wang Tanming, Xu Jing, Zhou Shuai. Path Following Control and Simulation Analysis of Multi-articulated Vehicles [J]. Journal of System Simulation, 2024, 36(10): 2300-2313.
[4]	Xinghua Liu, Chen Geng, Shenghan Xie, Jiaqiang Tian, Hui Cao. Day Ahead Thermal-photovoltaic Economic Dispatch Considering Uncertainty of Photovoltaic Power Generation [J]. Journal of System Simulation, 2022, 34(8): 1874-1884.
[5]	Gao Shengdong, Yu Xin. Co-simulation of Parallel Computing of Disc Temperature Distributed Parameter System Cooling Rate Control [J]. Journal of System Simulation, 2021, 33(8): 1839-1845.
[6]	Mei Panpan, Wu Lianghong, Zhang Hongqiang, Wang Huiying. Adaptive Differential Crisscross Optimization Algorithm for Dynamic Economic Emission Dispatch Considering Wind Power [J]. Journal of System Simulation, 2020, 32(6): 1179-1187.
[7]	Zhang Qing, Xia Hua, Yang Ning, Qin Xianrong, Sun Yuantao. Modeling and Co-simulation of Novel Electric Heave Compensation System [J]. Journal of System Simulation, 2018, 30(8): 2973-2981.
[8]	Sun Weiqing, Ye Lei, Tu Yiyun. GAMS-based Active Power Optimization with Reactive Power Compensation [J]. Journal of System Simulation, 2018, 30(8): 3082-3091.
[9]	Gao Cong, Wu Dinghui, Pan Tinglong, Ji Zhicheng. Optimal Economic Dispatch of Microgrid Based on Immune Particle Swarm Optimization Algorithm [J]. Journal of System Simulation, 2018, 30(2): 636-646.
[10]	Wei Xiang, Li Benwei, Yang Xinyi, Wang Xingbo. Fault Co-simulation of Fuel Regulator in a Certain Type of Turbofan Engine [J]. Journal of System Simulation, 2018, 30(10): 3923-3932.
[11]	Chen Gang, Zhou Qicai, Wu Jing, Yan Nan. Research for Co-simulation of 7DOF Manipulator Based on Adams and Matlab [J]. Journal of System Simulation, 2017, 29(1): 99-106.
[12]	Liu Zhenze, Wang Fengming, Zhang Xuange, Yin Cangqiong. Co-simulation of Intelligent Lower Limb Assist Device Virtual Prototype [J]. Journal of System Simulation, 2015, 27(6): 1247-1254.
[13]	Cai Pu, Lin Muyi, Zheng Xin, Wen Jian. Co-simulation and Experiment of Vehicle Electro-Hydraulic Dynamic Braking System [J]. Journal of System Simulation, 2015, 27(4): 893-899.