基于动态仿真与强化学习的化工装置调控研究

doi:10.16182/j.issn1004731x.joss.26-0140

摘要/Abstract

摘要：

针对蓄热式热力氧化炉（regenerative thermal oxidizer， RTO）在实际中废气浓度与气量波动以及工况变化使系统运行参数难以保持在经济最优区间，提出了一种基于强化学习的面向RTO运行控制的离线训练与现场部署相结合的智能控制策略框架。在离线阶段利用强化学习实现控制策略的自主学习与全局优化；在在线阶段，引入基于实时数据的策略自适应修正机制。将强化学习控制策略与运行监控与可视化系统集成，实现运行状态与控制效果的直观呈现。仿真结果表明：该单位蒸汽CO₂排放降低约1.4%，6个月运行周期内累计经济收益提升0.83%。在实际工业条件下系统蒸汽产出提升5.18%。

关键词: 蓄热式热力氧化炉, 强化学习, 动态仿真, 智能控制, 离线训练与现场部署

Abstract:

An intelligent control strategy framework combining offline training and on-site deployment for regenerative thermal oxidizer (RTO) operation control based on RL is proposed to solve the problem that fluctuations in exhaust gas concentration and flow, as well as changes in working conditions, make it difficult for RTO to maintain the system operating parameters within the economically optimal range in practical applications. In the offline phase, RL is employed to enable autonomous policy learning and global optimization. In the online phase, an adaptive policy correction mechanism based on real-time data is introduced. RL control strategy is integrated with the operation monitoring and visualization system to intuitively present operational status and control effects. Simulation results demonstrate that the CO₂ emissions per unit of steam are reduced by approximately 1.4%, and the cumulative economic benefit over a six-month operation period is increased by 0.83%. The steam production is increased by 5.18% in actual industrial conditions.

Key words: regenerative thermal oxidizer, reinforcement learning, dynamic simulation, intelligent control, offline training and on-site deployment

中图分类号:

TP391

李雨新,李海杰,韩尚伯等 . 基于动态仿真与强化学习的化工装置调控研究[J]. 系统仿真学报, 2026, 38(10): 2802-2818.

Li Yuxin,Li Haijie,Han Shangbo,et al . Chemical Plant Control Based on Dynamic Simulation and Reinforcement Learning[J]. Journal of System Simulation, 2026, 38(10): 2802-2818.

图/表 15

图1

图2

图3

表1

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

参考文献 36

[1]	Wang Jin, He Qinghua. Methane Removal from Air: Challenges and Opportunities[J]. Methane, 2023, 2(4): 404-414.
[2]	班鹏. 旋转蓄热氧化装置蓄热室的研究及优化[D]. 哈尔滨: 哈尔滨工业大学, 2022.
	Ban Peng. Study and Optimization on the Regenerator of a Regenerative Rotary Thermal Oxidizer Furnace[D]. Harbin: Harbin Institute of Technology, 2022.
[3]	宋宜凯, 申炜杰, 曾博, 等. 计及[火用]分析的矿山综合能源系统多目标运行方法[J]. 太阳能学报, 2025, 46(9): 650-660.
	Song Yikai, Shen Weijie, Zeng Bo, et al. Multi-objective Operation Method for Coal Mine Integrated Energy Systems Considering Exergy Analysis[J]. Acta Energiae Solaris Sinica, 2025, 46(9): 650-660.
[4]	赵旭生, 马国龙. 煤矿瓦斯智能抽采关键技术研究进展及展望[J]. 煤炭科学技术, 2021, 49(5): 27-34.
	Zhao Xusheng, Ma Guolong. Research Progress and Prospect of Key Technology of Intelligent Gas Drainage in Coal Mine[J]. Coal Science and Technology, 2021, 49(5): 27-34.
[5]	Procacci Alberto, Amaduzzi Ruggero, Coussement Axel, et al. Adaptive Digital Twins of Combustion Systems Using Sparse Sensing Strategies[J]. Proceedings of the Combustion Institute, 2023, 39(4): 4257-4266.
[6]	Bottari Agustín, Braccia Lautaro. Optimal Control Structure Selection Based on Economics for Continuous Cross-flow Grain Dryer[J]. Drying Technology, 2023, 41(10): 1605-1619.
[7]	黄代存. 蓄热式氧化炉在炼油污水处理厂中的应用[J]. 化工环保, 2024, 44(4): 589-594.
	Huang Daicun. Application of Regenerative Thermal Oxidizer in Refinery Wastewater Treatment Plant[J]. Environmental Protection of Chemical Industry, 2024, 44(4): 589-594.
[8]	张磊. 同步Buck DC-DC变换器的模糊控制研究[D]. 贵阳: 贵州大学, 2024.
[9]	Milojkovic Marko T, Djordjevic Andjela D, Stanisa Lj Peric, et al. Model Predictive Control of Nonlinear MIMO Systems Based on Adaptive Orthogonal Polynomial Networks[J]. Elektronika Ir Elektrotechnika, 2021, 27(2): 4-10.
[10]	Borase Rakesh P, Maghade D K, Sondkar S Y, et al. A Review of PID Control, Tuning Methods and Applications[J]. International Journal of Dynamics and Control, 2021, 9(2): 818-827.
[11]	Schwenzer Max, Ay Muzaffer, Bergs Thomas, et al. Review on Model Predictive Control: an Engineering Perspective[J]. The International Journal of Advanced Manufacturing Technology, 2021, 117(5): 1327-1349.
[12]	Darby M L, Nikolaou M, Jones J, et al. RTO: An Overview and Assessment of Current Practice[J]. Journal of Process Control, 2011, 21(6): 874-884.
[13]	Glauce De Souza, Odloak Darci, Zanin Antônio C. Real Time Optimization (RTO) with Model Predictive Control (MPC)[J]. Computers & Chemical Engineering, 2010, 34(12): 1999-2006.
[14]	Eduardo A Graciano José, Jäschke Johannes, Galo A C Le Roux, et al. Integrating Self-optimizing Control and Real-time Optimization Using zone Control MPC[J]. Journal of Process Control, 2015, 34: 35-48.
[15]	Adetola V, Guay M. Integration of Real-time Optimization & and Model Predictive Control[J]. IFAC Proceedings Volumes, 2009, 42(11): 63-68.
[16]	孙雁伯, 袁兴海, 林琳, 等. 三室蓄热式热力氧化炉热工性能数值模拟研究[J]. 力学与实践, 2024, 46(5): 1023-1036.
	Sun Yanbo, Yuan Xinghai, Lin Lin, et al. Numerical Simulation of Thermal Performance of Three-canister Regenerative Thermal Oxidizers[J]. Mechanics in Engineering, 2024, 46(5): 1023-1036.
[17]	陈相煜, 王波, 周秋平, 等. 煤矿乏风蓄热氧化过程数值模拟[J]. 化学工程, 2023, 51(1): 78-83.
	Chen Xiangyu, Wang Bo, Zhou Qiuping, et al. Simulation on Regenerative Thermal Oxidation of Coal Mine Ventilation Air Methane[J]. Chemical Engineering, 2023, 51(1): 78-83.
[18]	张睿智, 罗永浩, 赵善辉, 等. 低浓度VOCs气体漩涡燃烧分解过程的数值模拟研究[J]. 工业锅炉, 2023(5): 7-11.
	Zhang Ruizhi, Luo Yonghao, Zhao Shanhui, et al. Numerical Simulation Study on the Decomposition Process of Low-concentration VOCs by Vortex Combustion[J]. Industrial Boilers, 2023(5): 7-11.
[19]	萧琦, 姜泽毅, 张欣欣. 蓄热式有机废气焚烧炉的数值模拟和应用[J]. 北京科技大学学报, 2011, 33(5): 632-635.
	Xiao Qi, Jiang Zeyi, Zhang Xinxin. Numerical Simulation and Application of a Regenerative Thermal Oxidizer[J]. Journal of University of Science and Technology Beijing, 2011, 33(5): 632-635.
[20]	Sun Bo, Zhao Tong, Li Lu, et al. Optimization of Regenerator Operating Parameters and Thermal Insulation Construction for Rotary Regenerative Thermal Oxidizer (r-RTO) Based on Thermal-fluid Coupling Method and Quadratic Regression Model[J]. Case Studies in Thermal Engineering, 2022, 37: 102314.
[21]	刘丽娜, 尹浩东. 延误场景下基于强化学习的城轨列车运行调度优化[C]//2025世界交通运输大会(WTC2025). 北京: 人民交通出版社, 2025: 35-40.
	Liu Lina, Yin Haodong. Reinforcement Learning-based Optimization of Urban Rail Transit Train Rescheduling Under Delay Scenarios[C]//2025 World Transport Convention (WTC2025). Beijing: China Communications Press, 2025: 35-40.
[22]	姜嘉成, 贾政轩, 徐钊, 等. 基于博弈对抗复杂系统的决策建模与求解[J]. 系统仿真学报, 2025, 37(1): 66-78.
	Jiang Jiacheng, Jia Zhengxuan, Xu Zhao, et al. Decision Modeling and Solution Based on Game Adversarial Complex Systems[J]. Journal of System Simulation, 2025, 37(1): 66-78.
[23]	Park Joonsoo, Jung Hyein, Woo Kim Jong, et al. Reinforcement Learning for Process Control: Review and Benchmark Problems[J]. International Journal of Control Automation and Systems, 2025, 23(1): 1-40.
[24]	李文礼, 徐洋, 王云卿. 基于强化学习的路径跟踪控制方法[J]. 重庆理工大学学报(自然科学), 2025, 39(12): 19-28.
	Li Wenli, Xu Yang, Wang Yunqing. Path Tracking Control Method Based on Reinforcement Learning[J]. Journal of Chongqing University of Technology(Natural Science), 2025, 39(12): 19-28.
[25]	Yang Taoran, Guo Hongwei, Liang Helan, et al. Intelligent Combustion Control of the Hot Blast Stove: A Reinforcement Learning Approach[J]. Processes, 2023, 11(11): 3140.
[26]	Mathew Mithra Noel, Jaganatha Pandian B. Control of a Nonlinear Liquid Level System Using a New Artificial Neural Network Based Reinforcement Learning Approach[J]. Applied Soft Computing, 2014, 23: 444-451.
[27]	Dutta Debaprasad, Upreti Simant R. A Reinforcement Learning-based Transformed Inverse Model Strategy for Nonlinear Process Control[J]. Computers & Chemical Engineering, 2023, 178: 108386.
[28]	Mendiola-Rodriguez Tannia A, Ricardez-Sandoval Luis A. Robust Control for Anaerobic Digestion Systems of Tequila Vinasses Under Uncertainty: A Deep Deterministic Policy Gradient Algorithm[J]. Digital Chemical Engineering, 2022, 3: 100023.
[29]	Syauqi Ahmad, Kim Heehyang, Lim Hankwon. Optimizing Olefin Purification: An Artificial Intelligence-based Process-conscious PI Controller Tuning for Double Dividing Wall Column Distillation[J]. Chemical Engineering Journal, 2024, 500: 156645.
[30]	Zhang Yang, Wang Peng, Yu Liying, et al. Adaptive Tuning of Dynamic Matrix Control for Uncertain Industrial Systems with Deep Reinforcement Learning[J]. IEEE Transactions on Automation Science and Engineering, 2025, 22: 8695-8708.
[31]	Petsagkourakis P, Sandoval I O, Bradford E, et al. Constrained Reinforcement Learning for Dynamic Optimization under Uncertainty[J]. IFAC-PapersOnLine, 2020, 53(2): 11264-11270.
[32]	Xia Kaishu, Sacco C, Kirkpatrick M, et al. A Digital Twin to Train Deep Reinforcement Learning Agent for Smart Manufacturing Plants: Environment, Interfaces and Intelligence[J]. Journal of Manufacturing Systems, 2021, 58: 210-230.
[33]	Liu Chao, Ding Jinliang, Sun Jiyuan. Reinforcement Learning Based Decision Making of Operational Indices in Process Industry Under Changing Environment[J]. IEEE Transactions on Industrial Informatics, 2021, 17(4): 2727-2736.
[34]	Zhou Ping, Wang Xuan, Chai Tianyou. Multiobjective Operation Optimization of Wastewater Treatment Process Based on Reinforcement Self-learning and Knowledge Guidance[J]. IEEE Transactions on Cybernetics, 2023, 53(11): 6896-6909.
[35]	Zhao Mingming, Wang Ding, Ren Jin, et al. Integrated Online Q-learning Design for Wastewater Treatment Processes[J]. IEEE Transactions on Industrial Informatics, 2025, 21(2): 1833-1842.
[36]	东方仿真科技(北京)有限公司. AI+仿真实验平台[EB/OL]. (2026-01-26) [2026-02-01]. .

参数描述	变量名	空间
甲烷入口浓度/%	VCH4.OP	状态
主风机入口压力/Pa	PTC101.OP	状态
引风机入口压力/Pa	PTC301.OP	状态
主风机频率/Hz	B201HZ.OP	动作
引风机频率/Hz	B301HZ.OP	动作
热旁通阀门开度/%	FV202.OP	动作