Research on Fire Emergency Evacuation Simulation Based on Cooperative Deep Reinforcement Learning

doi:10.16182/j.issn1004731x.joss.21-0108

Abstract

Abstract:

The fire accident is a major threat to the public safety, in which the high temperature, toxic and harmful gases seriously interfer the selection of the evacuation routes. Deep reinforcement learning is introduced into the research of emergency evacuation simulation, and a cooperative double deep Q network algorithm is proposed for the multi-agent environment. A fire scene model that changes dynamically over time is established to provide the real-time information on the distribution of the dangerous areas for the evacuation. The independent agent neural networks are integrated and the multi-agent unified deep neural network is established to realize the sharing of the neural network and experience among all agents, and improve the overall cooperative evacuation efficiency. The experimental comparison results show that the proposed method has the good stability and adaptability, improved training and learning efficiency, and good application value.

Key words: cooperative double deep Q network algorithm, deep reinforcement learning, multi-agent system, emergency evacuation simulation, fire scenario simulation

CLC Number:

TP391.9

Lingjia Ni, Xiaoxia Huang, Hongga Li, Zibo Zhang. Research on Fire Emergency Evacuation Simulation Based on Cooperative Deep Reinforcement Learning[J]. Journal of System Simulation, 2022, 34(6): 1353-1366.

Figures/Tables 19

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Table 1

Fig. 7

Fig. 8

Table 2

Fig. 9

Table 3

Table 4

Fig. 10

Fig. 11

Table 5

Table 6

Fig. 12

Fig. 13

References 33

1	王玲. 公共场所用品的卫生检测分析[J]. 中外医学研究, 2010(10): 98.
	Wang Ling. Hygienic Detection and Analysis of Articles in Public Places[J]. Chinese and Foreign Medical Research, 2010(10): 98.
2	储浩然. 5G新技术全面赋能智慧消防行业[J]. 中国安防, 2020(5): 47-49.
	Chu Haoran. 5G New Technology Fully Empowers the Smart Fire Protection Industry[J]. China Security, 2020(5): 47-49.
3	Bryan J L. A Review of the Examination and Analysis of the Dynamics of Human Behavior in the Fire at the MGM Grand Hotel, Clark County, Nevada as Determined from a Selected Questionnaire Population [J]. Fire Safety Journal(S0379-7112), 1983, 5(3/4): 233-240.
4	Cremer M, Ludwig J. A Fast Simulation Model for Traffic Flow on the Basis of Boolean Operations[J]. Mathematics and Computers in Simulation, 1986, 28(4): 297-303.
5	宋英华, 张宇, 霍非舟, 等. 考虑避让行为的人员疏散元胞自动机模型研究[J]. 系统仿真学报, 2020, 32(6): 975-981.
	Song Yinghua, Zhang Yu, Huo Feizhou, et al. Study on Evacuation Cellular Automaton Model Considering Avoidance Behavior[J]. Journal of System Simulation, 2020, 32(6): 975-981.
6	Helbing D, Molnar P. Social Force Model for Pedestrian Dynamics[J]. Physical Review E (S1539-3755), 1995, 51(5): 4282-4286.
7	邓媛媛, 郑利平, 蔡瑞文. 社会行为驱动的疏散仿真方法研究[J]. 系统仿真学报, 2020, 32(1): 130-141.
	Deng Yuanyuan, Zheng Liping, Cai Ruiwen. Research on Evacuation Simulation Method Considering Social Behavior[J]. Journal of System Simulation, 2020, 32(1): 130-141.
8	李昌华, 毕成功, 李智杰. 基于改进PSO算法的人群疏散模型[J]. 系统仿真学报, 2020, 32(6): 1000-1008.
	Li Changhua, Bi Chenggong, Li Zhijie. Crowd Evacuation Model Based on Improved PSO Algorithm[J]. Journal of System Simulation, 2020, 32(6): 1000-1008.
9	Huang X, Li H, Li X, et al. Fire Numerical Simulation Analysis for Large-scale Public Building in 3D GIS[C]// 2019 IEEE International Geoscience and Remote Sensing Symposium. Yokohama: IEEE, 2019: 7522-7525.
10	Berg J V D, Lin M, Manocha D. Reciprocal Velocity Obstacles for Real-Time Multi-Agent Navigation[C]//IEEE International Conference on Robotics and Automatics. Pasadena: IEEE, 2008: 1928-1935.
11	林海波, 郑利平, 王建伟, 等. 一种通用的层次化人群疏散仿真框架[J]. 系统仿真学报, 2020, 32(8): 1524-1530.
	Lin Haibo, Zheng Liping, Wang Jianwei, et al. A Common and Hierarchical Crowd Evacuation Simulation Framework[J]. Journal of System Simulation, 2020, 32(8): 1524-1530.
12	Mnih V, Kavukcuoglu K, Silver D, et al. Playing Atari with Deep Reinforcement Learning[J/OL]. Computer Ence, [2021-03-18]. .
13	Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous Control with Deep Reinforcement Learning[J/OL]. [2021-03-18]. .
14	Mnih V, Badia A P, Mirza M, et al. Asynchronous Methods for Deep Reinforcement Learning[C]//Interna-tional Conference on Machine Learning. PMLR: 2016: 1928-1937.
15	Schulman J, Wolski F, Dhariwal P, et al. Proximal Policy Optimization Algorithms[J/OL]. [2021-03-18]. .
16	Mnih V, Kavukcuoglu K, Silver D, et al. Human-Level Control Through Deep Reinforcement Learning[J]. Nature (S0028-0836), 2015, 518(7540): 529-533.
17	Levine S, Pastor P, Krizhevsky A, et al. Learning Hand-Eye Coordination for Robotic Grasping with Large-Scale Data Collection[C]//International Symposium on Experimental Robotics. 2017.
18	Satija H, Pineau J. Simultaneous Machine Translation Using Deep Reinforcement Learning[C]//Workshops of International Conference on Machine Learning. New York: 2016: 110-119.
19	Sallab A, Abdou M, Perot E, et al. Deep Reinforcement Learning Framework for Autonomous Driving[J]. Electronic Imaging(S1017-9909), 2017(19): 70-76.
20	Tai L, Paolo G, Liu M. Virtual-to-Real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. Vancouver: IEEE. 2017: 31-36.
21	Yu H, Zhang H, Xu W. A Deep Compositional Framework for Human-Like Language Acquisition in Virtual Environment[J/OL]. [2021-03-18]. .
22	Chen Y F, Liu M, Everett M F, et al. Decentralized Non-Communicating Multiagent Collision Avoidance with Deep Reinforcement Learning[C]//2017 IEEE Interna-tional Conference on Robotics and Automation. Singa-pore: IEEE, 2017: 285-292.
23	Van H H, Guez A, Silver D. Deep Reinforcement Learning with Double Q-Learning[C]//Thirtieth AAAI Conference on Aitificial Intelligence. Palo Alto, CA: AAAI, 2016: 2094-2100.
24	孙彧, 曹雷, 陈希亮, 等. 多智能体深度强化学习研究综述[J]. 计算机工程与应用, 2020, 56(5): 13-24.
	Sun Yu, Cao Lei, Chen Xiliang, et al. A Review of Multi-Agent Deep Reinforcement Learning[J]. Computer Engineering and Applications, 2020, 56(5): 13-24.
25	陈延敏, 李锦华. 国内外建筑信息模型 (BIM) 理论与实践研究综述[J]. 城市, 2013(10): 72-76.
	Chen Yanmin, Li Jinhua. A Summary of the Research on the Theory and Practice of Building Information Modeling (BIM) at Home and Abroad[J]. City, 2013(10): 72-76.
26	刘阳. 基于FDS的建筑火灾数值模拟及安全疏散研究[D]. 阜新: 辽宁工程技术大学, 2013.
	Liu Yang. Research on Building Fire Numerical Simulation and Safe Evacuation Based on FDS[D]. Fuxin: Liaoning Technical University, 2013.
27	毕伟民. 基于火灾动力学和疏散理论耦合的人员疏散研究[D]. 西安: 建筑科技大学, 2008.
	Bi Weimin. Evacuation Research Based on the Coupling of Fire Dynamics and Evacuation Theory[D]. Xi'an: Xi'an University of Architecture and Technology, 2008.
28	沙云飞. 人群疏散的微观仿真模型研究[D]. 北京: 清华大学, 2008.
	Sha Yunfei. Research on Microscopic Simulation Model of Crowd Evacuation[D]. Beijing: Tsinghua University, 2008.
29	张江涛. 基于FDS的办公大楼火灾数值模拟研究[D]. 邯郸: 河北工程大学, 2015.
	Zhang Jiangtao. Research on Numerical Simulation of Office Building Fire Based on FDS[D]. Handan: Hebei University of Engineering, 2015.
30	李引擎, 史毅. 建筑防火工程[M]. 北京: 化学工业出版社, 2004.
	Li Yinqing, Shi Yi. Building Fire Protection Engineering[M]. Beijing: Chemical Industry Press, 2004.
31	Li B, Liang H. Multi-Robot Path Planning Method Based on Prior Knowledge and Q-learning Algorithms[J]. Journal of Physics: Conference Series, 2020, 1624(4):042008.
32	Lhan E, Gow J, Perezliebana D. Teaching on a Budget in Multi-Agent Deep Reinforcement Learning[C]//2019 IEEE Conference on Games. London: IEEE, 2019: 1-8.
33	Koh S, Zhou B, Fang H, et al. Real-Time Deep Reinforcement Learning Based Vehicle Routing and Navigation[J]. Applied Soft Computing(S1568-4946), 2020, 96: 106694.

输入参数	数值/属性
环境温度/℃ 风速/(m/s)	20 0
燃料类型	聚氯乙烯
火源点面积/m²	1
模拟时间/s	90
有无喷淋条件有无机械排烟装置	无无
地板属性	绝热惰性

参数	数值
环境长度/m	51.2
环境宽度/m	52
网格边长/m	0.4
疏散人数	100
出口数训练轮数	3 1 000
经验回放池容量	20 000
批量学习样本数衰减系数	16 0.95
学习速率	0.000 3

1 000轮算法训练情况		无火灾场景		火灾场景
1 000轮算法训练情况		IDQN算法	协作式DDQN算法	IDQN算法	协作式DDQN算法
疏散仿真总时间/s	第1次实验	61 348	55 386	66 612	63 153
	第2次实验	50 231	56 727	69 642	60 514
	第3次实验	51 819	49 820	63 960	63 044
	3次实验平均	54 466	53 978	66 738	62 237
初始收敛轮数	第1次实验	610	570	670	650
	第2次实验	600	630	830	710
	第3次实验	640	620	700	680
	3次实验平均	617	607	733	680

算法疏散结果		无火灾场景		火灾场景
算法疏散结果		IDQN算法	协作式DDQN算法	IDQN算法	协作式DDQN算法
平均每人疏散花费时间/s	第1次实验	14.93	14.93	15.11	14.97
	第2次实验	14.93	14.92	15.05	14.96
	第3次实验	14.92	14.91	15.13	14.98
	3次实验平均	14.93	14.92	15.10	14.97
人员全部疏散花费时间	第1次实验	26.00	26.00	26.00	26.00
	第2次实验	26.00	26.00	27.00	26.00
	第3次实验	26.00	26.00	26.33	26.00
	3次实验平均	26.00	26.00	26.44	26.00

[1]	Sen Zhang, Mengyan Zhang, Jingping Shao, Jiexin Pu. Multi-UAVs 3D Path Planning Method Based on Random Strategy Search [J]. Journal of System Simulation, 2022, 34(6): 1286-1295.
[2]	Xin Zhou, Weiping Wang, Yifan Zhu, Tao Wang, Tian Jing. An Unmanned Swarm Search Method Based on Human-Robot Cooperation [J]. Journal of System Simulation, 2022, 34(4): 735-744.
[3]	Hongwei Wang, Peng Yang. Research on Optimization of Airport Cargo Business Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2022, 34(3): 651-660.
[4]	Qirui Li, Xinyi Peng. Job Scheduling and Simulation in Cloud Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2022, 34(2): 258-268.
[5]	Qi Xiaolong, Yang Xuguang. Time-varying Output Formation Tracking Control of Discrete-time Heterogeneous Multi-agent Systems [J]. Journal of System Simulation, 2022, 34(1): 36-44.
[6]	Gui Xindong, Ji Hongjiang, Fan Lingling, Liu Shida. Application of Trust Driven Adaptive Cooperative Control Algorithm [J]. Journal of System Simulation, 2021, 33(8): 1809-1817.
[7]	Guo Haixiang, Zeng Yang, Chen Weiming. Research on Indoor Emergency Evacuation Simulation of Multi-Exit Based on social Force Model [J]. Journal of System Simulation, 2021, 33(3): 721-731.
[8]	Gao Ang, Dong Zhiming, Zhang Guohui, Liang Tao, Guo Qisheng. Research on Generation Technology of Computer Generated Force in LVC Training System [J]. Journal of System Simulation, 2021, 33(3): 745-752.
[9]	Zeng Bi, Fang Xiao, Kong Deshuai, Song Xiangxiang, Jia Zhengxuan, Lin Tingyu. A Data-Driven Modeling Method for Game Adversity Agent [J]. Journal of System Simulation, 2021, 33(12): 2838-2845.
[10]	Liang Wei, Huang Yanyan, Wang Jianyu. Modeling Method of Emergency Response based on Internal and External Areas of Disaster Zone [J]. Journal of System Simulation, 2020, 32(4): 669-677.
[11]	Fu Yuewen, Wang Yuancheng, Chen Zhen, Fan Wenlan. Target Decision in Collaborative Air Combats Using Multi-agent Particle Swarm Optimization [J]. Journal of System Simulation, 2018, 30(11): 4151-4157.
[12]	Jiang Wuxue, Hu Xuanzi, Liu Minxia, Chen Yuqiang. Information Collaboration Model of Cloud Computing Supply Chain Based on Multi-Agent [J]. Journal of System Simulation, 2016, 28(1): 51-56.
[13]	Zhao Yuan, Cheng Jiachang, Wang Lu, Hu Yueming. GPU-Accelerated Simulation for Class of Multi-Agent Based Models [J]. Journal of System Simulation, 2015, 27(2): 396-403.