基于协作式深度强化学习的火灾应急疏散仿真研究

doi:10.16182/j.issn1004731x.joss.21-0108

系统仿真学报 ›› 2022, Vol. 34 ›› Issue (6): 1353-1366.doi: 10.16182/j.issn1004731x.joss.21-0108

基于协作式深度强化学习的火灾应急疏散仿真研究

倪凌佳^1,²(), 黄晓霞^1,³(), 李红旮¹, 张子博^1,²

^1.中国科学院空天信息创新研究院, 北京 100094
^2.中国科学院大学, 北京 100049
^3.自然资源部城市国土资源监测与仿真重点实验室, 深圳 518034

收稿日期:2021-02-04 修回日期:2021-05-18 出版日期:2022-06-30 发布日期:2022-06-16
通讯作者: 黄晓霞 E-mail:13476143753@163.com;huangxx@aircas.ac.cn
作者简介:倪凌佳(1996-)，男，硕士生，研究方向为火灾应急疏散。E-mail：13476143753@163.com
基金资助:
国家自然科学基金(41971363);自然资源部城市国土资源监测与仿真重点实验室开放基金资助课题(KF-2018-03-032);国家重点研发计划(2017YFB0503905)

Research on Fire Emergency Evacuation Simulation Based on Cooperative Deep Reinforcement Learning

Lingjia Ni^1,²(), Xiaoxia Huang^1,³(), Hongga Li¹, Zibo Zhang^1,²

^1.Aerospace Information Research Institute, Chinese Academy of Sciences, Beijing 100094, China
^2.University of Chinese Academy of Sciences, Beijing 100049, China
^3.Key Laboratory of Urban Land Resources Monitoring and Simulation, MNR, Shenzhen 518034, China

Received:2021-02-04 Revised:2021-05-18 Online:2022-06-30 Published:2022-06-16
Contact: Xiaoxia Huang E-mail:13476143753@163.com;huangxx@aircas.ac.cn

摘要/Abstract

摘要：

火灾是威胁公共安全的主要灾害之一，火灾产生的高温和有毒有害烟气严重影响了疏散路径的选择。将深度强化学习引入到应急疏散仿真研究，针对多智能体环境提出了协作式双深度Q网络算法。建立随时间动态变化的火灾场景模型，为人员疏散提供实时的危险区域分布信息；对各自独立的智能体神经网络进行整合，建立多智能体统一的深度神经网络，实现所有智能体之间的神经网络和经验共享，提高整体协作疏散效率。结果表明：所提方法具有良好的稳定性和适应性，训练和学习效率得到提升，具有良好的应用价值。

关键词: 协作式双深度Q网络算法, 深度强化学习, 多智能体系统, 应急疏散仿真, 火灾场景仿真

Abstract:

The fire accident is a major threat to the public safety, in which the high temperature, toxic and harmful gases seriously interfer the selection of the evacuation routes. Deep reinforcement learning is introduced into the research of emergency evacuation simulation, and a cooperative double deep Q network algorithm is proposed for the multi-agent environment. A fire scene model that changes dynamically over time is established to provide the real-time information on the distribution of the dangerous areas for the evacuation. The independent agent neural networks are integrated and the multi-agent unified deep neural network is established to realize the sharing of the neural network and experience among all agents, and improve the overall cooperative evacuation efficiency. The experimental comparison results show that the proposed method has the good stability and adaptability, improved training and learning efficiency, and good application value.

Key words: cooperative double deep Q network algorithm, deep reinforcement learning, multi-agent system, emergency evacuation simulation, fire scenario simulation

中图分类号:

TP391.9

倪凌佳, 黄晓霞, 李红旮, 张子博. 基于协作式深度强化学习的火灾应急疏散仿真研究[J]. 系统仿真学报, 2022, 34(6): 1353-1366.

Lingjia Ni, Xiaoxia Huang, Hongga Li, Zibo Zhang. Research on Fire Emergency Evacuation Simulation Based on Cooperative Deep Reinforcement Learning[J]. Journal of System Simulation, 2022, 34(6): 1353-1366.

图/表 19

图1

图2

图3

图4

图5

图6

表1

图7

图8

表2

图9

表3

表4

图10

图11

表5

表6

图12

图13

参考文献 33

1	王玲. 公共场所用品的卫生检测分析[J]. 中外医学研究, 2010(10): 98.
	Wang Ling. Hygienic Detection and Analysis of Articles in Public Places[J]. Chinese and Foreign Medical Research, 2010(10): 98.
2	储浩然. 5G新技术全面赋能智慧消防行业[J]. 中国安防, 2020(5): 47-49.
	Chu Haoran. 5G New Technology Fully Empowers the Smart Fire Protection Industry[J]. China Security, 2020(5): 47-49.
3	Bryan J L. A Review of the Examination and Analysis of the Dynamics of Human Behavior in the Fire at the MGM Grand Hotel, Clark County, Nevada as Determined from a Selected Questionnaire Population [J]. Fire Safety Journal(S0379-7112), 1983, 5(3/4): 233-240.
4	Cremer M, Ludwig J. A Fast Simulation Model for Traffic Flow on the Basis of Boolean Operations[J]. Mathematics and Computers in Simulation, 1986, 28(4): 297-303.
5	宋英华, 张宇, 霍非舟, 等. 考虑避让行为的人员疏散元胞自动机模型研究[J]. 系统仿真学报, 2020, 32(6): 975-981.
	Song Yinghua, Zhang Yu, Huo Feizhou, et al. Study on Evacuation Cellular Automaton Model Considering Avoidance Behavior[J]. Journal of System Simulation, 2020, 32(6): 975-981.
6	Helbing D, Molnar P. Social Force Model for Pedestrian Dynamics[J]. Physical Review E (S1539-3755), 1995, 51(5): 4282-4286.
7	邓媛媛, 郑利平, 蔡瑞文. 社会行为驱动的疏散仿真方法研究[J]. 系统仿真学报, 2020, 32(1): 130-141.
	Deng Yuanyuan, Zheng Liping, Cai Ruiwen. Research on Evacuation Simulation Method Considering Social Behavior[J]. Journal of System Simulation, 2020, 32(1): 130-141.
8	李昌华, 毕成功, 李智杰. 基于改进PSO算法的人群疏散模型[J]. 系统仿真学报, 2020, 32(6): 1000-1008.
	Li Changhua, Bi Chenggong, Li Zhijie. Crowd Evacuation Model Based on Improved PSO Algorithm[J]. Journal of System Simulation, 2020, 32(6): 1000-1008.
9	Huang X, Li H, Li X, et al. Fire Numerical Simulation Analysis for Large-scale Public Building in 3D GIS[C]// 2019 IEEE International Geoscience and Remote Sensing Symposium. Yokohama: IEEE, 2019: 7522-7525.
10	Berg J V D, Lin M, Manocha D. Reciprocal Velocity Obstacles for Real-Time Multi-Agent Navigation[C]//IEEE International Conference on Robotics and Automatics. Pasadena: IEEE, 2008: 1928-1935.
11	林海波, 郑利平, 王建伟, 等. 一种通用的层次化人群疏散仿真框架[J]. 系统仿真学报, 2020, 32(8): 1524-1530.
	Lin Haibo, Zheng Liping, Wang Jianwei, et al. A Common and Hierarchical Crowd Evacuation Simulation Framework[J]. Journal of System Simulation, 2020, 32(8): 1524-1530.
12	Mnih V, Kavukcuoglu K, Silver D, et al. Playing Atari with Deep Reinforcement Learning[J/OL]. Computer Ence, [2021-03-18]. .
13	Lillicrap T P, Hunt J J, Pritzel A, et al. Continuous Control with Deep Reinforcement Learning[J/OL]. [2021-03-18]. .
14	Mnih V, Badia A P, Mirza M, et al. Asynchronous Methods for Deep Reinforcement Learning[C]//Interna-tional Conference on Machine Learning. PMLR: 2016: 1928-1937.
15	Schulman J, Wolski F, Dhariwal P, et al. Proximal Policy Optimization Algorithms[J/OL]. [2021-03-18]. .
16	Mnih V, Kavukcuoglu K, Silver D, et al. Human-Level Control Through Deep Reinforcement Learning[J]. Nature (S0028-0836), 2015, 518(7540): 529-533.
17	Levine S, Pastor P, Krizhevsky A, et al. Learning Hand-Eye Coordination for Robotic Grasping with Large-Scale Data Collection[C]//International Symposium on Experimental Robotics. 2017.
18	Satija H, Pineau J. Simultaneous Machine Translation Using Deep Reinforcement Learning[C]//Workshops of International Conference on Machine Learning. New York: 2016: 110-119.
19	Sallab A, Abdou M, Perot E, et al. Deep Reinforcement Learning Framework for Autonomous Driving[J]. Electronic Imaging(S1017-9909), 2017(19): 70-76.
20	Tai L, Paolo G, Liu M. Virtual-to-Real Deep Reinforcement Learning: Continuous Control of Mobile Robots for Mapless Navigation[C]//2017 IEEE/RSJ International Conference on Intelligent Robots and Systems. Vancouver: IEEE. 2017: 31-36.
21	Yu H, Zhang H, Xu W. A Deep Compositional Framework for Human-Like Language Acquisition in Virtual Environment[J/OL]. [2021-03-18]. .
22	Chen Y F, Liu M, Everett M F, et al. Decentralized Non-Communicating Multiagent Collision Avoidance with Deep Reinforcement Learning[C]//2017 IEEE Interna-tional Conference on Robotics and Automation. Singa-pore: IEEE, 2017: 285-292.
23	Van H H, Guez A, Silver D. Deep Reinforcement Learning with Double Q-Learning[C]//Thirtieth AAAI Conference on Aitificial Intelligence. Palo Alto, CA: AAAI, 2016: 2094-2100.
24	孙彧, 曹雷, 陈希亮, 等. 多智能体深度强化学习研究综述[J]. 计算机工程与应用, 2020, 56(5): 13-24.
	Sun Yu, Cao Lei, Chen Xiliang, et al. A Review of Multi-Agent Deep Reinforcement Learning[J]. Computer Engineering and Applications, 2020, 56(5): 13-24.
25	陈延敏, 李锦华. 国内外建筑信息模型 (BIM) 理论与实践研究综述[J]. 城市, 2013(10): 72-76.
	Chen Yanmin, Li Jinhua. A Summary of the Research on the Theory and Practice of Building Information Modeling (BIM) at Home and Abroad[J]. City, 2013(10): 72-76.
26	刘阳. 基于FDS的建筑火灾数值模拟及安全疏散研究[D]. 阜新: 辽宁工程技术大学, 2013.
	Liu Yang. Research on Building Fire Numerical Simulation and Safe Evacuation Based on FDS[D]. Fuxin: Liaoning Technical University, 2013.
27	毕伟民. 基于火灾动力学和疏散理论耦合的人员疏散研究[D]. 西安: 建筑科技大学, 2008.
	Bi Weimin. Evacuation Research Based on the Coupling of Fire Dynamics and Evacuation Theory[D]. Xi'an: Xi'an University of Architecture and Technology, 2008.
28	沙云飞. 人群疏散的微观仿真模型研究[D]. 北京: 清华大学, 2008.
	Sha Yunfei. Research on Microscopic Simulation Model of Crowd Evacuation[D]. Beijing: Tsinghua University, 2008.
29	张江涛. 基于FDS的办公大楼火灾数值模拟研究[D]. 邯郸: 河北工程大学, 2015.
	Zhang Jiangtao. Research on Numerical Simulation of Office Building Fire Based on FDS[D]. Handan: Hebei University of Engineering, 2015.
30	李引擎, 史毅. 建筑防火工程[M]. 北京: 化学工业出版社, 2004.
	Li Yinqing, Shi Yi. Building Fire Protection Engineering[M]. Beijing: Chemical Industry Press, 2004.
31	Li B, Liang H. Multi-Robot Path Planning Method Based on Prior Knowledge and Q-learning Algorithms[J]. Journal of Physics: Conference Series, 2020, 1624(4):042008.
32	Lhan E, Gow J, Perezliebana D. Teaching on a Budget in Multi-Agent Deep Reinforcement Learning[C]//2019 IEEE Conference on Games. London: IEEE, 2019: 1-8.
33	Koh S, Zhou B, Fang H, et al. Real-Time Deep Reinforcement Learning Based Vehicle Routing and Navigation[J]. Applied Soft Computing(S1568-4946), 2020, 96: 106694.

输入参数	数值/属性
环境温度/℃ 风速/(m/s)	20 0
燃料类型	聚氯乙烯
火源点面积/m²	1
模拟时间/s	90
有无喷淋条件有无机械排烟装置	无无
地板属性	绝热惰性

参数	数值
环境长度/m	51.2
环境宽度/m	52
网格边长/m	0.4
疏散人数	100
出口数训练轮数	3 1 000
经验回放池容量	20 000
批量学习样本数衰减系数	16 0.95
学习速率	0.000 3

1 000轮算法训练情况		无火灾场景		火灾场景
1 000轮算法训练情况		IDQN算法	协作式DDQN算法	IDQN算法	协作式DDQN算法
疏散仿真总时间/s	第1次实验	61 348	55 386	66 612	63 153
	第2次实验	50 231	56 727	69 642	60 514
	第3次实验	51 819	49 820	63 960	63 044
	3次实验平均	54 466	53 978	66 738	62 237
初始收敛轮数	第1次实验	610	570	670	650
	第2次实验	600	630	830	710
	第3次实验	640	620	700	680
	3次实验平均	617	607	733	680

算法疏散结果		无火灾场景		火灾场景
算法疏散结果		IDQN算法	协作式DDQN算法	IDQN算法	协作式DDQN算法
平均每人疏散花费时间/s	第1次实验	14.93	14.93	15.11	14.97
	第2次实验	14.93	14.92	15.05	14.96
	第3次实验	14.92	14.91	15.13	14.98
	3次实验平均	14.93	14.92	15.10	14.97
人员全部疏散花费时间	第1次实验	26.00	26.00	26.00	26.00
	第2次实验	26.00	26.00	27.00	26.00
	第3次实验	26.00	26.00	26.33	26.00
	3次实验平均	26.00	26.00	26.44	26.00

[1]	张森, 张孟炎, 邵敬平, 普杰信. 基于随机策略搜索的多机三维路径规划方法[J]. 系统仿真学报, 2022, 34(6): 1286-1295.
[2]	王红微, 杨鹏. 基于深度强化学习的机场货运业务优化研究[J]. 系统仿真学报, 2022, 34(3): 651-660.
[3]	李启锐, 彭心怡. 基于深度强化学习的云作业调度及仿真研究[J]. 系统仿真学报, 2022, 34(2): 258-268.
[4]	桂欣冬, 吉鸿江, 范玲玲, 刘世达. 信任驱动的自适应协调控制算法及应用[J]. 系统仿真学报, 2021, 33(8): 1809-1817.
[5]	郭海湘, 曾杨, 陈卫明. 基于社会力模型的多出口室内应急疏散仿真研究[J]. 系统仿真学报, 2021, 33(3): 721-731.
[6]	高昂, 董志明, 张国辉, 梁涛, 郭齐胜. LVC训练系统中计算机生成兵力生成技术研究[J]. 系统仿真学报, 2021, 33(3): 745-752.
[7]	曾贲, 房霄, 孔德帅, 宋祥祥, 贾政轩, 林廷宇. 一种数据驱动的对抗博弈智能体建模方法[J]. 系统仿真学报, 2021, 33(12): 2838-2845.
[8]	梁魏, 黄炎焱, 王建宇. 灾场内、外域相结合的应急响应建模方法研究[J]. 系统仿真学报, 2020, 32(4): 669-677.
[9]	付跃文, 王元诚, 陈珍, 范文澜. 基于多智能体粒子群的协同空战目标决策研究[J]. 系统仿真学报, 2018, 30(11): 4151-4157.
[10]	赵元, 程家昌, 王璐, 胡月明. 基于GPU的一类地理多智能体系统并行仿真研究[J]. 系统仿真学报, 2015, 27(2): 396-403.

基于协作式深度强化学习的火灾应急疏散仿真研究

Research on Fire Emergency Evacuation Simulation Based on Cooperative Deep Reinforcement Learning

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 19

参考文献 33

相关文章 10

编辑推荐

Metrics

本文评价

疏散时间/s	无火灾场景剩余人数	火灾场景剩余人数
0	100	100
10	84	85
20	14	14
26	0	0