Research on Optimization of Airport Cargo Business Based on Deep Reinforcement Learning

doi:10.16182/j.issn1004731x.joss.20-0794

Abstract

Abstract:

An intelligent agent technology architecture is adopted to the simulation model development of airport cargo business. Aiming at the optimization of airport cargo resources, a decision support system framework combining deep reinforcement learning (DRL) and airport cargo business simulation model is proposed. The simulated results are applied as the training data of the DRL network, and the DRL is used to optimize operation parameter of the simulation model. The mature system can be run online, which can provide optimized operation order in real time. In order to verify the effectiveness of the architecture, model development and experiments are conducted in Anylogic simulation platform, and the performances of DRL and OptQuest are compared. The results show that DRL can better optimize airport cargo business on the basis of ensuring orderly airport cargo operations.

Key words: online simulation, airport cargo business, deep reinforcement learning, simulation model optimization

CLC Number:

TP391.9

Hongwei Wang, Peng Yang. Research on Optimization of Airport Cargo Business Based on Deep Reinforcement Learning[J]. Journal of System Simulation, 2022, 34(3): 651-660.

Figures/Tables 17

Table 1

Fig. 1

Fig. 2

Table 2

Table 3

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 4

Table 5

Fig. 9

Fig. 10

Fig. 11

Fig. 12

References 11

1	2019年民航行业发展统计公报[R]. 北京: 中国民用航空局, 2020.
	Statistics Bulletin of Civil Aviation Industry Development in 2019[R]. Beijing: CAAC, 2020.
2	白杨, 朱金福. 基于随机Petri网的航空货运出港系统分析[J]. 数理统计与管理, 2011, 31(2): 199-206.
	Bai Yang, Zhu Jinfu. Analysis of Air Cargo Departure System Based on Stochastic Petri Net[J]. Mathematical Statistics and Management, 2011, 31(2): 199-206.
3	赵悦. 航空快递中心作业流程优化和布局规划研究[D]. 北京: 北京交通大学, 2017.
	Zhao Yue. Research on Operation Process Optimization and Layout Planning of Air Express Center[D]. Beijing: Beijing Jiaotong University, 2017.
4	孟钊. 基于随机时间Petri网的万翔航空冷链物流流程优化研究[D]. 天津: 中国民航大学, 2015.
	Meng Zhao. Research on Cold Chain Logistics Process Optimization of Wanxiang Aviation Based on stochastic time Petri Net[D]. Tianjin: Civil Aviation University of China, 2015.
5	Miquel Angel Piera, Novikov Alexey, Trapote Cesar, et al. A Simulation Model to Improve Air Cargo Operations in Passenger Aircraft[C]// Summer Computer Simulation Conference. Ottawa, Canada: SCSC, 2010: 446-451.
6	周泓, 欧建新, 李政道. 航空货运中心物流系统建模及仿真研究[J]. 系统仿真学报, 2008, 20(12): 3314-3320.
	Zhou Hong, Jianxin Ou, Li Zhengdao. Research on the Modeling and Simulation of Air Cargo Center Logistics System[J]. Journal of System Simulation, 2008, 20(12): 3314-3320.
7	崔华春, 尹纯建, 贾焜, 等. 基于实体流程图和Witness仿真的航空货运离港系统仿真分析与优化研究[J]. 物流科技, 2016, 39(10): 83-89.
	Cui Huachun, Yin Chunjian, Jia Kun, et al. Simulation Analysis and Optimization of Air Cargo Departure System Based on Entity Flow Chart and Witness Simulation[J]. Logistics Technology, 2006, 39(10): 83-89.
8	Nsakanda Aaton L., Turcotte Michel. Air Cargo Operations Evaluation and Analysis Through Simulation[C]// 2004 Winter Simulation Conference. Washington, USA: IEEE, 2004: 1790-1798.
9	Mnih V, Kavukcuoglu K, Silver D, et al. Human-Level Control Through Deep Reinforcement Learning[J]. Nature, 2015, 518(7540): 529-533.
10	Law A M, Kelton W D. Simulation Modeling and Analysis[M]. New York, USA: MeGraw-Hill, 2000.
11	刘庆鸿, 陈德源, 王子才. 建模与仿真校核、验证与确认综述[J]. 系统仿真学报, 2003, 15(7): 925-930.
	Liu Qinghong, Chen Deyuan, Wang Zicai. Modeling and Reviews of Verification, Validation of Simulation Models[J]. Journal of System Simulation, 2003, 15(7): 925-930.

机场货运仿真系统	DQN
货物量观测值	环境状态
货运业务系统	学习主体
相关设备增减	决策行为
采取行动的反馈	环境奖励
观测值变化	系统状态变化
货物流动	触发事件启动

时段	客运	货运	合计
00:00—00:59	1	2	3
01:00—01:59	2	2	4
02:00—02:59	4	2	6
03:00—03:59	2	2	4
04:00—04:59	4	2	6
05:00—05:59	3	2	5

参数	初始值
安检口数量/个	4
安检速度/(m·s^-1)	Triangular(3，9，6)
每票所需拖板数量(个)	Triangular(1,8,1)
拖车数量(台)	42
拖车速度/(m·min^-1)	仓库内74.405 仓库外148.81

参数	最优值
安检通道数量	4
进港仓库内拖车数量	2
进港仓库外拖车数量	15

[1]	Sen Zhang, Mengyan Zhang, Jingping Shao, Jiexin Pu. Multi-UAVs 3D Path Planning Method Based on Random Strategy Search [J]. Journal of System Simulation, 2022, 34(6): 1286-1295.
[2]	Lingjia Ni, Xiaoxia Huang, Hongga Li, Zibo Zhang. Research on Fire Emergency Evacuation Simulation Based on Cooperative Deep Reinforcement Learning [J]. Journal of System Simulation, 2022, 34(6): 1353-1366.
[3]	Qirui Li, Xinyi Peng. Job Scheduling and Simulation in Cloud Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2022, 34(2): 258-268.
[4]	Gao Ang, Dong Zhiming, Zhang Guohui, Liang Tao, Guo Qisheng. Research on Generation Technology of Computer Generated Force in LVC Training System [J]. Journal of System Simulation, 2021, 33(3): 745-752.
[5]	Zeng Bi, Fang Xiao, Kong Deshuai, Song Xiangxiang, Jia Zhengxuan, Lin Tingyu. A Data-Driven Modeling Method for Game Adversity Agent [J]. Journal of System Simulation, 2021, 33(12): 2838-2845.