Research on Optimization of Airport Cargo Business Based on Deep Reinforcement Learning

doi:10.16182/j.issn1004731x.joss.20-0794

Abstract

Abstract:

An intelligent agent technology architecture is adopted to the simulation model development of airport cargo business. Aiming at the optimization of airport cargo resources, a decision support system framework combining deep reinforcement learning (DRL) and airport cargo business simulation model is proposed. The simulated results are applied as the training data of the DRL network, and the DRL is used to optimize operation parameter of the simulation model. The mature system can be run online, which can provide optimized operation order in real time. In order to verify the effectiveness of the architecture, model development and experiments are conducted in Anylogic simulation platform, and the performances of DRL and OptQuest are compared. The results show that DRL can better optimize airport cargo business on the basis of ensuring orderly airport cargo operations.

Key words: online simulation, airport cargo business, deep reinforcement learning, simulation model optimization

CLC Number:

TP391.9

Hongwei Wang, Peng Yang. Research on Optimization of Airport Cargo Business Based on Deep Reinforcement Learning[J]. Journal of System Simulation, 2022, 34(3): 651-660.

Figures/Tables 17

Table 1

Fig. 1

Fig. 2

Table 2

Table 3

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Table 4

Table 5

Fig. 9

Fig. 10

Fig. 11

Fig. 12

References 11

1	2019年民航行业发展统计公报[R]. 北京: 中国民用航空局, 2020.
	Statistics Bulletin of Civil Aviation Industry Development in 2019[R]. Beijing: CAAC, 2020.
2	白杨, 朱金福. 基于随机Petri网的航空货运出港系统分析[J]. 数理统计与管理, 2011, 31(2): 199-206.
	Bai Yang, Zhu Jinfu. Analysis of Air Cargo Departure System Based on Stochastic Petri Net[J]. Mathematical Statistics and Management, 2011, 31(2): 199-206.
3	赵悦. 航空快递中心作业流程优化和布局规划研究[D]. 北京: 北京交通大学, 2017.
	Zhao Yue. Research on Operation Process Optimization and Layout Planning of Air Express Center[D]. Beijing: Beijing Jiaotong University, 2017.
4	孟钊. 基于随机时间Petri网的万翔航空冷链物流流程优化研究[D]. 天津: 中国民航大学, 2015.
	Meng Zhao. Research on Cold Chain Logistics Process Optimization of Wanxiang Aviation Based on stochastic time Petri Net[D]. Tianjin: Civil Aviation University of China, 2015.
5	Miquel Angel Piera, Novikov Alexey, Trapote Cesar, et al. A Simulation Model to Improve Air Cargo Operations in Passenger Aircraft[C]// Summer Computer Simulation Conference. Ottawa, Canada: SCSC, 2010: 446-451.
6	周泓, 欧建新, 李政道. 航空货运中心物流系统建模及仿真研究[J]. 系统仿真学报, 2008, 20(12): 3314-3320.
	Zhou Hong, Jianxin Ou, Li Zhengdao. Research on the Modeling and Simulation of Air Cargo Center Logistics System[J]. Journal of System Simulation, 2008, 20(12): 3314-3320.
7	崔华春, 尹纯建, 贾焜, 等. 基于实体流程图和Witness仿真的航空货运离港系统仿真分析与优化研究[J]. 物流科技, 2016, 39(10): 83-89.
	Cui Huachun, Yin Chunjian, Jia Kun, et al. Simulation Analysis and Optimization of Air Cargo Departure System Based on Entity Flow Chart and Witness Simulation[J]. Logistics Technology, 2006, 39(10): 83-89.
8	Nsakanda Aaton L., Turcotte Michel. Air Cargo Operations Evaluation and Analysis Through Simulation[C]// 2004 Winter Simulation Conference. Washington, USA: IEEE, 2004: 1790-1798.
9	Mnih V, Kavukcuoglu K, Silver D, et al. Human-Level Control Through Deep Reinforcement Learning[J]. Nature, 2015, 518(7540): 529-533.
10	Law A M, Kelton W D. Simulation Modeling and Analysis[M]. New York, USA: MeGraw-Hill, 2000.
11	刘庆鸿, 陈德源, 王子才. 建模与仿真校核、验证与确认综述[J]. 系统仿真学报, 2003, 15(7): 925-930.
	Liu Qinghong, Chen Deyuan, Wang Zicai. Modeling and Reviews of Verification, Validation of Simulation Models[J]. Journal of System Simulation, 2003, 15(7): 925-930.

机场货运仿真系统	DQN
货物量观测值	环境状态
货运业务系统	学习主体
相关设备增减	决策行为
采取行动的反馈	环境奖励
观测值变化	系统状态变化
货物流动	触发事件启动

时段	客运	货运	合计
00:00—00:59	1	2	3
01:00—01:59	2	2	4
02:00—02:59	4	2	6
03:00—03:59	2	2	4
04:00—04:59	4	2	6
05:00—05:59	3	2	5

参数	初始值
安检口数量/个	4
安检速度/(m·s^-1)	Triangular(3，9，6)
每票所需拖板数量(个)	Triangular(1,8,1)
拖车数量(台)	42
拖车速度/(m·min^-1)	仓库内74.405 仓库外148.81

参数	最优值
安检通道数量	4
进港仓库内拖车数量	2
进港仓库外拖车数量	15

[1]	Jiang Ming, He Tao. Solving the Vehicle Routing Problem Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(9): 2177-2187.
[2]	Ni Peilong, Mao Pengjun, Wang Ning, Yang Mengjie. Robot Path Planning Based on Improved A-DDQN Algorithm [J]. Journal of System Simulation, 2025, 37(9): 2420-2430.
[3]	Chen Zhen, Wu Zhuoyi, Zhang Lin. Research on Policy Representation in Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(7): 1753-1769.
[4]	Wu Guohua, Zeng Jiaheng, Wang Dezhi, Zheng Long, Zou Wei. A Quadrotor Trajectory Tracking Control Method Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(5): 1169-1187.
[5]	Bai Zhenzu, Hou Yizhi, He Zhangming, Wei Juhui, Zhou Haiyin, Wang Jiongqi. Optimization of Dynamic Weapon Target Assignment Considering Random Disturbances [J]. Journal of System Simulation, 2025, 37(12): 2967-2980.
[6]	Zheng Jiayu, Mai Zhuxue, Chen Zheyi. Optimization of Service Caching and Computation Offloading in Digital Twin Cloud-edge Networks [J]. Journal of System Simulation, 2025, 37(11): 2741-2753.
[7]	Di Jian, Wan Xue, Jiang Limei. Evolutionary Reinforcement Learning Based on Elite Instruction and Random Search [J]. Journal of System Simulation, 2025, 37(11): 2877-2887.
[8]	Xu Zhongkai, Chu Chenyang, Xie Kai, Zhao Ruizhuo, Ke Wenjun. Optimization Dispatch Method for High-proportion Renewable Energy Power Systems Based on SC-PPO [J]. Journal of System Simulation, 2025, 37(10): 2511-2521.
[9]	Liang Xiuman, Liu Ziliang, Liu Zhendong. Path Planning of Improved RRT Algorithm Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(10): 2578-2593.
[10]	Jiang Jiachen, Jia Zhengxuan, Xu Zhao, Lin Tingyu, Zhao Pengpeng, Ou Yiming. Decision Modeling and Solution Based on Game Adversarial Complex Systems [J]. Journal of System Simulation, 2025, 37(1): 66-78.
[11]	Qin Baoxin, Zhang Yuxiao, Wu Sirui, Cao Weichong, Li Zhan. Intelligent Optimization of Coal Terminal Unloading Scheduling Based on Improved D3QN Algorithm [J]. Journal of System Simulation, 2024, 36(3): 770-781.
[12]	Li Ming, Ye Wangzhong, Yan Jiehua. Path Planning of Desert Robot Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2024, 36(12): 2917-2925.
[13]	Zhang Yongfu, Liu Yang, Yuan He. A Method for Key Node Identification in Operational Target System Based on War Gaming [J]. Journal of System Simulation, 2024, 36(11): 2654-2661.
[14]	An Jing, Si Guangya, Zhang Lei. Strategy Optimization Method of Multi-dimension Projection Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2024, 36(1): 39-49.
[15]	Laiyi Yang, Jing Bi, Haitao Yuan. Intelligent Path Planning for Mobile Robots Based on SAC Algorithm [J]. Journal of System Simulation, 2023, 35(8): 1726-1736.