Task Scheduling for Internet of Vehicles Based on Deep Reinforcement Learning in Edge Computing

doi:10.16182/j.issn1004731x.joss.22-0841

Abstract

Abstract:

Aiming at the offloading and execution of delay-constrained computing tasks for internet of vehicles in edge computing, a task scheduling method based on deep reinforcement learning is proposed. In multi-edge server scenario, a software-defined network-aided internet of vehicles task offloading system is built. On this basis, the task scheduling model of vehicle computation offloading is given. According to the characteristics of task scheduling, a scheduling method based on an improved pointer network is designed. Considering the complexity of task scheduling and computing resource allocation, the deep reinforcement learning algorithm is used to train the pointer network. The vehicle offloading tasks is scheduled by the trained pointer network. The simulation results show that with the same computing resources of edge servers, the proposed method is better than other methods in processing the number of delay-constrained computing tasks, and effectively improves the service capability of the internet of vehicles task offloading system.

Key words: internet of vehicles, edge computing, task scheduling, pointer network, deep reinforcement learning

CLC Number:

TP391.9

Ju Xiang, Su Shengchao, Xu Chaojie, He Beibei. Task Scheduling for Internet of Vehicles Based on Deep Reinforcement Learning in Edge Computing[J]. Journal of System Simulation, 2023, 35(12): 2550-2559.

Figures/Tables 10

Fig. 1

Fig. 2

Fig. 3

Table 1

Default parameters setting

参数	描述	取值
$D i i n$ /KB	$Ψ i$ 的数据量	400~1 000
$D i o u t$ /KB	$Ψ i$ 执行结果的数据量	50~200
$C i$ /MHz	$Ψ i$ 执行所需的CPU cycles	200~1 500
$τ$ /s	$Ψ i$ 时间延迟约束	0.5~1.0
$f s$ /GHz	服务器默认CPU频率	4
$N$	服务器默认个数	4
$B u p$ /MHz	车辆的带宽	10~20
$B d o w n$ /MHz	RSU的带宽	30~40
$ε$	路径损失指数	4
$p i$ /mW	车辆 $i$ 发送功率	100
$p j$ /mW	RSU- $j$ 发送功率	100
$h 0$	复高斯通道系数	$h 0 ~ C N (0,1)$
$N 0$ /dBm	加性高斯白噪声	100

Table 1

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

References 20

1	Zhai Yanlong, Sun Wenxin, Wu Jianqing, et al. An Energy Aware Offloading Scheme for Interdependent Applications in Software-defined IoV with Fog Computing Architecture[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(6): 3813-3823.
2	Wang Jingjing, Jiang Chunxiao, Zhang Kai, et al. Vehicular Sensing Networks in a Smart City: Principles, Technologies and Applications[J]. IEEE Wireless Communications, 2018, 25(1): 122-132.
3	Cao Xiaowen, Wang Feng, Xu Jie, et al. Joint Computation and Communication Cooperation for Energy-efficient Mobile Edge Computing[J]. IEEE Internet of Things Journal, 2019, 6(3): 4188-4200.
4	Zhu Tongxin, Shi Tuo, Li Jianzhong, et al. Task Scheduling in Deadline-aware Mobile Edge Computing Systems[J]. IEEE Internet of Things Journal, 2019, 6(3): 4854-4866.
5	刘雷, 陈晨, 冯杰, 等. 车载边缘计算中任务卸载和服务缓存的联合智能优化[J]. 通信学报, 2021, 42(1): 18-26.
	Liu Lei, Chen Chen, Feng Jie, et al. Joint Intelligent Optimization of Task Offloading and Service Caching for Vehicular Edge Computing[J]. Journal on Communications, 2021, 42(1): 18-26.
6	Li Qiuping, Zhao Junhui, Gong Yi, et al. Energy-efficient Computation Offloading and Resource Allocation in Fog Computing for Internet of Everything[J]. China Communications, 2019, 16(3): 32-41.
7	谢人超, 廉晓飞, 贾庆民, 等. 移动边缘计算卸载技术综述[J]. 通信学报, 2018, 39(11): 138-155.
	Xie Renchao, Lian Xiaofei, Jia Qingmin, et al. Survey on Computation Offloading in Mobile Edge Computing[J]. Journal on Communications, 2018, 39(11): 138-155.
8	Feng Jie, Richard Yu F, Pei Qingqi, et al. Cooperative Computation Offloading and Resource Allocation for Blockchain-enabled Mobile-edge Computing: A Deep Reinforcement Learning Approach[J]. IEEE Internet of Things Journal, 2020, 7(7): 6214-6228.
9	Xing Hong, Liu Liang, Xu Jie, et al. Joint Task Assignment and Resource Allocation for D2D-enabled Mobile-edge Computing[J]. IEEE Transactions on Communications, 2019, 67(6): 4193-4207.
10	Liu Tong, Ni Shenggang, Li Xiaoqiang, et al. Deep Reinforcement Learning Based Approach for Online Service Placement and Computation Resource Allocation in Edge Computing[J]. IEEE Transactions on Mobile Computing, 2023, 22(7): 3870-3881.
11	Jiang Hongbo, Dai Xingxia, Xiao Zhu, et al. Joint Task Offloading and Resource Allocation for Energy-constrained Mobile Edge Computing[J]. IEEE Transactions on Mobile Computing, 2023, 22(7): 4000-4015.
12	Liu Liqing, Chang Zheng, Guo Xijuan, et al. Multiobjective Optimization for Computation Offloading in Fog Computing[J]. IEEE Internet of Things Journal, 2018, 5(1): 283-294.
13	Bozorgchenani A, Mashhadi F, Tarchi D, et al. Multi-objective Computation Sharing in Energy and Delay Constrained Mobile Edge Computing Environments[J]. IEEE Transactions on Mobile Computing, 2021, 20(10): 2992-3005.
14	Zhang Jie, Guo Hongzhi, Liu Jiajia, et al. Task Offloading in Vehicular Edge Computing Networks: A Load-balancing Solution[J]. IEEE Transactions on Vehicular Technology, 2020, 69(2): 2092-2104.
15	Hou Xiangwang, Ren Zhiyuan, Wang Jingjing, et al. Reliable Computation Offloading for Edge-computing-enabled Software-defined IoV[J]. IEEE Internet of Things Journal, 2020, 7(8): 7097-7111.
16	Kaur K, Garg S, Aujla G S, et al. Edge Computing in the Industrial Internet of Things Environment: Software-defined-networks-based Edge-cloud Interplay[J]. IEEE Communications Magazine, 2018, 56(2): 44-51.
17	Ma Yu, Liang Weifa, Li Jing, et al. Mobility-aware and Delay-sensitive Service Provisioning in Mobile Edge-cloud Networks[J]. IEEE Transactions on Mobile Computing, 2022, 21(1): 196-210.
18	Vinyals O, Fortunato M, Jaitly N. Pointer Networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2015: 2692-2700.
19	Bello I, Pham H, Le Q V, et al. Neural Combinatorial Optimization with Reinforcement Learning[EB/OL]. (2017-01-12) [2021-12-10]. .
20	Williams R J. Simple Statistical Gradient-following Algorithms for Connectionist Reinforcement Learning[J]. Machine Learning, 1992, 8(3): 229-256.

[1]	Peng Yong, Zhang Miao, Hu Yue. Cloud-Edge Collaborative Service Architecture for LVC Training System [J]. Journal of System Simulation, 2023, 35(9): 1825-1836.
[2]	Junqiang Lin, Hongjun Wang, Xiangjun Zou, Po Zhang, Chengen Li, Yipeng Zhou, Shujie Yao. Obstacle Avoidance Path Planning and Simulation of Mobile Picking Robot Based on DPPO [J]. Journal of System Simulation, 2023, 35(8): 1692-1704.
[3]	Jiayi Liu, Gang Wang, Qiang Fu, Xiangke Guo, Siyuan Wang. Intelligent Air Defense Task Assignment Based on Assignment Strategy Optimization Algorithm [J]. Journal of System Simulation, 2023, 35(8): 1705-1716.
[4]	Laiyi Yang, Jing Bi, Haitao Yuan. Intelligent Path Planning for Mobile Robots Based on SAC Algorithm [J]. Journal of System Simulation, 2023, 35(8): 1726-1736.
[5]	Fei Ding, Yuchen Sha, Ying Hong, Xiao Kuai, Dengyin Zhang. Joint Optimization Strategy of Computing Offloading and Edge Caching for Intelligent Connected Vehicles [J]. Journal of System Simulation, 2023, 35(6): 1203-1214.
[6]	Yuxuan Dai, Chenggang Cui. Deep Reinforcement Learning-Based Control Strategy for Boost Converter [J]. Journal of System Simulation, 2023, 35(5): 1109-1119.
[7]	Haotian Xu, Long Qin, Junjie Zeng, Yue Hu, Qi Zhang. Research Progress of Opponent Modeling Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2023, 35(4): 671-694.
[8]	Jiajie Shi, Peng Yang, Yannan Pi. Machine Learning-based Simulation Research of On-line Subway Pedestrian Flow Control [J]. Journal of System Simulation, 2023, 35(2): 386-395.
[9]	Zou Fumin, Guo Feng, Luo Sijie, Liao Lüchao, Li Nan, Xing Yue. Research and Design of ETC Simulation Platform for Expressway [J]. Journal of System Simulation, 2023, 35(12): 2624-2640.
[10]	Jianxu Zhang, Shuai Hu, Hongyi Jin. Modeling of Traffic Flow Velocity Control Strategy for Human-machine Mixed Driving at Signalized Intersections [J]. Journal of System Simulation, 2022, 34(8): 1697-1709.
[11]	Xuewen Wu, Jingxian Liao. Game-Based Resource Allocation and Task Offloading Scheme in Collaborative Cloud-Edge Computing System [J]. Journal of System Simulation, 2022, 34(7): 1468-1481.
[12]	Sen Zhang, Mengyan Zhang, Jingping Shao, Jiexin Pu. Multi-UAVs 3D Path Planning Method Based on Random Strategy Search [J]. Journal of System Simulation, 2022, 34(6): 1286-1295.
[13]	Peng Cheng, Wenzhu Zhang, Shuhan Xie, Zixuan Yang. Research and Simulation of Internet of Vehicles Task Offloading Based on Mobile Edge Computing [J]. Journal of System Simulation, 2022, 34(6): 1304-1311.
[14]	Lingjia Ni, Xiaoxia Huang, Hongga Li, Zibo Zhang. Research on Fire Emergency Evacuation Simulation Based on Cooperative Deep Reinforcement Learning [J]. Journal of System Simulation, 2022, 34(6): 1353-1366.
[15]	Fei Ye, Ziqing Li, Yuanjun Laili. Simulation Optimization on Joint Production and Preventive Maintenance Scheduling for Distributed Job-shop [J]. Journal of System Simulation, 2022, 34(4): 688-699.