Task Scheduling for Internet of Vehicles Based on Deep Reinforcement Learning in Edge Computing

doi:10.16182/j.issn1004731x.joss.22-0841

Abstract

Abstract:

Aiming at the offloading and execution of delay-constrained computing tasks for internet of vehicles in edge computing, a task scheduling method based on deep reinforcement learning is proposed. In multi-edge server scenario, a software-defined network-aided internet of vehicles task offloading system is built. On this basis, the task scheduling model of vehicle computation offloading is given. According to the characteristics of task scheduling, a scheduling method based on an improved pointer network is designed. Considering the complexity of task scheduling and computing resource allocation, the deep reinforcement learning algorithm is used to train the pointer network. The vehicle offloading tasks is scheduled by the trained pointer network. The simulation results show that with the same computing resources of edge servers, the proposed method is better than other methods in processing the number of delay-constrained computing tasks, and effectively improves the service capability of the internet of vehicles task offloading system.

Key words: internet of vehicles, edge computing, task scheduling, pointer network, deep reinforcement learning

CLC Number:

TP391.9

Ju Xiang, Su Shengchao, Xu Chaojie, He Beibei. Task Scheduling for Internet of Vehicles Based on Deep Reinforcement Learning in Edge Computing[J]. Journal of System Simulation, 2023, 35(12): 2550-2559.

Figures/Tables 10

Fig. 1

Fig. 2

Fig. 3

Table 1

Default parameters setting

参数	描述	取值
$D i i n$ /KB	$Ψ i$ 的数据量	400~1 000
$D i o u t$ /KB	$Ψ i$ 执行结果的数据量	50~200
$C i$ /MHz	$Ψ i$ 执行所需的CPU cycles	200~1 500
$τ$ /s	$Ψ i$ 时间延迟约束	0.5~1.0
$f s$ /GHz	服务器默认CPU频率	4
$N$	服务器默认个数	4
$B u p$ /MHz	车辆的带宽	10~20
$B d o w n$ /MHz	RSU的带宽	30~40
$ε$	路径损失指数	4
$p i$ /mW	车辆 $i$ 发送功率	100
$p j$ /mW	RSU- $j$ 发送功率	100
$h 0$	复高斯通道系数	$h 0 ~ C N (0,1)$
$N 0$ /dBm	加性高斯白噪声	100

Table 1

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

References 20

1	Zhai Yanlong, Sun Wenxin, Wu Jianqing, et al. An Energy Aware Offloading Scheme for Interdependent Applications in Software-defined IoV with Fog Computing Architecture[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(6): 3813-3823.
2	Wang Jingjing, Jiang Chunxiao, Zhang Kai, et al. Vehicular Sensing Networks in a Smart City: Principles, Technologies and Applications[J]. IEEE Wireless Communications, 2018, 25(1): 122-132.
3	Cao Xiaowen, Wang Feng, Xu Jie, et al. Joint Computation and Communication Cooperation for Energy-efficient Mobile Edge Computing[J]. IEEE Internet of Things Journal, 2019, 6(3): 4188-4200.
4	Zhu Tongxin, Shi Tuo, Li Jianzhong, et al. Task Scheduling in Deadline-aware Mobile Edge Computing Systems[J]. IEEE Internet of Things Journal, 2019, 6(3): 4854-4866.
5	刘雷, 陈晨, 冯杰, 等. 车载边缘计算中任务卸载和服务缓存的联合智能优化[J]. 通信学报, 2021, 42(1): 18-26.
	Liu Lei, Chen Chen, Feng Jie, et al. Joint Intelligent Optimization of Task Offloading and Service Caching for Vehicular Edge Computing[J]. Journal on Communications, 2021, 42(1): 18-26.
6	Li Qiuping, Zhao Junhui, Gong Yi, et al. Energy-efficient Computation Offloading and Resource Allocation in Fog Computing for Internet of Everything[J]. China Communications, 2019, 16(3): 32-41.
7	谢人超, 廉晓飞, 贾庆民, 等. 移动边缘计算卸载技术综述[J]. 通信学报, 2018, 39(11): 138-155.
	Xie Renchao, Lian Xiaofei, Jia Qingmin, et al. Survey on Computation Offloading in Mobile Edge Computing[J]. Journal on Communications, 2018, 39(11): 138-155.
8	Feng Jie, Richard Yu F, Pei Qingqi, et al. Cooperative Computation Offloading and Resource Allocation for Blockchain-enabled Mobile-edge Computing: A Deep Reinforcement Learning Approach[J]. IEEE Internet of Things Journal, 2020, 7(7): 6214-6228.
9	Xing Hong, Liu Liang, Xu Jie, et al. Joint Task Assignment and Resource Allocation for D2D-enabled Mobile-edge Computing[J]. IEEE Transactions on Communications, 2019, 67(6): 4193-4207.
10	Liu Tong, Ni Shenggang, Li Xiaoqiang, et al. Deep Reinforcement Learning Based Approach for Online Service Placement and Computation Resource Allocation in Edge Computing[J]. IEEE Transactions on Mobile Computing, 2023, 22(7): 3870-3881.
11	Jiang Hongbo, Dai Xingxia, Xiao Zhu, et al. Joint Task Offloading and Resource Allocation for Energy-constrained Mobile Edge Computing[J]. IEEE Transactions on Mobile Computing, 2023, 22(7): 4000-4015.
12	Liu Liqing, Chang Zheng, Guo Xijuan, et al. Multiobjective Optimization for Computation Offloading in Fog Computing[J]. IEEE Internet of Things Journal, 2018, 5(1): 283-294.
13	Bozorgchenani A, Mashhadi F, Tarchi D, et al. Multi-objective Computation Sharing in Energy and Delay Constrained Mobile Edge Computing Environments[J]. IEEE Transactions on Mobile Computing, 2021, 20(10): 2992-3005.
14	Zhang Jie, Guo Hongzhi, Liu Jiajia, et al. Task Offloading in Vehicular Edge Computing Networks: A Load-balancing Solution[J]. IEEE Transactions on Vehicular Technology, 2020, 69(2): 2092-2104.
15	Hou Xiangwang, Ren Zhiyuan, Wang Jingjing, et al. Reliable Computation Offloading for Edge-computing-enabled Software-defined IoV[J]. IEEE Internet of Things Journal, 2020, 7(8): 7097-7111.
16	Kaur K, Garg S, Aujla G S, et al. Edge Computing in the Industrial Internet of Things Environment: Software-defined-networks-based Edge-cloud Interplay[J]. IEEE Communications Magazine, 2018, 56(2): 44-51.
17	Ma Yu, Liang Weifa, Li Jing, et al. Mobility-aware and Delay-sensitive Service Provisioning in Mobile Edge-cloud Networks[J]. IEEE Transactions on Mobile Computing, 2022, 21(1): 196-210.
18	Vinyals O, Fortunato M, Jaitly N. Pointer Networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2015: 2692-2700.
19	Bello I, Pham H, Le Q V, et al. Neural Combinatorial Optimization with Reinforcement Learning[EB/OL]. (2017-01-12) [2021-12-10]. .
20	Williams R J. Simple Statistical Gradient-following Algorithms for Connectionist Reinforcement Learning[J]. Machine Learning, 1992, 8(3): 229-256.

[1]	Jiang Ming, He Tao. Solving the Vehicle Routing Problem Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(9): 2177-2187.
[2]	Ni Peilong, Mao Pengjun, Wang Ning, Yang Mengjie. Robot Path Planning Based on Improved A-DDQN Algorithm [J]. Journal of System Simulation, 2025, 37(9): 2420-2430.
[3]	Chen Zhen, Wu Zhuoyi, Zhang Lin. Research on Policy Representation in Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(7): 1753-1769.
[4]	Wu Guohua, Zeng Jiaheng, Wang Dezhi, Zheng Long, Zou Wei. A Quadrotor Trajectory Tracking Control Method Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(5): 1169-1187.
[5]	Yang Chao, Zheng Ruiqun, Li Zhen, Zhang Hongwei, Tang Yanqun, Li Dongze. Parallel Task Transmission and Processing Optimization Scheme for UAV-assisted Internet of Vehicles [J]. Journal of System Simulation, 2025, 37(3): 635-645.
[6]	Bai Zhenzu, Hou Yizhi, He Zhangming, Wei Juhui, Zhou Haiyin, Wang Jiongqi. Optimization of Dynamic Weapon Target Assignment Considering Random Disturbances [J]. Journal of System Simulation, 2025, 37(12): 2967-2980.
[7]	Zheng Jiayu, Mai Zhuxue, Chen Zheyi. Optimization of Service Caching and Computation Offloading in Digital Twin Cloud-edge Networks [J]. Journal of System Simulation, 2025, 37(11): 2741-2753.
[8]	Di Jian, Wan Xue, Jiang Limei. Evolutionary Reinforcement Learning Based on Elite Instruction and Random Search [J]. Journal of System Simulation, 2025, 37(11): 2877-2887.
[9]	Xu Zhongkai, Chu Chenyang, Xie Kai, Zhao Ruizhuo, Ke Wenjun. Optimization Dispatch Method for High-proportion Renewable Energy Power Systems Based on SC-PPO [J]. Journal of System Simulation, 2025, 37(10): 2511-2521.
[10]	Liang Xiuman, Liu Ziliang, Liu Zhendong. Path Planning of Improved RRT Algorithm Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2025, 37(10): 2578-2593.
[11]	Huang Zhiqin, Lu Tianying, Chen Zheyi. Multi-UAV Deployment and Collaborative Offloading for Large-scale IoT Systems [J]. Journal of System Simulation, 2025, 37(1): 25-39.
[12]	Jiang Jiachen, Jia Zhengxuan, Xu Zhao, Lin Tingyu, Zhao Pengpeng, Ou Yiming. Decision Modeling and Solution Based on Game Adversarial Complex Systems [J]. Journal of System Simulation, 2025, 37(1): 66-78.
[13]	Li Chao, Li Jiabao, Ding Caichang, Ye Zhiwei, Zuo Fangwei. Edge Surveillance Task Offloading and Resource Allocation Algorithm Based on DRL [J]. Journal of System Simulation, 2024, 36(9): 2113-2126.
[14]	Qin Baoxin, Zhang Yuxiao, Wu Sirui, Cao Weichong, Li Zhan. Intelligent Optimization of Coal Terminal Unloading Scheduling Based on Improved D3QN Algorithm [J]. Journal of System Simulation, 2024, 36(3): 770-781.
[15]	Li Ming, Ye Wangzhong, Yan Jiehua. Path Planning of Desert Robot Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2024, 36(12): 2917-2925.