边缘计算下基于深度强化学习的车联网任务调度

doi:10.16182/j.issn1004731x.joss.22-0841

系统仿真学报 ›› 2023, Vol. 35 ›› Issue (12): 2550-2559.doi: 10.16182/j.issn1004731x.joss.22-0841

边缘计算下基于深度强化学习的车联网任务调度

琚翔(), 苏圣超(), 徐超杰, 何蓓蓓

上海工程技术大学电子电气工程学院，上海 201620

收稿日期:2022-07-20 修回日期:2022-08-22 出版日期:2023-12-15 发布日期:2023-12-12
通讯作者: 苏圣超 E-mail:juxiang@sues.edu.cn;jnssc@sues.edu.cn
第一作者简介:琚翔(1997-)，男，硕士生，研究方向为边缘计算、车联网。E-mail：juxiang@sues.edu.cn
基金资助:
国家自然科学基金(61603241)

Task Scheduling for Internet of Vehicles Based on Deep Reinforcement Learning in Edge Computing

Ju Xiang(), Su Shengchao(), Xu Chaojie, He Beibei

School of Electronic and Electrical Engineering, Shanghai University of Engineering Science, Shanghai 201620, China

Received:2022-07-20 Revised:2022-08-22 Online:2023-12-15 Published:2023-12-12
Contact: Su Shengchao E-mail:juxiang@sues.edu.cn;jnssc@sues.edu.cn

摘要/Abstract

摘要：

针对边缘计算下车联网中时延约束型计算任务的卸载执行问题，提出一种基于深度强化学习的任务调度方法。在多边缘服务器场景下，构建软件定义网络辅助的车联网任务卸载系统，给出车辆计算卸载的任务调度模型；根据任务调度的特点，设计一种基于改进指针网络的调度方法，综合考虑任务调度和计算资源分配的复杂性，采用深度强化学习算法对指针网络进行训练；运用训练好的指针网络对车辆卸载任务进行调度。仿真结果表明：在边缘服务器计算资源相同的情况下，该方法在处理时延约束型计算任务的数量方面优于其他方法，有效提高了车联网任务卸载系统的服务能力。

关键词: 车联网, 边缘计算, 任务调度, 指针网络, 深度强化学习

Abstract:

Aiming at the offloading and execution of delay-constrained computing tasks for internet of vehicles in edge computing, a task scheduling method based on deep reinforcement learning is proposed. In multi-edge server scenario, a software-defined network-aided internet of vehicles task offloading system is built. On this basis, the task scheduling model of vehicle computation offloading is given. According to the characteristics of task scheduling, a scheduling method based on an improved pointer network is designed. Considering the complexity of task scheduling and computing resource allocation, the deep reinforcement learning algorithm is used to train the pointer network. The vehicle offloading tasks is scheduled by the trained pointer network. The simulation results show that with the same computing resources of edge servers, the proposed method is better than other methods in processing the number of delay-constrained computing tasks, and effectively improves the service capability of the internet of vehicles task offloading system.

Key words: internet of vehicles, edge computing, task scheduling, pointer network, deep reinforcement learning

中图分类号:

TP391.9

琚翔,苏圣超,徐超杰等 . 边缘计算下基于深度强化学习的车联网任务调度[J]. 系统仿真学报, 2023, 35(12): 2550-2559.

Ju Xiang,Su Shengchao,Xu Chaojie,et al . Task Scheduling for Internet of Vehicles Based on Deep Reinforcement Learning in Edge Computing[J]. Journal of System Simulation, 2023, 35(12): 2550-2559.

图/表 10

图 1

图2

图 3

表1

默认参数设置

参数	描述	取值
$D i i n$ /KB	$Ψ i$ 的数据量	400~1 000
$D i o u t$ /KB	$Ψ i$ 执行结果的数据量	50~200
$C i$ /MHz	$Ψ i$ 执行所需的CPU cycles	200~1 500
$τ$ /s	$Ψ i$ 时间延迟约束	0.5~1.0
$f s$ /GHz	服务器默认CPU频率	4
$N$	服务器默认个数	4
$B u p$ /MHz	车辆的带宽	10~20
$B d o w n$ /MHz	RSU的带宽	30~40
$ε$	路径损失指数	4
$p i$ /mW	车辆 $i$ 发送功率	100
$p j$ /mW	RSU- $j$ 发送功率	100
$h 0$	复高斯通道系数	$h 0 ~ C N (0,1)$
$N 0$ /dBm	加性高斯白噪声	100

表1

图 4

图 5

图 6

图 7

图 8

图9

参考文献 20

1	Zhai Yanlong, Sun Wenxin, Wu Jianqing, et al. An Energy Aware Offloading Scheme for Interdependent Applications in Software-defined IoV with Fog Computing Architecture[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 22(6): 3813-3823.
2	Wang Jingjing, Jiang Chunxiao, Zhang Kai, et al. Vehicular Sensing Networks in a Smart City: Principles, Technologies and Applications[J]. IEEE Wireless Communications, 2018, 25(1): 122-132.
3	Cao Xiaowen, Wang Feng, Xu Jie, et al. Joint Computation and Communication Cooperation for Energy-efficient Mobile Edge Computing[J]. IEEE Internet of Things Journal, 2019, 6(3): 4188-4200.
4	Zhu Tongxin, Shi Tuo, Li Jianzhong, et al. Task Scheduling in Deadline-aware Mobile Edge Computing Systems[J]. IEEE Internet of Things Journal, 2019, 6(3): 4854-4866.
5	刘雷, 陈晨, 冯杰, 等. 车载边缘计算中任务卸载和服务缓存的联合智能优化[J]. 通信学报, 2021, 42(1): 18-26.
	Liu Lei, Chen Chen, Feng Jie, et al. Joint Intelligent Optimization of Task Offloading and Service Caching for Vehicular Edge Computing[J]. Journal on Communications, 2021, 42(1): 18-26.
6	Li Qiuping, Zhao Junhui, Gong Yi, et al. Energy-efficient Computation Offloading and Resource Allocation in Fog Computing for Internet of Everything[J]. China Communications, 2019, 16(3): 32-41.
7	谢人超, 廉晓飞, 贾庆民, 等. 移动边缘计算卸载技术综述[J]. 通信学报, 2018, 39(11): 138-155.
	Xie Renchao, Lian Xiaofei, Jia Qingmin, et al. Survey on Computation Offloading in Mobile Edge Computing[J]. Journal on Communications, 2018, 39(11): 138-155.
8	Feng Jie, Richard Yu F, Pei Qingqi, et al. Cooperative Computation Offloading and Resource Allocation for Blockchain-enabled Mobile-edge Computing: A Deep Reinforcement Learning Approach[J]. IEEE Internet of Things Journal, 2020, 7(7): 6214-6228.
9	Xing Hong, Liu Liang, Xu Jie, et al. Joint Task Assignment and Resource Allocation for D2D-enabled Mobile-edge Computing[J]. IEEE Transactions on Communications, 2019, 67(6): 4193-4207.
10	Liu Tong, Ni Shenggang, Li Xiaoqiang, et al. Deep Reinforcement Learning Based Approach for Online Service Placement and Computation Resource Allocation in Edge Computing[J]. IEEE Transactions on Mobile Computing, 2023, 22(7): 3870-3881.
11	Jiang Hongbo, Dai Xingxia, Xiao Zhu, et al. Joint Task Offloading and Resource Allocation for Energy-constrained Mobile Edge Computing[J]. IEEE Transactions on Mobile Computing, 2023, 22(7): 4000-4015.
12	Liu Liqing, Chang Zheng, Guo Xijuan, et al. Multiobjective Optimization for Computation Offloading in Fog Computing[J]. IEEE Internet of Things Journal, 2018, 5(1): 283-294.
13	Bozorgchenani A, Mashhadi F, Tarchi D, et al. Multi-objective Computation Sharing in Energy and Delay Constrained Mobile Edge Computing Environments[J]. IEEE Transactions on Mobile Computing, 2021, 20(10): 2992-3005.
14	Zhang Jie, Guo Hongzhi, Liu Jiajia, et al. Task Offloading in Vehicular Edge Computing Networks: A Load-balancing Solution[J]. IEEE Transactions on Vehicular Technology, 2020, 69(2): 2092-2104.
15	Hou Xiangwang, Ren Zhiyuan, Wang Jingjing, et al. Reliable Computation Offloading for Edge-computing-enabled Software-defined IoV[J]. IEEE Internet of Things Journal, 2020, 7(8): 7097-7111.
16	Kaur K, Garg S, Aujla G S, et al. Edge Computing in the Industrial Internet of Things Environment: Software-defined-networks-based Edge-cloud Interplay[J]. IEEE Communications Magazine, 2018, 56(2): 44-51.
17	Ma Yu, Liang Weifa, Li Jing, et al. Mobility-aware and Delay-sensitive Service Provisioning in Mobile Edge-cloud Networks[J]. IEEE Transactions on Mobile Computing, 2022, 21(1): 196-210.
18	Vinyals O, Fortunato M, Jaitly N. Pointer Networks[C]//Proceedings of the 28th International Conference on Neural Information Processing Systems. Cambridge, MA, USA: MIT Press, 2015: 2692-2700.
19	Bello I, Pham H, Le Q V, et al. Neural Combinatorial Optimization with Reinforcement Learning[EB/OL]. (2017-01-12) [2021-12-10]. .
20	Williams R J. Simple Statistical Gradient-following Algorithms for Connectionist Reinforcement Learning[J]. Machine Learning, 1992, 8(3): 229-256.

[1]	林俊强, 王红军, 邹湘军, 张坡, 李承恩, 周益鹏, 姚书杰. 基于DPPO的移动采摘机器人避障路径规划及仿真[J]. 系统仿真学报, 2023, 35(8): 1692-1704.
[2]	刘家义, 王刚, 付强, 郭相科, 王思远. 基于分配策略优化算法的智能防空任务分配[J]. 系统仿真学报, 2023, 35(8): 1705-1716.
[3]	杨来义, 毕敬, 苑海涛. 基于SAC算法的移动机器人智能路径规划[J]. 系统仿真学报, 2023, 35(8): 1726-1736.
[4]	丁飞, 沙宇晨, 洪莹, 蒯晓, 张登银. 智能网联汽车计算卸载与边缘缓存联合优化策略[J]. 系统仿真学报, 2023, 35(6): 1203-1214.
[5]	戴宇轩, 崔承刚. 基于深度强化学习的Boost变换器控制策略[J]. 系统仿真学报, 2023, 35(5): 1109-1119.
[6]	徐浩添, 秦龙, 曾俊杰, 胡越, 张琪. 基于深度强化学习的对手建模方法研究综述[J]. 系统仿真学报, 2023, 35(4): 671-694.
[7]	史佳洁, 杨鹏, 皮雁南. 基于机器学习的地铁行人流在线优化控制研究[J]. 系统仿真学报, 2023, 35(2): 386-395.
[8]	邹复民, 郭峰, 罗思杰, 廖律超, 李楠, 邢悦. 高速公路ETC仿真平台研究与设计[J]. 系统仿真学报, 2023, 35(12): 2624-2640.
[9]	王宇琨, 王泽, 董力维, 李妮. 基于分层的智能建模方法的多机空战行为建模[J]. 系统仿真学报, 2023, 35(10): 2249-2261.
[10]	张建旭, 胡帅, 金宏意. 信号交叉口人机混驾交通流速度控制策略建模[J]. 系统仿真学报, 2022, 34(8): 1697-1709.
[11]	吴学文, 廖婧贤. 云边协同系统中基于博弈论的资源分配与任务卸载方案[J]. 系统仿真学报, 2022, 34(7): 1468-1481.
[12]	马文臻, 邹自明, 黎建辉, 于勤思, 佟继周, 李菁菁. 空间科学卫星数据处理框架研究与系统实现[J]. 系统仿真学报, 2022, 34(7): 1605-1618.
[13]	张森, 张孟炎, 邵敬平, 普杰信. 基于随机策略搜索的多机三维路径规划方法[J]. 系统仿真学报, 2022, 34(6): 1286-1295.
[14]	程鹏, 张文柱, 谢书翰, 杨子轩. 基于移动边缘计算的车联网任务卸载研究与仿真[J]. 系统仿真学报, 2022, 34(6): 1304-1311.
[15]	倪凌佳, 黄晓霞, 李红旮, 张子博. 基于协作式深度强化学习的火灾应急疏散仿真研究[J]. 系统仿真学报, 2022, 34(6): 1353-1366.

边缘计算下基于深度强化学习的车联网任务调度

Task Scheduling for Internet of Vehicles Based on Deep Reinforcement Learning in Edge Computing

RichHTML

PDF (PC)

可视化

摘要/Abstract

引用本文

使用本文

图/表 10

参考文献 20

相关文章 15

编辑推荐

Metrics

本文评价