Online PPO-based Multi-hop Task Offloading Strategy for Vehicular Edge Computing

doi:10.16182/j.issn1004731x.joss.25-0752

Abstract

Abstract:

To address the problems of frequent communication link interruptions caused by dynamic network topologies and the sharp increase in computational complexity triggered by high-dimensional decision spaces in the vehicular edge computing (VEC) environment, a multi-hop task offloading strategy for VEC based on an online PPO algorithm was proposed. A multi-hop task offloading optimization model simultaneously considering link effective time, transmission rate, and computing resource constraints was constructed; a multi-hop $A *$ path search algorithm integrating link stability and end-to-end delay was designed; an online offloading decision framework based on PPO was proposed, which transformed the 0-1 mixed integer nonlinear programming problem into a Markov process and provided complexity and convergence analyses. Simulation results indicate that the strategy significantly outperforms the Greedy-MinDelay, DQN, and A3C benchmark algorithms in latency-energy balance and task completion rate, effectively improving the real-time performance and robustness of the system in highly dynamic vehicular scenarios.

Key words: vehicular edge computing, multi-hop task offloading, PPO algorithm, Markov decision, integer nonlinear programming

CLC Number:

TP391

Zhang Wenzhu, Bian Yuewei, Xiong Fuli, Cai Siqi. Online PPO-based Multi-hop Task Offloading Strategy for Vehicular Edge Computing[J]. Journal of System Simulation, 2026, 38(7): 1901-1921.

Figures/Tables 17

Fig. 1

Table 1

Vehicle speed sampling parameter settings

交通状态	$μ$ /(m/s)	$ϱ$	$ω$	$a m a x$ /(m/s²)
畅通	35	3	0.9	2.0
微拥	25	5	0.9	2.0
拥堵	10	3	0.9	1.5

Table 1

Table 2

Simulation environment parameter settings

仿真参数	数值
道路长度/m	1 200
道路宽度/m	50
车辆数量	10
车辆移动速度/( $m / s$ )	$[30,40]$
车辆计算能力 $f v$ /( $G c y c l e s / s$ )	$[20,25]$
车辆初始队列积压/ $M B$	$[8,20]$
RSU数量	3
RSU计算能力 $f m$ /( $G c y c l e s / s$ )	$[25,30]$
RSU初始队列积压/ $M B$	$[4,12]$
任务大小 $d m$ / $M B$	$[2,5]$
最大容忍时延 $δ m$ / $s$	$[20,25]$
任务计算强度/( $C y c l e s / b i t$ )	$300$
车辆通信范围 $ρ$ /m	200
RSU通信范围 $λ$ /m	400
车辆发射功率 $P V$ / $d B m$	$10$
RSU发射功率 $P R$ / $d B m$	$50$
V2V链路带宽 $W V 2 V$ / $Μ Η z$	$20$
R2V链路带宽 $W R 2 V$ / $Μ Η z$	$20$
噪声功率 $N$ /( $W / H z$ )	$1 × 10 - 3$
固定损耗 $K$	$30$
路径损耗因子 $γ$	2
R2R链路传输速率 $r R 2 R$ /( $M B / s$ )	$100$
R2C链路传输速率 $r R 2 C$ /( $M B / s$ )	$200$
R2R链路时延/ $m s$	$1$
R2C链路时延/ $m s$	$2$
车辆计算能耗系数 $ξ v$ /( $J / c y c l e$ )	$1 × 10 - 9$
RSU计算能耗系数 $ξ m$ /( $J / c y c l e$ )	$5 × 10 - 10$

Table 2

Fig. 2

Table 3

PPO algorithm hyperparameter settings

参数	数值
最大训练回合数	1 000
折扣因子 $η$	0.92
GAE参数 $ξ$	0.90
截断比率 $ε$	0.10
每次更新训练周期	10
更新间隔(采样步数)	1 024
学习率 $r$	$5 × 10 - 4$
隐藏层维度	256
熵系数	0.05

Table 3

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Fig. 11

Fig. 12

Fig. 13

Fig. 14

References 31

[1]	Lin Zhuangxing, Cui Haixia, Liu Yong. Distributed Deep Learning Based on Edge Computing Over Internet of Vehicles: Overview, Applications, and Challenges[J]. IEEE Access, 2024, 12: 133734-133747.
[2]	Zhang Ming. Development of Analytical Offloading for Innovative Internet of Vehicles Based on Mobile Edge Computing[J]. Journal of Grid Computing, 2024, 22(1): 4.
[3]	Luo Jiabin, Song Qinyu, Guo Fusen, et al. Joint Deep Reinforcement Learning Strategy in MEC for Smart Internet of Vehicles Edge Computing Networks[J]. Sustainable Computing: Informatics and Systems, 2025, 46: 101121.
[4]	Yang Ananjie, Chen Yingyang, Lin Zhijian, et al. Distributed Computation Offloading in Autonomous Driving Vehicular Networks: A Stochastic Geometry Approach[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(1): 2701-2713.
[5]	Quan Haoyu, Zhang Qingmiao, Zhao Junhui. Federated Learning Assisted Intelligent IoV Mobile Edge Computing[J]. IEEE Transactions on Green Communications and Networking, 2025, 9(1): 228-241.
[6]	Zhou Yingbo, Chai Zhengyi, Li Yalun, et al. Parking Vehicle-assisted Task Offloading in Edge Computing: A Dynamic Multi-objective Evolutionary Algorithm with Multi-strategy Fusion Response[J]. Swarm and Evolutionary Computation, 2025, 94: 101900.
[7]	Cao Dun, Wang Wenqian, Wu Meihua, et al. A General Task Offloading and Resources Allocation Strategy for multi-RSUs with Load Unbalance and Priority Awareness[J]. Ad Hoc Networks, 2025, 167: 103690.
[8]	Zhao Liang, Zhao Zijia, Zhang Enchao, et al. A Digital Twin-assisted Intelligent Partial Offloading Approach for Vehicular Edge Computing[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(11): 3386-3400.
[9]	Abbas Zahir, Xu Shihe, Zhang Xinming. An Efficient Partial Task Offloading and Resource Allocation Scheme for Vehicular Edge Computing in a Dynamic Environment[J]. IEEE Transactions on Intelligent Transportation Systems, 2025, 26(2): 2488-2502.
[10]	刘佳慧, 邹渊, 孙巍, 等. 基于混合深度强化学习的ICV任务卸载与资源分配[J]. 汽车工程, 2025, 47(1): 35-43.
	Liu Jiahui, Zou Yuan, Sun Wei, et al. ICV Task Offloading and Resource Allocation Based on Hybrid Deep Reinforcement Learning[J]. Automotive Engineering, 2025, 47(1): 35-43.
[11]	李致远, 彭二帅, 徐晓萍, 等. 基于最优控制的车载边缘算力网络负载均衡策略[J]. 北京邮电大学学报, 2024, 47(5): 122-127.
	Li Zhiyuan, Peng Ershuai, Xu Xiaoping, et al. Optimal Control-based Load Balancing Strategy for Vehicular Edge Computing Power Network[J]. Journal of Beijing University of Posts and Telecommunications, 2024, 47(5): 122-127.
[12]	Zhang Zheng, Zeng Feng. Efficient Task Allocation for Computation Offloading in Vehicular Edge Computing[J]. IEEE Internet of Things Journal, 2023, 10(6): 5595-5606.
[13]	Tonini Federico, Khorsandi Bahare M, Amato Elisabetta, et al. Scalable Edge Computing Deployment for Reliable Service Provisioning in Vehicular Networks[J]. Journal of Sensor and Actuator Networks, 2019, 8(4): 51.
[14]	Zhang Qixia, Chen Hao, Phuong Hoai Ha. Cost-efficient Vehicular Edge Computing Deployment for Mobile Air Pollution Monitoring[C]//2024 IEEE Wireless Communications and Networking Conference (WCNC). Piscataway: IEEE, 2024: 1-6.
[15]	Yang Kun, Sun Peng, Yang Dingkang, et al. A Novel Hierarchical Distributed Vehicular Edge Computing Framework for Supporting Intelligent Driving[J]. Ad Hoc Networks, 2024, 153: 103343.
[16]	Wu Zhoupeng, Jia Zongpu, Pang Xiaoyan, et al. Deep Reinforcement Learning-based Task Offloading and Load Balancing for Vehicular Edge Computing[J]. Electronics, 2024, 13(8): 1511.
[17]	Lu Lingyu, Yu Jing, Du Haifeng, et al. A3C-based Load-balancing Solution for Computation Offloading in SDN-enabled Vehicular Edge Computing Networks[J]. Peer-to-Peer Networking and Applications, 2023, 16(2): 1242-1256.
[18]	Liu Weiqi, Du Qingwei. Multi-hop Propagation-based Computation Offloading Scheme in Vehicular Communication Environment[C]//2021 IEEE 4th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). Piscataway: IEEE, 2021: 311-315.
[19]	Wang Yueqing, Hu Xi, Guo Lijun, et al. Research on V2I/V2V Hybrid Multi-hop Edge Computing Offloading Algorithm in IoV Environment[C]//2020 IEEE 5th International Conference on Intelligent Transportation Engineering (ICITE). Piscataway: IEEE, 2020: 336-340.
[20]	Ahmed Manzoor, Raza Salman, Ahmad Haseeb, et al. Deep Reinforcement Learning Approach for Multi-hop Task Offloading in Vehicular Edge Computing[J]. Engineering Science and Technology, an International Journal, 2024, 59: 101854.
[21]	Xu Jincheng, Yang Bo, Liu Yuxiang, et al. Joint Task Offloading and Resource Allocation for Multihop Industrial Internet of Things[J]. IEEE Internet of Things Journal, 2022, 9(21): 22022-22033.
[22]	Zhang Ziyang, Gu Keyu, Xu Zijie. DRL-based Task and Computational Offloading for Internet of Vehicles in Decentralized Computing[J]. Journal of Grid Computing, 2024, 22(1): 18.
[23]	Deng Zizhen, Cai Zhen, Liang Mangui. A Multi-hop VANETs-assisted Offloading Strategy in Vehicular Mobile Edge Computing[J]. IEEE Access, 2020, 8: 53062-53071.
[24]	Cao Dun, Gu Ning, Wu Meihua, et al. Cost-effective Task Partial Offloading and Resource Allocation for Multi-vehicle and Multi-MEC on B5G/6G Edge Networks[J]. Ad Hoc Networks, 2024, 156: 103438.
[25]	Zhao Wei, Cheng Yu, Liu Zhi, et al. Asynchronous DRL-based Multi-hop Task Offloading in RSU-assisted IoV Networks[J]. IEEE Transactions on Cognitive Communications and Networking, 2025, 11(1): 546-555.
[26]	Anzola-Rojas Camilo, Ignacio de Miguel, Merayo Noemí, et al. Optimizing Road Side Units Connectivity in Intelligent Transportation Systems with Optical Network Solutions[J]. Vehicular Communications, 2025, 56: 100975.
[27]	Syed Aizaz Ul Haq, Imran Muqaddas, Shah Nadir, et al. SDN-based Edge Computing in Vehicular Communication Networks: A Survey of Existing Approaches[J]. IEEE Access, 2025, 13: 74252-74287.
[28]	Zhang Jie, Wang Qiao, Mitchell P, et al. An Integrated Access and Backhaul Approach to Sustainable Dense Small Cell Network Planning[J]. Information, 2024, 15(1): 19.
[29]	Fayad Abdulhalim, Cinkler Tibor, Rak Jacek. Toward 6G Optical Fronthaul: A Survey on Enabling Technologies and Research Perspectives[J]. IEEE Communications Surveys & Tutorials, 2025, 27(1): 629-666.
[30]	Gui Jinsong, Yin Long, Deng Xiaoheng, et al. Joint Access and Relay-assisted Backhaul Resource Allocation for Dense mmWave Multiple Access Networks[J]. IEEE Transactions on Vehicular Technology, 2024, 73(1): 1289-1305.
[31]	Pagin Matteo, Zugno Tommaso, Polese M, et al. Resource Management for 5G NR Integrated Access and Backhaul: A Semi-centralized Approach[J]. IEEE Transactions on Wireless Communications, 2022, 21(2): 753-767.