线上PPO车载边缘计算多跳任务卸载策略

doi:10.16182/j.issn1004731x.joss.25-0752

摘要/Abstract

摘要：

针对车载边缘计算(vehicular edge computing， VEC)环境中，由动态网络拓扑导致的通信链路频繁中断，以及高维决策空间引发的计算复杂度剧增问题，提出了一种基于线上PPO算法的车载边缘计算多跳任务卸载策略。构建了同时考虑链路有效时间、传输速率与计算资源约束的多跳任务卸载优化模型；设计了融合链路稳定性与端到端时延的多跳 $A *$ 路径搜索算法；提出了基于PPO的在线卸载决策框架，将0-1混合整数非线性规划问题转化为马尔可夫过程并给出复杂度与收敛性分析。仿真结果表明：该策略在时延-能耗平衡和任务完成率方面均显著优于Greedy-MinDelay、DQN和A3C基准算法，有效提升了系统在高动态车载场景下的实时性与鲁棒性。

关键词: 车载边缘计算, 多跳任务卸载, PPO算法, 马尔可夫决策, 整数非线性规划

Abstract:

To address the problems of frequent communication link interruptions caused by dynamic network topologies and the sharp increase in computational complexity triggered by high-dimensional decision spaces in the vehicular edge computing (VEC) environment, a multi-hop task offloading strategy for VEC based on an online PPO algorithm was proposed. A multi-hop task offloading optimization model simultaneously considering link effective time, transmission rate, and computing resource constraints was constructed; a multi-hop $A *$ path search algorithm integrating link stability and end-to-end delay was designed; an online offloading decision framework based on PPO was proposed, which transformed the 0-1 mixed integer nonlinear programming problem into a Markov process and provided complexity and convergence analyses. Simulation results indicate that the strategy significantly outperforms the Greedy-MinDelay, DQN, and A3C benchmark algorithms in latency-energy balance and task completion rate, effectively improving the real-time performance and robustness of the system in highly dynamic vehicular scenarios.

Key words: vehicular edge computing, multi-hop task offloading, PPO algorithm, Markov decision, integer nonlinear programming

中图分类号:

TP391

张文柱,边跃伟,熊福力等 . 线上PPO车载边缘计算多跳任务卸载策略[J]. 系统仿真学报, 2026, 38(7): 1901-1921.

Zhang Wenzhu,Bian Yuewei,Xiong Fuli,et al . Online PPO-based Multi-hop Task Offloading Strategy for Vehicular Edge Computing[J]. Journal of System Simulation, 2026, 38(7): 1901-1921.

图/表 17

图1

表1

车辆速度采样参数设置

交通状态	$μ$ /(m/s)	$ϱ$	$ω$	$a m a x$ /(m/s²)
畅通	35	3	0.9	2.0
微拥	25	5	0.9	2.0
拥堵	10	3	0.9	1.5

表1

表2

仿真环境参数设置

仿真参数	数值
道路长度/m	1 200
道路宽度/m	50
车辆数量	10
车辆移动速度/( $m / s$ )	$[30,40]$
车辆计算能力 $f v$ /( $G c y c l e s / s$ )	$[20,25]$
车辆初始队列积压/ $M B$	$[8,20]$
RSU数量	3
RSU计算能力 $f m$ /( $G c y c l e s / s$ )	$[25,30]$
RSU初始队列积压/ $M B$	$[4,12]$
任务大小 $d m$ / $M B$	$[2,5]$
最大容忍时延 $δ m$ / $s$	$[20,25]$
任务计算强度/( $C y c l e s / b i t$ )	$300$
车辆通信范围 $ρ$ /m	200
RSU通信范围 $λ$ /m	400
车辆发射功率 $P V$ / $d B m$	$10$
RSU发射功率 $P R$ / $d B m$	$50$
V2V链路带宽 $W V 2 V$ / $Μ Η z$	$20$
R2V链路带宽 $W R 2 V$ / $Μ Η z$	$20$
噪声功率 $N$ /( $W / H z$ )	$1 × 10 - 3$
固定损耗 $K$	$30$
路径损耗因子 $γ$	2
R2R链路传输速率 $r R 2 R$ /( $M B / s$ )	$100$
R2C链路传输速率 $r R 2 C$ /( $M B / s$ )	$200$
R2R链路时延/ $m s$	$1$
R2C链路时延/ $m s$	$2$
车辆计算能耗系数 $ξ v$ /( $J / c y c l e$ )	$1 × 10 - 9$
RSU计算能耗系数 $ξ m$ /( $J / c y c l e$ )	$5 × 10 - 10$

表2

图2

表3

PPO算法超参数设置

参数	数值
最大训练回合数	1 000
折扣因子 $η$	0.92
GAE参数 $ξ$	0.90
截断比率 $ε$	0.10
每次更新训练周期	10
更新间隔(采样步数)	1 024
学习率 $r$	$5 × 10 - 4$
隐藏层维度	256
熵系数	0.05

表3

图3

图4

图5

图6

图7

图8

图9

图10

图11

图12

图13

图14

参考文献 31

[1]	Lin Zhuangxing, Cui Haixia, Liu Yong. Distributed Deep Learning Based on Edge Computing Over Internet of Vehicles: Overview, Applications, and Challenges[J]. IEEE Access, 2024, 12: 133734-133747.
[2]	Zhang Ming. Development of Analytical Offloading for Innovative Internet of Vehicles Based on Mobile Edge Computing[J]. Journal of Grid Computing, 2024, 22(1): 4.
[3]	Luo Jiabin, Song Qinyu, Guo Fusen, et al. Joint Deep Reinforcement Learning Strategy in MEC for Smart Internet of Vehicles Edge Computing Networks[J]. Sustainable Computing: Informatics and Systems, 2025, 46: 101121.
[4]	Yang Ananjie, Chen Yingyang, Lin Zhijian, et al. Distributed Computation Offloading in Autonomous Driving Vehicular Networks: A Stochastic Geometry Approach[J]. IEEE Transactions on Intelligent Vehicles, 2024, 9(1): 2701-2713.
[5]	Quan Haoyu, Zhang Qingmiao, Zhao Junhui. Federated Learning Assisted Intelligent IoV Mobile Edge Computing[J]. IEEE Transactions on Green Communications and Networking, 2025, 9(1): 228-241.
[6]	Zhou Yingbo, Chai Zhengyi, Li Yalun, et al. Parking Vehicle-assisted Task Offloading in Edge Computing: A Dynamic Multi-objective Evolutionary Algorithm with Multi-strategy Fusion Response[J]. Swarm and Evolutionary Computation, 2025, 94: 101900.
[7]	Cao Dun, Wang Wenqian, Wu Meihua, et al. A General Task Offloading and Resources Allocation Strategy for multi-RSUs with Load Unbalance and Priority Awareness[J]. Ad Hoc Networks, 2025, 167: 103690.
[8]	Zhao Liang, Zhao Zijia, Zhang Enchao, et al. A Digital Twin-assisted Intelligent Partial Offloading Approach for Vehicular Edge Computing[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(11): 3386-3400.
[9]	Abbas Zahir, Xu Shihe, Zhang Xinming. An Efficient Partial Task Offloading and Resource Allocation Scheme for Vehicular Edge Computing in a Dynamic Environment[J]. IEEE Transactions on Intelligent Transportation Systems, 2025, 26(2): 2488-2502.
[10]	刘佳慧, 邹渊, 孙巍, 等. 基于混合深度强化学习的ICV任务卸载与资源分配[J]. 汽车工程, 2025, 47(1): 35-43.
	Liu Jiahui, Zou Yuan, Sun Wei, et al. ICV Task Offloading and Resource Allocation Based on Hybrid Deep Reinforcement Learning[J]. Automotive Engineering, 2025, 47(1): 35-43.
[11]	李致远, 彭二帅, 徐晓萍, 等. 基于最优控制的车载边缘算力网络负载均衡策略[J]. 北京邮电大学学报, 2024, 47(5): 122-127.
	Li Zhiyuan, Peng Ershuai, Xu Xiaoping, et al. Optimal Control-based Load Balancing Strategy for Vehicular Edge Computing Power Network[J]. Journal of Beijing University of Posts and Telecommunications, 2024, 47(5): 122-127.
[12]	Zhang Zheng, Zeng Feng. Efficient Task Allocation for Computation Offloading in Vehicular Edge Computing[J]. IEEE Internet of Things Journal, 2023, 10(6): 5595-5606.
[13]	Tonini Federico, Khorsandi Bahare M, Amato Elisabetta, et al. Scalable Edge Computing Deployment for Reliable Service Provisioning in Vehicular Networks[J]. Journal of Sensor and Actuator Networks, 2019, 8(4): 51.
[14]	Zhang Qixia, Chen Hao, Phuong Hoai Ha. Cost-efficient Vehicular Edge Computing Deployment for Mobile Air Pollution Monitoring[C]//2024 IEEE Wireless Communications and Networking Conference (WCNC). Piscataway: IEEE, 2024: 1-6.
[15]	Yang Kun, Sun Peng, Yang Dingkang, et al. A Novel Hierarchical Distributed Vehicular Edge Computing Framework for Supporting Intelligent Driving[J]. Ad Hoc Networks, 2024, 153: 103343.
[16]	Wu Zhoupeng, Jia Zongpu, Pang Xiaoyan, et al. Deep Reinforcement Learning-based Task Offloading and Load Balancing for Vehicular Edge Computing[J]. Electronics, 2024, 13(8): 1511.
[17]	Lu Lingyu, Yu Jing, Du Haifeng, et al. A3C-based Load-balancing Solution for Computation Offloading in SDN-enabled Vehicular Edge Computing Networks[J]. Peer-to-Peer Networking and Applications, 2023, 16(2): 1242-1256.
[18]	Liu Weiqi, Du Qingwei. Multi-hop Propagation-based Computation Offloading Scheme in Vehicular Communication Environment[C]//2021 IEEE 4th Advanced Information Management, Communicates, Electronic and Automation Control Conference (IMCEC). Piscataway: IEEE, 2021: 311-315.
[19]	Wang Yueqing, Hu Xi, Guo Lijun, et al. Research on V2I/V2V Hybrid Multi-hop Edge Computing Offloading Algorithm in IoV Environment[C]//2020 IEEE 5th International Conference on Intelligent Transportation Engineering (ICITE). Piscataway: IEEE, 2020: 336-340.
[20]	Ahmed Manzoor, Raza Salman, Ahmad Haseeb, et al. Deep Reinforcement Learning Approach for Multi-hop Task Offloading in Vehicular Edge Computing[J]. Engineering Science and Technology, an International Journal, 2024, 59: 101854.
[21]	Xu Jincheng, Yang Bo, Liu Yuxiang, et al. Joint Task Offloading and Resource Allocation for Multihop Industrial Internet of Things[J]. IEEE Internet of Things Journal, 2022, 9(21): 22022-22033.
[22]	Zhang Ziyang, Gu Keyu, Xu Zijie. DRL-based Task and Computational Offloading for Internet of Vehicles in Decentralized Computing[J]. Journal of Grid Computing, 2024, 22(1): 18.
[23]	Deng Zizhen, Cai Zhen, Liang Mangui. A Multi-hop VANETs-assisted Offloading Strategy in Vehicular Mobile Edge Computing[J]. IEEE Access, 2020, 8: 53062-53071.
[24]	Cao Dun, Gu Ning, Wu Meihua, et al. Cost-effective Task Partial Offloading and Resource Allocation for Multi-vehicle and Multi-MEC on B5G/6G Edge Networks[J]. Ad Hoc Networks, 2024, 156: 103438.
[25]	Zhao Wei, Cheng Yu, Liu Zhi, et al. Asynchronous DRL-based Multi-hop Task Offloading in RSU-assisted IoV Networks[J]. IEEE Transactions on Cognitive Communications and Networking, 2025, 11(1): 546-555.
[26]	Anzola-Rojas Camilo, Ignacio de Miguel, Merayo Noemí, et al. Optimizing Road Side Units Connectivity in Intelligent Transportation Systems with Optical Network Solutions[J]. Vehicular Communications, 2025, 56: 100975.
[27]	Syed Aizaz Ul Haq, Imran Muqaddas, Shah Nadir, et al. SDN-based Edge Computing in Vehicular Communication Networks: A Survey of Existing Approaches[J]. IEEE Access, 2025, 13: 74252-74287.
[28]	Zhang Jie, Wang Qiao, Mitchell P, et al. An Integrated Access and Backhaul Approach to Sustainable Dense Small Cell Network Planning[J]. Information, 2024, 15(1): 19.
[29]	Fayad Abdulhalim, Cinkler Tibor, Rak Jacek. Toward 6G Optical Fronthaul: A Survey on Enabling Technologies and Research Perspectives[J]. IEEE Communications Surveys & Tutorials, 2025, 27(1): 629-666.
[30]	Gui Jinsong, Yin Long, Deng Xiaoheng, et al. Joint Access and Relay-assisted Backhaul Resource Allocation for Dense mmWave Multiple Access Networks[J]. IEEE Transactions on Vehicular Technology, 2024, 73(1): 1289-1305.
[31]	Pagin Matteo, Zugno Tommaso, Polese M, et al. Resource Management for 5G NR Integrated Access and Backhaul: A Semi-centralized Approach[J]. IEEE Transactions on Wireless Communications, 2022, 21(2): 753-767.