Intelligent Service Migration towards MEC-based IoV Systems

doi:10.16182/j.issn1004731x.joss.23-1142

Abstract

Abstract:

To address the problem of QoS degradation during the vehicle movement, a novel service migration via convex-optimization-enabled deep reinforcement learning (SeMiR) method is proposed. The optimization problem is decomposed into two sub-problems and solved separately. For the service migration sub-problem, an improved deep reinforcement learning based service migration method is designed to explore the optimal migration policy. For the resource allocation sub-problem, a convex optimization based resource allocation method is developed to derive the optimal resource allocation for each MEC server under the given migration decisions, thereby improving the performance of service migration. Experimental results show that the SeMiR method can achieve better QoS and superior service migration performance than benchmark methods under various scenarios.

Key words: MEC, Internet-of-Vehicles, service migration, resource allocation, DRL, convex optimization

CLC Number:

TP393

Huang Sijin, Wen Jia, Chen Zheyi. Intelligent Service Migration towards MEC-based IoV Systems[J]. Journal of System Simulation, 2025, 37(2): 379-391.

Figures/Tables 10

Fig. 1

Fig. 2

Table 1

Settings of simulation parameters

仿真参数	设置
车辆 $u$ 的发射功率 $P u$ /W	[0.4, 0.6]
任务数据量 $D t (u)$ /MB	[0.5, 1.5]^[25]
任务处理密度 $C t (u)$ /(cycles/bit)	[200, 1 000]^{[24, 26]}
服务实例数据量 $S t (u)$ /MB	[0.5, 50]^{[14, 24]}
回程链路带宽 $χ$ /Mbps	500^[24]
单位传输延迟系数 $μ h$ /s	0.2
单位迁移延迟系数 $μ m$ /s	1
高斯噪声功率 $σ 2$ /W	10^-13[26]
参考信道增益 $α$	10^-5[26]

Table 1

Fig. 3

Fig. 4

Fig. 5

Fig. 6

Table 2

Fig. 7

Fig. 8

References 26

1	Pliatsios Dimitrios, Sarigiannidis Panagiotis, Lagkas Thomas D, et al. Joint Wireless Resource and Computation Offloading Optimization for Energy Efficient Internet of Vehicles[J]. IEEE Transactions on Green Communications and Networking, 2022, 6(3): 1468-1480.
2	丁飞, 沙宇晨, 洪莹, 等. 智能网联汽车计算卸载与边缘缓存联合优化策略[J]. 系统仿真学报, 2023, 35(6): 1203-1214.
	Ding Fei, Sha Yuchen, Hong Ying, et al. Joint Optimization Strategy of Computing Offloading and Edge Caching for Intelligent Connected Vehicles[J]. Journal of System Simulation, 2023, 35(6): 1203-1214.
3	Zhao Gongming, Xu Hongli, Zhao Yangming, et al. Offloading Dependent Tasks in Mobile Edge Computing with Service Caching[C]//IEEE INFOCOM 2020 - IEEE Conference on Computer Communications. Piscataway: IEEE, 2020: 1997-2006.
4	李智勇, 王琦, 陈一凡, 等. 车辆边缘计算环境下任务卸载研究综述[J]. 计算机学报, 2021, 44(5): 963-982.
	Li Zhiyong, Wang Qi, Chen Yifan, et al. A Survey on Task Offloading Research in Vehicular Edge Computing[J]. Chinese Journal of Computers, 2021, 44(5): 963-982.
5	Rejiba Zeineb, Masip-Bruin Xavier, Marín-Tordera Eva. A Survey on Mobility-induced Service Migration in the Fog, Edge, and Related Computing Paradigms[J]. ACM Computing Surveys, 2020, 52(5): 90.
6	Kim T, Sathyanarayana S D, Chen Siqi, et al. MoDEMS: Optimizing Edge Computing Migrations for User Mobility[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(3): 675-689.
7	Chen Xiangyi, Bi Yuanguo, Chen Xueping, et al. Dynamic Service Migration and Request Routing for Microservice in Multicell Mobile-edge Computing[J]. IEEE Internet of Things Journal, 2022, 9(15): 13126-13143.
8	Velrajan Saravanan, Ceronmani Sharmila V. QoS-aware Service Migration in Multi-access Edge Compute Using Closed-loop Adaptive Particle Swarm Optimization Algorithm[J]. Journal of Network and Systems Management, 2023, 31(1): 17.
9	Ouyang Tao, Zhou Zhi, Chen Xu. Follow Me at the Edge: Mobility-aware Dynamic Service Placement for Mobile Edge Computing[J]. IEEE Journal on Selected Areas in Communications, 2018, 36(10): 2333-2345.
10	Arulkumaran K, Deisenroth M P, Brundage M, et al. Deep Reinforcement Learning: A Brief Survey[J]. IEEE Signal Processing Magazine, 2017, 34(6): 26-38.
11	Wang Shangguang, Guo Yan, Zhang Ning, et al. Delay-aware Microservice Coordination in Mobile Edge Computing: A Reinforcement Learning Approach[J]. IEEE Transactions on Mobile Computing, 2021, 20(3): 939-951.
12	Rami Akrem Addad, Diego Leonel Cadette Dutra, Taleb Tarik, et al. Toward Using Reinforcement Learning for Trigger Selection in Network Slice Mobility[J]. IEEE Journal on Selected Areas in Communications, 2021, 39(7): 2241-2253.
13	Zhang Cheng, Zheng Zixuan. Task Migration for Mobile Edge Computing Using Deep Reinforcement Learning[J]. Future Generation Computer Systems, 2019, 96: 111-118.
14	Zhang Yaqiang, Li Rengang, Zhao Yaqian, et al. Deep Reinforcement Learning Based Mobility-aware Service Migration for Multi-access Edge Computing Environment[C]//2022 IEEE Symposium on Computers and Communications (ISCC). Piscataway: IEEE, 2022: 1-6.
15	Lan Dapeng, Taherkordi Amir, Eliassen Frank, et al. Deep Reinforcement Learning for Intelligent Migration of Fog Services in Smart Cities[C]//Algorithms and Architectures for Parallel Processing. Cham: Springer International Publishing, 2020: 230-244.
16	Chen Yan, Sun Yanjing, Wang Chenyang, et al. Dynamic Task Allocation and Service Migration in Edge-cloud IoT System Based on Deep Reinforcement Learning[J]. IEEE Internet of Things Journal, 2022, 9(18): 16742-16757.
17	Du Jianbo, Zhao Liqiang, Feng Jie, et al. Computation Offloading and Resource Allocation in Mixed Fog/Cloud Computing Systems with Min-max Fairness Guarantee[J]. IEEE Transactions on Communications, 2018, 66(4): 1594-1608.
18	Deng Ruilong, Lu Rongxing, Lai Chengzhe, et al. Optimal Workload Allocation in Fog-cloud Computing Toward Balanced Delay and Power Consumption[J]. IEEE Internet of Things Journal, 2016, 3(6): 1171-1181.
19	Li Qun, Shao Hanqin. Cooperative Resource Allocation for Computation Offloading in Mobile-edge Computing Networks[C]//2021 IEEE Wireless Communications and Networking Conference (WCNC). Piscataway: IEEE, 2021: 1-6.
20	Jošilo Slađana, Dán György. Wireless and Computing Resource Allocation for Selfish Computation Offloading in Edge Computing[C]//IEEE INFOCOM 2019 - IEEE Conference on Computer Communications. Piscataway: IEEE, 2019: 2467-2475.
21	Liang Zezu, Liu Yuan, Lok T M, et al. Multi-cell Mobile Edge Computing: Joint Service Migration and Resource Allocation[J]. IEEE Transactions on Wireless Communications, 2021, 20(9): 5898-5912.
22	Liu Haolin, Long Xiaoling, Li Zhetao, et al. Joint Optimization of Request Assignment and Computing Resource Allocation in Multi-access Edge Computing[J]. IEEE Transactions on Services Computing, 2023, 16(2): 1254-1267.
23	Boyd S P, Vandenberghe L. Convex Optimization[M]. Cambridge: Cambridge University Press, 2004.
24	Wang Jin, Hu Jia, Min Geyong, et al. Online Service Migration in Mobile Edge with Incomplete System Information: A Deep Recurrent Actor-critic Learning Approach[J]. IEEE Transactions on Mobile Computing, 2023, 22(11): 6663-6675.
25	Chai Rong, Li Mingzhu, Yang Tiantian, et al. Dynamic Priority-based Computation Scheduling and Offloading for Interdependent Tasks: Leveraging Parallel Transmission and Execution[J]. IEEE Transactions on Vehicular Technology, 2021, 70(10): 10970-10985.
26	Wang Yunpeng, Fang Weiwei, Ding Yi, et al. Computation Offloading Optimization for UAV-assisted Mobile Edge Computing: A Deep Deterministic Policy Gradient Approach[J]. Wireless Networks, 2021, 27(4): 2991-3006.

方法	车辆数目
方法	15	20	25	30	35
SeMiR	1.715 6	2.140 6	2.701 4	3.185 1	3.646 7
DDPG	1.798 3	2.226 5	2.795 6	3.307 8	3.770 9
DQN	1.820 1	2.336 3	2.853 3	3.344 8	3.945 9
GA	1.878 7	2.377 3	3.158 4	3.662 4	4.179 8
NM	1.858 2	2.554 1	3.343 1	4.004 9	4.522 9
AM	2.029 3	2.595 4	3.340 6	3.931 9	4.608 7
PM	2.210 7	2.771 3	3.616 5	4.272 6	4.908 6

[1]	He Lian, Huang Kexiang, Yan Dapeng, Tang Ruida, Ding Gangyi. Research on Modeling and Simulation of Collective Mechanical Props Performance Behavior [J]. Journal of System Simulation, 2025, 37(1): 299-310.
[2]	Li Chao, Li Jiabao, Ding Caichang, Ye Zhiwei, Zuo Fangwei. Edge Surveillance Task Offloading and Resource Allocation Algorithm Based on DRL [J]. Journal of System Simulation, 2024, 36(9): 2113-2126.
[3]	Liu Peijin, Fu Xuefeng, Sun Haofeng, He Lin, Liu Shujie. A Highly Robust Target Tracking Algorithm Merging CNN and Transformer [J]. Journal of System Simulation, 2024, 36(8): 1854-1868.
[4]	Lei Zhen, Liu Yuhua, Ding Kai, Chen Haoxiang, Li Dongwei. Digital Twin-Driven Structural Thermal Deformation Compensation System for Radio Telescopes [J]. Journal of System Simulation, 2024, 36(8): 1869-1883.
[5]	Luo Tianyu, Xing Lining, Wang Rui, Wang Ling, Shi Jianmai, Sun Xin. Dynamic Air Defense Resource Allocation Optimization Based on Improved Differential Evolution Algorithm [J]. Journal of System Simulation, 2024, 36(6): 1285-1297.
[6]	Lu Yang, Liu Pengfei, Xu Siyuan, Liu Qiwang, Gu Fuqian, Wang Peng. Simulation of Rice Disease Recognition Based on Improved Attention Mechanism Embedded in PR-Net Model [J]. Journal of System Simulation, 2024, 36(6): 1322-1333.
[7]	Wang Fei, Chang Daofang, Wen Furong. Classification Cooperative Scheduling of U-automated Container Terminal Based on Container Markers [J]. Journal of System Simulation, 2024, 36(6): 1392-1403.
[8]	Zhu Zilu, Liu Yongkui, Zhang Lin, Wang Lihui, Lin Tingyu. Simulation of Robotic Peg-in-hole Assembly Strategy Based on DRL [J]. Journal of System Simulation, 2024, 36(6): 1414-1424.
[9]	Wang Hongjun, Lin Junqiang, Zou Xiangjun, Zhang Po, Zhou Mingxuan, Zou Weirui, Tang Yunchao, Luo Lufeng. Construction of a Virtual Interactive System for Orchards Based on Digital Twin [J]. Journal of System Simulation, 2024, 36(6): 1493-1508.
[10]	Liu Jinhui, Chen Mengyuan, Han Pengpeng, Chen Hebao, Zhang Yukun. A Graph Neural Network Visual SLAM Algorithm for Large-angle View Motion [J]. Journal of System Simulation, 2024, 36(5): 1043-1060.
[11]	Wang Yuan, Xu Lin, Gong Xiaoze, Zhang Yongliang, Wang Yongli. Gradient-based Deep Reinforcement Learning Interpretation Methods [J]. Journal of System Simulation, 2024, 36(5): 1130-1140.
[12]	Hu Yang, Wang Weiran, Fang Fang, Song Ziqiu, Xu Yuhan, Liu Jizhen. Dynamic Digital Twin Modelling and Semi-Physical Simulation of Wind Turbine Operation [J]. Journal of System Simulation, 2024, 36(3): 636-648.
[13]	Zhao Yingying, Dong Pusen, Zhu Tianchen, Li Fan, Su Yun, Tai Zhenying, Sun Qingyun, Fan Hang. Efficiency Optimization Method for Data Sampling in Power Grid Topology Scheduling Simulation [J]. Journal of System Simulation, 2024, 36(2): 283-295.
[14]	Wang Xinpeng, Fu Huiqiao, Deng Guizhou, Tang Kaiqiang, Chen Chunlin, Liu Canghai. Research on Motion Planning of Hexapod Robot Based on DRL and Free Gait [J]. Journal of System Simulation, 2024, 36(2): 373-384.
[15]	Pan Hainan, Chen Bailiang, Huang Kaihong, Ren Junkai, Cheng Chuang, Lu Huimin, Zhang Hui. Flipper Control Method for Tracked Robot Based on Deep Reinforcement Learning [J]. Journal of System Simulation, 2024, 36(2): 405-414.