Optimization of Service Caching and Computation Offloading in Digital Twin Cloud-edge Networks

doi:10.16182/j.issn1004731x.joss.24-0574

Abstract

Abstract:

In mobile edge computing (MEC), to satisfy diverse user demands by jointly optimizing service caching and computation offloading and address low-efficiency resource utilization caused by irrational resource allocation, this paper proposed a novel joint optimization of service caching and computation offloading with a convex-optimization-enabled deep reinforcement learning (JCO-CR) method. Additionally, a new model for digital twin cloud-edge networks (DTCEN) was constructed. The joint optimization of service caching and computation offloading was decoupled into two sub-problems, which were solved by an improved deep reinforcement learning method and convex optimization theory, respectively. Simulation experiments demonstrate that the proposed JCO-CR method can reduce long-term service latency and achieve better performance under different scenarios.

Key words: mobile edge computing, digital twin(DT), service caching, computation offloading, deep reinforcement learning, convex optimization

CLC Number:

TP393

Zheng Jiayu, Mai Zhuxue, Chen Zheyi. Optimization of Service Caching and Computation Offloading in Digital Twin Cloud-edge Networks[J]. Journal of System Simulation, 2025, 37(11): 2741-2753.

Figures/Tables 7

Fig. 1

Fig. 2

Table 1

Settings of simulation parameters

参数	值
BS数量I	3^[23]
BS的计算能力F/GHz	{12,18,24}
BS的存储容量C/GB	{10,15,20}
BS单时隙收到请求数N	[2,10]
训练回合E	5 000
时隙数T	144^[24]
任务大小d/kb	[200,400]
高斯噪声功率 $σ 2$ /dBm	-50^[18]
通信干扰功率 $I j', i$ /dBm	-60

Table 1

Fig. 3

Fig. 4

Fig. 5

Fig. 6

References 26

[1]	Mario Sanz Rodrigo, Rivera Diego, José Ignacio Moreno, et al. Digital Twins for 5G Networks: A Modeling and Deployment Methodology[J]. IEEE Access, 2023, 11: 38112-38126.
[2]	李超, 李贾宝, 丁才昌, 等. 基于DRL的边缘监控任务卸载与资源分配算法[J]. 系统仿真学报, 2024, 36(9): 2113-2126.
	Li Chao, Li Jiabao, Ding Caichang, et al. Edge Surveillance Task Offloading and Resource Allocation Algorithm Based on DRL[J]. Journal of System Simulation, 2024, 36(9): 2113-2126.
[3]	Mach Pavel, Becvar Zdenek. Mobile Edge Computing: A Survey on Architecture and Computation Offloading[J]. IEEE Communications Surveys & Tutorials, 2017, 19(3): 1628-1656.
[4]	周娴玮, 龚启旭, 余松森. 基于Stackelberg博弈与深度强化学习的计算卸载策略[J]. 系统仿真学报, 2023, 35(2): 372-385.
	Zhou Xianwei, Gong Qixu, Yu Songsen. Computation Offloading Strategy Based on Stackelberg Game and DRL[J]. Journal of System Simulation, 2023, 35(2): 372-385.
[5]	Yang Song, Liu Jintian, Zhang Fei, et al. Caching-enabled Computation Offloading in Multi-region MEC Network via Deep Reinforcement Learning[J]. IEEE Internet of Things Journal, 2022, 9(21): 21086-21098.
[6]	Bi Suzhi, Huang Liang, Zhang Yingjun. Joint Optimization of Service Caching Placement and Computation Offloading in Mobile Edge Computing Systems[J]. IEEE Transactions on Wireless Communications, 2020, 19(7): 4947-4963.
[7]	Zhang Guanglin, Zhang Shun, Zhang Wenqian, et al. Joint Service Caching, Computation Offloading and Resource Allocation in Mobile Edge Computing Systems[J]. IEEE Transactions on Wireless Communications, 2021, 20(8): 5288-5300.
[8]	Zhao Gongming, Xu Hongli, Zhao Yangming, et al. Offloading Tasks with Dependency and Service Caching in Mobile Edge Computing[J]. IEEE Transactions on Parallel and Distributed Systems, 2021, 32(11): 2777-2792.
[9]	Zhou Ruiting, Wu Xiaoyi, Tan Haisheng, et al. Two Time-scale Joint Service Caching and Task Offloading for UAV-assisted Mobile Edge Computing[C]//IEEE INFOCOM 2022 - IEEE Conference on Computer Communications. Piscataway: IEEE, 2022: 1189-1198.
[10]	Wu Huaming, Chen Junqi, Nguyen T N, et al. Lyapunov-guided Delay-aware Energy Efficient Offloading in IIoT-MEC Systems[J]. IEEE Transactions on Industrial Informatics, 2023, 19(2): 2117-2128.
[11]	Xu Xiaolong, Liu Zhongjian, Bilal Muhammad, et al. Computation Offloading and Service Caching for Intelligent Transportation Systems with Digital Twin[J]. IEEE Transactions on Intelligent Transportation Systems, 2022, 23(11): 20757-20772.
[12]	Huang Binbin, Liu Xiao, Xiang Yuanyuan, et al. Reinforcement Learning for Cost-effective IoT Service Caching at the Edge[J]. Journal of Parallel and Distributed Computing, 2022, 168(C): 120-136.
[13]	Wang Xiaofei, Wang Chenyang, Li Xiuhua, et al. Federated Deep Reinforcement Learning for Internet of Things with Decentralized Cooperative Edge Caching[J]. IEEE Internet of Things Journal, 2020, 7(10): 9441-9455.
[14]	Zhao Junhui, He Lin, Zhang Danyang, et al. A TP-DDPG Algorithm Based on Cache Assistance for Task Offloading in Urban Rail Transit[J]. IEEE Transactions on Vehicular Technology, 2023, 72(8): 10671-10681.
[15]	Zhou Tianqing, Xu Ming, Qin Dong, et al. Computing Offloading Based on TD3 Algorithm in Cache-assisted Vehicular NOMA-MEC Networks[J]. Sensors, 2023, 23(22): 9064.
[16]	Zhou Huan, Wang Zhenning, Zheng Hantong, et al. Cost Minimization-oriented Computation Offloading and Service Caching in Mobile Cloud-edge Computing: An A3C-based Approach[J]. IEEE Transactions on Network Science and Engineering, 2023, 10(3): 1326-1338.
[17]	Wei Yifei, Yu F R, Song Mei, et al. Joint Optimization of Caching, Computing, and Radio Resources for Fog-enabled IoT Using Natural Actor-critic Deep Reinforcement Learning[J]. IEEE Internet of Things Journal, 2019, 6(2): 2061-2073.
[18]	Yao Zhixiu, Xia Shichao, Li Yun, et al. Cooperative Task Offloading and Service Caching for Digital Twin Edge Networks: A Graph Attention Multi-agent Reinforcement Learning Approach[J]. IEEE Journal on Selected Areas in Communications, 2023, 41(11): 3401-3413.
[19]	Dong Rui, She Changyang, Hardjawana W, et al. Deep Learning for Hybrid 5G Services in Mobile Edge Computing Systems: Learn from a Digital Twin[J]. IEEE Transactions on Wireless Communications, 2019, 18(10): 4692-4707.
[20]	Tang Fengxiao, Chen Xuehan, Tiago Koketsu Rodrigues, et al. Survey on Digital Twin Edge Networks (DITEN) Toward 6G[J]. IEEE Open Journal of the Communications Society, 2022, 3: 1360-1381.
[21]	Bøhn Eivind, Coates Erlend M, Moe Signe, et al. Deep Reinforcement Learning Attitude Control of Fixed-Wing UAVs Using Proximal Policy optimization[C]//2019 International Conference on Unmanned Aircraft Systems (ICUAS). Piscataway: IEEE, 2019: 523-533.
[22]	黄思进, 文佳, 陈哲毅. 面向边缘车联网系统的智能服务迁移方法[J]. 系统仿真学报, 2025, 37(2): 379-391.
	Huang Sijin, Wen Jia, Chen Zheyi. Intelligent Service Migration Towards MEC-based IoV Systems[J]. Journal of System Simulation, 2025, 37(2): 379-391.
[23]	Barlacchi Gianni, Marco De Nadai, Larcher Roberto, et al. A Multi-source Dataset of Urban Life in the City of Milan and the Province of Trentino[J]. Scientific Data, 2015, 2: 150055.
[24]	Chen Zheyi, Zhang Junjie, Zheng Xianghan, et al. Profit-aware Cooperative Offloading in UAV-enabled MEC Systems Using Lightweight Deep Reinforcement Learning[J]. IEEE Internet of Things Journal, 2024, 11(12): 21325-21336.
[25]	Zhou Huan, Zhang Zhenyu, Wu Yuan, et al. Energy Efficient Joint Computation Offloading and Service Caching for Mobile Edge Computing: A Deep Reinforcement Learning Approach[J]. IEEE Transactions on Green Communications and Networking, 2023, 7(2): 950-961.
[26]	Huang Jiwei, Wan Jiangyuan, Bofeng Lü, et al. Joint Computation Offloading and Resource Allocation for Edge-cloud Collaboration in Internet of Vehicles via Deep Reinforcement Learning[J]. IEEE Systems Journal, 2023, 17(2): 2500-2511.