Resource-efficient Continuous Learning Framework for Edge Real-time Video Analytics

doi:10.16182/j.issn1004731x.joss.25-0590

Abstract

Abstract:

By deploying lightweight models at the network edge, edge systems can provide services of real-time video analytics. However, due to the data drift caused by the discrepancy between model training and actual deployment, it is challenging to construct lightweight models that match real-world environments. To address this challenge, a resource-efficient continuous learning framework for edge real-time video analytics (CL4VA) was proposed. A region of interest-granularity predictor for accuracy degradation was introduced to efficiently select key samples from real-time video streams.A two-layer mixed sample pool was constructed to adaptively trigger the model's continuous learning and avoid the issue of catastrophic forgetting.A DRL-based controller was designed to determine the appropriate time to complete model re-training. The simulation results show that compared to the baseline method, CL4VA can reduce the average latency by 8.65% and increase the accuracy by up to 5.57%. Moreover, the core components of CL4VA require extremely low online overhead.

Key words: edge computing, real-time video analytics, resource optimization, continuous learning, DRL

CLC Number:

TP393

Wu Shuxia, Zhang Junjie, Chen Delong, Chen Zheyi. Resource-efficient Continuous Learning Framework for Edge Real-time Video Analytics[J]. Journal of System Simulation, 2026, 38(2): 294-306.

Figures/Tables 9

Fig. 1

Table 1

Fig. 2

Table 2

Fig. 3

Fig. 4

Fig. 5

Table 3

Fig. 6

References 28

[1]	Xu Renjie, Razavi Saiedeh, Zheng Rong. Edge Video Analytics: A Survey on Applications, Systems and Enabling Techniques[J]. IEEE Communications Surveys & Tutorials, 2023, 25(4): 2951-2982.
[2]	袁牧, 张兰, 姚云昊, 等. 面向智能物联网的资源高效模型推理综述[J]. 计算机学报, 2024, 47(10): 2247-2273.
	Yuan Mu, Zhang Lan, Yao Yunhao, et al. Resource-efficient Model Inference for AIoT: A Survey[J]. Chinese Journal of Computers, 2024, 47(10): 2247-2273.
[3]	李超, 李贾宝, 丁才昌, 等. 基于DRL的边缘监控任务卸载与资源分配算法[J]. 系统仿真学报, 2024, 36(9): 2113-2126.
	Li Chao, Li Jiabao, Ding Caichang, et al. Edge Surveillance Task Offloading and Resource Allocation Algorithm Based on DRL[J]. Journal of System Simulation, 2024, 36(9): 2113-2126.
[4]	Gao Xitong, Zhao Yiren, Dudziak Łukasz, et al. Dynamic Channel Pruning: Feature Boosting and Suppression[EB/OL]. (2019-01-28) [2025-06-12]. .
[5]	Banitalebi-Dehkordi Amin, Vedula Naveen, Pei Jian, et al. Auto-split: A General Framework of Collaborative Edge-cloud AI[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery & Data Mining. New York: Association for Computing Machinery, 2021: 2543-2553.
[6]	Huang Kai, Gao Wei. Real-time Neural Network Inference on Extremely Weak Devices: Agile Offloading with Explainable AI[C]//Proceedings of the 28th Annual International Conference on Mobile Computing and Networking. New York: Association for Computing Machinery, 2022: 200-213.
[7]	Chen Gang, He Shengyu, Meng Haitao, et al. PhoneBit: Efficient GPU-accelerated Binary Neural Network Inference Engine for Mobile Phones[C]//2020 Design, Automation & Test in Europe Conference & Exhibition (DATE). Piscataway: IEEE, 2020: 786-791.
[8]	Chen Yuhan, Ravindranath L, Deng Shuo, et al. Glimpse: Continuous, Real-time Object Recognition on Mobile Devices[C]//Proceedings of the 13th ACM Conference on Embedded Networked Sensor Systems. New York: Association for Computing Machinery, 2015: 155-168.
[9]	Yuan Mu, Zhang Lan, He Fengxiang, et al. InFi: End-to-end Learnable Input Filter for Resource-efficient Mobile-centric Inference[C]//Proceedings of the 28th Annual International Conference on Mobile Computing And Networking. New York: Association for Computing Machinery, 2022: 228-241.
[10]	Li Yuanqi, Padmanabhan A, Zhao Pengzhan, et al. Reducto: On-camera Filtering for Resource-efficient Real-time Video Analytics[C]//Proceedings of the Annual conference of the ACM Special Interest Group on Data Communication on the applications, technologies, architectures, and protocols for computer communication. New York: Association for Computing Machinery, 2020: 359-376.
[11]	Ling Neiwen, Wang Kai, He Yuze, et al. RT-mDL: Supporting Real-time Mixed Deep Learning Tasks on Edge Platforms[C]//Proceedings of the 19th ACM Conference on Embedded Networked Sensor Systems. New York: Association for Computing Machinery, 2021: 1-14.
[12]	Jiang Shiqi, Lin Zhiqi, Li Yuanchun, et al. Flexible High-resolution Object Detection on Edge Devices with Tunable Latency[C]//Proceedings of the 27th Annual International Conference on Mobile Computing and Networking. New York: Association for Computing Machinery, 2021: 559-572.
[13]	Yang Zheng, Wang Xu, Wu Jiahang, et al. EdgeDuet: Tiling Small Object Detection for Edge Assisted Autonomous Mobile Vision[J]. IEEE/ACM Transactions on Networking, 2023, 31(4): 1765-1778.
[14]	Mullapudi R T, Chen S, Zhang Keyi, et al. Online Model Distillation for Efficient Video Inference[C]//2019 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2019: 3572-3581.
[15]	Khani M, Ananthanarayanan G, Hsieh K, et al. RECL: Responsive Resource-efficient Continuous Learning for Video Analytics[C]//20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23). Boston: USENIX Association, 2023: 917-932.
[16]	Zhang Jingwei, Cheng Long, Liu Cong, et al. Cost-aware Scheduling Systems for Real-time Workflows in Cloud: An Approach Based on Genetic Algorithm and Deep Reinforcement Learning[J]. Expert Systems with Applications, 2023, 234: 120972.
[17]	Cai Huaiguang, Zhou Zhi, Huang Qianyi. Online Resource Allocation for Edge Intelligence with Colocated Model Retraining and Inference[C]//IEEE INFOCOM 2024 - IEEE Conference on Computer Communications. Piscataway: IEEE, 2024: 1900-1909.
[18]	Kong Yuxin, Yang Peng, Cheng Yan. Edge-assisted On-device Model Update for Video Analytics in Adverse Environments[C]//Proceedings of the 31st ACM International Conference on Multimedia. New York: Association for Computing Machinery, 2023: 9051-9060.
[19]	Shubha S S, Shen Haiying. AdaInf: Data Drift Adaptive Scheduling for Accurate and SLO-guaranteed Multiple-model Inference Serving at Edge Servers[C]//Proceedings of the ACM SIGCOMM 2023 Conference. New York: Association for Computing Machinery, 2023: 473-485.
[20]	Zhang Ziyang, Zhao Yang, Chang Mingching, et al . E 4: Energy-efficient DNN Inference for Edge Video Analytics Via Early Exiting and DVFS[C]//Proceedings of the Thirty-ninth AAAI Conference on Artificial Intelligence and Thirty-seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence. Palo Alto: AAAI Press, 2025: 1165-1173.
[21]	Wang Liang, Qu Xiaoyang, Wang Jianzong, et al. Gecko: Resource-efficient and Accurate Queries in Real-time Video Streams at the Edge[C]//IEEE INFOCOM 2024 - IEEE Conference on Computer Communications. Piscataway: IEEE, 2024: 481-490.
[22]	Bhardwaj R, Xia Zhengxu, Ananthanarayanan G, et al. Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers[C]//19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22). Renton: USENIX Association, 2022: 119-135.
[23]	Schulman J, Wolski F, Dhariwal P, et al. Proximal Policy Optimization Algorithms[EB/OL]. (2017-08-28) [2025-06-12]. .
[24]	Zeng Xiao, Fang Biyi, Shen Haichen, et al. Distream: Scaling Live Video Analytics with Workload-adaptive Distributed Edge Intelligence[C]//Proceedings of the 18th Conference on Embedded Networked Sensor Systems. New York: Association for Computing Machinery, 2020: 409-421.
[25]	Huang Wenhui, Zhou Yanxin, He Xiangkun, et al. Goal-guided Transformer-enabled Reinforcement Learning for Efficient Autonomous Navigation[J]. IEEE Transactions on Intelligent Transportation Systems, 2024, 25(2): 1832-1845.
[26]	Leal-Taixé Laura, Milan A, Reid I, et al. MOTChallenge 2015: Towards a Benchmark for Multi-Target Tracking[EB/OL]. (2015-04-08) [2025-06-12]. .
[27]	Yu F, Chen Haofeng, Wang Xin, et al. BDD100K: A Diverse Driving Dataset for Heterogeneous Multitask Learning[C]//2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). Piscataway: IEEE, 2020: 2633-2642.
[28]	Wen Longyin, Du Dawei, Cai Zhaowei, et al. UA-DETRAC: A New Benchmark and Protocol for Multi-object Detection and Tracking[J]. Computer Vision and Image Understanding, 2020, 193: 102907.

特征	含义	作用
area	候选RoI面积与整帧图像面积之比	刻画了RoI在图像中的占比，当RoI较小或被部分遮挡时，其在低分辨率场景下容易丢失
ratio	候选RoI的外接矩形宽高比	描述了RoI的形态特征，反映了遮挡场景带来的比例异常
circularity	RoI中所有前景轮廓平均圆形度的反值	反映了RoI轮廓的几何规则，在光照不均或遮挡导致轮廓断裂时，该值将升高
convexity	RoI中所有前景轮廓的平均凸度	反映了RoI轮廓的凹陷特征，在遮挡场景下RoI轮廓会出现凹陷，导致该值下降
position	候选RoI中心点相对整帧图像中心的归一化位置	表示RoI在图像中的分布情况，位于边缘位置的 RoI 更容易受到拍摄角度、光照不均或遮挡的影响而难以被检测到

数据集	指标	Vanilla	Ekya	AdaInf	CL4VA	Oracle
UA-DETRAC	延迟/ms	19.98	34.48	35.65	30.23	41.42
UA-DETRAC	mAp/%	20.20	58.32	58.44	63.89	70.15
BDD100K	延迟/ms	21.78	38.07	42.18	33.39	46.45
BDD100K	mAp/%	28.68	32.46	33.48	35.89	36.50
MOT	延迟/ms	16.45	28.83	31.24	29.42	39.20
MOT	mAp/%	73.69	78.23	79.98	80.64	78.93
Real-Camera	延迟/ms	17.12	30.74	30.61	27.51	39.89
Real-Camera	mAp/%	59.14	94.66	97.25	97.75	100