A Transfer Learning-based Hybrid Model for PM2.5Concentration Prediction

doi:10.16182/j.issn1004731x.joss.23-1532

Abstract

Abstract:

In order to solve the problems of increased computational cost due to irrelevant features and decreased prediction accuracy due to the difference in probability distribution caused by the change of data distribution over time in PM_2.5 concentration prediction, this paper constructs a hybrid deep learning model TraTCN-LSTM-BiGRU based on migration learning. The meteorological factors related to PM_2.5 concentration are selected as the model input using the mean-value heat map algorithm features; the source domain data and target domain data are divided by KL scatter and an adaptive layer is introduced into the model to achieve inter-domain distribution adaptation; the TCN-LSTM-BiGRU model is designed to extract high-level spatial features in multivariate variables using TCN, and the extracted features are fed into the LSTM for extracting time series features. The extracted features are fed into the LSTM to extract time-series features, and the features are fused by residual connection and fed into the BiGRU for prediction. Simulation results show that the model proposed in this paper can effectively predict the future trend of PM_2.5, and effectively weaken the influence of the difference in data distribution.

Key words: transfer learning, PM_2.5 concentration, mean-value heat map, probability distribution difference, TraTCN-LSTM-BiGRU

CLC Number:

TP273+.4

Lu Xinbiao, Ye Chunlin, Chen Yisen, Wu Wen, Chen Yudan. A Transfer Learning-based Hybrid Model for PM_2.5Concentration Prediction[J]. Journal of System Simulation, 2025, 37(4): 882-894.

Figures/Tables 19

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Fig. 5

Table 1

Table 2

Fig. 6

Fig. 7

Fig. 8

Fig. 9

Fig. 10

Table 3

Fig. 11

Fig. 12

Fig. 13

Table 4

Table 5

Table 6

References 39

1	Kobza Joanna, Geremek Mariusz, Dul Lechosław. Characteristics of Air Quality and Sources Affecting High Levels of PM₁₀ and PM_2.5 in Poland, Upper Silesia Urban Area[J]. Environmental Monitoring and Assessment, 2018, 190(9): 515.
2	胡冬梅, 吴建平, 田琨, 等. 北京重霾期空气污染物、气象和交通的时空特征[J]. 系统仿真学报, 2019, 31(3): 403-412.
	Hu Dongmei, Wu Jianping, Tian Kun, et al. Spatiotemporal Characteristics of Air Pollutants, Meteorology and Traffic During Heavy Haze Pollution in Beijing[J]. Journal of System Simulation, 2019, 31(3): 403-412.
3	Yu Haofei, Stuart A L. Impacts of Compact Growth and Electric Vehicles on Future Air Quality and Urban Exposures May be Mixed[J]. Science of the Total Environment, 2017, 576: 148-158.
4	Winterbottom C J, Shah R J, Patterson K C, et al. Exposure to Ambient Particulate Matter is Associated with Accelerated Functional Decline in Idiopathic Pulmonary Fibrosis[J]. Chest, 2018, 153(5): 1221-1228.
5	Sanguineti Pamela B, Lanzaco Bethania L, María Laura López, et al. PM_2.5 Monitoring During a 10-year Period: Relation Between Elemental Concentration and Meteorological Conditions[J]. Environmental Monitoring and Assessment, 2020, 192(5): 313.
6	Zhu Suling, Lian Xiuyuan, Liu Haixia, et al. Daily Air Quality Index Forecasting with Hybrid Models: A Case in China[J]. Environmental Pollution, 2017, 231, Part 2: 1232-1244.
7	Md Mazharul Islam, Sharmin Mowshumi, Ahmed Faroque. Predicting Air Quality of Dhaka and Sylhet Divisions in Bangladesh: A Time Series Modeling Approach[J]. Air Quality, Atmosphere & Health, 2020, 13(5): 607-615.
8	Man Tat Lei, Monjardino Joana, Mendes Luisa, et al. Macao Air Quality Forecast Using Statistical Methods[J]. Air Quality, Atmosphere & Health, 2019, 12(9): 1049-1057.
9	Jian Wei Koo, Shin Wee Wong, Selvachandran Ganeshsree, et al. Prediction of Air Pollution Index in Kuala Lumpur Using Fuzzy Time Series and Statistical Models[J]. Air Quality, Atmosphere & Health, 2020, 13(1): 77-88.
10	Luciana Maria Baptista Ventura, Fellipe de Oliveira Pinto, Laiza Molezon Soares, et al. Forecast of Daily PM_2.5 Concentrations Applying Artificial Neural Networks and Holt-winters Models[J]. Air Quality, Atmosphere & Health, 2019, 12(3): 317-325.
11	Ida Kalate Ahani, Salari Majid, Shadman Alireza. An Ensemble Multi-step-ahead Forecasting System for Fine Particulate Matter in Urban Areas[J]. Journal of Cleaner Production, 2020, 263: 120983.
12	Zheng Tongshu, Bergin M H, Hu Shijia, et al. Estimating Ground-level PM_2.5 Using Micro-satellite Images by a Convolutional Neural Network and Random Forest Approach[J]. Atmospheric Environment, 2020, 230: 117451.
13	Liu Hui, Chen Chao. Spatial Air Quality Index Prediction Model Based on Decomposition, Adaptive Boosting, and Three-stage Feature Selection: A Case Study in China[J]. Journal of Cleaner Production, 2020, 265: 121777.
14	Zaremba W, Sutskever I, Vinyals O. Recurrent Neural Network Regularization[EB/OL]. (2015-02-19) [2023-12-01]. .
15	Gers Felix A, Schmidhuber Jürgen, Cummins Fred. Learning to Forget: Continual Prediction with LSTM[J]. Neural Computation, 2000, 12(10): 2451-2471.
16	Hochreiter Sepp, Schmidhuber Jürgen. Long Short-term Memory[J]. Neural Computation, 1997, 9(8): 1735-1780.
17	Cho Kyunghyun, van Merrienboer Bart, Gulcehre Caglar, et al. Learning Phrase Representations Using RNN Encoder-decoder for Statistical Machine Translation[EB/OL]. (2014-09-03) [2023-12-01]. .
18	Abbasimehr Hossein, Paki Reza. Improving Time Series Forecasting Using LSTM and Attention Models[J]. Journal of Ambient Intelligence and Humanized Computing, 2022, 13(1): 673-691.
19	Gundu Venkateswarlu, Simon Sishaj P. PSO-LSTM for Short Term Forecast of Heterogeneous Time Series Electricity Price Signals[J]. Journal of Ambient Intelligence and Humanized Computing, 2021, 12(2): 2375-2385.
20	Hu Juntao, Chen Yiyuan, Wang Wei, et al. An Optimized Hybrid Deep Learning Model for PM_2.5 and O₃ Concentration Prediction[J]. Air Quality, Atmosphere & Health, 2023, 16(4): 857-871.
21	Ganesh S S, Arulmozhivarman P, Tatavarti V S N R. Prediction of PM_2.5 Using an Ensemble of Artificial Neural Networks and Regression Models[J/OL]. Journal of Ambient Intelligence and Humanized Computing. (2018-04-30) [2023-12-01]. .
22	Zhao Jiachen, Deng Fang, Cai Yeyun, et al. Long Short-term Memory-fully Connected (LSTM-FC) Neural Network for PM_2.5 Concentration Prediction[J]. Chemosphere, 2019, 220: 486-492.
23	Liu D R, Hsu Y K, Chen H Y, et al. Air Pollution Prediction Based on Factory-aware Attentional LSTM Neural Network[J]. Computing, 2021, 103(1): 75-98.
24	Zhang Mingmin, Wu Dihua, Xue Rongna. Hourly Prediction of PM_2.5 Concentration in Beijing Based on Bi-LSTM Neural Network[J]. Multimedia Tools and Applications, 2021, 80(16): 24455-24468.
25	Ding Chen, Wang Guizhi, Zhang Xinyue, et al. A Hybrid CNN-LSTM Model for Predicting PM_2.5 in Beijing Based on Spatiotemporal Correlation[J]. Environmental and Ecological Statistics, 2021, 28(3): 503-522.
26	Liu Hui, Dong Shuqin. A Novel Hybrid Ensemble Model for Hourly PM_2.5 Forecasting Using Multiple Neural Networks: A Case Study in China[J]. Air Quality, Atmosphere & Health, 2020, 13(12): 1411-1420.
27	Yang Guangfei, Zhang Qiang, Yuan Erbiao, et al. GAT-EGRU: A Deep Learning Prediction Model for PM_2.5Coupled with Empirical Modal Decomposition Algorithm[J]. Journal of Systems Science and Systems Engineering, 2023, 32(2): 246-263.
28	Yu Xiang, Wang Jian, Hong Qingqi, et al. Transfer Learning for Medical Images Analyses: A Survey[J]. Neurocomputing, 2022, 489: 230-254.
29	An Guangzhou, Akiba Masahiro, Omodaka Kazuko, et al. Hierarchical Deep Learning Models Using Transfer Learning for Disease Detection and Classification Based on Small Number of Medical Images[J]. Scientific Reports, 2021, 11(1): 4250.
30	Xie Junyao, Zhang Laibin, Duan Lixiang, et al. On Cross-domain Feature Fusion in Gearbox Fault Diagnosis Under Various Operating Conditions Based on Transfer Component Analysis[C]//2016 IEEE International Conference on Prognostics and Health Management (ICPHM). Piscataway: IEEE, 2016: 1-6.
31	Kaur Taranjit, Tapan Kumar Gandhi. Deep Convolutional Neural Networks with Transfer Learning for Automated Brain Image Classification[J]. Machine Vision and Applications, 2020, 31(3): 20.
32	Rostami M, Kolouri S, Eaton E, et al. Deep Transfer Learning for Few-shot SAR Image Classification[J]. Remote Sensing, 2019, 11(11): 1374.
33	Mai Bui Huynh Thuy, Vinh Truong Hoang. Fusing of Deep Learning, Transfer Learning and GAN for Breast Cancer Histopathological Image Classification[C]//Advanced Computational Methods for Knowledge Engineering. Cham: Springer International Publishing, 2020: 255-266.
34	Du Yuntao, Wang Jindong, Feng Wenjie, et al. AdaRNN: Adaptive Learning and Forecasting of Time Series[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management. New York: ACM, 2021: 402-411.
35	Hosny M, Zhu Minwei, Gao Wenpeng, et al. A Novel Deep LSTM Network for Artifacts Detection in Microelectrode Recordings[J]. Biocybernetics and Biomedical Engineering, 2020, 40(3): 1052-1063.
36	Bengio Y, Simard P, Frasconi P. Learning Long-term Dependencies with Gradient Descent Is Difficult[J]. IEEE Transactions on Neural Networks, 1994, 5(2): 157-166.
37	Wang Yusen, Liao Wenlong, Chang Yuqing. Gated Recurrent Unit Network-based Short-term Photovoltaic Forecasting[J]. Energies, 2018, 11(8): 2163.
38	Wang Baowei, Kong Weiwen, Guan Hui, et al. Air Quality Forecasting Based on Gated Recurrent Long Short Term Memory Model in Internet of Things[J]. IEEE Access, 2019, 7: 69524-69534.
39	王增平, 赵兵, 纪维佳, 等. 基于GRU-NN模型的短期负荷预测方法[J]. 电力系统自动化, 2019, 43(5): 53-58.
	Wang Zengping, Zhao Bing, Ji Weijia, et al. Short-term Load Forecasting Method Based on GRU-NN Model[J]. Automation of Electric Power Systems, 2019, 43(5): 53-58.

特征	相关系数	特征	相关系数
PM_2.5	1	大气压强	0.002 149
PM₁₀	0.889 705	雨水	-0.008 090
CO	0.776 079	风速	-0.028 003
NO₂	0.635 951	温度	-0.120 102
SO₂	0.461 935	O₃	-0.126 368
露点温度	0.118 777	风速功率	-0.268 076

比例	KL散度	比例	KL散度
5:5	753.936 0	8:2	308.428 2
6:4	959.548 7	9:1	143.932 9
7:3	548.312 3

指标	窗口值
指标	2	4	8	16	32	64
RMSE	12.574	10.026	4.868	7.317	7.226	7.907
MAE	6.149	5.393	3.449	5.102	4.715	4.675
MAPE	0.093	0.077	0.083	0.076	0.081	0.079
R²	0.978	0.977	0.994	0.991	0.989	0.993

模型	RMSE	MAE	MAPE	R²
本文	5.731	3.925	0.074	0.993
LSTM	13.057	8.706	0.137	0.963
CNN-LSTM	12.180	7.720	0.131	0.968
GRU-Attention	12.965	9.598	0.186	0.964
LSTM-GRU	15.534	11.157	0.171	0.948
Stacked LSTM	14.354	9.991	0.154	0.956
TCN	12.211	7.794	0.126	0.968
TCN-LSTM	16.732	10.179	0.148	0.940

模型	RMSE	MAE	MAPE	R²
本文	6.690	4.134	0.088	0.992
LSTM	22.182	11.626	0.267	0.910
CNN-LSTM	16.707	7.980	0.146	0.949
GRU-Attention	19.272	9.419	0.158	0.932
LSTM-GRU	17.943	9.118	0.197	0.941
Stacked LSTM	16.833	8.610	0.152	0.948
TCN	18.013	8.776	0.207	0.941
TCN-LSTM	17.968	8.609	0.164	0.941