Research on Active Learning Method and Application Based on Covariance Matrix

doi:10.16182/j.issn1004731x.joss.20-0788

Abstract

Abstract:

Since the data collected from industrial processes often contain a large number of unlabeled samples, while the number of labeled samples is small and the cost of manual labeling is high, an active learning method based on covariance matrix is proposed. This method uses labeled samples to establish a Gaussian process regression model, and constructs the covariance matrix between the unlabeled samples, using the value of the determinant of the covariance matrix as an evaluation indicator. While selecting informative unlabeled samples, the similarity between samples is measured to avoid redundant addition of samples, which finally improves model prediction accuracy at the same labeling cost. The application simulation of the algorithm based on industrial process data verifies the feasibility and effectiveness of the proposed method.

Key words: active learning, Gaussian process regression, covariance matrix, similarity words

CLC Number:

TP274

Bowen Zhou, Weili Xiong. Research on Active Learning Method and Application Based on Covariance Matrix[J]. Journal of System Simulation, 2022, 34(3): 452-460.

Figures/Tables 13

Fig. 1

Fig. 2

Fig. 3

Fig. 4

Table 1

Fig. 5

Fig. 6

Table 2

Fig. 7

Fig. 8

Fig. 9

Table 3

Fig. 10

References 21

1	周志华. 基于分歧的半监督学习[J].自动化学报, 2013,39(11): 1871-1878.
	Zhou Zhihua. Disagreement-Based Semi-Supervised Learning[J].Acta Automatica Sinica, 2013, 39(11): 1871-1878.
2	Zhou Zhihua, Li Ming. Semi Supervised Regression with Cotraining-Style Algorithms[J]. IEEE Transactions on Knowledge and Data Engineering(S1041-4347) 2007, 19(11): 1479-1493.
3	程玉虎, 冀杰, 王雪松. 基于Help-Training的半监督支持向量回归[J].控制与决策, 2012, 27(2): 205-210, 226.
	Cheng Yuhu, Ji Jie, Wang Xuesong. Semi-Supervised Support Vector Regression Based on Help-Training[J]. Control and Decision, 2012, 27(2): 205-210, 226.
4	盛高斌, 姚明海. 基于半监督回归的选择性集成算法[J].计算机仿真, 2009, 26(10): 198-201, 318.
	Sheng Gaobin, Yao Minghai. An Ensemble Selection Algorithm Based on Semi-Supervised Regression[J]. Computer Simulation, 2009, 26(10): 198-201, 318.
5	Raju G, Cooney C. Active Learning from Process Data[J]. AIChE Journal (S0001-1541), 1998, 44: 2199-2211
6	Li Jiayi, Xin Huang, Chang Xiaoyu. A label-Noise Robust Active Learning Sample Collection Method for Multi-Temporal Urban Land-Cover Classification and Change Analysis[J]. ISPRS Journal of Photogrammetry and Remote Sensing (S0924-2716), 2020, 163: 1-17.
7	Chiu S C, Jin Z, Gu Y. Active Learning Combining Uncertainty and Diversity for Multi-Class Image Classification[J]. IET Computer Vision (S1751-9632), 2015, 9(3): 400-407.
8	Ge zhiqiang. Active Learning Strategy for Smart Soft Sensor Development under a Small Number of Labeled Data Samples[J]. Journal of Process Control (S0959-1524), 2014, 24(9): 1454-1461.
9	Shi Xudong, Xiong Weili. Approximate linear Dependence Criteria with Active Learningfor Smart Soft Sensor Design[J]. Chemometrics and Intelligent Laboratory Systems (S0169-7639), 2018, 180: 88-95.
10	FREUND Y, SEUNG H S, SHAMIR E, et al. Selective Sampling Using the Query by Committee Algorithm[J]. Machine Learning (S0885-6125), 1997, 28(2/3): 133-168.
11	Cohn D, Atlas L, Ladner R. Improving Generalization with Active Learning[J]. Machine Learning (S0885-6125), 1994, 15(2): 201-221.
12	MUSLEA I, MINTON S, KNOBLOCK C A. Active Learning with Multiple Views[J].Journal of Artificial Intelligence Research (S1615-147X), 2006, 27: 203-233.
13	Yong Z, Meng J E. Sequential Active Learming Using Meta-Cognitive Extreme Learning Machine[J]. Neurocomputing (S0925-2312), 2016, 173(3): 835-844.
14	Ge Zhiqiang. Active Probabilistic Sample Selection for Intelligent Soft Sensing of Industrial Processes[J]. Chemometrics & Intelligent Laboratory Systems (S0169-7439), 2016, 151: 181-189.
15	Tong Jun, Rui Hu, Xi Jiangtao, et al. Linear Shrinkage Estimation of Covariance Matrices Using Low-Complexity Cross-Validation[J]. Signal Processing (S0165-1684), 2018, 148: 223-233.
16	Qian Junhui, He Zishu, Xie Julan, et al. Null Broadening Adaptive Beamforming Based on Covariance Matrix Reconstruction and Similarity Constraint[J] Signal Processing (S1687-6172), 2017, 2017(1): 1-10.
17	何志昆, 刘光斌, 赵曦晶, 等. 高斯过程回归方法综述[J].控制与决策, 2013, 28(8): 1121-1129.
	He Zhikun, Liu Guangbin, Zhao Xijing, et al. Overview of Gaussian Process Regression[J]. Control and Decision, 2013, 28(8): 1121-1129, 1137.
18	Fernando D S, Adriana N A. Biomass Estimation in Batch Biotechnological Processes by Bayesian Gausian Process Regression[J]. Computers & Chemical Engineering(S0098-1354), 2008, 32(12): 3264-3273.
19	Alexandra G, Jus K, Tor A. Explicit Stochastic Predictive Control Of Combustion Plants Based on Gaussian Process [J].Automatica(S 0005-1098), 2008,44(6): 1621-1631.
20	曹鹏飞, 罗雄麟. 化工过程软测量建模方法研究进展[J].化工学报, 2013, 64(3): 788-800.
	Cao Junfei, Luo Xionglin. Modeling of Soft Sensor for Chemical Process[J]. CIESC Journal,2013, 64(3): 788-800.
21	Fortuna L, Graziani S, Xibilia M G. Soft Sensors for Product Quality Monitoring in Debutanizer Distillation Columns[J]. Control Engineering Practice (S0967-0661), 2005, 13(4): 499-508.

距离均值	Variance	Proposed
点1	0.443	4.911
点2	0.715	3.424
点3	0.389	3.145
点4	0.432	5.194
点5	0.711	3.486

输入变量	描述	输入变量	描述
X₁	塔顶温度	X₅	第6块塔板温度
X₂	塔底温度	X₆	塔底温度1
X₃	回流量	X₇	塔底温度2
X₄	下一级流量

Method	标记样本数	RMSE	标记样本数	RMSE
Random	100	0.158 9	280	0.115 7
Variance	100	0.136 1	280	0.092 6
Proposed	100	0.121 8	280	0.080 3

[1]	Hailong Zhu, Ruxia Jia, Liang Zhang, Wei He. Turbofan Engine Fault Prediction Based on Evidential Reasoning and Belief Rule Base [J]. Journal of System Simulation, 2022, 34(9): 2074-2086.
[2]	Qi Cheng, Xiong Weili. A Just-in-time Learning Soft Sensing Modeling Method Based on Bayesian Gaussian Mixture Model [J]. Journal of System Simulation, 2019, 31(8): 1555-1561.
[3]	Peng Huilai, Xiong Weili. GPR Modeling Method Based on Kernel Slow Feature Analysis and Time Delay Estimation [J]. Journal of System Simulation, 2019, 31(8): 1562-1571.
[4]	Zhao Shuai, Shi Xudong, Xiong Weili. A Hierarchical Integrated Soft Sensing Modeling Method for Gauss Process Regression [J]. Journal of System Simulation, 2019, 31(10): 2042-2051.