双流框架下的改进Transformer软测量建模

doi:10.16182/j.issn1004731x.joss.24-0490

摘要/Abstract

摘要：

为解决工业过程信息因具有高度非线性和动态性，数据间存在长期依赖关系，而使时序特征难以被充分提取，提出一种双流框架下的改进Transformer软测量模型。将数据进行分割扩充处理；采用结合自注意力机制的卷积神经网络和改进Transformer模型的双流结构并行提取特征；将双流特征融合用于软测量回归；引入残差连接加快模型收敛速度，采用基于正交随机特征算法的改进多头注意力机制解决传统循环神经网络面临的梯度消失和无法并行等问题。仿真实验验证了所提方法的有效性和优越性。

关键词: 软测量, 深度学习, 卷积神经网络, Transformer, 残差连接, 脱丁烷塔

Abstract:

Industrial process information is highly nonlinear and dynamic, with long-term dependencies between data, making it difficult to adequately extract time-series features. To address this issue, an improved Transformer-based soft sensor model in a dual-stream framework was proposed. The data were segmented and expanded. The features were extracted in parallel using a dual-stream structure combining a convolutional neural network with a self-attention mechanism and the improved Transformer model. The dual-stream features were fused for soft sensor regression. Residual connections were further introduced to accelerate the convergence speed of the model, and an orthogonal random features-based improved multi-head attention mechanism was adopted to solve the vanishing gradient and non-parallelization issues faced by traditional recurrent neural networks. Numerical simulations have verified the effectiveness and superiority of the proposed method.

Key words: soft sensor, deep learning, convolutional neural network, Transformer, residual connection, debutanizer

中图分类号:

TP274

顾皓,王佳宇,熊伟丽 . 双流框架下的改进Transformer软测量建模[J]. 系统仿真学报, 2025, 37(10): 2594-2604.

Gu Hao,Wang Jiayu,Xiong Weili . Soft Sensor Modeling Based on Improved Transformer in Dual-stream Framework[J]. Journal of System Simulation, 2025, 37(10): 2594-2604.

图/表 14

图1

图2

图3

表1

表2

图4

图5

表3

图6

表4

辅助变量选择

变量	描述	变量	描述
$x 1$	顶部温度	$x 5$	第六塔板温度
$x 2$	顶部压力	$x 6$	底部温度1
$x 3$	回流流量	$x 7$	底部温度2
$x 4$	流向下一过程流量

表4

表5

图7

图8

表6

参考文献 26

[1]	Chiplunkar R, Huang Biao. Siamese Neural Network-based Supervised Slow Feature Extraction for Soft Sensor Application[J]. IEEE Transactions on Industrial Electronics, 2021, 68(9): 8953-8962.
[2]	Fortuna L, Graziani S, Rizzo A, et al. Soft Sensors for Monitoring and Control of Industrial Processes[M]. London: Springer, 2007: 27-51.
[3]	熊伟丽, 孙文心, 马君霞. 带自相关约束的NARX动态软测量模型[J]. 控制与决策, 2020, 35(4): 816-822.
	Xiong Weili, Sun Wenxin, Ma Junxia. Autocorrelation Constrained NARX Dynamic Soft Sensing Model[J]. Control and Decision, 2020, 35(4): 816-822.
[4]	代学志, 熊伟丽. 基于概率选择的主动学习智能软测量建模[J]. 系统仿真学报, 2021, 33(6): 1350-1357.
	Dai Xuezhi, Xiong Weili. Active Learning Intelligent Soft Sensor Based on Probability Selection[J]. Journal of System Simulation, 2021, 33(6): 1350-1357.
[5]	Guo Runyuan, Liu Han. A Hybrid Mechanism-and Data-driven Soft Sensor Based on the Generative Adversarial Network and Gated Recurrent Unit[J]. IEEE Sensors Journal, 2021, 21(22): 25901-25911.
[6]	周平, 张丽, 李温鹏, 等. 集成自编码与PCA的高炉多元铁水质量随机权神经网络建模[J]. 自动化学报, 2018, 44(10): 1799-1811.
	Zhou Ping, Zhang Li, Li Wenpeng, et al. Autoencoder and PCA Based RVFLNs Modeling for Multivariate Molten Iron Quality in Blast Furnace Ironmaking[J]. Acta Automatica Sinica, 2018, 44(10): 1799-1811.
[7]	Xie Xiaochen, Sun Wei, Cheung K C. An Advanced PLS Approach for Key Performance Indicator-related Prediction and Diagnosis in Case of Outliers[J]. IEEE Transactions on Industrial Electronics, 2016, 63(4): 2587-2594.
[8]	耿增显, 柴天佑. 基于LS-SVM的浮选过程工艺技术指标软测量[J]. 系统仿真学报, 2008, 20(23): 6321-6324.
	Geng Zengxian, Chai Tianyou. Soft Sensor of Technical Indices Based on LS-SVM for Flotation Process[J]. Journal of System Simulation, 2008, 20(23): 6321-6324.
[9]	Yan Weiwu, Tang Di, Lin Yujun. A Data-driven Soft Sensor Modeling Method Based on Deep Learning and Its Application[J]. IEEE Transactions on Industrial Electronics, 2017, 64(5): 4237-4245.
[10]	Yao Le, Ge Zhiqiang. Deep Learning of Semisupervised Process Data with Hierarchical Extreme Learning Machine and Soft Sensor Application[J]. IEEE Transactions on Industrial Electronics, 2018, 65(2): 1490-1498.
[11]	徐芳萍, 杨辉, 陈俊, 等. 基于迁移学习与残差注意力卷积网络的稀土元素组分含量软测量[J]. 化工学报, 2025, 76(4): 1647-1660.
	Xu Fangping, Yang Hui, Chen Jun, et al. Soft Sensor of Rare Earth Element Content with Transfer Learning and Residual Attention Convolutional Neural Network[J]. CIESC Journal, 2025, 76(4): 1647-1660.
[12]	Ma Fangyuan, Ji Cheng, Wang Jingde, et al. Early Identification of Process Deviation Based on Convolutional Neural Network[J]. Chinese Journal of Chemical Engineering, 2023, 56: 104-118.
[13]	Yuan Xiaofeng, Qi Shuaibin, A W Shardt Yuri, et al. Soft Sensor Model for Dynamic Processes Based on Multichannel Convolutional Neural Network[J]. Chemometrics and Intelligent Laboratory Systems, 2020, 203: 104050.
[14]	Zhao Yingying, He Yigang, Xing Zhikai, et al. Multibranch 1-D CNN Based on Attention Mechanism for the DAB Converter Fault Diagnosis[J]. IEEE Transactions on Instrumentation and Measurement, 2022, 71: 1-12.
[15]	Fang Chenyu, He Dakuo, Li Kang, et al. Image-based Thickener Mud Layer Height Prediction with Attention Mechanism-based CNN[J]. ISA Transactions, 2022, 128, Part B: 677-689.
[16]	黄婉蓉, 何凯, 刘坤, 等. 基于注意力机制的手写体中文字符识别[J]. 激光与光电子学进展, 2020, 57(8): 29-34.
	Huang Wanrong, He Kai, Liu Kun, et al. Handwritten Chinese Character Recognition Based on Attention Mechanism[J]. Laser & Optoelectronics Progress, 2020, 57(8): 29-34.
[17]	Vaswani A, Shazeer N, Parmar N, et al. Attention is All You Need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 6000-6010.
[18]	Geng Zhiqiang, Chen Zhiwei, Meng Qingchao, et al. Novel Transformer Based on Gated Convolutional Neural Network for Dynamic Soft Sensor Modeling of Industrial Processes[J]. IEEE Transactions on Industrial Informatics, 2022, 18(3): 1521-1529.
[19]	Chang Peng, Zhang Shirao, Wang Zichen. Soft Sensor of the Key Effluent Index in the Municipal Wastewater Treatment Process Based on Transformer[J]. IEEE Transactions on Industrial Informatics, 2024, 20(3): 4021-4028.
[20]	Tay Y, Dehghani Mostafa, Bahri D. Efficient Transformers: A Survey[J]. ACM Computing Surveys, 2022, 55(6): 109.
[21]	Zha Wenshu, Liu Yuping, Wan Yujin, et al. Forecasting Monthly Gas Field Production Based on the CNN-LSTM Model[J]. Energy, 2022, 260: 124889.
[22]	Li Chang, Huang Xiaoyang, Song Rencheng, et al. EEG-based Seizure Prediction via Transformer Guided CNN[J]. Measurement, 2022, 203: 111948.
[23]	Friedman J H. Multivariate Adaptive Regression Splines[J]. The Annals of Statistics, 1991, 19(1): 1-67.
[24]	Meng Yanmei, Lan Qiliang, Qin J, et al. Data-driven Soft Sensor Modeling Based on Twin Support Vector Regression for Cane Sugar Crystallization[J]. Journal of Food Engineering, 2019, 241: 159-165.
[25]	何罗苏阳, 熊伟丽. 助训练策略下的多模型软测量建模[J]. 系统仿真学报, 2024, 36(1): 249-259.
	He Luosuyang, Xiong Weili. Multi-model Soft Sensor Modeling Under Help-training Strategy[J]. Journal of System Simulation, 2024, 36(1): 249-259.
[26]	Fortuna L, Graziani S, Xibilia M G. Soft Sensors for Product Quality Monitoring in Debutanizer Distillation Columns[J]. Control Engineering Practice, 2005, 13(4): 499-508.

模型	层数	隐藏层节点数	学习率	迭代次数
CNN	2	32，64	0.001	100
SAT-CNN	2	72, /	0.001	100
Transformer	1	72	0.001	100
F-Transformer	1	72	0.005	100
CNN-LSTM	2	32, 64	0.005	100
CNN-Transformer	2	72, 72	0.001	100
DSF-Transformer	2	72, 72	0.005	100

模型	RMSE	MAE	R²
CNN	2.070 9	1.677 6	0.706 6
SAT-CNN	1.974 8	1.598 2	0.733 2
Transformer	1.974 1	1.548 9	0.733 4
F-Transformer	1.886 2	1.493 1	0.756 6
CNN-LSTM	1.813 5	1.432 0	0.775 0
CNN-Transformer	1.771 7	1.391 6	0.785 3
DSF-Transformer	1.642 1	1.283 5	0.815 5

模型	运行时间/s	内存占用/MB
CNN	0.07	367.60
SAT-CNN	0.06	366.72
Transformer	0.37	401.16
F-Transformer	0.16	382.20
CNN-LSTM	0.13	404.22
CNN-Transformer	0.25	405.66
DSF-Transformer	0.17	387.94

模型	层数	隐藏层节点数	学习率	迭代次数
CNN	2	32, 64	0.001	100
SAT-CNN	2	72, /	0.001	100
Transformer	1	72	0.005	50
F-Transformer	1	72	0.005	50
CNN-LSTM	2	64, 128	0.001	100
CNN-Transformer	2	72, 72	0.001	50
DSF-Transformer	2	72, 72	0.005	50

模型	RMSE	MAE	R²
CNN	0.049	0.040	0.942
SAT-CNN	0.042	0.034	0.956
Transformer	0.059	0.048	0.915
F-Transformer	0.053	0.040	0.932
CNN-LSTM	0.045	0.036	0.950
CNN-Transformer	0.042	0.031	0.957
DSF-Transformer	0.028	0.021	0.981