Prediction of Industrial Concentration Parameters Based on CauDformer Model

doi:10.16182/j.issn1004731x.joss.25-0785

Abstract

Abstract:

To solve the problems in the industrial concentration process of traditional Chinese medicine that traditional prediction methods are difficult to deal with complex characteristics such as nonlinearity, high-dimensional coupling, and highly skewed distribution and that existing deep learning models insufficiently consider causal relationships among variables, ignore frequency-domain periodic characteristics, and lack long-range dependency modeling capabilities, an improved multivariate time series prediction model CauDformer was proposed in this paper. Based on the iTransformer architecture, a causal multi-head self-attention mechanism (C-MHSA) was introduced to compulsorily constrain the dependency direction among variables by utilizing a lower triangular causal mask, guiding the model to focus on physically reasonable causal chains; a frequency-domain enhancement module was designed to extract the periodic and fluctuation characteristics of sequences using the real fast Fourier transform (RFFT), assisting the model in capturing long-range dependencies; an adaptive time-frequency fusion loss mechanism was proposed to dynamically adjust the weights of time-domain reconstruction loss and frequency-domain auxiliary loss during the training process, to mitigate the fitting instability in the early stage of model training. The Optuna framework was adopted for Bayesian hyperparameter optimization. Based on the real data of the honeysuckle triple-effect concentration process, a multi-step prediction task including 12 key variables was constructed. The experimental results indicate that under the prediction horizons of 10, 30, and 60 min, the mean absolute error and mean squared error of CauDformer are significantly superior to those of advanced models such as PatchTST, DLinear, and TimeXer, which verifies the high precision and strong robustness of this model in complex industrial process modeling.

Key words: time series prediction, industrial concentration, causal attention, frequency domain enhancement, Bayesian optimization

CLC Number:

TP391.1

Xu Kaipeng, Wang Yan, Zhang Xin, Liu Yang, Wang Zhenzhong, Liu Xiang, Ji Zhicheng. Prediction of Industrial Concentration Parameters Based on CauDformer Model[J]. Journal of System Simulation, 2026, 38(7): 2105-2118.

Figures/Tables 12

Fig. 1

Fig. 2

Table 1

Critical variables and their description

变量名称	具体描述	变量名称	具体描述
$u 1$	蒸汽压力	$u 7$	三效压力
$u 2$	进液流速	$u 8$	三效温度
$u 3$	一效压力	$v 1$	大循环出液温度
$u 4$	一效温度	$v 2$	大循环出液累计
$u 5$	二效压力	$v 3$	大循环出液流速
$u 6$	二效温度	$v 4$	大循环出液密度

Table 1

Fig. 3

Fig. 4

Fig. 5

Table 2

Fig. 6

Fig. 7

Table 3

Fig. 8

Table 4

References 27

[1]	李舒艺, 伍振峰, 岳鹏飞, 等. 中药提取液浓缩工艺和设备现状及问题分析[J]. 世界科学技术-中医药现代化, 2016, 18(10): 1782-1787.
	Li Shuyi, Wu Zhenfeng, Yue Pengfei, et al. Analysis of Current Situation and Problems in Technologies and Equipments for the Concentration of Liquid Extraction of Chinese Materia Medica (CMM)[J]. Modernization of Traditional Chinese Medicine and Materia Medica-World Science and Technology, 2016, 18(10): 1782-1787.
[2]	陈全. 中药生产浓缩工艺智能控制系统的研制[D]. 长沙: 中南大学, 2005.
[3]	Shumway R H, Stoffer D S. ARIMA Models[M]//Shumway R H, Stoffer D S. Time Series Analysis and Its Applications: With R Examples. Cham: Springer International Publishing, 2017: 75-163.
[4]	程跃. 中药制药过程控制及集成化生产若干关键问题研究[D]. 成都: 西南交通大学, 2010.
	Cheng Yue. Research on Several Key Issues for Chinese Pharmaceutical Manufacturing Process Control and Integrated Manufacture[D]. Chengdu: Southwest Jiaotong University, 2010.
[5]	张晗筱, 王瑞琪, 张亚婷. 基于CNN-LSTM的换热器污垢因子预测研究[J]. 化工学报, 2025, 76(4): 1671-1679.
	Zhang Hanxiao, Wang Ruiqi, Zhang Yating. Prediction of Scale Factor of Heat Exchangers Based on CNN-LSTM Neural Network[J]. CIESC Journal, 2025, 76(4): 1671-1679.
[6]	孟安波, 陈顺, 王陈恩, 等. 基于混沌CSO优化时序注意力GRU模型的超短期风电功率预测[J]. 电网技术, 2021, 45(12): 4692-4700.
	Meng Anbo, Chen Shun, Wang Chenen, et al. Ultra-short-term Wind Power Prediction Based on Chaotic CSO Optimized Temporal Attention GRU Model[J]. Power System Technology, 2021, 45(12): 4692-4700.
[7]	周红福, 孙凯文. 基于改进LSTM的多时隙工业时序数据预测方法研究[J]. 自动化仪表, 2025, 46(4): 86-91.
	Zhou Hongfu, Sun Kaiwen. Research on Multi-timeslot Industrial Timing Series Data Prediction Method Based on Improved LSTM[J]. Process Automation Instrumentation, 2025, 46(4): 86-91.
[8]	Vaswani A, Shazeer N, Parmar N, et al. Attention Is All You Need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 6000-6010.
[9]	黄文琦, 梁凌宇, 王鑫, 等. 基于变量选择与Transformer模型的中长期电力负荷预测方法[J]. 浙江大学学报(理学版), 2024, 51(4): 483-491, 500.
	Huang Wenqi, Liang Lingyu, Wang Xin, et al. Mid-long Term Power Load Forecasting Based Variable Selection and Transformer Model[J]. Journal of Zhejiang University(Science Edition), 2024, 51(4): 483-491, 500.
[10]	徐武, 范鑫豪, 沈智方, 等. 多尺度特征提取的Transformer短期风电功率预测[J]. 太阳能学报, 2025, 46(2): 640-648.
	Xu Wu, Fan Xinhao, Shen Zhifang, et al. Short-term Wind Power Prediction Using Transformer with Multi-scale Feature Extraction[J]. Acta Energiae Solaris Sinica, 2025, 46(2): 640-648.
[11]	Liu Yong, Hu Tengge, Zhang Haoran, et al. iTransformer: Inverted Transformers are Effective for Time Series Forecasting[EB/OL]. (2024-03-14) [2025-05-12]. .
[12]	孟祥福, 石皓源. 基于Transformer模型的时序数据预测方法综述[J]. 计算机科学与探索, 2025, 19(1): 45-64.
	Meng Xiangfu, Shi Haoyuan. Survey of Transformer-based Model for Time Series Forecasting[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(1): 45-64.
[13]	Hu Jie, Shen Li, Sun Gang. Squeeze-and-excitation Networks[C]//2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 7132-7141.
[14]	Chen Chunfu, Fan Quanfu, Panda R. CrossViT: Cross-attention Multi-scale Vision Transformer for Image Classification[C]//2021 IEEE/CVF International Conference on Computer Vision (ICCV). Piscataway: IEEE, 2021: 347-356.
[15]	Kong Lingbai, Li Wengen, Yang Hanchen, et al. CausalFormer: An Interpretable Transformer for Temporal Causal Discovery[J]. IEEE Transactions on Knowledge and Data Engineering, 2025, 37(1): 102-115.
[16]	Nauta Meike, Bucur Doina, Seifert Christin. Causal Discovery with Attention-based Convolutional Neural Networks[J]. Machine Learning and Knowledge Extraction, 2019, 1(1): 312-340.
[17]	Shi Lihui. Feedforward Control and Process Improvement for Some Time Series Disturbance Models[D]. Seattle: University of Washington, 2012.
[18]	Krupakar Hans, Kandappan V A. A Review of the Long Horizon Forecasting Problem in Time Series Analysis[EB/OL]. (2025-06-15) [2025-07-11]. .
[19]	Patro B N, Namboodiri V P, Agneeswaran V S. SpectFormer: Frequency and Attention is What You Need in a Vision Transformer[C]//2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Piscataway: IEEE, 2025: 9543-9554.
[20]	Han Honggui, Tang Zecheng, Wu Xiaolong, et al. Time-aware Fuzzy Neural Network Based on Frequency-enhanced Modulation Mechanism[J]. IEEE Transactions on Fuzzy Systems, 2024, 32(8): 4772-4786.
[21]	Nie Yuqi, Nguyen N H, Sinthong P, et al. A Time Series is Worth 64 Words: Long-term Forecasting with Transformers[EB/OL]. (2023-03-05) [2025-05-12]. .
[22]	Zeng Ailing, Chen Muxi, Zhang Lei, et al. Are Transformers Effective for Time Series Forecasting?[C]//Proceedings of the Thirty-Seventh AAAI Conference on Artificial Intelligence and Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence and Thirteenth Symposium on Educational Advances in Artificial Intelligence. Palo Alto: AAAI Press, 2023: 11121-11128.
[23]	Wang Yuxuan, Wu Haixu, Dong Jiaxiang, et al. TimeXer: Empowering Transformers for Time Series Forecasting with Exogenous Variables[EB/OL]. (2024-11-11) [2025-06-12]. .
[24]	Das A, Kong Weihao, Leach A, et al. Long-term Forecasting with TiDE: Time-series Dense Encoder[EB/OL]. (2024-04-04) [2025-06-12]. .
[25]	Ayuningtyas Hari Fristiana, Syukron Abu Ishaq Alfarozi, Adhistya Erna Permanasari, et al. A Survey on Hyperparameters Optimization of Deep Learning for Time Series Classification[J]. IEEE Access, 2024, 12: 191162-191198.
[26]	Akiba Takuya, Sano Shotaro, Yanase Toshihiko, et al. Optuna: A Next-generation Hyperparameter Optimization Framework[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining. New York: ACM, 2019: 2623-2631.
[27]	Li Jince, Shi Yilin, Li Hongguang, et al. TC-GATN: Temporal Causal Graph Attention Networks with Nonlinear Paradigm for Multivariate Time-series Forecasting in Industrial Processes[J]. IEEE Transactions on Industrial Informatics, 2023, 19(6): 7592-7601.

超参数名称	取值范围
学习率	10^-5~10^-3
损失权重起始值	0.1~0.9
损失权重终止值	0.9~0.1
批次大小	16、32、64、128
模型隐藏维度	128、256、512
编码器层数	2、3、4
Dropout率	0.05~0.25

序号	模型	10 min		30 min		60 min
序号	模型	MAE	MSE	MAE	MSE	MAE	MSE
a	TimeXer	0.043 4	0.250 8	0.092 5	1.033 6	0.137 0	1.397 8
b	TiDE	0.059 7	0.422 4	0.096 7	0.830 0	0.139 5	1.327 2
c	PatchTST	0.041 3	0.248 3	0.087 1	0.703 1	0.134 8	1.373 2
d	DLinear	0.273 3	0.508 5	0.359 9	1.028 5	0.625 8	2.253 2
e	iTransformer	0.042 4	0.283 0	0.092 3	0.764 6	0.153 7	1.355 2
f	Ours	0.027 8	0.243 4	0.057 9	0.638 1	0.095 7	1.104 8

变体	因果注意力机制	频域增强模块	自适应时频融合	结果
变体	因果注意力机制	频域增强模块	自适应时频融合	MAE	MSE
Ⅰ	×	×	×	0.153 7	1.355 2
Ⅱ	√	×	×	0.125 2	1.285 7
Ⅲ	√	√	×	0.097 0	1.113 7
Ⅳ	√	√	√	0.095 7	1.104 8
Ⅴ	×	√	√	0.110 9	1.189 8