Robust Identification of Black-box Nonlinear SSM Using Expectation-maximization

doi:10.16182/j.issn1004731x.joss.25-0736

Abstract

Abstract:

To address the robust identification problem of nonlinear state space models (SSM) with outliers, missing observations, and unknown state equations, this paper proposes a modeling method based on eigenfunction expansion, Gaussian-process state-space models (GP-SSM), and Student-t distribution. The proposed approach consists of: modeling the state transition function using eigenfunctions and pre-encoding the priors of basis function coefficients via GP-SSM to enhance flexibility; modeling observations as a Student-t distribution with unknown parameters to enhance robustness against outliers; proposing the enhanced particle Gibbs with ancestor sampling (EPGAS) algorithm to adapt to state estimation in scenarios with missing observations; and deriving unknown model parameters based on the expectation maximization (EM) method. The simulation examples and benchmark model test results show that the proposed method has better performance compared to existing literature methods, and can significantly improve the model identification accuracy when there are outliers and missing observations.

Key words: robust identification, unknown system dynamics, Student-t distribution, missing and abnormal observations, expectation-maximization

CLC Number:

TP391.9

Li Xiaonan, Chao Tao, Ma Ping, Yang Ming, Wang Yuxuan. Robust Identification of Black-box Nonlinear SSM Using Expectation-maximization[J]. Journal of System Simulation, 2026, 38(5): 1146-1158.

Figures/Tables 10

Fig. 1

Table 1

Complexity of algorithm

算法名称	复杂度
EPGAS方法	$O (T N (d x + d y 3))$
鲁棒辨识方法	$O (M K 3 + M T N (d x + d y 3))$

Table 1

Table 2

Error corresponding to different iteration numbers

指标	20	50	80	100	200	300
$M A E$	2.12	1.77	1.60	1.51	1.50	1.50
$R M S E$	2.73	2.31	2.08	1.97	1.95	1.95

Table 2

Fig. 2

Table 3

Error corresponding to different abnormal ratios

指标	方法	$p a b n$
指标	方法	0.05	0.10	0.20
$M A E$	Svensson-RML	1.81	2.49	3.56
	Svensson-Bayesian	1.67	2.16	2.78
	Adhau	1.62	2.11	2.69
	本文	1.33	1.50	1.72
$R M S E$	Svensson-RML	2.58	3.71	4.70
	Svensson-Bayesian	2.25	3.01	3.78
	Adhau	2.18	2.98	3.66
	本文	1.74	1.95	2.28

Table 3

Table 4

Comparison of errors under different missing and abnormal ratios

指标	方法	$p a b n = 0.1 p m i s = 0.1$	$p a b n = 0.2 p m i s = 0.2$
$M A E$	Svensson-RML	2.78	4.52
	Svensson-Bayesian	2.55	3.44
	Adhau	2.48	3.29
	本文	1.61	1.98
$R M S E$	Svensson-RML	3.88	6.07
	Svensson-Bayesian	3.64	4.60
	Adhau	3.53	4.41
	本文	2.12	2.61

Table 4

Fig. 3

Table 5

Running time of different methods

方法	运行时间/s
Svensson-RML	$54$
Svensson-Bayesian	$55$
Adhau	$82$
本文	$60$

Table 5

Table 6

Identification errors of different methods

误差	方法	含20%异常值
$M A E$	Svensson-RML	0.27
	Svensson-Bayesian	0.23
	Adhau	0.22
	本文	0.14
$R M S E$	Svensson-RML	0.34
	Svensson-Bayesian	0.30
	Adhau	0.29
	本文	0.18

Table 6

Table 7

Algorithm errors under 20% outliers and 10% missing data

方法	$R M S E$
Svensson-RML	0.36
Svensson-Bayesian	0.32
Adhau	0.31
本文	0.21

Table 7

References 25

[1]	Courts Jarrad, Wills Adrian G, Schön Thomas B, et al. Variational System Identification for Nonlinear State-space Models[J]. Automatica, 2023, 147: 110687.
[2]	Weitkämper Felix Q. Scaling the Weight Parameters in Markov Logic Networks and Relational Logistic Regression Models[J]. Machine Learning, 2025, 114(3): 85.
[3]	Urošević Vladimir, Savić Andrej M. Temporal Clustering for Accurate Short-term Load Forecasting Using Bayesian Multiple Linear Regression[J]. Applied Intelligence, 2024, 55(1): 19.
[4]	Lu Weiguo, Ding Deng, Wu Fengyan, et al. An Efficient Gaussian Mixture Model and Its Application to Neural Networks[J]. Knowledge-Based Systems, 2025, 310: 112942.
[5]	Wang Bingchuan, Mao Yangyang, Wang Yong, et al. Gaussian Process-accelerated Multiobjective Evolutionary Design of Charging Process Considering Multiple User Preferences[J]. IEEE Transactions on Industrial Informatics, 2024, 20(8): 10123-10133.
[6]	Svensson Andreas, Schön Thomas B. A Flexible State-space Model for Learning Nonlinear Dynamical Systems[J]. Automatica, 2017, 80: 189-199.
[7]	Berntorp K. Online Bayesian Inference and Learning of Gaussian-process State-space Models[J]. Automatica, 2021, 129: 109613.
[8]	Kullberg Anton, Skog Isaac, Hendeby Gustaf. Learning Driver Behaviors Using a Gaussian Process Augmented State-space Model[C]//2020 IEEE 23rd International Conference on Information Fusion (FUSION). Piscataway: IEEE, 2020: 1-7.
[9]	Järvenpää Marko, Corander J. Approximate Bayesian Inference from Noisy Likelihoods with Gaussian Process Emulated MCMC[J]. The Journal of Machine Learning Research, 2024, 25(1): 366.
[10]	Chen Xiang, Wang Xiaogang, Liu Fei. Identification of Linear Parameter-varying System with Missing Measurement Data and Outliers[J]. Journal of the Franklin Institute, 2025, 362(4): 107547.
[11]	Kullberg Anton, Skog Isaac, Hendeby Gustaf. Online Joint State Inference and Learning of Partially Unknown State-space Models[J]. IEEE Transactions on Signal Processing, 2021, 69: 4149-4161.
[12]	Zheng Jian, Yan Ming, Li Yun, et al. An Online Identification Approach for a Nonlinear Ship Motion Model Based on a Receding Horizon[J]. Transactions of the Institute of Measurement and Control, 2021, 43(13): 3000-3012.
[13]	Chen Xin, Zhao Shunyi, Liu Fei. Robust Identification of Linear ARX Models with Recursive EM Algorithm Based on Student's T-distribution[J]. Journal of the Franklin Institute, 2021, 358(1): 1103-1121.
[14]	Karthik Raghavan Ramaswamy, Bottegal Giulio, Paul M J Van den Hof. Learning Linear Modules in a Dynamic Network Using Regularized Kernel-based Methods[J]. Automatica, 2021, 129: 109591.
[15]	Zhao Shunyi, Li Ke, Choon Ki Ahn, et al. Tuning-free Bayesian Estimation Algorithms for Faulty Sensor Signals in State-space[J]. IEEE Transactions on Industrial Electronics, 2023, 70(1): 921-929.
[16]	Yang Xianqiang, Liu Xin, Yin Shen. Robust Identification of Nonlinear Systems with Missing Observations: The Case of State-space Model Structure[J]. IEEE Transactions on Industrial Informatics, 2019, 15(5): 2763-2774.
[17]	Du Siyuan, Zhu Fengchi, Wang Zhao, et al. A Novel Lie Group Framework-based Student's t Robust Filter and its Application to INS/DVL Tightly Integrated Navigation[J]. IEEE Transactions on Instrumentation and Measurement, 2024, 73: 1-21.
[18]	Zhang Tianyu, Zhao Shunyi, Luan Xiaoli, et al. Bayesian Inference for State-space Models with Student-t Mixture Distributions[J]. IEEE Transactions on Cybernetics, 2023, 53(7): 4435-4445.
[19]	Gopaluni R B. A Particle Filter Approach to Identification of Nonlinear Processes Under Missing Observations[J]. The Canadian Journal of Chemical Engineering, 2008, 86(6): 1081-1092.
[20]	Li Xiaonan, Ma Ping, Chao Tao, et al. System Identification of Nonlinear Dynamical System with Missing Observations[C]//Methods and Applications for Modeling and Simulation of Complex Systems. Singapore: Springer Nature Singapore, 2022: 117-128.
[21]	Liu Xinpeng, Yang Xianqiang. Identification of Nonlinear State-space Systems with Skewed Measurement Noises[J]. IEEE Transactions on Circuits and Systems I: Regular Papers, 2022, 69(11): 4654-4662.
[22]	Lindsten F, Jordan M I, Schön Thomas B. Particle Gibbs with Ancestor Sampling[J]. Journal of Machine Learning Research, 2014, 15(1): 2145-2184.
[23]	Adhau Saket, Gros Sébastien, Skogestad Sigurd. Reinforcement Learning Based MPC with Neural Dynamical Models[J]. European Journal of Control, 2024, 80, Part A: 101048.
[24]	Xu Jun, Huang Xiaolin, Wang Shuning. Adaptive Hinging Hyperplanes and Its Applications in Dynamic System Identification[J]. Automatica, 2009, 45(10): 2325-2332.
[25]	Roll Jacob, Nazin Alexander, Ljung Lennart. Nonlinear System Identification via Direct Weight Optimization[J]. Automatica, 2005, 41(3): 475-490.